Filter Previously Processed Items with a Supplied ID Ledger and Output the Updated Ledger

How it works

1. 1.Manual Trigger starts the run
2. 2.Code node emits a fictional batch and ledger
3. 3.Set node defines id_field and max_ledger_size
4. 4.Classify node normalizes IDs and tags each item
5. 5.IF routes new items to accepted, others to excluded
6. 6.Merge appends both routes
7. 7.Code node builds one report and the trimmed processed_ids

Use cases

• Prevent duplicate messages across repeated runs
• Clean overlapping order batches before processing
• Inspect and bound the dedup history size

Requirements

• JSON input containing items and processed_ids
• Set id_field and max_ledger_size in Dedup Settings
• Persist the output ledger between runs outside this workflow

Setup and testing in n8n

1. Import the JSON into n8n
2. Run manually and verify new_count=3 and excluded_count=3
3. Check processed_ids holds only 5 IDs
4. Set items to an empty array and confirm zero counts with the old ledger
5. Change max_ledger_size to 2 and verify trimming

Service Value

Ready-made workflow template for automation delivery and service execution.