Process large datasets in controlled batches
Workflow Description
Automated workflow that splits large data collections into manageable batches, processes each batch sequentially, and controls processing speed through configurable limits and wait times between executions.
How it works
- 1.Trigger the workflow manually or automatically when data arrives
- 2.Divide data into batches according to the specified batch size
- 3.Process each batch individually with pauses between iterations
- 4.Send processed data via HTTP requests or to connected applications
Use cases
- Handle large customer or product lists without overwhelming backend systems
- Send bulk emails or notifications to thousands of users in organized waves
- Synchronize data between multiple systems reliably and efficiently
Requirements
- A data source containing lists or arrays to process
- Configured batch size appropriate for your target system's capacity
Service Value
Ready-made workflow template for automation delivery and service execution.
Apps Used
Details
How to Use
- 1.Click "Download Template"
- 2.Open your n8n dashboard
- 3.Go to Workflows > Import from File
- 4.Select downloaded file and configure credentials
Nodes Used (17)
When clicking ‘Test workflow’
Manual Trigger
Wait
Wait
Sticky Note36
Sticky Note
Sticky Note28
Sticky Note
Connect to your own data source
No Op
Get urls from own data source
No Op
Example fields from data source
Set
Sticky Note33
Sticky Note
Sticky Note34
Sticky Note
Sticky Note35
Sticky Note
Sticky Note37
Sticky Note
40 items at a time
Limit
10 at a time
Split In Batches
Markdown data and Links
Set
Split out page URLs
Split Out
Retrieve Page Markdown and Links
HTTP Request
Sticky Note38
Sticky Note