Process large data batches and send them incrementally
Workflow Description
A workflow that divides large datasets into manageable batches, controlling execution rate and delays between steps. Supports HTTP requests to transfer data to external systems efficiently while preventing server overload.
How it works
- 1.Trigger the workflow manually or based on a specific event
- 2.Split the data into batches of consistent size
- 3.Apply time delays between processing each batch
- 4.Send processed data via HTTP requests to the destination endpoint
Use cases
- Synchronize large contact lists between different platforms
- Process sales records or customer data without overwhelming the server
- Export database records to another application in an organized manner
Requirements
- A data source containing multiple records or entries
- An API endpoint or HTTP receiver that accepts requests
- Proper batch size configuration based on target system capacity
Service Value
Ready-made workflow template for automation delivery and service execution.
Apps Used
Details
How to Use
- 1.Click "Download Template"
- 2.Open your n8n dashboard
- 3.Go to Workflows > Import from File
- 4.Select downloaded file and configure credentials
Nodes Used (17)
When clicking ‘Test workflow’
Manual Trigger
Wait
Wait
Sticky Note36
Sticky Note
Sticky Note28
Sticky Note
Connect to your own data source
No Op
Get urls from own data source
No Op
Example fields from data source
Set
Sticky Note33
Sticky Note
Sticky Note34
Sticky Note
Sticky Note35
Sticky Note
Sticky Note37
Sticky Note
40 items at a time
Limit
10 at a time
Split In Batches
Markdown data and Links
Set
Split out page URLs
Split Out
Retrieve Page Markdown and Links
HTTP Request
Sticky Note38
Sticky Note