Convert web pages and extract links automatically
Workflow Description
Automation that transforms web page content into different formats and extracts links efficiently. Processes data in controlled batches and regulates requests to ensure optimal performance without server overload.
How it works
- 1.Manual trigger to start content processing
- 2.Convert pages to desired format through HTTP requests
- 3.Process data in separate batches with a defined maximum limit
- 4.Extract all links from the converted content
- 5.Pause between requests to prevent system overload
- 6.Return the list of extracted links
Use cases
- Extract links from multiple pages and save them to a database
- Convert HTML content to other formats and detect broken links
- Automatically gather links from search results or product listings
Requirements
- Web address or list of links to process
- API or server capable of converting pages
- Define reasonable batch limits to prevent connection failures
Service Value
Ready-made workflow template for automation delivery and service execution.
Apps Used
Details
How to Use
- 1.Click "Download Template"
- 2.Open your n8n dashboard
- 3.Go to Workflows > Import from File
- 4.Select downloaded file and configure credentials
Nodes Used (17)
When clicking ‘Test workflow’
Manual Trigger
Wait
Wait
Sticky Note36
Sticky Note
Sticky Note28
Sticky Note
Connect to your own data source
No Op
Get urls from own data source
No Op
Example fields from data source
Set
Sticky Note33
Sticky Note
Sticky Note34
Sticky Note
Sticky Note35
Sticky Note
Sticky Note37
Sticky Note
40 items at a time
Limit
10 at a time
Split In Batches
Markdown data and Links
Set
Split out page URLs
Split Out
Retrieve Page Markdown and Links
HTTP Request
Sticky Note38
Sticky Note