Transform site crawl data into AI-ready files
Loading workflow…
Workflow Description
Automation that receives website crawl data through a form, classifies and summarizes it using OpenAI, then converts it into structured files ready for immediate AI model processing without manual intervention.
How it works
- 1.Receive crawled site data via dedicated web form
- 2.Automatically classify content by type and relevance
- 3.Summarize lengthy text passages using OpenAI
- 4.Extract key information from uploaded documents
- 5.Generate formatted files ready for immediate AI use
Use cases
- Prepare crawled website content for training custom AI models
- Automate market research and competitor analysis data processing
- Extract structured data from web pages to feed machine learning systems
Requirements
- Active OpenAI API key with sufficient quota
- Web form configured to receive crawl data
- File processing capability to extract text content
Service Value
Ideal as a smart automation service combining integrations and AI to produce ready-to-use results.
Apps Used
Details
How to Use
- 1.Click "Download Template"
- 2.Open your n8n dashboard
- 3.Go to Workflows > Import from File
- 4.Select downloaded file and configure credentials
Nodes Used (11)
Sticky Note
Sticky Note
Sticky Note1
Sticky Note
Sticky Note2
Sticky Note
Set useful fields
Set
Text Classifier
If
OpenAI Chat Model
OpenAI
No Operation, do nothing
No Op
Sticky Note3
Sticky Note
Filter URLs
Filter
Sticky Note4
Sticky Note
Sticky Note5
Sticky Note
Sticky Note6
Sticky Note
Sticky Note7
Sticky Note
Sticky Note8
Sticky Note
Sticky Note9
Sticky Note
Summarize - Concatenate
Summarize
Set Fields - llms.txt Content
Set
Set Field - llms.txt Row
Set
Sticky Note10
Sticky Note
Form - Screaming frog internal_html.csv upload
Form Trigger
Extract data from Screaming Frog file
Extract From File
Generate llms.txt file
Convert To File
Fresh workflows, straight to your inbox
Get newly published DROPIDEA workflows, a clear overview of what they do, and a link to explore them.
- New workflows
- Clear overviews
- Setup guides