Preparing web content for Bedrock knowledge bases with Crawl4AI
What happened
On 3 November 2026, the AWS Machine Learning Blog published an article on using the open-source Python crawler Crawl4AI to pre-process web sources that need fine-grained control in Amazon Bedrock Knowledge Bases: each web page is written as a Markdown file plus a metadata sidecar file and uploaded to Amazon S3, then ingested through the knowledge base's S3 data source. The report does not mention earlier implementations or previous progress.
Written by AI from the reports · updated 3 h ago
Timeline
Follow the reports to see the event from each side.
- AWS Machine Learning BlogPrepare web content for Amazon Bedrock Knowledge Bases with Crawl4AI
This post shows how to use Crawl4AI, an open source web crawler, as a content preparation layer for Amazon Bedrock Knowledge Bases, handling JavaScript rendering, selective extraction, and custom metadata enrichment before web content reaches an Amazon S3 data source.
Heat of this event
Current heat 9·Peak in the comparable range 9（Nov 3 04:00）·Change over 24 hours in the comparable range –
The trend compares only the same accounts observed without a break, so its range can be narrower than the current heat count. Move the pointer or tap the chart to see each hour; with a keyboard, use the left and right arrow keys.