AWS Machine Learning Blog · Daxeshkumar Patel · 3 小时前模型评分48
用 Crawl4AI 为 Amazon Bedrock Knowledge Bases 准备网页内容
Prepare web content for Amazon Bedrock Knowledge Bases with Crawl4AI
导读
针对需要精细控制网页内容的场景，可在 Amazon Bedrock Knowledge Bases 的托管网页爬虫之外，用开源 Python 爬虫 Crawl4AI 做预处理：只提取指定页面区域、去除导航等模板内容，并为每页生成 Markdown 和元数据文件写入 Amazon S3，再由知识库的 S3 数据源摄取。该方案适合批量导入已知网站，不适用于推理时的实时网页访问。
分类置信度
高
- 官方或准官方来源，且无冲突类别。
- 正文、可信日期、主分类和证据摘录完整。
来源级别：官方一手
事件类型：分析与教程
证据摘录
- “In this post, we show you how to choose the appropriate web ingestion path and implement custom preparation for content that needs additional processing.”
- “This is batch ingestion of known web properties, not real-time web access at inference time such as Web Grounding with Amazon Nova.”
- “The workflow processes public pages only and doesn’t implement authenticated sessions.”
- “The maxDepth and maxPages values limit the first crawl to one link level and 25 pages.”
- “Each run incurs Fargate charges for the configured task size, which defaults to 2 vCPU and 4 GB.”
来源：AWS Machine Learning Blog · aws.amazon.com