mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-02-06 04:45:10 -05:00
cda9572acd
Add WebFetchComponent for fast HTTP-based page fetching without browser overhead. Uses trafilatura for intelligent content extraction. Commands: - fetch_webpage: Extract main content as text/markdown/xml - Removes navigation, ads, boilerplate automatically - Extracts page metadata (title, description, author, date) - Extracts and lists page links - Much faster than Selenium-based read_webpage - fetch_raw_html: Get raw HTML for structure inspection - Optional truncation for large pages Features: - Trafilatura-powered content extraction (best-in-class accuracy) - Automatic link extraction with relative URL resolution - Page metadata extraction (OG tags, meta tags) - Configurable timeout, max content length, max links - Proper error handling for timeouts and HTTP errors - 19 comprehensive tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🚀 AutoGPT-Forge: Build Your Own AutoGPT Agent! 🧠
🌌 Dive into the Universe of AutoGPT Creation! 🌌
Ever dreamt of becoming the genius behind an AI agent? Dive into the Forge, where you become the creator!
🛠️ Why AutoGPT-Forge?
- 💤 No More Boilerplate! Don't let the mundane tasks stop you. Fork and build without the headache of starting from scratch!
- 🧠 Brain-centric Development! All the tools you need so you can spend 100% of your time on what matters - crafting the brain of your AI!
- 🛠️ Tooling ecosystem! We work with the best in class tools to bring you the best experience possible!
🚀 Get Started!
The getting started tutorial series will guide you through the process of setting up your project all the way through to building a generalist agent.