mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-02-10 06:45:28 -05:00
c1a1767034
- Add generate_block_docs.py script that introspects block code to
generate markdown
- Support manual content preservation via <!-- MANUAL: --> markers
- Add migrate_block_docs.py to preserve existing manual content from git
HEAD
- Add CI workflow (docs-block-sync.yml) to fail if docs drift from code
- Add Claude PR review workflow (docs-claude-review.yml) for doc changes
- Add manual LLM enhancement workflow (docs-enhance.yml)
- Add GitBook configuration (.gitbook.yaml, SUMMARY.md)
- Fix non-deterministic category ordering (categories is a set)
- Add comprehensive test suite (32 tests)
- Generate docs for 444 blocks with 66 preserved manual sections
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
<!-- Clearly explain the need for these changes: -->
### Changes 🏗️
<!-- Concisely describe all of the changes made in this pull request:
-->
### Checklist 📋
#### For code changes:
- [x] I have clearly listed my changes in the PR description
- [x] I have made a test plan
- [x] I have tested my changes according to the test plan:
<!-- Put your test plan here: -->
- [x] Extensively test code generation for the docs pages
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces an automated documentation pipeline for blocks and
integrates it into CI.
>
> - Adds `scripts/generate_block_docs.py` (+ tests) to introspect blocks
and generate `docs/integrations/**`, preserving `<!-- MANUAL: -->`
sections
> - New CI workflows: **docs-block-sync** (fails if docs drift),
**docs-claude-review** (AI review for block/docs PRs), and
**docs-enhance** (optional LLM improvements)
> - Updates existing Claude workflows to use `CLAUDE_CODE_OAUTH_TOKEN`
instead of `ANTHROPIC_API_KEY`
> - Improves numerous block descriptions/typos and links across backend
blocks to standardize docs output
> - Commits initial generated docs including
`docs/integrations/README.md` and many provider/category pages
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
631e53e0f6. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2.0 KiB
2.0 KiB
Jina Search
Blocks for extracting website content and performing web searches using Jina AI.
Extract Website Content
What it is
This block scrapes the content from the given web URL.
How it works
The block sends a request to the given URL, downloads the HTML content, and uses content extraction algorithms to identify and extract the main text content of the page.
Inputs
| Input | Description | Type | Required |
|---|---|---|---|
| url | The URL to scrape the content from | str | Yes |
| raw_content | Whether to do a raw scrape of the content or use Jina-ai Reader to scrape the content | bool | No |
Outputs
| Output | Description | Type |
|---|---|---|
| error | Error message if the content cannot be retrieved | str |
| content | The scraped content from the given URL | str |
Possible use case
A data analyst could use this block to automatically extract article content from news websites for sentiment analysis or topic modeling.
Search The Web
What it is
This block searches the internet for the given search query.
How it works
The block sends the search query to a search engine API, processes the results, and returns them in a structured format.
Inputs
| Input | Description | Type | Required |
|---|---|---|---|
| query | The search query to search the web for | str | Yes |
Outputs
| Output | Description | Type |
|---|---|---|
| error | Error message if the operation failed | str |
| results | The search results including content from top 5 URLs | str |
Possible use case
A content creator could use this block to research trending topics in their field, gathering ideas for new articles or videos.