mirror of
https://github.com/Significant-Gravitas/AutoGPT.git
synced 2026-04-08 03:00:28 -04:00
fdb7ff8111
Move remaining block docs to block-integrations/ subdirectory: - Delete old docs from docs/integrations/ root - Add new docs under docs/integrations/block-integrations/ - Add guides/ directory with LLM and voice provider docs - Update SUMMARY.md with correct navigation structure Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1.4 KiB
1.4 KiB
Jina Chunking
Blocks for splitting text into semantic chunks using Jina AI.
Jina Chunking
What it is
Chunks texts using Jina AI's segmentation service
How it works
This block uses Jina AI's segmentation service to split texts into semantically meaningful chunks. Unlike simple splitting by character count, Jina's chunking preserves semantic coherence, making it ideal for RAG applications.
Configure maximum chunk length and optionally return token information for each chunk.
Inputs
| Input | Description | Type | Required |
|---|---|---|---|
| texts | List of texts to chunk | List[Any] | Yes |
| max_chunk_length | Maximum length of each chunk | int | No |
| return_tokens | Whether to return token information | bool | No |
Outputs
| Output | Description | Type |
|---|---|---|
| error | Error message if the operation failed | str |
| chunks | List of chunked texts | List[Any] |
| tokens | List of token information for each chunk | List[Any] |
Possible use case
RAG Preprocessing: Chunk documents for retrieval-augmented generation systems.
Embedding Preparation: Split long texts into optimal chunks for embedding generation.
Document Processing: Break down large documents for analysis or storage in vector databases.