Chunking hooks

Chunks are the retrievable units of unstructured content. A chunking hook changes how a source's content is split, so retrieval returns passages that match how your documents are actually organized.

Implementing a ChunkingHook

A chunking hook extends ChunkingHook and implements execute, which receives the content and returns the list of chunks. Optional pre and post steps let you clean content before splitting or adjust the chunks afterward.

class HeadingChunkingHook(ChunkingHook):
    async def execute(self, content: str) -> list[str]:
        # Split on headings so each section is its own retrievable chunk.
        return split_by_heading(content, level=2)

Register it by overriding chunking_hook() on the connector or on a specific entity. Several built-in connectors do this already, including Confluence, Notion, and Google Meet.

When to customize chunking

The default splitting works for most prose. Customize it when a source has structure worth preserving: a runbook split by procedure, a transcript split by speaker turn, an API reference split by endpoint. Better boundaries mean more precise retrieval.

Related

• Chunks: the concept behind the unit.
• Entity extraction hooks: control what is pulled from each chunk.