Choosing Chunk Size and Overlap

There's no chunk size that's correct for every document set — but there are starting points worth testing before you tune further.

Start from your content, not a default

• Dense technical text (docs, legal, code) often works better with smaller chunks (200–400 tokens) — a paragraph usually contains one complete idea.
• Conversational or narrative text (transcripts, support tickets) often needs larger chunks (500–800 tokens) to preserve enough context to be useful alone.
• Structured content (FAQs, tables) often chunks best along its natural boundaries — one FAQ entry or one table row-group per chunk — rather than a fixed token count.

Overlap: enough to preserve context, not so much you triple your cost

Overlap exists so an idea that spans a chunk boundary isn't split awkwardly in half. 10–20% of chunk size is a common starting range. Overlap above ~30% usually adds embedding cost and near-duplicate chunks without much retrieval benefit.

How to actually tune it

1. Pick a reasonable starting point (e.g. 500-token chunks, 50-token overlap).
2. Run your real queries against it and look at what gets retrieved.
3. If answers miss context that's split across chunks, increase chunk size or overlap.
4. If retrieved chunks contain irrelevant mixed topics, decrease chunk size.

Use the RAG Chunk Calculator to see how a candidate chunk size and overlap translate into chunk count and cost before you commit to re-indexing a large document set.