What Is RAG Chunking? (Chunk Size and Overlap)

Before a document can be embedded and retrieved, it's split into smaller pieces called chunks. Chunking exists because embedding models and retrieval systems work better over focused passages than entire documents — a whole 50-page PDF embedded as one vector loses most of its specific meaning.

Chunk size

Chunk size is the target length of each piece, usually measured in tokens. Smaller chunks retrieve more precisely but produce more chunks (and more embedding cost, plus more vectors to search). Larger chunks reduce chunk count but can dilute relevance by mixing multiple topics into one vector.

Chunk overlap

Overlap repeats a portion of the previous chunk at the start of the next one, so ideas that span a chunk boundary aren't split awkwardly in half. The "effective step" — how far each chunk advances past the last — is chunk size minus overlap:

If overlap is greater than or equal to chunk size, the effective step becomes zero or negative and chunking can never progress through the document — that combination is always invalid.

There's no universally correct chunk size

The right chunk size and overlap depend on your documents (dense technical text vs. conversational transcripts), your embedding model, and how your retrieval step uses the results. Common starting points are chunk sizes in the 200–800 token range with 10–20% overlap, but treat that as a starting point to test, not a rule.

Use the RAG Chunk Calculator to see how a given chunk size and overlap translate into chunk count for your document size.