Chunking Strategies for Better RAG

Why it's make-or-break

Retrieval can only return chunks you created. Bad chunks — too big, too small, or split mid-thought — cap how good your answers can ever be.

Strategies, simplest first

• Fixed-size with overlap — e.g., 400 tokens, 50 overlap. Easy and surprisingly strong.
• Recursive/structural — split on headings → paragraphs → sentences, keeping related text together.
• Semantic — group sentences by similarity so each chunk is one coherent idea.

Practical advice

Start with fixed-size + overlap, attach metadata (source, title) to every chunk, and only move to fancier strategies if evaluation shows retrieval is the bottleneck. Measure, don't guess.