← All posts

Building a production RAG pipeline: lessons from the trenches

Chunking, evaluation and the retrieval bugs nobody warns you about.

Luqmaan

Nov 12, 2026 · 3 min read · Updated Nov 20, 2026

On this page

My first RAG prototype took an afternoon. It answered every question I threw at it, and I was convinced the hard part was done. It wasn't.

Why prototypes lie

When you write the test questions yourself, you phrase them the way your documents are phrased. Real users don't.

Key idea: build your eval set from real questions, not your own guesses.

Chunking is a product decision

Split on document structure first, then on size.

python
def chunk(doc, max_tokens=400):
    for section in split_by_headings(doc):
        yield from split_by_size(section, max_tokens, prefix=section.heading)

Evaluate before you tune

Measure retrieval and generation separately — a perfect answer from the wrong source is still a bug.

Takeaways

Start with real questions, chunk by structure, and build the eval set before you tune.

Written by

Luqmaan

I design and ship end-to-end web applications and LLM-powered features — from database schema to polished interface.

Enjoyed this?

Follow new posts via RSS, or get in touch.

← PreviousType-safe APIs end to end with tRPC and Zod

Keep reading

All posts →
↑↓ to navigate↵ to openesc to close