01The problem
Teams keep knowledge in PDFs and docs that nobody can search well. Keyword search misses questions phrased differently from the source text.
02Approach
- Structure-aware chunking (headings kept with their paragraphs)
- Hybrid retrieval: vector + keyword, then reranking
- Answers stream with inline citations to the exact source chunk
03Architecture
code
Upload → Parse → Chunk → Embed → pgvector
Question → Retrieve (hybrid) → Rerank → LLM → Streamed answer + citations04Trade-offs & decisions
- Hybrid search vs. vectors only: more moving parts, but far better on exact names and codes.
- Smaller model for reranking: keeps latency down at a small quality cost.
05Results & evaluation
Replace this with real numbers once you have them — e.g. retrieval hit-rate on your eval set, latency, cost per question.
06What I’d do differently
Build the eval set first. Several "improvements" I made before measuring turned out to make answers worse.