Hola, I’m Hieu, a Software Engineer

The usual process is: open a leaderboard, sort by average score, take the top model that fits the budget, move on. It produces a defensible choice roughly half the time, and the half where it fails, it fails expensively — because switching embedding models means re-embedding everything and rebuilding every index. Here are the questions that predict the outcome better than rank does. Does it work on your text? A benchmark average is a weighted mixture of tasks, most of which are not yours. A model that leads overall can trail badly on the one thing you need — legal clauses, product SKUs, Vietnamese customer support tickets, code. The evaluation that matters takes an afternoon: Collect 50-100 real queries from your logs (or write them, if you have no traffic yet). For each, mark which documents in your corpus should be retrieved. This labelling is the actual work. Embed the corpus with each candidate model, run the queries, measure recall@k for the k you actually feed to the mo...

A retrieval system that answers “what is our refund window?” correctly and fails on “does the refund window differ for enterprise customers?” usually does not have a model problem. It has a chunk that contains the general policy and a different chunk, retrieved never, that contains the exception. Chunking decides what a single retrievable unit is . Get it wrong and no amount of reranking, prompt engineering or model upgrading recovers the information you split apart. Fixed-size splitting is a baseline, not a strategy The default everyone starts with — split every N characters with M overlap — is worth understanding precisely, because it is the thing you will be comparing against. def fixed_chunks (text, size = 1000 , overlap = 200 ): chunks, start = [], 0 while start < len (text): chunks.append(text[start:start + size]) start += size - overlap return chunks What it gets right: uniform chunks, predictable cost, no assumptions about docum...