Chuyển đến nội dung chính

Bài đăng

Hiển thị các bài đăng có nhãn Reliability

Shipping an LLM feature: the demo is 20% of the work

The demo always works. You paste a good input, the model returns something impressive, you show it in standup, and everyone agrees it should ship. Two weeks later it is in production and you are reading a support ticket where the model told a customer their refund was approved. The demo is the easy 20%. The other 80% is everything that stops a probabilistic function from taking your application down with it — not because the model is bad, but because you wired a non-deterministic component into a system that was designed on the assumption that functions return what their signature says. What follows is the order I actually build these things in. The order matters more than any individual technique, because each step tells you whether the next one is even worth doing. If you skip the eval set you will spend three days tuning a prompt and have no way to know if you improved it. If you skip the guardrails you will find out about your output-format bug from a user. None of this is speci...