The pitch is genuinely attractive: no API key, no per-token cost, no network round trip, and user data that never leaves the device. For a note-taking app, a keyboard, or anything handling health or financial records, that last point alone can decide the architecture. Then you look at the numbers. A small quantised language model is a download of one to three gigabytes, needs most of that resident in RAM while it runs, and generates tokens at a rate that would embarrass a 2019 API. Both of those pictures are accurate. The question is which one applies to your feature. What actually runs well on a phone The tasks where a small local model is genuinely good are narrower than the demos suggest, and they cluster around one property: short input, short output, no world knowledge required. Task On-device viability Why Text classification, intent detection Excellent Tiny models, often not LLMs at all Embeddings for local semantic...