Shipping an on-device LLM in a consumer iPhone app.

Ragondin Assistant is a private AI assistant that runs entirely on the phone: no account, no backend, no data collection. The interesting engineering is not the chat UI. It is two questions that sound small and are not: which model do you put on the phone, and how do you get several gigabytes of weights there without ruining anyone’s day? These are our notes on both.

One model is the wrong number

The single biggest lesson: do not ship a model. iPhones in the wild range from devices that struggle to spare 1.5 GB of RAM to devices carrying Apple’s own foundation model. Any single choice either abandons the low end or wastes the high end. So the app ships a menu, and the device decides which entries are real options:

Model	Download	Needs	Why it exists
Apple Intelligence	Nothing	A phone that has it	Built into the OS. A real AI answer in the first minute, with zero downloaded bytes.
Qwen 3.5 0.8B	533 MB	1.5 GB RAM	The floor. Ultra-light, fastest responses, runs on old phones.
Qwen 3.5 2B	1.3 GB	2.5 GB RAM	Light and fast, noticeably smarter.
Qwen 3.5 4B	2.7 GB	4 GB RAM	Balanced quality for mid-range devices.
Gemma 4 E2B	3.1 GB + 1 GB	4 GB RAM	Can see images. The second gigabyte is the vision adapter, shipped as its own file.
Bonsai 27B	3.8 GB	8 GB RAM	Flagship-class quality, compressed to 1-bit. High-end devices only.

The menu framing changes how the app greets a new user. On a phone with Apple Intelligence, the built-in model is offered first: the user is talking to a real model before they have downloaded anything. Everyone else picks a rung that matches their device and their patience. Nobody stares at a progress bar before they have seen any value.

The RAM number on the box is not the RAM number

Every model in the menu carries a required-RAM figure, and the app refuses to load a model past it. But the check is subtler than a comparison, because the phone lies twice.

First, iOS reports total memory slightly under the nominal spec, so a naive check would tell a 4 GB phone it has too little RAM for a 4 GB model. The gate allows about half a gigabyte of slack for that. Second, and more important, total RAM is not available RAM. Before a load, the app also asks the OS how much this process may still allocate and compares that to the model’s expected peak. Model weights end up wired and unswappable: loading past that line does not degrade gracefully, it takes the app down mid-answer, and users experience that as “the app is broken”, not “my phone is small”. When a model does not fit, the app says so in plain words and points at a smaller one.

A 27B model on a phone, and what it actually costs

The top of the menu is the instructive case. Bonsai 27B is a 27-billion-parameter model quantized to 1-bit, which squeezes the weights into a 3.8 GB file, smaller than Gemma’s 4-billion-parameter download. The file size is a lie of omission: at inference the model peaks around 5.2 GB, which is why it is only offered on 8 GB devices.

It also broke our defaults. The app’s device-tuned settings, a 32K context with 8-bit KV cache on high-end phones, are sized for 2 to 4 GB models; with 27B weights they sail past the iOS memory limit as the context fills. This one model overrides them: an 8K context and a 4-bit KV cache, which keeps the peak near the weights-plus-overhead baseline. The lesson generalizes: a model’s cost is not its file size, it is weights plus context plus cache, and the biggest model in your menu will be the one that finds out whether you know that.

Weights are content, not code

None of the models live in the app binary. The binary stays a normal-sized App Store download; weights come later, one time, from a CDN bucket, with the user’s consent and a progress bar. Downloads are resumable, because a 3.8 GB file over a phone connection will be interrupted, and restarting from zero is how you teach users to give up.

Every file’s SHA256 hash is pinned in the app release, and every completed download is verified against it before the file is ever loaded. If a file is rotated upstream, the hash must ship in the same release or downloads fail verification, which is the point: the alternative is silently loading whatever the network handed you into memory. One honest UI detail this forces: hashing gigabytes takes real seconds and fires no progress events, so the progress bar needs an explicit “verifying” phase, or the user watches a full bar do nothing and assumes a hang.

The same pipeline treats sources as an implementation detail. Most weights are mirrored on the app’s own CDN; one model still ships straight from its publisher until the mirror lands, and the speech models come from a third place. The manifest is data. Swapping a URL is not an app release.

The download can be perfect and still kill you

A verified file on disk is not the end. The native load itself can die: the OS reclaims memory at the wrong moment, the GPU hangs, the process is killed with no exception to catch. From the app’s point of view there is nothing to handle, because the app no longer exists.

So the load is wrapped in a dead-man’s switch. A marker is persisted just before a multi-gigabyte load starts and cleared when it succeeds. If the marker is still there at next launch, the previous session died mid-load, and that death counts against the model. Two consecutive mid-load deaths and the app refuses to load that model again until it is re-downloaded, rather than letting the user relaunch into the same crash forever. A crash loop you walked into once is a bug; one you let the user repeat is a design decision.

What generalizes

• Ship a menu of models, not a model. Let the device rule out the ones it cannot carry.
• Gate on memory twice: the device’s total RAM, and what the OS will actually still give you.
• A model’s cost is weights plus context plus KV cache. File size alone will mislead you exactly once, at the top of your menu.
• Weights are content, not code: CDN, resume, checksum, consent, and a progress bar that admits the verify phase exists.
• Plan for the load that kills the process. Persist intent before, clear it after, and stop repeating a load that keeps dying.

None of this is exotic. It is the ordinary discipline of shipping consumer software, applied to a stack young enough that the defaults do not exist yet. That, more than any single technique, is what on-device AI work is right now: deciding what the defaults should be, on hardware your users already own.