The Last Mile: Shipping a .NET App With a 2 GB Brain Inside

It runs on my machine. Professor Maier believes, deflects, and can be honestly beaten (last post). Five posts of building, and now the part nobody puts in the tutorial: getting it onto someone else's machine.

This is where a local-AI app stops behaving like normal software. A typical .NET app is a few megabytes. This one has to drag a 1.5–2 GB model along with it, and that single fact breaks almost every shipping assumption you have — your installer size, your update flow, your store listing, your first-run experience. Let's walk the last mile.

The central decision: bundle or download

You have two ways to get the model onto the user's disk, and it's the first real choice.

Bundle it in the installer. Everything arrives in one package. The app works fully offline the instant it's installed — no network, no waiting. The cost: a 2 GB installer that people hesitate to download, most app stores won't accept, and — the quiet killer — every app update risks re-shipping the entire model. Push a one-line bug fix, ship 2 GB again.

Download it on first run. Ship a small, normal-sized app; fetch the model the first time it launches. The installer stays lean, stores are happy. The cost: your "offline" app needs the internet exactly once, you now own a download UI, and you depend on wherever the file is hosted staying up.

For anything you actually distribute, first-run download wins — but with one non-negotiable rule that saves you from the update trap: the model lives separately from the app, downloaded once, and app updates never touch it.

Where the model lives on disk

This is a small decision that bites hard if you get it wrong. Do not put a 2 GB file in Program Files — it needs admin rights to write, and a reinstall can wipe it. Put it in the per-user local app-data folder, which needs no elevation and survives app updates:

var modelDir = Path.Combine(
    Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData),
    "SchwurblerGame", "models");
Directory.CreateDirectory(modelDir);

var modelPath = Path.Combine(modelDir, "gemma-2-2b-it-Q4_K_M.gguf");

Now updating the app is a small download, and the brain stays put. Download once, run forever.

Verify what you downloaded — or pay for it later

Remember the cryptic native load errors from post 2? A truncated or corrupted GGUF is a first-class source of them. A download that drops at 94% leaves you a file that looks fine and fails to load with an error that points nowhere near the real cause.

So never load a model you haven't verified. Publish the SHA-256 of the file, and check it after download:

static bool VerifyChecksum(string path, string expectedSha256)
{
    using var stream = File.OpenRead(path);
    using var sha = SHA256.Create();
    var hash = Convert.ToHexString(sha.ComputeHash(stream));
    return hash.Equals(expectedSha256, StringComparison.OrdinalIgnoreCase);
}

If it doesn't match, delete and re-download. Bonus: use HTTP range requests so an interrupted download resumes instead of starting over — on a 2 GB file over a hotel WiFi, your users will feel the difference.

Will this machine even run it?

You picked your model on your dev box. Your user's five-year-old laptop is a different story. A model that needs ~2 GB of RAM on a thin machine doesn't politely decline — it OOMs and dumps a stack trace, which is a terrible first impression.

Precise cross-platform "how much free RAM is there" detection in .NET is fiddlier than it should be, so I lean on two pragmatic guards instead. First, a pre-flight sanity check against the model file size. Second — and this is the one that matters — never let the load crash raw. Wrap it and translate failure into something human:

try
{
    using var model = LLamaWeights.LoadFromFile(parameters);
    // ...
}
catch (Exception ex)
{
    Console.Error.WriteLine(
        "The model couldn't be loaded on this machine — most likely not enough " +
        "free memory. Try closing other apps, or grab the smaller model build.");
    // log ex for yourself; show the human sentence to the user
}

If you want to go further, ship two quantizations — a Q4_K_M and a smaller Q3 — and pick based on the machine. But at minimum, fail with a sentence, not a stack trace.

Be honest about the one-time internet

There's an irony worth naming right in your UI: an "offline AI" app that needs the internet on first launch. Don't hide it — frame it. "One-time 1.8 GB download. After that, everything runs on your machine, fully offline, forever." Users forgive a big download they understood in advance; they don't forgive a spinner with no explanation.

And a whole saga I'll only wave at here, because it's its own post: on Windows, an unsigned downloaded .exe greets your user with a SmartScreen warning that scares off a good chunk of them. Code signing is the fix, and it's a rabbit hole of certificates and costs. Worth knowing it's waiting for you before you ship, not after.

What six posts of a stubborn professor actually taught

Step back and the series has a shape, and it isn't really about a flat-earther.

We started by picking the weakest model on purpose, because its confident nonsense was the feature, not the bug. We spent a whole post learning that a small model needs fewer rules, not more. We built a referee out of boring deterministic code and refused, twice, to let the model judge itself. And when the naive version got gamed, we didn't out-clever the players — we made the model's hidden state explicit and let plain code decide against a threshold.

One thread runs through all of it: treat the model as a creative engine, and put everything that has to be correct in code you can test. The model is there to be interesting. Determinism is there to be right. Get that division of labour straight and a tiny model running on a laptop, for free, offline, becomes genuinely useful — not despite its limits, but by casting them in the right role.

That's the whole trick, and it's not specific to games. It's how I'd wire an LLM into anything that matters.

The professor is on GitHub — [repo link] — model file, grammar, and the deterministic oracle included. Clone it, break it, give him a new conspiracy to defend. If you build something with it, I'd love to see what you make him say.

Thanks for reading all six. Now go make a small model believe something ridiculous.