The open-weight landscape — what is actually shippable

So far this course has compared three frontier APIs: Claude, GPT, Gemini. They share a property — closed weights, hosted by the vendor, charged per token. The other half of the prompt-engineering landscape is open-weight models that you can download and run yourself, on your own infrastructure or on a third-party host that does not own the weights. Llama, Mistral, Qwen, DeepSeek, Phi. The list keeps growing.

You need to know this part of the landscape because for many of the tasks Hagar's CTO is paying Claude for, an open-weight model running on a small GPU could do the job for a fraction of the cost.

The families that matter, and how to read them

Specific model names and sizes turn over every few months; the families and the licences move much more slowly. So the durable version of this table is who makes it and what to check, with the sizes read off the model card on the day you choose.

Family	Maker	What to know before you pick a size
Llama	Meta	Llama 4 moved the family to Mixture-of-Experts — Scout and Maverick both activate ~17B parameters per token while holding far more in memory. Do not carry Llama 3's dense size ladder (8B / 70B / 405B) across to Llama 4; the sizes and the hardware maths are both different.
Mistral	Mistral AI	Mistral Large 3 is a large MoE released under Apache 2.0 — permissive licence, but frontier-scale VRAM. Mistral also ships much smaller models under the same licence; the licence is not a size signal.
Qwen	Alibaba	Both dense and MoE variants, most under Apache 2.0. Strong on code and on Chinese/English. The widest size range of any family, which makes it the usual first stop when you have a fixed VRAM budget.
DeepSeek	DeepSeek	MoE, aggressive cost/quality. The reasoning variant is the open-weight counterpart to the hosted reasoning modes in Module 4. Licence terms vary by release — check each one.
Phi	Microsoft	The small-model family. Phi-4 itself is a 14B dense model; the sub-4B member is Phi-4-mini. Pick the right family member — the names differ by one word and the hardware requirement differs by ~4×.

Check current sizes, context windows, and licences on each family's own model cards — Meta Llama, Mistral, Qwen, DeepSeek, Phi. Those pages are corrected when a release changes; a table in a course is not.

What "shippable" actually means

A model is shippable for your task if it satisfies all four:

1. Can it actually run on the hardware budget you have? Work it out from the model card, not from the family name. For a dense model the floor is roughly parameters × bytes-per-parameter — about 2 bytes each at fp16, about 0.5 at int4 — plus KV cache and runtime overhead, so a 70B dense model needs tens of GB at int4 and well over a hundred at fp16. For an MoE model, size the VRAM off total parameters, because every expert has to be resident, and expect the speed and per-token cost of the much smaller active count. This is why "it activates 17B parameters" tells you nothing about whether it fits. Pick the smallest model that does the job.

2. Does the prompt that worked on Claude / GPT / Gemini still work here? Often, no. Open-weight models tend to be more sensitive to prompt phrasing and to follow few-shot examples more literally. The next lessons cover this.

3. What is the licence? "Open-weight" is not one licence. Llama ships under Meta's own community licence, which carries conditions — including a threshold above which very large deployments need a separate agreement. Apache 2.0, which several Mistral and Qwen releases use, has no such condition. DeepSeek's terms vary by release. Read the licence file next to the weights, per model, before you ship — and note that the licence can differ between two models from the same maker.

4. What is the inference cost end-to-end? A free model is not free if you pay AWS for the GPU. Compare against the per-token price of the frontier API for the same task volume. At low volumes, frontier APIs are cheaper. At high volumes (millions of requests per day), open-weight on your own infrastructure wins.

The honest tradeoff

The gap between the best open-weight models and the hosted frontier has narrowed a lot, and it narrows unevenly. Where open-weight tends to hold up: single-turn generation, classification, summarisation of text that is already in the prompt. Where it tends not to: sustained instruction-following across many simultaneous constraints, long-context reasoning, and multi-step tool use.

Resist quoting a headline percentage for that gap, including one from us. Published leaderboards move weekly, they are usually reported on benchmarks that do not resemble your task, and by the time a number reaches a course it describes a release that has been superseded. The gap that matters is the one on your own prompts — which is exactly what the capstone measures. Run your real prompt against a hosted model and an open-weight candidate, score both with the same rubric, and you will have a number that is worth more than any leaderboard because it is about your workload.

Hagar's pragmatic plan takes the same shape: keep a frontier model for customer-facing copy, keep a cheap hosted model for high-volume drafting, and test whether a self-hosted open-weight model can absorb the bulk-classification job that runs every night. Classification is the strongest candidate for the swap — it is high volume, the output is short and checkable, and a wrong label is cheap to catch downstream. Whether the savings cover the GPU is arithmetic she can only do after measuring, and that is the point.

Next: how the prompt budget changes — open-weight models behave differently when prompts get long. :::