You don't always need a frontier model. You need one that fits.

I asked a 2-billion-parameter model for a 150-word LinkedIn post and got 105 words. I asked for 300 and got 210. Every run came back short.

No leaderboard would have told me that. It took a small, representative evaluation on the task I cared about, and that one finding said more about whether the model fit my workload than any benchmark score.

The full article builds a working app twice with Gemma 4, once against a hosted endpoint and once on a laptop. This page is the argument without the code.

The trade is generality against fit

The usual framing is that big models are smart and small models are not. That stopped being useful a while ago. Small models have improved along with the frontier, and on bounded tasks a current small model can beat a much larger old one.

A frontier model is a generalist. You pay for that breadth in compute, latency, money and a network trip to someone else's data center. A small model deployed well is a specialist. It does fewer things, and in exchange it can give you properties a hosted API cannot.

Four properties worth paying attention to

• Privacy. When inference runs inside a boundary you control, prompts and outputs can stay there. I have watched deals stall for months on data residency questions that get easier once that is true. You still owe the access controls, logging and governance.
• Cost. Self-hosting swaps per-token spend for capacity you pay for whether or not it is busy. For high-volume, low-complexity work such as classification or extraction, that can be cheaper to sustain. It is never free.
• Latency. No network round trip. Measure time to first token on the device you will ship on, not the one on your desk.
• Availability. This is the one I care about most. A tutoring app that needs a stable connection to a US-hosted API is not a product for a student in rural Ghana. A model that runs offline on a $200 device is a different product, and it reaches people the cloud version leaves out by design.

A rule you can use on Monday

Reach for a frontier model when the task needs deep multi-step reasoning, very broad knowledge, or the best possible open-ended writing.

Reach for a small one when the task is bounded, repetitive, sensitive, latency-critical, or has to run somewhere with poor connectivity. A lot of the production features I see teams build sit in that second list, and they are running on a frontier model because it was the default.

So: use the smallest model that passes a representative evaluation, and escalate only when the workload makes you. The same app can have two right answers. In the article, the hosted demo uses a 26B mixture-of-experts model and the laptop build uses the 2B edge model, on purpose.

One note on terms

Open source, open weights and local are three different things. Open-source software publishes code you can study and change. An open-weight model publishes trained parameters, often without the full training recipe. A local model is one you run on infrastructure you control, whatever its license. Gemma 4 is an Apache 2.0, open-weight family. Keeping those apart saves a lot of confused meetings.