Local models are part of the platform

A local model is most useful when the application gives it one bounded responsibility.

Redaction, classification, routing, and offline summarization are all plausible jobs. None of them becomes operationally simple just because the weights are running on the same machine as the data. The application still needs to know which model ran, what happens when it is uncertain, and whether the input is allowed to leave the device.

For example, a request can make the boundary explicit:

{
  "task": "classify_note",
  "input": "raw note text",
  "model": "phi-3-mini",
  "fallback": "remote-summary",
  "retain": false
}

That small contract answers more useful questions than “is the model local?” It identifies the task, records the selected model, names the fallback, and states that the input should not be retained.

The platform still has to decide:

• what is stored locally
• what can sync later
• what is discarded after inference
• which results are final
• which results are only a first pass

These decisions affect the user-facing behavior. A local classifier can keep a private note on the device. A remote fallback may need an explicit boundary. An offline summarizer needs a way to show that its result was produced without the usual service.

Small models are not automatically uncomplicated models. They are useful when they make one path through the system more controlled. The model is a platform component when its version, fallback, retention policy, and failure mode are visible enough to operate.