On-device AI fits private workflows

On-device AI changes where the model sits in the user’s life.

A hosted model moves context away, waits for a response, and returns an answer. That is appropriate for some work. It is less appropriate for local notes, private photos, screenshots, and files that should not leave the device.

Privacy is a data-movement property. Ask what leaves the device, what is logged, what is retained, and which derived artifacts sync later. On-device inference reduces how often raw material moves.

Small local models work best with narrow jobs:

• classify a note as task, reference, or draft
• extract contact information from a card
• suggest tags for local files
• redact obvious secrets before remote submission
• summarize a short transcript

A hybrid pipeline can make the boundary explicit:

{
  "pipeline": [
    {"step": "capture", "model": "small local classifier"},
    {"step": "tag", "model": "small local extractor"},
    {"step": "escalate", "model": "remote general model", "only_if": "confidence_low"}
  ]
}

Derived state needs the same privacy review. Embeddings, summaries, tags, memories, logs, and cached outputs can leak information even when the original file stays local. The product should say what syncs, what can be deleted, and what expires.

Keep private and fast steps local. Escalate only when the user needs broader context or stronger reasoning. On-device AI is useful when the boundary stays close to the person doing the work.