Model routing is a systems problem

Model routing is not a model picker. It is a control plane.

A refund summary, a CI failure, and a high-risk policy check are different product paths. They may need different context, latency, logging, approval, and fallback behavior.

Routing should eliminate ineligible paths before ranking quality. Restricted data may disqualify a hosted model. A JSON contract may disqualify a route without structured output. A tight latency budget may eliminate a slow model.

The decision should be inspectable:

type RouteDecision = {
  routeId: string
  reason: string
  confidence: number
  constraintsApplied: string[]
  expectedCostCents: number
  expectedLatencyMs: number
  fallbackRouteId?: string
}

When a result is bad, that record helps separate misclassification, policy violation, route failure, and fallback failure.

Fallbacks need human-readable behavior. A cheap route should not retry forever. A high-risk route should not silently degrade into an unsafe one. If the system cannot recover cleanly, it should stop and say what remains.

The useful analogy is a load balancer: inspect the request, apply constraints, record the decision, and move on. Model routing needs the same discipline.