Winning the Opir Challenge by testing the edges

I was one of the first-place winners of Wordcab’s OPIR Challenge. The part worth keeping was the testing method.

A semantic guardrail is a classifier with boundaries, false positives, false negatives, calibration, and coverage gaps. Obvious unsafe input being blocked proves little. The useful cases sit near the boundary: paraphrased, translated, quoted, encoded, or surrounded by harmless context.

Controlled variation makes that boundary visible:

{
  "family": "same_intent_multiform",
  "variants": [
    "plain_english",
    "translated_low_resource_language",
    "quoted_instruction",
    "code_comment_framing",
    "encoded_payload_reference"
  ],
  "expected_behavior": "stable safety decision",
  "review_focus": "decision movement across semantically nearby variants"
}

The architectural seam matters too. A classifier and a generator may represent the same request differently. One can see noise while the other recovers the intent. That disagreement is a system failure, not only a prompt failure.

False positives need paired probes:

unsafe intent, direct phrasing
unsafe intent, transformed phrasing
benign nearby intent, direct phrasing
benign nearby intent, transformed phrasing

This shows whether the boundary is stable, too broad, or too narrow. It also protects the product from a guardrail that blocks ordinary security education or research and teaches users to route around it.

Practical safety layers may need to run locally, cheaply, and before data leaves a trust boundary. Smaller classifiers are useful there, but their coverage gaps need measurement.

Track decision stability across probe families, false positives on nearby benign inputs, transformed unsafe misses, threshold movement, taxonomy drift, and classifier-generator disagreement.

The award was fun. The durable lesson was to test semantic guardrails as classifiers embedded in systems. Semantic distance is not safety distance.