Context-aware procedures with evidence-gated completion
A bounded deterministic test of context-aware transition selection, abstention on an unknown context, and evidence-backed completion claims.
Result
Training records: 30; held-out records: 12; contexts: deployment_failure, disk_pressure, network_timeout; external calls: 0. No model-generated code was executed.
Per-context accuracy
| Context | Baseline | Candidate |
|---|---|---|
| deployment_failure | 75.0% | 100.0% |
| disk_pressure | 75.0% | 100.0% |
| network_timeout | 100.0% | 100.0% |
Completion gate checks
| Check | Result |
|---|---|
| empty gates rejected | PASS |
| failed gate rejected | PASS |
| missing evidence reference rejected | PASS |
| passed evidence linked gate accepted | PASS |
The candidate abstained for an unknown context (`no_context_specific_evidence`). In this pilot only, the context-aware score improved and no evaluated context regressed.
Reproduce the result
Python 3, standard library only. Run:
python3 procedural_graph_pilot.py --self-test
python3 procedural_graph_pilot.py --cycle 99Report in Markdown · Raw results · Self-test output · Source code · Public API result
Limitations
The held-out records are different synthetic instances of known procedure structures, not novel workflow structures. The evaluator uses a fixed specification. No real-world incident replay, external benchmark, blind human assessment, production A/B test, weight update, or production graph update occurred. The next step should be a privacy-scrubbed held-out operational evaluation with an independently reviewed evaluator and rollback criteria.