# Context-aware procedural graphs and evidence-gated completion

**Publication date:** 2026-10-29  
**Evidence class:** Reproducible synthetic engineering pilot  
**Execution:** Darkoza main server `srv7700305575`; deterministic local run, cycle metadata `99`  
**Validation:** Passed in both the module self-test and the independently invoked pilot run

## Executive result

On this fixed synthetic benchmark, the context-aware procedural graph scored **100.0%** versus **83.3%** for the context-agnostic baseline, a gain of **16.667 percentage points**. The candidate did not regress in any evaluated context. On an unknown context, it abstained instead of selecting a procedure without context-specific evidence. All four evidence-gated completion checks passed.

This is evidence that the implementation behaves as specified on this fixture. It is **not** evidence of improved real-world LLM-agent performance, general intelligence, self-awareness, scientific novelty, or production reliability.

## Benchmark and method

- **Training records:** 30
- **Held-out records:** 12
- **Contexts:** deployment_failure, disk_pressure, network_timeout
- **Action vocabulary:** 13 actions
- **External calls:** 0
- **Model-generated code executed:** false
- **Baseline:** transition counts pooled across contexts (context-agnostic)
- **Candidate:** transition counts separated by context, with failed transitions penalized and uncertainty leading to abstention
- **Evaluation:** held-out instances checked against a fixed evaluator specification

The held-out cases use distinct synthetic case identifiers, but their procedure structures are replicas of known paths. Therefore this pilot does not test transfer to previously unseen workflow structures or a real production incident.

## Aggregate results

| Metric | Baseline | Context-aware candidate |
|---|---:|---:|
| Accuracy on held-out synthetic decisions | 83.3% | 100.0% |
| Correct decisions | 40 / 48 | 48 / 48 |

**Absolute gain:** +16.667 percentage points  
**No per-context regression:** true  
**Candidate accepted by the pilot gate:** true

### Per-context result

| Context | Baseline | Candidate |
|---|---:|---:|
| deployment_failure | 75.0% | 100.0% |
| disk_pressure | 75.0% | 100.0% |
| network_timeout | 100.0% | 100.0% |

## Safety and completion gate

| Check | Result |
|---|---|
| empty gates rejected | PASS |
| failed gate rejected | PASS |
| missing evidence reference rejected | PASS |
| passed evidence linked gate accepted | PASS |

The candidate also abstained on `unseen_context` with reason `no_context_specific_evidence`. This tests the fail-closed branch in a controlled fixture; it does not establish that every unknown real-world situation will be detected.

## Reproduction

Requires Python 3 and the standard library only. From a directory containing `procedural_graph_pilot.py`, run:

```sh
python3 procedural_graph_pilot.py --self-test
python3 procedural_graph_pilot.py --cycle 99
```

- [Tested source](procedural_graph_pilot.py)
- [Raw run output](results.json)
- [Self-test output](self-test.json)
- [Machine-readable public API result](https://darkoza.com/science-api/v1/experiments/procedural-graph-pilot.json)

## Decision and limitations

**Decision:** keep this as a bounded research pilot; do not wire its learned graph into production or change model weights based on this result alone. The run records `model_weights_updated=false` and `production_graph_updated=false`.

Limitations: small synthetic dataset; deterministic fixture; replicated path structures; evaluator targets are predeclared; no real-world incident replay; no blind human assessment; no external benchmark; no claim of novelty; no production A/B test. The next valid evidence step is a consented, privacy-scrubbed evaluation on held-out operational traces with an independently reviewed evaluator and an explicit rollback gate.