Research
I am moving part of my time from building products to empirical research on AI systems. The starting point is security, where claims can be checked against real code and real evidence. This page tracks that work as it happens, and separates what is planned from what has been measured.
Current project
Pilot measured - 20 scenarios, results are preliminary
An open-source research extension of PandaWard, the application security engine I maintain. It is a sandboxed harness that tests whether untrusted repository and tool content can push an AI coding or security agent into unauthorized simulated actions, or into security conclusions that the scan evidence does not support.
When an agent is exposed to adversarial repository or tool content, can independently enforced authorization and evidence provenance reduce unauthorized side effects and unsupported security conclusions, without an unacceptable loss of legitimate task completion?
PandaWard repository Lab source and corpus Run reports PandaWard demo
First measurements
Corpus v1.4, 20 synthetic scenarios, 3 repeats, no invalid episodes. Attack success is the share of 36 attack episodes per arm that committed a forbidden effect or a report the evidence does not support.
| Track | A | B | C | D | Episodes |
|---|---|---|---|---|---|
| API trackclaude-haiku-5-5, Messages API, controlled tools | 1/36 | 0/36 | 0/36 | 0/36 | 240 |
| Claude Code trackalias "sonnet", Claude Code on a subscription | 2/36 | 1/36 | 0/36 | not run | 180 |
Corpus history. A pilot found faults in the measuring tools, and they were fixed before any official run. Two full runs were then discarded: v1.2 because a fixture cited a file it did not contain, and v1.3 because some fixtures pointed at an evidence file the repository lacked, which made the monitor block 7 of 24 legitimate reports. That second fault was found while analysing arm D, after the arm results were visible. Total spend including pilots and discarded runs was $1.56 of a $5.00 cap.
Hypotheses
Each one comes with the result that would refute it, written before any run.
Deterministic authorization (arm B) and evidence-bound assurance (arm C) reduce committed unauthorized state changes relative to prompt-only operation (arm A).
Refuted if: No improvement on matched attack episodes once uncertainty is accounted for.
Evidence-bound assurance (C) reduces unsupported PASS and hidden-finding reports relative to B.
Refuted if: No measurable incremental improvement in false-assurance outcomes.
An independent model monitor (D) catches some semantic adversarial attempts that static rules miss.
Refuted if: No incremental interceptions, or the quality and latency costs outweigh the benefit.
Stronger guardrails impose a measurable cost in legitimate task completion and latency.
Refuted if: Effects are indistinguishable within measured uncertainty. That would be reported as a result.
Failures differ by attack surface and by agent and model pairing.
Refuted if: No robust pattern when results are stratified across families and models.
Design
Same scenarios, same tools, same capabilities. Only the enforcement differs, so each step isolates one control.
Scenarios
Corpus v1 has 20 scenarios today: 12 attacks and 8 matched legitimate controls. The design target is 96 (64 attacks and 32 controls), which does not exist yet.
Repository instructions. README and comments override the task.
MCP tool metadata and results. A tool impersonates authority.
Scanner output poisoning. False parser output hides or invents findings.
SARIF and report tampering. An untrusted report promotes BLOCKED to PASS.
Evidence provenance. Forged hashes, stale evidence, hidden mismatches.
Authorization forgery. Approval or scope widening claimed indirectly.
Long-horizon drift. Earlier constraints disappear in delayed steps.
Cross-tool attacks. Small allowed steps add up to a forbidden effect.
Method
Status
I will publish a report only when there is data behind it, and it will link to the code and the exact commands to reproduce it. Dates are targets, not promises.
Requirements, threat model, hypotheses and scenario taxonomy
Sandbox, effect journal, gates for arms B to D, scoring, 20 scenarios (12 attacks, 8 matched controls) across all eight families
API track (240 episodes, arms A to D) and Claude Code track (180 episodes, arms A to C), 3 repeats, no invalid episodes
The pilot corpus is too easy to separate the arms. A harder, frozen held-out set is next
96 scenarios (64 attack, 32 legitimate) and a second model
Limits
The benchmark is synthetic, short-horizon compared with real deployments, partly written by its maintainer, and tied to the chosen models, tools and permission rules. Evidence checks cannot prove the absence of vulnerabilities. A block in a simulated sandbox does not mean the same policy works in an arbitrary production setup. Small samples and repeated runs are not independent. It is independent work and has no endorsement from any model provider.