# ApexClaw Agent-Readiness Conformance Vectors

Runnable adversarial test vectors for ARB/1.1 (Agent Readiness Benchmark) and
AGA/1.1 (Agent Governance Audit) — the two deterministic engines behind
[apexclawai.com](https://apexclawai.com/) and its audit tool. Published
CC-BY-4.0. This is the artifact referenced when either engine's methodology
is described to a standards body: every check is not just described in
prose, it is testable by anyone with a Python interpreter, against a fixed
clean/dirty fixture pair, with a fixed expected outcome.

## Why this exists

Feedback on an earlier proposal (via the W3C's AI Agent Verifiable Statements
/ Agent Trust Protocol community discussions) raised two real gaps in how
this work was presented: no evidence-state distinction between "we checked
and it failed" and "we could not check," and no recomputable, independent
vector set to actually verify a claimed check against. Both are addressed
directly by ARB/1.1 (three-state VERIFIED/UNVERIFIED/UNOBSERVABLE verdicts,
see `arb_v11/`) and by this directory existing at all.

## Layout

```
arb_v11/
  engine.py, engine_v11.py   -- the engines themselves, bundled so this
                                 directory runs standalone, no other source
                                 needed.
  vectors.json                -- declarative: fixture URL, check id,
                                 expected verdict (and independence
                                 classification, where the check reports one).
  run.py                      -- loads vectors.json, audits each fixture,
                                 compares actual vs expected, exits nonzero
                                 on any mismatch.
  fixtures/                   -- the clean/dirty HTML fixtures and the tiny
                                 local HTTP/HTTPS servers that serve them.

aga_v11/
  aga.py, aga_v11.py, vectors.json, run.py, fixtures/  -- same shape, for
  the governance-audit engine. No servers needed; fixtures are plain file
  trees.
```

## Running the ARB/1.1 vectors

```
cd arb_v11
bash fixtures/start_servers.sh
ALLOW_PRIVATE=1 python3 run.py
bash fixtures/stop_servers.sh
```

`ALLOW_PRIVATE=1` is required only because the fixtures are loopback
addresses (127.0.0.1 / test-a.local / test-b.local / test-c.local, the
latter three aliased to 127.0.0.1 — add `127.0.0.1 test-a.local test-b.local
test-c.local` to `/etc/hosts` if they are not already present). A real audit
target is never loopback and this flag has no effect on auditing one.

`fixtures/tls_dirty_cert/` generates a fresh, throwaway self-signed
certificate the first time `start_servers.sh` runs (via `openssl req`); no
private key is shipped in this package.

## Running the AGA/1.1 vectors

```
cd aga_v11
python3 run.py
```

No setup required — every fixture is a static file tree, audited directly.

## Two declared network exceptions

Every fixture above is loopback-only, with two exceptions, both disclosed in
`arb_v11/vectors.json`'s `network_exceptions` field and inherent to what the
check actually measures, not a shortcut:

- **`offsite.wikidata`** calls the real, public, unauthenticated Wikidata
  API (`www.wikidata.org/w/api.php`). Faking this locally would mean
  testing a fake API, not the check.
- **`sec.cert_valid`**'s VERIFIED vector uses the real, live
  `https://apexclawai.com/`, because certificate chain-of-trust validity is
  inherently rooted in real, publicly-trusted certificate authorities. A
  self-signed local certificate can only ever correctly demonstrate the
  UNVERIFIED path — which the vector set also tests, locally, via
  `tls_dirty_cert/`.

## What VERIFIED / UNVERIFIED / UNOBSERVABLE mean

- **VERIFIED** — evidence was observed and it satisfies the check.
- **UNVERIFIED** — evidence was observed and it does **not** satisfy the
  check.
- **UNOBSERVABLE** — the evidence required to decide is absent (a network
  failure, a check that requires runtime access nobody supplied, a page with
  nothing for the check to evaluate). UNOBSERVABLE is excluded from the
  score denominator, exactly as it would be if simply skipped — but unlike a
  silent skip, it is a named state on the face of every report, and every
  report states how many checks landed there.

Missing evidence must never resolve to a pass by default. That is the one
rule every vector here exists to hold the code to.

## What this is not

Passing these vectors proves the *engine's own logic* is internally
consistent with its documented checks. It is not proof that the engine's
checks are the *right* checks, and running it against your own site is not a
certification of anything — see the scope statement embedded in every
`audit()` result for what ARB and AGA do and do not measure.