docs(ttp): mark E.2.9–E.2.14b as done in design doc
Each section gets a Status: ✅ done block summarising what's GREEN
today vs xfail-gated and noting any divergence from the doc's
original wording (E.2.9 lossy observable phases; E.2.13 db_backends
fixture landed alongside; E.2.14a Jaeger-skip + tracing-enabled
plumbing; E.2.14b NamedTuple AttributeError vs FrozenInstanceError).
This commit is contained in:
@@ -2651,6 +2651,15 @@ edit of `tests/api/ttp/schemas/endpoints.placeholder.json`).
|
|||||||
|
|
||||||
**E.2.9 — UKC bridge bijection tests** (`tests/clustering/test_ukc_bridge.py`)
|
**E.2.9 — UKC bridge bijection tests** (`tests/clustering/test_ukc_bridge.py`)
|
||||||
|
|
||||||
|
**Status:** ✅ done. The full inverse claim (every observable phase
|
||||||
|
round-trips) is overstated — `EXPLOITATION`, `PIVOTING`, and
|
||||||
|
`OBJECTIVES` are observable but UKC-only concepts that ATT&CK lacks
|
||||||
|
matching tactics for. The test pins them as observable-but-lossy
|
||||||
|
alongside the pre-target lossy phases via a single
|
||||||
|
`_LOSSY_INVERSE_REFERENCE` table; round-trip is asserted only over
|
||||||
|
`OBSERVABLE_PHASES − _LOSSY_INVERSE_REFERENCE`. All assertions GREEN
|
||||||
|
today; no xfail.
|
||||||
|
|
||||||
- Every tactic key in `ATTACK_TACTIC_TO_UKC` is a valid
|
- Every tactic key in `ATTACK_TACTIC_TO_UKC` is a valid
|
||||||
TA-prefixed string.
|
TA-prefixed string.
|
||||||
- Every value is a member of `UKCPhase`.
|
- Every value is a member of `UKCPhase`.
|
||||||
@@ -2663,6 +2672,13 @@ edit of `tests/api/ttp/schemas/endpoints.placeholder.json`).
|
|||||||
|
|
||||||
**E.2.10 — Confidence model tests** (`tests/ttp/test_confidence.py`)
|
**E.2.10 — Confidence model tests** (`tests/ttp/test_confidence.py`)
|
||||||
|
|
||||||
|
**Status:** ✅ done. Pure-arithmetic adjustment property
|
||||||
|
(`confidence × multiplier ≤ base` for `multiplier ∈ [0, 1]`) +
|
||||||
|
known-input table + floor-constant pinning + invalid-multiplier
|
||||||
|
guard GREEN today via Hypothesis. `insert_tags`-side drop-below-0.3
|
||||||
|
xfail-gated behind E.3.3; AbuseIPDB-30 worked-example xfail-gated
|
||||||
|
behind E.3.10.
|
||||||
|
|
||||||
- `confidence × multiplier` never raises the value above the rule's
|
- `confidence × multiplier` never raises the value above the rule's
|
||||||
base (downward-only adjustment property).
|
base (downward-only adjustment property).
|
||||||
- A computed confidence below 0.3 is dropped — `insert_tags()`
|
- A computed confidence below 0.3 is dropped — `insert_tags()`
|
||||||
@@ -2672,6 +2688,13 @@ edit of `tests/api/ttp/schemas/endpoints.placeholder.json`).
|
|||||||
|
|
||||||
**E.2.11 — Multi-mapping property tests** (`tests/ttp/test_multi_mapping.py`)
|
**E.2.11 — Multi-mapping property tests** (`tests/ttp/test_multi_mapping.py`)
|
||||||
|
|
||||||
|
**Status:** ✅ done. UUID-distinctness property over N×M cartesian
|
||||||
|
product GREEN today (exercised via `compute_tag_uuid` directly +
|
||||||
|
Hypothesis). One-rule / two-techniques worked example pinned as a
|
||||||
|
fixture. Engine-level fan-out and engine-replay-safety
|
||||||
|
xfail-gated behind E.3.7 (`RuleEngine.evaluate` returns `[]` from
|
||||||
|
its empty body).
|
||||||
|
|
||||||
- Hypothesis: given a synthetic event matched by N rules each
|
- Hypothesis: given a synthetic event matched by N rules each
|
||||||
emitting M techniques, the engine produces exactly N×M tag rows
|
emitting M techniques, the engine produces exactly N×M tag rows
|
||||||
(with idempotent UUIDs so a re-run produces zero new rows).
|
(with idempotent UUIDs so a re-run produces zero new rows).
|
||||||
@@ -2680,6 +2703,14 @@ edit of `tests/api/ttp/schemas/endpoints.placeholder.json`).
|
|||||||
|
|
||||||
**E.2.12 — Bus integration** (`tests/ttp/test_worker_bus.py`)
|
**E.2.12 — Bus integration** (`tests/ttp/test_worker_bus.py`)
|
||||||
|
|
||||||
|
**Status:** ✅ done. `_TOPICS` frozenset equality against the
|
||||||
|
documented set + module-level constant pinning + every-pattern
|
||||||
|
self-match (or wildcard-extension match) + `run_ttp_worker_loop`
|
||||||
|
async-signature surface GREEN today. Worker→engine wiring,
|
||||||
|
loop-prevention invariant, attacker.enriched/email.received
|
||||||
|
catch-up asymmetry, subscription-introspection xfail-gated behind
|
||||||
|
E.3.14.
|
||||||
|
|
||||||
- Subscribed topics from `_TOPICS` constant match the documented
|
- Subscribed topics from `_TOPICS` constant match the documented
|
||||||
set exactly.
|
set exactly.
|
||||||
- Worker started against an in-memory bus and given a faked
|
- Worker started against an in-memory bus and given a faked
|
||||||
@@ -2704,6 +2735,15 @@ edit of `tests/api/ttp/schemas/endpoints.placeholder.json`).
|
|||||||
|
|
||||||
**E.2.13 — Repository tests** (`tests/web/db/test_ttp_repo.py`)
|
**E.2.13 — Repository tests** (`tests/web/db/test_ttp_repo.py`)
|
||||||
|
|
||||||
|
**Status:** ✅ done. The `db_backends` fixture didn't exist at the
|
||||||
|
time of this commit — it lands here under `tests/web/db/conftest.py`
|
||||||
|
parametrizing SQLite (always) + MySQL (gated on
|
||||||
|
`DECNET_TEST_MYSQL_URL` env var per project memory: skip heavy
|
||||||
|
suites in dev). Mixin-method async-coroutine introspection +
|
||||||
|
mixin-presence-on-repo GREEN today; `insert_tags` idempotency,
|
||||||
|
identity-rollup projection, attacker-rollup exclusion of
|
||||||
|
NULL-attacker tags xfail-gated behind E.3.3.
|
||||||
|
|
||||||
- Per dual-DB-backend project convention: every repo test runs
|
- Per dual-DB-backend project convention: every repo test runs
|
||||||
against both SQLite and MySQL. Use the existing `db_backends`
|
against both SQLite and MySQL. Use the existing `db_backends`
|
||||||
parametrize fixture.
|
parametrize fixture.
|
||||||
@@ -2715,6 +2755,20 @@ edit of `tests/api/ttp/schemas/endpoints.placeholder.json`).
|
|||||||
|
|
||||||
**E.2.14a — Observability** (`tests/ttp/test_tracing.py`)
|
**E.2.14a — Observability** (`tests/ttp/test_tracing.py`)
|
||||||
|
|
||||||
|
**Status:** ✅ done. Per-test `InMemorySpanExporter` + fresh
|
||||||
|
`TracerProvider` (OTEL forbids overriding the global once set, so
|
||||||
|
no global mutation). Session-scoped autouse fixture in
|
||||||
|
`tests/ttp/conftest.py` sets `DECNET_DEVELOPER_TRACING=true` and
|
||||||
|
forces `decnet.telemetry._ENABLED = True` so the no-op tracer
|
||||||
|
doesn't silently swallow spans. The `span_exporter` fixture also
|
||||||
|
monkeypatches `decnet.telemetry.get_tracer` so production code
|
||||||
|
under test lands spans in the in-memory exporter. The whole module
|
||||||
|
skips when the configured Jaeger / OTLP endpoint
|
||||||
|
(`DECNET_OTEL_ENDPOINT`, default `localhost:4317`) is not reachable
|
||||||
|
— tracing tests need an observability backend or they have nothing
|
||||||
|
meaningful to assert. Span-emission assertions xfail-gated behind
|
||||||
|
E.3.5/E.3.6/E.3.7/E.3.9–E.3.13.
|
||||||
|
|
||||||
OTEL spans are not optional decoration; they're a stated design
|
OTEL spans are not optional decoration; they're a stated design
|
||||||
property. Tests pin the span hierarchy:
|
property. Tests pin the span hierarchy:
|
||||||
|
|
||||||
@@ -2735,6 +2789,33 @@ property. Tests pin the span hierarchy:
|
|||||||
|
|
||||||
**E.2.14b — RuleStore conformance** (`tests/ttp/store/test_*.py`)
|
**E.2.14b — RuleStore conformance** (`tests/ttp/store/test_*.py`)
|
||||||
|
|
||||||
|
**Status:** ✅ done. Three test files under `tests/ttp/store/`:
|
||||||
|
|
||||||
|
* `test_conformance.py` — cross-backend assertions parametrized via
|
||||||
|
the `rule_store` fixture in `conftest.py`. `get_state` default
|
||||||
|
for unknown rule_id is GREEN on `FilesystemRuleStore` (the
|
||||||
|
in-memory cache returns `RuleState()` for empty lookup); the
|
||||||
|
`DatabaseRuleStore` parametrization xfails until E.3.6. Other
|
||||||
|
conformance assertions (`load_compiled` corpus equality,
|
||||||
|
`set_state` isolation/round-trip, `subscribe_changes` per-rule
|
||||||
|
fan-out, `expires_at` auto-revert, `set_state` failure
|
||||||
|
semantics) xfail-gated behind E.3.5/E.3.6.
|
||||||
|
* `test_filesystem.py` — Linux-only (skipped wholesale on macOS /
|
||||||
|
Windows). Inotify mask + canonical kernel values + 9
|
||||||
|
scratch-filename rejections + 4 valid-filename acceptances +
|
||||||
|
fullmatch-anchor pinning + tmp_path construction +
|
||||||
|
`CompiledRule` immutability GREEN today. Doc references
|
||||||
|
`dataclasses.FrozenInstanceError` for the immutability smoke
|
||||||
|
signal but the actual implementation uses NamedTuple, which
|
||||||
|
raises `AttributeError` on assignment — the test pins
|
||||||
|
`AttributeError` and the test docstring calls out the
|
||||||
|
divergence. Per-save-style + filter-ordering + atomic-swap
|
||||||
|
concurrency xfail-gated behind E.3.5.
|
||||||
|
* `test_database.py` — class-level surface (no platform guard, all
|
||||||
|
ABC methods concrete, async coroutines) GREEN today;
|
||||||
|
`ttp_rule_state` writes + filesystem→DB sync xfail-gated
|
||||||
|
behind E.3.6.
|
||||||
|
|
||||||
The crucial property: both backends satisfy the **same** ABC
|
The crucial property: both backends satisfy the **same** ABC
|
||||||
contract observably. Tests are parametrized over
|
contract observably. Tests are parametrized over
|
||||||
`(FilesystemRuleStore, DatabaseRuleStore)` and assert identical
|
`(FilesystemRuleStore, DatabaseRuleStore)` and assert identical
|
||||||
|
|||||||
Reference in New Issue
Block a user