chore: drop credential-reuse handoff TODO
Dashboard tab + drawer + bidirectional badge shipped, secret enrichment landed, systemd unit wired, DEBT.md follow-up closed. The handoff doc has no remaining work.
This commit is contained in:
210
TODO.md
210
TODO.md
@@ -1,210 +0,0 @@
|
||||
# TODO — credential reuse + vectorstore (handoff)
|
||||
|
||||
This document hands off in-progress work on the **credential reuse
|
||||
patterns** task from `development/DEVELOPMENT.md` (under *Service-Level
|
||||
Behavioral Profiling*) plus the **`decnet/vectorstore/`** scaffolding
|
||||
that prepares the substrate for a future statistical re-identification
|
||||
engine over behavioral fingerprints. See
|
||||
`/home/anti/.claude/plans/ah-excellent-alright-claude-vivid-thimble.md`
|
||||
for the full approved plan and motivation.
|
||||
|
||||
## Done in the previous session
|
||||
|
||||
Foundation is shipped + tested (26 new tests passing, no regressions):
|
||||
|
||||
- **Schema** — `decnet/web/db/models/logs.py`
|
||||
- `Credential.attacker_uuid: Optional[str]` FK to `attackers.uuid`,
|
||||
nullable. Backfilled by the profiler post-write.
|
||||
- `CredentialReuse` table (UUID PK; JSON list columns for
|
||||
`attacker_uuids`, `attacker_ips`, `deckies`, `services`;
|
||||
`target_count`, `attempt_count`, `confidence` reserved for future
|
||||
fuzzy matching). Unique key: `(secret_sha256, secret_kind,
|
||||
principal_key)`.
|
||||
- `CredentialReuseResponse` Pydantic DTO.
|
||||
- **Repo** — `decnet/web/db/sqlmodel_repo.py` + `repository.py`
|
||||
- `upsert_credential_reuse(...)`,
|
||||
`list_credential_reuses(limit, offset, min_target_count, secret_kind)`,
|
||||
`get_credential_reuse_by_id(id)`,
|
||||
`update_credential_attacker_uuid(attacker_ip, attacker_uuid) -> int`.
|
||||
- **Rename**: pre-existing `get_credential_reuse(secret_sha256)` →
|
||||
`get_credential_attempts_for_secret(secret_sha256)`. All callers
|
||||
updated.
|
||||
- **Bus topics** — `decnet/bus/topics.py`
|
||||
- `CREDENTIAL_CAPTURED = "captured"` (one per Credential upsert).
|
||||
- `CREDENTIAL_REUSE_DETECTED = "reuse.detected"` (correlator emits
|
||||
on insert/grow).
|
||||
- `credential(event_type)` builder.
|
||||
- **Vectorstore** — `decnet/vectorstore/` (NEW; flat layout mirroring
|
||||
`decnet/bus/`)
|
||||
- `base.py` — `BaseVectorStore` ABC, `VectorRecord`, `Neighbor`,
|
||||
`VECTORSTORE_SCHEMA_VERSION`. Methods: `initialize`, `close`,
|
||||
`health`, `insert`, `get`, `delete`, `knn`. Keyed by `(kind, id)`.
|
||||
- `fake.py` — `FakeVectorStore` (in-memory, brute-force L2 KNN) +
|
||||
`NullVectorStore` (no-op when `DECNET_VECTORSTORE_ENABLED=false`).
|
||||
- `sqlite_vec.py` — `SqliteVecVectorStore`; lazy-loads the
|
||||
`sqlite_vec` extension; one `vec0` virtual table per `kind` so
|
||||
new feature families don't require schema migration. Per-kind
|
||||
dim is locked on first insert.
|
||||
- `factory.py` — `get_vectorstore()` env-driven dispatch
|
||||
(`DECNET_VECTORSTORE_TYPE` ∈ {sqlite_vec, fake};
|
||||
`DECNET_VECTORSTORE_ENABLED`; `DECNET_VECTORSTORE_PATH`). On
|
||||
missing `sqlite_vec` extension: logs a warning and returns
|
||||
`FakeVectorStore` so workers don't crash.
|
||||
- **Tests**
|
||||
- `tests/db/test_credential_reuse.py` — 11 tests (upsert idempotency,
|
||||
list filters/pagination, FK backfill semantics, null-principal
|
||||
uniqueness, JSON-list merging).
|
||||
- `tests/vectorstore/test_factory.py` (6) +
|
||||
`tests/vectorstore/test_fake.py` (9) — factory dispatch + fallback,
|
||||
round-trip, dim-mismatch raises, KNN ordering, NullStore no-op.
|
||||
- Updated `tests/db/test_base_repo.py` and
|
||||
`tests/db/test_credentials.py` for the rename.
|
||||
|
||||
## Not yet done — what the next agent should pick up
|
||||
|
||||
Tasks below are roughly in dependency order. Backend first, dashboard
|
||||
last (it's the largest unknown and benefits from a fresh context).
|
||||
|
||||
### 1. Profiler backfill of `Credential.attacker_uuid`
|
||||
|
||||
Smallest task; do this first to validate the FK column end-to-end.
|
||||
|
||||
- File: `decnet/profiler/` — find the spot where the profiler
|
||||
mints/updates an `Attacker` row from observed events. There's
|
||||
likely an `upsert_attacker(...)` call that produces the `(ip, uuid)`
|
||||
pair.
|
||||
- Add immediately after a successful upsert:
|
||||
```python
|
||||
await repo.update_credential_attacker_uuid(ip, uuid)
|
||||
```
|
||||
- Test in `tests/profiler/` (whatever the existing test file is) that
|
||||
after the profiler processes events for an IP, all `Credential`
|
||||
rows for that IP have their `attacker_uuid` populated. Use the
|
||||
pattern from `tests/db/test_credential_reuse.py::
|
||||
test_update_credential_attacker_uuid_backfills_only_nulls`.
|
||||
|
||||
### 2. Correlator engine + worker wiring
|
||||
|
||||
- File: `decnet/correlation/engine.py` — add
|
||||
`correlate_credential_reuse(min_targets: int = 2)` to
|
||||
`CorrelationEngine`. Signature suggested in the plan:
|
||||
```sql
|
||||
SELECT secret_sha256, secret_kind, principal,
|
||||
COUNT(DISTINCT decky_name||':'||service) AS target_count
|
||||
FROM credentials
|
||||
GROUP BY secret_sha256, secret_kind, principal
|
||||
HAVING target_count >= :min_targets
|
||||
```
|
||||
For each group, fetch the underlying credential rows and call
|
||||
`repo.upsert_credential_reuse(...)` per row. The repo upsert
|
||||
recomputes `target_count` from the `credentials` table on each
|
||||
update, so you don't need to pass aggregates in.
|
||||
- On insert/grow (`out["inserted"] is True or out["changed"] is True`),
|
||||
publish `bus.publish(topics.credential(topics.CREDENTIAL_REUSE_DETECTED), {...})`
|
||||
with payload `{id, secret_kind, target_count, attacker_uuids,
|
||||
attacker_ips, deckies, services}`.
|
||||
- Worker file: `decnet/correlation/main.py` (or wherever
|
||||
`CorrelationEngine` is loop-driven). Subscribe to:
|
||||
- `attacker.observed` — re-runs reuse pass for that IP.
|
||||
- `credential.captured` — re-runs reuse pass for that secret.
|
||||
- Heartbeat tick every 60s as a fallback (mirror the mutator's
|
||||
bus-wake + slow-tick pattern).
|
||||
- Where is `credential.captured` emitted? Find the credential ingest
|
||||
path — probably `decnet/collector/` or wherever
|
||||
`repo.upsert_credential(...)` is called. Add a `bus.publish(
|
||||
topics.credential(topics.CREDENTIAL_CAPTURED), {secret_sha256,
|
||||
secret_kind, attacker_ip, decky, service})` after a successful
|
||||
upsert. Bus is fire-and-forget — don't block on it.
|
||||
- Tests:
|
||||
- `tests/correlation/test_credential_reuse.py` — engine emits the
|
||||
right `CredentialReuse` rows from synthetic credentials; bus
|
||||
event published exactly once per insert/grow.
|
||||
- Use `decnet.bus.fake.FakeBus` in tests; collect published
|
||||
events for assertion.
|
||||
|
||||
### 3. API routes — `GET /api/v1/credential-reuse`
|
||||
|
||||
- File: probably `decnet/web/api/routes/` — see how existing
|
||||
credentials routes are organized (recent commit
|
||||
`feat(api): GET /credentials endpoint` → `4566146`).
|
||||
- Endpoints:
|
||||
- `GET /api/v1/credential-reuse?limit=50&offset=0&min_target_count=2&secret_kind=plaintext`
|
||||
→ `CredentialReuseResponse` (already in models).
|
||||
- `GET /api/v1/credential-reuse/{id}` → single row dict, 404 if
|
||||
missing.
|
||||
- JWT-gated like all other routes. Use the existing dependency.
|
||||
- No POST/PUT/PATCH — read-only this release. Per the
|
||||
`feedback_schemathesis_400` memory there's no 400 contract to
|
||||
document since there's no body parsing.
|
||||
- Tests: `tests/api/test_credential_reuse_routes.py` — JWT gate,
|
||||
pagination, filters, 404 for missing id.
|
||||
|
||||
### 4. Dashboard — Credentials Reuse tab + drawer
|
||||
|
||||
The big unknown. Next agent should:
|
||||
|
||||
1. Survey `decnet/web/dashboard/` (React app) — how the existing
|
||||
Credentials view is structured (commit `4ea4b0b feat(web):
|
||||
Credentials view + inspector`).
|
||||
2. Add a "Reuse" tab/filter that lists `CredentialReuse` rows sorted
|
||||
by `target_count desc`.
|
||||
3. Drawer on row-click showing decky×service breakdown,
|
||||
`attacker_uuid` list (link to `/attackers/:id`), timeline. Reuse
|
||||
the existing drawer pattern (see `feedback_react_stop_propagation_native_delegation`
|
||||
memory — backdrop click closes via `target===currentTarget`,
|
||||
never `stopPropagation`).
|
||||
4. On the existing Credentials list, add a "seen on N targets"
|
||||
badge when a credential has a corresponding `CredentialReuse`
|
||||
row, so the connection is bidirectional.
|
||||
|
||||
### 5. DEVELOPMENT.md
|
||||
|
||||
Tick `[x] Credential reuse patterns` under *Service-Level Behavioral
|
||||
Profiling*. Add a one-liner under *Attacker Intelligence Collection*
|
||||
noting `decnet/vectorstore/` is scaffolded for the future statistical
|
||||
re-ID engine (no behavioural change yet).
|
||||
|
||||
## Architectural decisions worth knowing
|
||||
|
||||
These came out of the design conversation that produced the plan; the
|
||||
next agent should respect them:
|
||||
|
||||
- **Classical statistics, not ML**, for attacker re-identification.
|
||||
Cosine/Mahalanobis/KS-test over per-kind feature vectors, weighted
|
||||
voting, versioned thresholds. Reproducible, explainable, no model
|
||||
drift. ML is reserved for a future *advisory* layer behind the
|
||||
factory, never primary.
|
||||
- **Provider factory pattern is mandatory** for any new pluggable
|
||||
backend (storage, transport, similarity). Mirror `decnet/web/db/`
|
||||
and `decnet/bus/` — never let workers import concrete backends.
|
||||
- **`kind` discriminator is the extension point** for new feature
|
||||
families. Adding `kind="cmd_ngram"` later does not require schema
|
||||
changes — the `vec_<kind>` table is created lazily on first insert.
|
||||
- **`Credential.attacker_uuid` is nullable on write** by design — the
|
||||
credential capture path runs before the profiler mints `Attacker`,
|
||||
so coupling them would create a chicken-and-egg ordering bug. The
|
||||
profiler backfills.
|
||||
- **`CredentialReuse.confidence` is always 1.0 today** (exact-secret
|
||||
match). The column exists so a future fuzzy-credential pass
|
||||
(`hunter2` ≈ `hunter22`) can write 0.x rows without schema work.
|
||||
|
||||
## Verification checklist for the next agent
|
||||
|
||||
After finishing each chunk:
|
||||
|
||||
- `pytest tests/<area> --timeout=30 --timeout-method=thread` — must
|
||||
be green before moving on.
|
||||
- Don't run fuzz/bench/live/stress in the dev loop (memory:
|
||||
`feedback_skip_heavy_tests`).
|
||||
- Don't pre-clear with custom bandit/ruff flags (memory:
|
||||
`feedback_trust_git_hooks`) — the pre-commit hook is authoritative.
|
||||
- Commit per task, not batched (memory: `feedback_commit_per_task`).
|
||||
Don't add Co-Authored-By to commit messages.
|
||||
|
||||
## Open questions to surface to ANTI before tackling §4
|
||||
|
||||
- Should the dashboard "Reuse" surface live as a tab on the existing
|
||||
Credentials page, or as a sibling page? (The plan said tab, but
|
||||
worth confirming once you've seen the code.)
|
||||
- Pagination size for the reuse list — match the existing Credentials
|
||||
view default, or use a smaller page since the rows are wider?
|
||||
Reference in New Issue
Block a user