Rewrites the architecture section for the full current module tree and adds
new sections for the REST API, swarm/agent mode, service bus, attacker
intelligence stack (profiler, clustering, correlation, GeoIP/ASN),
MazeNET topology, canary tokens, and TTP tagging/export. Updates the CLI
reference table, test count (478 → 5050), and Python version constraints.
Replaces LICENSE (GPLv3 -> AGPLv3) and prepends
`SPDX-License-Identifier: AGPL-3.0-or-later` to every source file
across decnet/, decnet_web/, tests/, scripts/, and tools/.
Rationale: closes the GPLv3 ASP loophole so any party operating a
modified DECNET as a network service must offer their modified
source. Personal copyright (Samuel Paschuan) + inbound=outbound
contributions make a future unilateral relicense infeasible.
- LICENSE: full AGPL-3.0 text (gnu.org/licenses/agpl-3.0.txt)
- COPYRIGHT: project copyright notice
- tools/add_spdx_headers.py: idempotent header injector
(shebang- and PEP 263-aware)
Touches 1565 source files (.py, .ts, .tsx, .js, .jsx, .css, .sh).
No behavior change; comments only.
Every compose invocation used -p decnet so fleet + every topology
lived in one docker compose project. --remove-orphans, run during
fleet pre-up cleanup and on every topology teardown / rollback, then
swept every container in the project not listed in the current compose
file — wiping sibling topologies and the flat fleet along with the
intended target.
Parameterize project on _compose / _compose_with_retry / _compose_ps
(default FLEET_COMPOSE_PROJECT="decnet"). Add _topology_compose_project
that returns decnet-topo-<id8>, and pass it through every topology
compose call site (master deploy_topology + rollback + post-deploy ps,
master teardown_topology, agent apply, agent teardown, all four live
service mutations on topology deckies). Fleet calls keep the default
and are unaffected.
Migration: live containers from before this fix remain in the shared
"decnet" project and need a one-time manual cleanup before they're
reachable to the new topology code paths.
The wizard POSTs only the new decky on each submit. The handler used to
treat every INI as the complete desired fleet (config.deckies = INI) so
the reconciler tore down prior deckies as orphans — deploying a second
Windows workstation silently wiped the first.
Add replace_fleet to DeployIniRequest (default false). Default path
merges new deckies into existing config and rejects name/IP collisions
with 409. replace_fleet=true preserves set-desired-state semantics for
CLI / declarative callers. Lifecycle rows are created only for the
deckies submitted in the current call, so /deckies/lifecycle?ids=...
reflects exactly what this submit deployed.
build_deckies_from_ini gains reserved_ips so additive auto-allocation
skips IPs already held by the existing fleet.
- New useLifecyclePolling(ids, intervalMs) hook: polls
GET /deckies/lifecycle?ids=... every 2s until every row is terminal,
surfaces transient HTTP failures without giving up.
- DeployWizard: drops the 180s axios timeout and the fake-log-driven
deployOk flag. After POST 202, sets lifecycle_ids -> the hook drives
the per-decky pill grid (PENDING / RUNNING / SUCCEEDED / FAILED).
Real terminal lines stream into the log as rows resolve. Auto-close
on all-success after 700ms.
- DeckyFleet.css: .lifecycle-grid + .lifecycle-pill in the existing
fleet vocabulary; running pill pulses, failed pill borders alert.
- Existing 4 wizard render tests still pass; 4 new hook tests cover
empty ids / single-success / polling-until-terminal / HTTP error.
GET /deckies/lifecycle?ids=<uuid>&ids=<uuid> returns the matching
DeckyLifecycle rows so the wizard can poll instead of holding an HTTP
request open across compose work. require_viewer gating -- read-only.
Startup sweep: on master boot, any pending/running row with
started_at older than 1h flips to failed with
error='master restarted during operation'. Pre-v1 substitute for a
durable task queue: if the master crashes mid-deploy, the wizard sees
FAILED on refresh and the operator retries. Idempotent + cheap; runs
unconditionally including in contract-test mode.
This is the unblock for the wizard hang. Both endpoints used to run
docker compose synchronously inside the HTTP handler -- on master
(unihost) or via asyncio.gather of worker /deploy POSTs at 600s
timeout each (swarm) -- blocking every other API request.
New flow:
1. Commit the new config shape to repo state (fast).
2. Create one DeckyLifecycle row per decky (status=pending).
3. Spawn asyncio.create_task(run_deploy / run_mutate) -- the
lifecycle runner drives rows through running -> succeeded|failed
and emits decky.<name>.lifecycle on the bus.
4. Return 202 with {lifecycle_ids: [...]}. Wizard polls
GET /deckies/lifecycle?ids=... (next commit).
mutator/engine.py gains pick_new_services() -- shared between the
async API path and the watch-loop's synchronous mutate_decky().
DeployResponse grows lifecycle_ids[]. The old dispatch_decnet_config
helper still exists for the CLI swarm-deploy command path; it just
isn't called from the API handler anymore.
Test changes: 200 -> 202, drop dispatch_decnet_config mocks (handler
no longer calls it), assert lifecycle_ids in response + committed
state matches expectations.
HeartbeatRequest grows an optional lifecycle field carrying per-decky
completion records from the worker:
[{decky_name, operation, status, error?, completed_at?}]
For each delta, the master finds the most-recently-started open
DeckyLifecycle row for (decky_name, operation, host_uuid) and flips
it to terminal with the worker's error text + timestamp. Stale
duplicates (row already sealed or never existed) are logged and
dropped -- not errors.
Each successful pivot also emits decky.<name>.lifecycle on the bus
so the dashboard sees the transition without waiting for its next
poll tick.
This is the master-side completion channel for the worker's 202
fire-and-forget /deploy and /mutate.
The wizard API used to hang because /deckies/deploy ran docker compose
build && up -d synchronously, holding the request thread for minutes.
The worker side of that pipeline now returns 202 Accepted immediately
and runs the deploy in an asyncio.create_task.
On task completion (success or failure) the worker pushes a one-off
heartbeat carrying a lifecycle delta per decky:
{decky_name, operation, status: succeeded|failed, error?, completed_at}
Master pivots these onto open DeckyLifecycle rows in the heartbeat
handler (next commit). The scheduled 30s heartbeat tick is the
fallback if the immediate push drops.
- decnet/agent/app.py: /deploy and /mutate return 202; dry_run mutate
still validates synchronously and returns 200.
- decnet/agent/executor.py: deploy_async + mutate_async wrap the work
and push the completion delta.
- decnet/agent/heartbeat.py: push_lifecycle_delta() helper builds a
one-off body and POSTs with the same mTLS context as the loop.
- decnet/swarm/client.py: revert deploy/mutate to control timeout
(master no longer holds the HTTP request open for compose work).
Worker state.json gains no lifecycle field -- master DeckyLifecycle is
the source of truth; the master sweep handles crashed-mid-deploy
recovery.
Add decnet.lifecycle package: pure orchestration layer that the
master API will invoke via asyncio.create_task to drive DeckyLifecycle
rows through pending -> running -> succeeded | failed without
holding an HTTP request open.
Strategy classes per (operation, transport):
- LocalDeployStrategy: master-resident, runs engine.deployer.deploy
in a thread.
- SwarmDeployStrategy: shards by host_uuid, dispatches via
AgentClient.deploy; worker drives terminal via heartbeat.
- LocalMutateStrategy: write_compose + compose up.
- SwarmMutateStrategy: AgentClient.mutate; worker drives terminal.
decnet.bus.topics gains decky_lifecycle(name) -> decky.<name>.lifecycle
plus DECKY_LIFECYCLE constant. Payload documented in the wiki
(separate commit). publish_safely keeps bus best-effort.
Nothing is wired to call this yet -- next commits convert worker
/deploy /mutate to 202, then heartbeat delta wiring, then master API.
One row per (decky, operation) attempt. State machine:
pending -> running -> succeeded | failed (+ error text). Rows are
append-only after terminal; retries write a new row.
Sibling of DeckyShard rather than a rework -- DeckyShard tracks
runtime container state observed via heartbeat, this tracks
operation lifecycle. New table, UUID PK.
Adds BaseRepository abstract methods (create_lifecycle,
update_lifecycle, get_lifecycle_by_ids, find_open_lifecycle,
sweep_stale_lifecycle) with SQLModelRepository mixin impl.
Backbone for the upcoming 202-Accepted async API.
- Implement /mutate handler: load_state, update services + last_mutated,
save_state, write_compose, compose up -d via asyncio.to_thread. 404
for missing state / unknown decky_id. dry_run short-circuits before
any side effect.
- Add AgentClient.mutate(decky_id, services, *, dry_run=False) using
_TIMEOUT_DEPLOY (compose up can pull/build, exceeds control timeout).
- mutator/engine.py: in swarm mode with decky.host_uuid set, resolve
worker via _resolve_swarm_host and dispatch through AgentClient.mutate
instead of writing a compose file on master. Master-resident deckies
(unihost mode, or swarm with host_uuid=None) keep the local path.
Adds state_path ServiceConfigField and passes DNS_STATE_PATH into the
container environment. Operator must mount the parent directory on a
volume for persistence to survive container recreation.
Switch burst deque from monotonic() to time.time() (wall-clock, serializable).
Add DNS_STATE_PATH env var: on startup _load_state() reads {src:[ts,...]} JSON
and prunes entries older than the burst window. _flush_state() write-then-renames
atomically; _state_flusher() coroutine flushes every 5s when dirty. Detection of
the 5th event also triggers an immediate flush. No-op when DNS_STATE_PATH is
unset, so the default deployment is unchanged.
Rename _txt_times -> _tunnel_times. Add TYPE_CNAME=5, TYPE_NULL=10,
TYPE_PRIVATE=65399 constants. Guard burst counter with _TUNNEL_QTYPES
frozenset instead of TYPE_TXT only. Mixed-type queries from one source
now share a single burst window, closing iodine NULL/CNAME downlink
and AAAA-encoded uplink evasion gaps.
_is_tunneling now returns str|None (the detection method) instead of bool.
Two new tunables _QNAME_TOTAL_LEN_THRESHOLD=50 and _QNAME_ENTROPY_THRESHOLD=3.5
catch attackers who split a high-entropy payload across multiple short labels.
tunnel_method field added to tunneling_suspect events for downstream correlation.
_parse_edns_size only extracted the requestor UDP size; every other field in
the OPT record (DO bit, EDNS version, extended RCODE, all sub-options) was
invisible. Replaced with _parse_opt_record returning a full dict:
udp_size, ext_rcode, version, do_bit, z, options[(code, len, data)]
NSID request (option code 3) is now detected as fingerprint_probe with
probe=edns_nsid and contributes to recon_burst. DO bit, COOKIE (10), and
other options are not escalated; udp_size continues to drive amp_probe.
Tools like fpdns send OPCODE=IQUERY/STATUS/NOTIFY/UPDATE or set the reserved
Z bit to fingerprint resolver behaviour. Previously all these were parsed as
standard queries with no signal.
- opcode!=0 → fingerprint_probe probe=opcode_<name>, NOTIMP response;
fired before qdcount check so qdcount=0 UPDATE packets are still caught.
- Z bit set OR (AD+CD without RD) → fingerprint_probe probe=header_flags;
AD alone with RD is ignored to avoid tagging DNSSEC-aware stubs.
- Both variants contribute to recon_burst.
qclass=255 in a standard query is unusual enough to be a fingerprinting probe
(fpdns, various scanner scripts). Previously it was logged as a plain query
with qclass=ANY in the event field; now it emits fingerprint_probe with
probe=qclass_any and returns REFUSED — consistent with how we treat other
probe types. Contributes to recon_burst.
The inline probe_map dict inside _handle made tests blind to the probe
catalogue and couldn't be extended without touching the hot path. It is now
module-level _CHAOS_PROBE_MAP. authors.bind. joins the three existing entries
so it gets named correctly instead of carrying the raw qname.
Packets with multiple questions were silently parsed at q0 only; the extra
questions were invisible. Now emits multi_question at severity=5 with the
qdcount and q0 qname, then falls through and answers q0 normally.
Silent drops on <12B packets, qdcount=0, and question-section ValueError gave
fuzzers and scanners a completely dark target. New events malformed_packet,
empty_question_section, and question_parse_error fire at severity=5 so these
probes are visible without counting toward recon_burst.
Adds DNS_FORWARD_BUDGET (default 50) and DNS_FORWARD_WINDOW (default 1.0s)
env vars. _can_forward() maintains a rolling deque of upstream call
timestamps; queries that exceed the budget within the window are answered
with the sinkhole (127.x) instead of being forwarded, making the honeypot
ineligible as a sustained amp vector even when real_recursive is enabled.
Rate limit is global (not per-source) so IP-spoofed amplification floods
hit the ceiling regardless of how many source addresses are rotated.
When DNS_REAL_RECURSIVE=true and DNS_ZONE_MODE=recursive, out-of-zone
queries are forwarded to DNS_UPSTREAM (default 8.8.8.8:53) via async
UDP. Upstream response is relayed as-is; on timeout or error the
already-computed sinkhole (127.x) is returned instead.
_handle() always runs first so logging, tunneling detection, flood
tracking, and recon-burst aggregation fire on every query regardless
of whether the response ultimately comes from upstream. _dispatch()
overlays forwarding on top of the sync handler.
Protocol handlers (UDP datagram_received, TCP session) are now async
via asyncio.ensure_future / await _dispatch(). Service class exposes
real_recursive (bool) and upstream (string) config fields.
RA=1 + empty answer section is immediately detectable as fake by any
open-resolver scanner. Recursive mode now behaves like open mode
(127.0.0.x sinkhole, deterministic on qname) with RA=1 and AA=0,
matching what a real recursive resolver returns.
- Add per-src QPS counter (_qps_window) with flood_suspect event at ≥50 qps/10s;
one event per src per 30s cooldown, does not suppress baseline query events.
- Add tracking_evicted telemetry every 100 LRU evictions so IP-rotation evasion
of _txt_times/_qps_window/_recon_window is observable, not silent.
- Shared _track_lru helper consolidates LRU touch + eviction signalling across
all three bounded OrderedDicts.
- Add TYPE_AAAA=28 support: _fake_ipv6() returns deterministic ULA (fd::/8)
addresses for in-zone names; extra_records parser now accepts and validates
AAAA entries via socket.inet_pton.
- Add per-src recon-burst aggregation (_recon_window): fingerprint_probe +
zone_transfer + amp_probe are tracked per source in a 60s window; recon_burst
fires when ≥2 distinct signal types seen, once per src per 120s cooldown.
- 47 tests passing (19 new across TestAAAARecords, TestFloodDetection, TestReconBurst).
Python asyncio DNS server on UDP+TCP/53 masquerading as BIND 9.x.
Emits four event_type values: query, fingerprint_probe (version.bind /
hostname.bind / id.server CHAOS), zone_transfer (AXFR/IXFR, always
REFUSED), amp_probe (qtype=ANY or EDNS udp_size>1232), and
tunneling_suspect (long high-entropy labels or rapid TXT burst).
Zone persona is generated per-decky from instance_seed (domain name,
SOA serial, NS, A, MX, TXT SPF); overridable via config_schema.
Three zone modes: auth (default), recursive, open (sinkhole).
AttackerData type gets bgp_prefix / rpki_status / rpki_source.
TimelineSection renders prefix inline next to AS number; RPKI status
shows as a green RPKI VALID / red RPKI INVALID badge, or dim
NO ROA for not-found. rpki-status-badge CSS added to Dashboard.css.
Export network block extended with the three new fields.
Import enrich_rpki from decnet.rpki and call it inline after the
ASN lookup. bgp_prefix, rpki_status, rpki_source added to the
record dict that feeds the Attacker upsert. enrich_rpki short-circuits
to (None, None) when asn is None, so private / unannounced IPs
never hit RIPE STAT.
bgp_prefix (max 43 chars, indexed) holds the covering CIDR from
the ASN lookup. rpki_status / rpki_source hold RIPE STAT validation
outcome. All nullable — null means enrichment was skipped or ASN
did not resolve.
RipeStatValidator makes two RIPE STAT calls per uncached IP:
network-info -> announced prefix, rpki-validation -> ROA state.
2-second timeout; any network failure returns status='unknown'.
SQLite cache keyed by IP, 12-hour TTL, pruned on validator init.
Cache avoids per-event HTTP for the high-churn attacker pool —
steady-state cost approaches zero for repeat offenders.
Synthesize the covering CIDR at lookup time from the matched iptoasn
range using ipaddress.summarize_address_range. AsnInfo.prefix is
populated per-query; not persisted in the pickle cache.
enrich_ip now returns (asn, as_name, bgp_prefix, provider_name).
Profiler worker updated to unpack the 4-tuple and write bgp_prefix
into the attacker record dict.
Four RFC 4443 stimuli (port-unreach, hop-limit-exceeded, unknown-NH,
bad-dest-option) produce a 4-char matrix + sha256 fingerprint for IPv6
attackers. Auto-registers via ActiveProbeMeta at priority=860 (after v4
icmp_error=850, before ipv6_leak=999). IPv4 targets fast-return None.
Sends four crafted stimuli (UDP/closed-port, TTL=1, DF+oversized,
bad IP option) and records which ICMP error classes come back, the
per-error RTT, and the bytes echoed in each ICMP body. Absence is
as informative as a reply — Linux rate-limiting is a fingerprint signal.
Returns None when no packets could be sent (no CAP_NET_RAW), so the
probe is a no-op in non-root test environments. Port-free ActiveProbe
subclass (priority=850), metaclass auto-registered in the registry.
Also fixes three sets of stale tests left over from the TlsCertProbe
migration (4b2759e0):
- test_active_probe_registry: closed name/order sets updated for
tls_certificate and icmp_error
- test_prober_rotation: dead patches on worker.fetch_leaf_cert removed
- test_prober_worker (TestProbeCycleTLSCert): rewritten to test
TlsCertProbe as an independent registry probe, patch target updated
from worker.fetch_leaf_cert to probes.tlscert_probe.fetch_leaf_cert
TLS cert capture was the last prober special-case that bypassed
ActiveProbeMeta. Moves logic into TlsCertProbe (priority=200, runs
after JARM) in probes/tlscert_probe.py; drops _capture_tls_cert,
the probe.probe_name=="jarm" name-check, and the direct
fetch_leaf_cert import from worker.py.
ActiveProbe.run/syslog_fields/publish_payload now accept port=None so
non-port-iterating probes can live in the registry. Ipv6LeakProbe replaces
the hand-rolled _ipv6_leak_phase special case in worker.py; it runs last
via priority=999. _probe_cycle no longer has an ad-hoc phase call.
Fixes three stale test files (test_prober_bus, test_prober_rotation,
test_prober_worker) that were broken since the 916b21b6 registry refactor.
_route_info() calls _ip_route_get once and returns (on_link, iface);
worker._ipv6_leak_phase now calls it instead of the two separate helpers.
Bare except clauses at _ip_route_get and response parse now log at debug.
Iterates every template with a Dockerfile, builds decnet/<svc>:latest
with DOCKER_BUILDKIT=1. Supports NO_CACHE=1 and FAIL_FAST=0 flags,
mirrors the style of test-all. Updated help target.
FingerprintGroup switch fell through to FpGeneric (raw JSON dump) for all
four new fingerprint_type values the ingester now produces. Add FpJa4h,
FpHttpSettings, FpJa4Quic components and wire them into the dispatcher;
also register their labels and icons in fpTypeLabel/fpTypeIcon.
ingester: wrap bootstrap get_state() in forever-retry loop — MySQL coming
up after the API process killed the ingestion task permanently before it
ever entered _run_loop. Regression test added.
deps: idna 3.13→3.15 (CVE-2026-45409), twisted 26.4.0rc2→26.4.0
(PYSEC-2026-160), pip 26.1→26.1.1 (CVE-2026-3219 resolved upstream),
behave-core/behave-shell renamed from decnet-behave-* and bumped to 0.1.1.
pre-commit hook updated to reflect current ignore list.
Replace _jarm_phase / _hassh_phase / _tcpfp_phase boilerplate (3×~50
lines of identical port-iteration logic) with a metaclass-registered ABC.
Adding a new port-iterating active probe is now one class + three methods.
- decnet/prober/base.py: ActiveProbeMeta auto-registers subclasses by
probe_name; ActiveProbe ABC enforces run/syslog_fields/publish_payload
with env-driven DECNET_PROBE_PORTS_<NAME> port override.
- decnet/prober/probes/{jarm,hassh,tcpfp}.py: concrete probe classes.
- decnet/prober/worker.py: single _run_probe driver replaces the three
phase functions; _probe_cycle iterates ActiveProbeMeta.all(); drops
the ports=/ssh_ports=/tcpfp_ports= kwargs from prober_worker.
- IPv6 leak and TLS cert capture stay as special cases (different call
shapes; intentionally outside the registry).
- tests/prober/test_active_probe_registry.py: registry contents, sort
order, priority-10 override, ABC contract per probe class.
- tests/prober/test_run_probe_driver.py: dedup, success, None-skip,
exception, rotation, publish paths for _run_probe.
- tests/prober/test_prober_worker.py: updated patch targets and
_probe_cycle call sites; port control via monkeypatch.setattr.
- Add "ipv6_leak" to KNOWN_SOURCE_KINDS in ttp/base.py
- Register Ipv6LeakLifter(store) in factory.py get_tagger()
- Subscribe worker to attacker.fingerprinted; route by Event.type
so JARM/HASSH/ipv6_leak share the topic without source_kind collision
- Add bump_attacker_ipv6_leak() to BaseRepository (abstract) +
TTPMixin (implementation): increments ipv6_leak_count, sets last_ipv6_*
denorm fields, appends-with-dedup to AttackerIdentity.ipv6_link_local_iids
- Call bump_attacker_ipv6_leak from _process_event after insert_tags
- Add DummyRepo stub + coverage call in tests/db/test_base_repo.py
Add inline documentation for all known kind= discriminators on the
fingerprinted topic including the new ipv6_leak variant so future
consumers know what fields to expect without reading the prober source.
Ipv6LeakLifter subscribes to source_kind="ipv6_leak" events from both
the passive sniffer and active prober. Emits T1090 (Proxy) under TA0011
(C2) when fe80:: source address is observed — the attacker's VPN only
tunnels IPv4 so their link-local IID leaks their NIC identity.
Rule R0059 sets base confidence 0.85; iid_kind in the evidence carries
the per-observation strength (eui64 = MAC-derived, deterministic;
stable_privacy = RFC 7217; temporary = RFC 4941).
Add ipv6_leak.py with solicit_ipv6_leak() — sends ICMPv6 Echo to
ff02::1 on the attacker's iface and returns fe80:: evidence when a
link-local response arrives. Gated on _is_on_link(): skips when
attacker is behind a router (no L2 adjacency).
Add _ipv6_leak_phase() to worker.py (Phase 4 in _probe_cycle).
Phase runs once per attacker IP per cycle (sentinel at port 0 in
ip_probed["ipv6_leak"]) and publishes kind="ipv6_leak" via publish_fn.
Add list_v6_addrs(iface) to network.py: returns [(addr, scope)] for
all IPv6 addresses on an interface, required for source-routing ICMPv6
from the correct link-local address.
Add _ipv6_iid_classify() to fingerprint EUI-64 vs stable-privacy IIDs
and derive the MAC OUI from EUI-64-encoded link-local addresses.
SnifferEngine._on_ipv6_packet() observes fe80::/10 sources destined for
known deckies and emits ipv6_link_local_leak syslog + bus events.
on_packet() now dispatches the IPv6 branch before the v4 TCP path.
BPF default widened from "tcp" to "tcp or ip6" so the sniff loop
captures IPv6 frames without config change.
Attacker gains five denormalized cache fields (ipv6_leak_count,
last_ipv6_leak_at, last_ipv6_link_local, last_ipv6_iid_kind,
last_ipv6_mac_oui) mirroring the rotation_count/last_rotation_at pattern.
AttackerIdentity gains ipv6_link_local_iids (JSON list[dict]) for
EUI-64-derived MAC cluster signals that survive VPN/IP rotation.
No ALTER TABLE helpers — direct SQLModel column additions per pre-v1 policy.
Pins the evidence shape for IPv6 link-local leakage findings. All fields
optional (total=False) so partial observation (passive sniffer vs active
solicitation) fills whatever the vector provides. Lifter lands in a
subsequent commit.
- Add dedicated test-schema Makefile target (xdist logical, 600s timeout,
-m fuzz) so schemathesis runs separately from test-fuzz, which was
spinning up competing uvicorn workers per xdist process
- Exclude all test_schemathesis*.py files from FUZZ_FLAGS via --ignore
- Add schema to _ALL_SUITES between api and fuzz
- Add SCHEMA_QUICK env var (default 0): caps every max_examples to 100
across all four schemathesis files (4520 -> 600 total examples)
- Fix pre-push hook: use .311 venv and delegate to make test-all FAIL_FAST=0
instead of hand-rolling five separate pytest invocations
@pytest.fixture on an async fixture ignores loop_scope, so mysql_repo
ran on the per-function loop while mysql_test_db_url's engine was bound
to the module loop — triggering 'Future attached to a different loop'.
Add mypy, bandit, vulture, pip-audit as Makefile targets and include
them in test-all. Also enable -n logical on SEQ_FLAGS so live/api/stress
suites run in parallel where async-safe.
After the ingester._sleep alias fix, three tests in test_service_isolation.py
still patched `decnet.web.ingester.asyncio.sleep` (the old global-singleton
path). The ingester now calls `_sleep` directly, so those patches no longer
controlled the ingester's sleep — the worker looped with real asyncio.sleep
and the tests hung indefinitely.
Also: four API lifespan tests had no tarpit_watcher_worker patch, letting the
real tarpit task start. And test_api_survives_db_init_failure patched
`decnet.web.api.asyncio.sleep` (the singleton) instead of the existing
`_retry_sleep` alias.
Fixes:
- patch("decnet.web.ingester._sleep", ...) in the three ingester tests
- add tarpit_watcher_worker patch to all four api lifespan tests
- patch("decnet.web.api._retry_sleep", ...) in db_init_failure test
Two interacting bugs caused asyncio.sleep to be mocked globally,
letting tarpit_watcher_worker spin the event loop on a non-async
mock and accumulate _increment_mock_call records without bound:
1. test_ingester.py patched `decnet.web.ingester.asyncio.sleep` via
the asyncio singleton — any code in the process using asyncio.sleep
(including the tarpit worker) hit the fake_sleep side_effect.
Fix: add `_sleep = asyncio.sleep` alias in ingester.py and patch
`decnet.web.ingester._sleep` instead — scopes the mock to ingester.
2. test_api_startup_guards.py called `_run_lifespan_startup` without
DECNET_CONTRACT_TEST=true, which started the real tarpit task in a
manually-constructed event loop that the tests never cancelled.
Fix: set DECNET_CONTRACT_TEST=true inside _run_lifespan_startup so
the lifespan skips all background workers.
asyncio_default_fixture_loop_scope was 'module', so all async tests in
a module share one event loop. test_lifespan_startup_and_shutdown patched
log_ingestion_worker/log_collector_worker/attacker_profile_worker but not
tarpit_watcher_worker — the real while-True coroutine was created as an
asyncio task on the shared loop and never cancelled. The xdist worker ran
for 4+ hours (confirmed via py-spy + etime=04:48) consuming 15+ GB before
OOM-kill.
Fixes:
- Patch tarpit_watcher_worker in both TestLifespan tests
- Change asyncio_default_fixture_loop_scope to 'function' so each test
gets its own loop; tasks cannot outlive their test
- Add loop_scope='module' to precision_engine which legitimately needs
a module-scoped event loop
Five list columns (greynoise_tags, abuseipdb_categories, threatfox_threat_types,
threatfox_ioc_types, threatfox_malware_families) and four dict columns
(*_raw) are now Column(JSON) with list/dict type annotations and
default_factory=list/dict. Providers return native Python objects; the
application-layer json.dumps/json.loads round-trip and _decode_json_list
helpers are gone. to_intel_event_payload() reads columns directly.
Also caps pytest xdist at -n 4 and excludes tests/api from norecursedirs
to prevent schemathesis workers from OOM-killing the dev loop.
- test_evidence_shape.py: replace broken (command, BehavioralLifter)
pairing with correct (http_fingerprint, HttpFingerprintLifter) case;
expand _LIFTER_CASES to 5-tuples with per-lifter payloads and rule
factories; wire StubRuleStore + _index.install() per lifter; remove
xfail marker — all 4 parametrized cases now pass
- factory.py: add _span() helper gated on _telemetry._ENABLED; wrap
each per-lifter dispatch in _tag_one() that opens a
ttp.lifter.{name} child span per call
- http_fingerprint_lifter.py: add missing name = "http_fingerprint"
- test_tracing.py: replace pytest.fail() stubs in
test_lifter_child_spans_emitted and test_no_pii_canary_in_span_attributes
with real test bodies; remove xfail markers
Removes the E.3.14b xfail marker and writes the test body:
- _StubRepo gains get_attacker_intel_row_by_uuid(uuid) backed by an
optional intel_rows dict; existing tests pass None (no catch-up, no
change to their behaviour).
- The test drives a session.ended event with NO intel.enriched published,
injects an AttackerIntel row into the stub repo, and asserts the
tagger is called with source_kind='intel' carrying the correct payload
fields (abuseipdb_score, greynoise_classification).
- Pins the asymmetry contract: email.received has no catch-up path
(sibling test already green); intel does.
On every attacker.session.ended event, the TTP worker now reads the
persisted AttackerIntel row (if any) and synthesizes an intel-source
TaggerEvent so intel-derived tags emit even when attacker.intel.enriched
was dropped or arrived before the worker started.
Key changes:
- AttackerIntel.to_intel_event_payload() — single source of truth for
the intel-row → lifter payload projection; shared by future callers
without importing decnet.intel.* (no-SPOF contract preserved).
- BaseRepository.get_attacker_intel_row_by_uuid() — returns the live
SQLModel instance so the catch-up path can call to_intel_event_payload().
- _build_intel_catchup_event() in ttp/worker.py — looks up the intel row,
builds the TaggerEvent, returns None on absent row (silence, not error).
- _process_event() extended: appends the catch-up event to tagger_events
when topic contains "session.ended". Deterministic source_id keeps
compute_tag_uuid idempotent across replays; INSERT OR IGNORE deduplicates
against any prior attacker.intel.enriched path.
DummyRepo stub + coverage call added per feedback_run_base_repo_test.md.
Replace pytest.fail() stub with actual test body: constructs IntelLifter
with R0054, feeds score=30 payload, asserts confidence=0.21 (0.70×0.30)
which is below CONFIDENCE_FLOOR. xfail marker removed.
Corrects docstring: R0054 T1110 base_conf=0.70, not 0.85 as originally written.
- TolerantTagger.tag validates evidence keys against EVIDENCE_SCHEMA TypedDicts;
TypeError (programmer error) propagates instead of being swallowed
- IntelEvidence and EmailEvidence expanded from stubs to full per-provider
key sets (total=False); IntelEvidence old stub fields replaced wholesale
- EVIDENCE_SCHEMA map added to models/ttp.py and imported by base.py
- TTPTag __table_args__ gains confidence [0,1] CheckConstraint (DB-enforced)
- xfail removed from test_confidence_outside_range_rejected_at_insert and
test_evidence_shape_violation_propagates_as_typeerror — both now pass
- TypeError removed from _SWALLOWED_EXCS fuzz list; test_intel_evidence_keys
updated to assert the real provider key set
import decnet.cli as _decnet_cli at module level guarantees the app singleton is
built in master mode before any test can set DECNET_MODE=agent. Without this,
test_defence_in_depth_direct_call_fails_in_agent_mode triggered a fresh import
of decnet.cli with DECNET_MODE=agent active, which stripped master-only commands
and wrote the stripped module to sys.modules[decnet].cli — a parent-attribute
corruption that no sys.modules dict restore can fix.
- SSH schema: add user + user_password fields (service extended post-test)
- TopologySummary: repo.get_topology() returns model now, not raw dict
- health live: tarpit_watcher added to get_background_tasks(), add to expected set
Four-part fix for the collection bottleneck that was blocking the dev loop:
1. Lazy mitreattack.stix20 import in attack_stix.py — deferred to first
_load() call (TYPE_CHECKING guard at top level)
2. Lazy misp_stix_converter import in both MISP export routers — moved
from module level into the route handler body
3. Lazy attack_catalog / attack_stix in ttp.py repo mixin — thin wrapper
functions so the import chain never fires at module load time
4. tests/api/conftest.py — `from decnet.web.api import app` moved inside
the `client()` fixture; `pytest_ignore_collect` broadened to skip all
test_schemathesis*.py variants (not just test_schemathesis.py), which
were launching a subprocess server at module-import time
5. pyproject.toml — `norecursedirs` for tests/live, tests/stress,
tests/service_testing, tests/docker, tests/perf so these directories
are never entered; `-m` filter removed from addopts (now redundant);
`--dist loadscope` → `--dist load` to unblock workers immediately
6. behave_core / behave_shell rename — BEHAVE packages dropped the
`decnet_` prefix; reinstalled editable installs and updated all 14
import sites across profiler, ttp, bus, and correlation modules
host:port in remote_addr was creating a distinct Attacker row per TCP
connection instead of per IP. Split on the last ':' in parse_rfc5424;
preserve the port as fields['remote_port'] so repeated source ports are
retained as fingerprint signal in bounty payloads.
quic-go v0.59.0 (shipped with Caddy v2.11.2) removed quic.Connection as
a public interface and quic-go/logging as a public package, breaking
H3App's connection-wrapping approach.
Resolution:
- Remove H3App (h3app.go) entirely; Caddy handles h3 natively when h3
is in the protocols list.
- Rewrite h3conn.go to keep only tryParseH3ControlStream + varint/name
utilities (tested, useful for future stream-level tapping if the API
ever re-exposes it).
- FPHandler.ServeHTTP: for h3 requests, type-assert ResponseWriter to
http3.Settingser (the public interface exposed by quic-go/http3 v0.59),
read the peer's Settings after ReceivedSettings channel closes, emit
h3_settings fp record.
- https/entrypoint.sh: include h3 in CADDY_PROTOCOLS (Caddy now owns
UDP/443); remove DECNET_H3_GLOBAL block.
- Update go.mod/go.sum to caddy v2.11.2 + quic-go v0.59.0.
- Update test_https_compose_h3_app.py to expect h3 in protocols when
http/3 is selected, and assert decnet_h3 block is absent.
- All Go tests (9) and Python tests (15) remain green.
- Renames caddy.listeners.decnet_h2fp → decnet_fp; adds h1 raw-byte
header capture (plainTappingConn) and h2 continuous HPACK decode loop
(parseH2HeadersLoop) so headers_ordered reflects actual wire order, not
Go map iteration order.
- Adds H3App Caddy module (decnet_h3) that owns UDP/443 via quic-go,
wraps accepted QUIC connections with h3SettingsTappingConn to intercept
the h3 control stream and extract RFC 9114 SETTINGS in wire order.
- Wires access_log emission from FPHandler.ServeHTTP via responseCapture.
- Updates syslog_bridge.py (canonical + per-service copies) with inline
_compute_ja4h and new fp socket record branches: http_request_headers,
h3_settings, access_log.
- Fixes ingester proto field alias (bridge emits 'proto', ingester expected
'protocol') and exposes _process_fingerprint_bounties test alias.
- Go tests: h1/h2/h3 golden-byte tests all green; h3_tracer_test covers
varint parser, GREASE detection, truncated-stream safety.
- Python tests: 15/15 green across bridge JA4H hash parity, ingester
compat (old + new event shapes), and Caddyfile h3 template assertions.
[decky.https] relied on ini_loader prefix-matching to propagate config
to decky-03/04/05 — silent and fragile. Now emits [decky-03.https],
[decky-04.https], [decky-05.https] explicitly so the INI is self-evident
and doesn't depend on pattern matching side-effects.
networks.list() returns bare objects — Containers is always empty
without a reload(). The active-endpoint guard from the prior commit
never fired because it was checking a stale empty dict.
Docker refuses network removal (403) when containers hold endpoints.
The old IPAM-drift path tried to disconnect+remove even with live
containers — disconnect silently failed, remove raised APIError.
Since DECNET assigns IPs explicitly in compose (never via Docker's
auto-assign pool), an ip_range mismatch on an existing same-driver
network is harmless. Bail out early and attach to the existing network
whenever Containers is non-empty.
Swap Werkzeug for Caddy as the protocol layer for http and https decoy
services. Flask keeps owning app logic (fake_app, custom_body, headers,
syslog) on 127.0.0.1:8080; Caddy terminates h1/h2/h2c/h3 on the wire
with real-world TLS/QUIC fingerprints.
- Add `multi_enum` FieldType to ServiceConfigField + _coerce
- Add `http_versions` field to HTTPService (h1/h2c) and HTTPSService
(h1/h2/h3); selecting h3 emits UDP/443 port mapping in compose
- Rewrite both Dockerfiles with multi-stage Caddy binary copy +
setcap for port binding as the logrelay user
- Entrypoints parse HTTP_VERSIONS JSON, render a Caddyfile, start
Flask in background, wait for it, then exec Caddy
- https/server.py drops direct TLS handling; Caddy owns the cert
- Add ProxyFix to both server.py so Flask sees real attacker IPs
- Frontend: multi_enum checkbox-group renderer in ServiceConfigFields;
FormValue union extended to string[]; compactPayload skips []
- Fix stale test_smtp_relay_schema_matches_smtp: relay schema is a
superset of smtp, not equal; update assertions accordingly
Remaining files from the fingerprint-bounties + characterizes-SRO commit:
misp_export, repository, bounties mixin, all 4 router endpoints, and test suite
updates. Prerequisite: previous commit added _extract_fingerprint_bounty_data
and the stix_export changes.
Wire fingerprint bounties (JARM hashes, HTTP header quirks) from the bounties
table into the DecnetActorFingerprintExt.protocol_fingerprints group so the
sniffer/profiler-captured HTTP fingerprinting data surfaces in every STIX export.
Add a stix2.Relationship(relationship_type="characterizes") SRO linking each
x-decnet-behave-profile SDO back to its ThreatActor so graph-traversal tools
can follow the edge without relying on the bare x_decnet_behave_profile_ref
custom string property alone.
New repo surface:
- get_fingerprint_bounties_by_ip(ip) -> list[dict]
- get_all_fingerprint_bounties_for_export() -> dict[str, list[dict]]
All 4 export endpoints (per-attacker + fleet, STIX + MISP) extended with the
new gather slot. 50/50 tests green, mypy clean.
Adds GET /api/v1/attackers/{uuid}/export/misp and
GET /api/v1/attackers/export/misp backed by misp_export.py, which
converts existing STIX bundles to MISP events via misp-stix
ExternalSTIX2toMISPParser. Fleet endpoint emits {response:[...]}
collection (one event per attacker). Frontend: STIX/MISP buttons on
AttackerDetail header and Attackers list. 13 new tests green.
GET /api/v1/attackers/{uuid}/export/stix returns a self-contained STIX
2.1 bundle: ip observation, threat-actor, ATT&CK attack-patterns with
canonical MITRE IDs, uses relationships, per-tag sightings, file SCOs
for artifacts, domain-name SCOs for SMTP targets, and a provider intel
note. Attack-pattern SDOs carry the MITRE bundle IDs so consumers
deduplicating against the public ATT&CK bundle get exact matches.
Every technique_id in TechniqueBar and TTPInspector now links to its
canonical attack.mitre.org page. The inspector drawer gains a GROUPS
subpanel that lazy-fetches the new /ttp/techniques/{id}/groups endpoint
and renders each MITRE-tracked intrusion-set with deeplink and aliases.
Centralizes TTP row interfaces into src/types/ttp.ts and API wrappers
into src/utils/ttpApi.ts to give the new GroupRef type a clean home and
avoid a third inline fetch declaration.
Surfaces the intrusion-set reverse index from the loaded ATT&CK
bundle: given a technique, returns the list of groups MITRE has
documented as using it. Read-only — explicitly NOT an attribution
claim about a DECNET attacker. The frontend pulls this lazily when
the operator expands a technique panel; payload-size cost on every
TTPTagDetailRow makes embedding wasteful for techniques with 50+
documented groups.
- decnet/web/router/ttp/api_get_groups_for_technique.py exposes
GET /api/v1/ttp/techniques/{technique_id}/groups, response_model
list[GroupRef]. Same JWT-viewer auth gating as the rest of the
TTP router. 404 when the technique_id doesn't resolve in the
bundle.
- Sub-techniques are queried directly (no auto-union with parent)
to match ATT&CK Navigator semantics; callers that want a broader
view query the parent themselves.
- tests/ttp/test_groups_for_technique.py covers happy path, 404,
sub-technique attribution independence, empty-list-on-zero-groups,
and that responses include mitre_url + aliases.
- tests/web/test_api_attackers.py: fix pre-existing fixture drift
introduced by a2a61b63 — three TestGetAttackerDetail cases were
missing AsyncMock for repo.latest_observation_per_primitive,
causing TypeError on await of MagicMock. The new groups endpoint
doesn't share code with attacker_detail; this is a drive-by fix
surfaced by the same suite run.
Phase 2 attached mitre_url to intel-emitted tags' evidence JSON;
Phase 3 promotes it to a real column populated for *every* tag —
intel, credential, behavioral, canary, identity, email, rule-engine —
from one source. Pre-v1, so the SQLModel field is added directly
without an Alembic migration.
- TTPTag gains mitre_url: Optional[str] (not indexed — derived
deeplink, not a query target; technique_id is already indexed).
- _emit.py and rule_engine._evaluate_rules both populate mitre_url
via attack_stix.mitre_url_for(sub_technique_id or technique_id).
Sub-technique URL when present, else parent. The two construction
sites stay separate because the rule_engine path carries per-emit
span instrumentation that emit_tags() can't preserve without
threading a span object through; minimal-change beats forced
refactor here.
- intel_lifter strips mitre_url from evidence_extra in all four
decision functions. The column is canonical now; duplicating in
the JSON column would drift when the bundle moves. The unused
TechniqueEmission import + tracking dicts removed too.
- IdentityTechniqueRow / TechniqueRollupRow / TTPTagDetailRow /
CampaignTechniqueRow gain mitre_url: Optional[str].
- sqlmodel_repo/ttp.py:_mitre_url_for added; the 5 row-builder sites
pass mitre_url=_mitre_url_for(sub_technique_id or technique_id)
alongside the existing technique_name resolution.
- api_get_tag_details.py needs no change — list_tags_by_scope_and
_technique already returns model_dump() rows that flow the new
column through **row spread to TTPTagDetailRow.
- tests/ttp/test_emit_attaches_mitre_url.py covers both construction
paths (top-level, sub-tech, unknown, multi-emit) and a regression
test that intel_lifter evidence dicts no longer contain mitre_url.
Suite is now 51 files / 259 tests, 25.68% lines / 21.43% branches.
Floor: lines 24->25, functions 21->22, branches 19->21,
statements 23->24. Inspector/index.tsx ends at 172 LOC, the only
other > 250 LOC file in MazeNET/ is NodeInspector (362) — the
node branch was the bulk of the original 606 LOC and its 7
add-service / tarpit form states stay co-located there.
Inspector.tsx (606 LOC) splits into Inspector/{NetInspector,
NodeInspector, EdgeInspector, ServiceInspector, index}.tsx plus
types.ts. The dispatcher (index.tsx) owns the title bar, the empty
state, the activeNetIds derivation, the pending-diff block, and the
topology-status block; each per-type panel takes only the props it
needs. NodeInspector keeps the 7 useStates for the add-service /
tarpit forms since they are node-only.
10 new dispatcher-level tests cover empty / node / net / edge /
service / observed-entity / internet-net / live-ops gating /
tarpit-controls / pending-diff. Selection type re-exported from
Inspector/index.tsx so MazeNET.tsx, Canvas.tsx, and
useMazeContextMenu.tsx keep their existing import path.
Two reusable bundle-derived lookups that the next two commits build
on:
- mitre_url_for(tid) returns the canonical attack.mitre.org URL by
reading external_references on the cached attack-pattern. Backed
by the existing lru-cached _attack_pattern_by_id so per-call cost
is constant. Handles top-level techniques and sub-techniques
(T1059.004 -> .../techniques/T1059/004).
- GroupRef + groups_using_technique(tid) surface the intrusion-set
reverse index from the loaded bundle: given a technique, return
the MITRE-tracked groups documented as using it. Sorted by
group_id for deterministic responses; lru-cached. Sub-technique
semantics match ATT&CK Navigator (do NOT auto-union with parent).
- decnet/ttp/data/intel_loader._mitre_url_for collapses to a thin
re-export of attack_stix.mitre_url_for; the loader keeps mitre_url
on TechniqueEmission for the eventual STIX export.
- tests/ttp/test_attack_url.py covers both helpers: top-level + sub
URLs, unknown -> None / (), GroupRef immutability + hashability,
deterministic ordering, sub-technique distinct from parent.
Drop unused icon/api/useEffect/Tag imports left behind by the
fingerprint, behaviour, and IntelPanel extractions. AttackerDetail.tsx
ends at 450 LOC across Phase 10 (down from 1652 / 73% reduction).
Coverage floor: lines 23->24, functions 20->21, branches 17->19,
statements 22->23.
Move IntelPanel + IntelRow type + ProviderRow + VERDICT_TONE/fmtTs
helpers into AttackerDetail/IntelPanel/. AttackerDetail.tsx drops
from 680 to 449 LOC. New IntelPanel.test.tsx covers the loading,
absent (404), error (500), and ok states with MSW handlers.
Move BehaviouralPrimitivesPanel + 8 sub-components (BehaviorHeadline,
BeaconBlock, DetectedToolsBlock, TcpStackBlock, TimingStatsBlock,
PhaseSequenceBlock, AttributionBadge, KeyValueRow, StatBlock) plus
the OS_LABELS / BEHAVIOR_LABELS / TOOL_LABELS / BEHAVIOUR_DOMAIN_*
lookup tables and fmtOpt/fmtSecs into AttackerDetail/behaviour/.
AttackerDetail.tsx drops from 1220 to 680 LOC; existing
behaviour_panel test moves to behaviour/BehaviouralPrimitivesPanel.test.tsx
and now imports from the canonical location. The shell still
re-exports BehaviouralPrimitivesPanel for source compatibility.
Move 12 Fp* components, FingerprintGroup, getPayload, seqClassColor,
HashRow, fpType lookups, and UA color tables into
AttackerDetail/fingerprints/. AttackerDetail.tsx drops from 1652
to 1220 LOC; the orchestrator now imports the same helpers it used
to define inline. 10 new tests covering UA / HTTP-quirks / resumption
/ certificate / spoofed-source / TCP-stack / dispatch fallback.
The four provider→technique tables (AbuseIPDB cat→techniques,
GreyNoise tag→techniques, ThreatFox threat_type→techniques, plus
the Feodo binary-listed signal) used to live as Final[dict] constants
in intel_lifter.py. Two real problems with that:
1. Drift between rules/ttp/R0054.yaml..R0058.yaml (which declare
the full slate per provider) and the Python dicts (which decide
which slate-member fires per signal). The v2 audit comment in
intel_lifter.py documented that they had silently drifted.
2. No ATT&CK provenance on emissions — the loaded STIX bundle has
rich external_references (canonical attack.mitre.org URLs) that
never surfaced because the lifter had no path back to them.
Mappings now live as YAML at decnet/ttp/data/intel/{provider}.yaml,
validated at load against the loaded ATT&CK bundle, with each entry
enriched by attack_stix._attack_pattern_by_id to attach the canonical
MITRE URL to every emission.
- decnet/ttp/data/intel_loader.py: pydantic-validated schema +
ProviderMapping/Signal/TechniqueEmission frozen dataclasses +
load_provider_mapping(provider) lru-cached.
- Per-technique high_score_threshold inlined into YAML
(collapses the separate _ABUSEIPDB_HIGH_SCORE_GATED dict).
- external_reference field follows the STIX 2.1 external-reference
shape (source_name + url + optional external_id) so the future
STIX/MISP exporter is a direct translation.
- intel_lifter.py: dicts deleted, decision functions read from
ProviderMapping accessors. Decision-flow constants (T1071/T1595
bare-classification fallbacks in _greynoise_decisions) stay in
code — they're not table rows.
- Each emit slot's evidence_extra now carries mitre_url for any
technique resolved in the bundle (every one in practice).
- tests/ttp/test_intel_mappings.py: snapshot equivalence vs the
legacy dicts, high-score gate behavior, every-signal-has-an-
external-reference, every-emission-has-a-mitre-url, negative
paths (unknown technique_id raises AttackBundleError, mismatched
provider field rejected, dir listing matches expected providers).
The YAML schema + mitre_url enrichment lays groundwork for the
future STIX exporter; this commit does NOT build that exporter.
MITRE's ATT&CK Terms of Use require reproducing their copyright +
license alongside any cached copy of ATT&CK data. Today we ship the
bundle but not the license — this commit closes that compliance gap.
- attack_version.py pins ATTACK_LICENSE_URL +
ATTACK_LICENSE_SHA256 + ATTACK_LICENSE_FILENAME, sourced from the
same attack-stix-data repo as the bundle.
- attack_stix.py:_fetch_license downloads LICENSE.txt next to the
bundle. License sha mismatch is logged + refreshed (license text
gets occasional formatting tweaks; not a security event), unlike
the bundle which stays fail-closed.
- _ensure_license is the compliance ratchet: resolve_bundle_path
refuses to return without LICENSE.txt on disk. Override-mode
(DECNET_ATTACK_BUNDLE) checks for a sibling LICENSE.txt first,
then DECNET_ATTACK_LICENSE, then the cache dir.
- python -m decnet.ttp.attack_stix license prints the cached license
to stdout for operator audit.
- loaded_license_path() exposes the active license path read-only.
- tests/ttp/test_attack_license.py covers happy paths (sibling +
explicit env), refusal when DECNET_ATTACK_LICENSE points at a
missing file, the CLI subcommand, and the pinned-sha shape.
Credentials.tsx: 487 -> 231 LOC. Page now composes CredsTable +
ReuseTable + useCredentials hook; URL-derived state (tab, query,
service, page) and selection/sort UI are the only concerns left
in the shell.
SwarmHosts.tsx: 513 -> 161 LOC. Page now composes EnrollmentWizard
+ useSwarmHosts hook; only the arm/confirm UI affordance and the
busy-set tracking remain in the shell.
Webhooks.tsx: 642 -> 387 LOC. Page now composes FormRow + SecretModal
+ useWebhooks hook; toast policy is the only UI concern left in the
shell. Multi-select delete uses the hook's reload internally.
Drift between the technique/tactic IDs hardcoded in the lifters and
what the loaded ATT&CK STIX bundle actually contains is silent in the
status quo: a renamed-or-retired technique just stops being tagged.
Every emission point now has an explicit validator that asserts its
IDs resolve in the loaded bundle, called once at TTP-worker boot.
- intel_lifter.all_emitted_technique_ids() collects every technique
the four provider tables (AbuseIPDB / GreyNoise / Feodo / ThreatFox)
plus the decision-flow constants in _greynoise_decisions and
_feodo_decisions can emit. validate_against_attack_bundle() runs it
through attack_stix.assert_known_technique_ids().
- ukc.validate_against_attack_bundle() asserts every key in
ATTACK_TACTIC_TO_UKC resolves, with TA0100..TA0106 documented as
_NON_ENTERPRISE_TACTICS (lives in the ICS bundle, not the
enterprise bundle DECNET loads).
- decnet/ttp/worker.py:run_ttp_worker_loop calls both validators
before subscribing to the bus. A bundle-vs-code mismatch refuses
to start the worker rather than silently mistagging events.
- tests/ttp/test_attack_bundle_validation.py covers the happy path
for both validators, the negative path (injected bogus tactic ID
raises AttackBundleError), the ICS exemption, and the lone T1078
reference in credential_lifter.
Replace the hand-maintained TECHNIQUE_NAMES dict (pinned to v15.1) with
a runtime loader that reads the official enterprise-attack-N.json STIX
bundle. Version bumps now require only updating attack_version.py;
sub-technique parents, tactic IDs, and kill-chain phases all come from
MITRE's published data.
- decnet/ttp/attack_version.py pins version 19.0 + sha256 + URL
- decnet/ttp/attack_stix.py is the lazy STIX loader. Resolution order:
DECNET_ATTACK_BUNDLE env -> ~/.cache/decnet/attack/ -> fetch from
the pinned MITRE GitHub URL. SHA-256 verified before parse;
mismatch fails closed.
- decnet/ttp/attack_catalog.py collapses to a shim re-exporting
technique_name() so the ~9 router/repo call sites don't churn.
- python -m decnet.ttp.attack_stix fetch warms the cache and can
print sha256 for version-bump workflows.
- test_attack_catalog.py now asserts every rule-emitted ID resolves
in the loaded bundle (same contract, real source) and exercises
the SHA-256-mismatch fail-closed path.
PersonaGeneration.tsx: 875 -> 357 LOC. Page now composes the data
hook + PersonaCard + PersonaEditor; bulk-import helpers stay in
helpers.ts; toast policy is the only UI concern left in the shell.
Final integration step. The MazeNET page shell is now a thinner
composition of the existing module-level hooks (useMazeApi,
useMazeInteraction, useTopologyEditor, useTopologyStream,
useLayoutPersistor) PLUS the three new ones from this phase
(useFullscreenMode, useTopologyData, useMazeContextMenu).
- MazeNET.tsx: 980 -> 715 LOC. The fullscreen + body-class
effects, the topology hydrate / SSE stream / deploy /
flashErr plumbing, and the four context-menu builders are
all gone from the shell.
- Page still owns the per-operation editor callbacks
(removeNet/Node/Edge, duplicateNode, addServiceToNode, etc.)
because they need direct access to setNodes/setEdges/setNets
for optimistic patches alongside their REST calls — those
setters are exposed by useTopologyData for that reason.
Coverage floor bumped after the phase:
lines 17 -> 19
functions 15 -> 17
branches 13 -> 14
statements 16 -> 18
Phase 5 final scoreboard: 37 test files, 172 tests, all green.
Lift the context-menu builder out of the page shell. The hook
owns ctxMenu open/close state and exposes one builder per
surface (node / net / edge / canvas); the actual operations come
in via callbacks so the page keeps its optimistic-patch logic
unchanged.
- New MazeNET/useMazeContextMenu.tsx
- useMazeContextMenu.test.ts covers menu lifecycle (open/close),
node-menu items, observed-entity locking, internet-net
delete-disabled, canvas-menu Add subnet/DMZ items, and the
edge-menu Remove invocation.
- Wiring into MazeNET.tsx lands next.
Lift the canvas data plane off the page shell. The hook owns:
GET /topologies/:id (hydrates nets/nodes/edges + meta)
GET services + archetypes (catalogs, with bundled fallback)
POST /topologies/:id/deploy
/topologies/:id/events SSE (open only when active/degraded)
flashErr() banner timer (auto-clears actionErr after 4s)
State setters for nets / nodes / edges are returned so the
per-operation callbacks living in the page can optimistically
patch local state alongside their REST calls (matches the
existing pattern; wholesale lift would mean dragging every
mutation along too).
- New MazeNET/useTopologyData.ts
- useTopologyData.test.ts covers hydrate, loadErr surfacing,
streamEnabled gating on active/degraded, onDeploy success +
error paths, and the flashErr 4s auto-clear with fake timers.
- Wiring into MazeNET.tsx lands in the next commit.
Lift the four fullscreen-related side-effects off the page shell.
The hook owns:
1. body class toggle so page CSS can hide its chrome
2. browser fullscreen API request/exit (failures ignored)
3. fullscreenchange listener so F11/Esc from outside our button
keeps internal state in sync
4. Esc keystroke handler
Returns { fullscreen, setFullscreen, toggle }.
- New MazeNET/useFullscreenMode.ts
- useFullscreenMode.test.ts (jsdom) covers initial toggle, body
class lifecycle, Esc-to-exit, and unmount cleanup.
- MazeNET.tsx loses ~30 LOC of inline state + effects.
Final integration. The page shell is now a thin composition of
useConfig + the previously-extracted children:
- Config.tsx: 989 -> 131 LOC. Page owns only the activeTab state
(and the "drop the users tab if the server didn't send users"
effect). Every form lives inside its tab; toast wiring lives
in AppearanceTab; window.alert calls live inside UsersTab.
- Tabs receive their `onSave* / onAddUser / ...` callbacks
directly from the hook — no intermediate wrapper handlers.
Coverage floor bumped after the split:
lines 14 -> 17
functions 13 -> 15
branches 11 -> 13
statements 13 -> 16
Phase 4 final scoreboard: 34 test files, 156 tests, all green.
APPEARANCE panel — accent-color picker — into its own tab. State
is local since no other tab cares about the value; localStorage
persistence + the document.documentElement[data-accent] mirror
move along with it.
- New Config/tabs/AppearanceTab.tsx
- AppearanceTab.test.tsx covers the matrix default, reading the
saved accent from localStorage on mount, and the click-to-flip
flow writing both localStorage and the html data-accent attr.
GLOBAL VALUES panel + the developer-mode-gated DANGER ZONE
(reinit) into one tab file. Two stacked panels because they're
the two pieces of UX you ever see together on the globals tab;
splitting them into separate components would force the page
shell to re-pick the gating predicate.
- New Config/tabs/GlobalsTab.tsx (mutation-interval + DangerZone
inline, since DangerZone is reinit-specific and won't be reused)
- GlobalsTab.test.tsx covers interval-format validation, the
DANGER ZONE gating on developer_mode, the two-step reinit
confirm flow, the totals chip ("PURGED: N logs, N bounties,
N attacker profiles") on success, and viewer-mode rendering.
USER MANAGEMENT panel into its own tab. Owns the per-row UI
state (delete-confirm, reset-password popup) plus the add-user
form state; mutations come in via prop. Errors on per-row
operations stay on window.alert (matches existing behavior); the
add form uses the inline FormMsg chip.
- New Config/tabs/UsersTab.tsx
- UsersTab.test.tsx covers row rendering with the must-change
badge, the two-step delete confirm flow, the add-user submit
payload (trimmed username + selected role), and the success
chip after a successful add.
DEPLOYMENT LIMITS panel into its own tab file. Owns the input
state, preset-button shortcuts, and the inline FormMsg chip; the
hook mutation is passed in via prop so this component is fully
reusable as a presentation-only piece.
- New Config/tabs/LimitsTab.tsx
- LimitsTab.test.tsx covers viewer-vs-admin rendering, the
1-500 validation message, and success/error chip display.
Lift the GET /config fetch and every admin-side mutation off the
page shell:
GET /config
PUT /config/deployment-limit
PUT /config/global-mutation-interval
POST /config/users
DELETE /config/users/:uuid
PUT /config/users/:uuid/role
PUT /config/users/:uuid/reset-password
DELETE /config/reinit (returns { logs, bounties, attackers })
Mutations return { ok: true } | { ok: false; reason: string } so
the upcoming tab components can render the inline FormMsg chip
without touching axios error shapes. reinit additionally returns
the deletion totals so the danger-zone confirmation can echo
"PURGED: N logs, N bounties, N attackers".
- New Config/useConfig.ts
- useConfig.test.ts MSW-covers initial load, isAdmin role
surfacing, setDeploymentLimit ok + 400 paths, addUser, deleteUser
refused, and reinit success.
- Wiring into Config.tsx + tab extractions land in follow-up commits.
Verbatim move of the worker-status pollster (~390 LOC) plus its
RealismBadge sidekick into its own file. Owns its own polling +
stop/start/start-all mutations; toast push comes in via prop so
the parent stays the one source of toast tone.
- New Config/WorkersPanel.tsx
- WorkersPanel.test.tsx (MSW) covers worker-row rendering, the
BUS OFFLINE banner, and the error panel on /workers 500.
- Config.tsx loses the inline WorkersPanel + RealismBadge plus
the now-unused icon imports (Square, RefreshCw, Play).
Foundation for the Config split. UserEntry / ConfigData move out
of the page so the upcoming hook + tab extractions can import
without reaching back through Config.tsx. New ConfigTab union and
FormMsg type for the inline success/error chip pattern that
repeats across every admin form on the page.
- New Config/types.ts (UserEntry, ConfigData, ConfigTab, FormMsg)
- Config.tsx loses the inline interfaces and the `as any` cast on
setActiveTab in the tab-switcher.
Final integration step. The page shell is now a thin composition
of useCanaryTokens + the previously-extracted children:
- CanaryTokens.tsx: 1,334 -> 210 LOC. Page owns only the
pure-UI state (tab, search/state/scope filters, modal
visibility, drawer selection, local fileDrops log) and the
thin handlers that translate hook results into confirm/alert
prompts. Initial parallel fetch + deleteBlob mutation moved
to useCanaryTokens in the prior commit.
- Modals plug directly into the hook's optimistic helpers
(prependToken / prependBlob / markTokenRevoked) so the page
doesn't reach into the data shape.
Coverage floor bumped after the split:
lines 11 -> 14
functions 10 -> 13
branches 8 -> 11
statements 11 -> 13
Phase 3 final scoreboard: 28 test files, 131 tests, all green.
Lift the three tab bodies — tokens, blobs, file drops — into
their own files. Each takes plain props (data + the operations
its rows need), so the page shell stops mixing tab markup with
data plumbing.
- New CanaryTokens/TokenListView.tsx (text search + state/scope
filter selectors + flat row grid; visibleTokens memo lives here
now). Exports StateFilter / ScopeFilter union types so the page
can declare its filter useState with the right shape.
- New CanaryTokens/BlobListView.tsx (delete refused while a token
references a blob; ref count badge reuses the disabled button).
- New CanaryTokens/FileDropListView.tsx (CLEAR LIST hidden when
the local log is empty).
- Three companion tests cover empty states, filter behavior,
delete refused-vs-allowed, and the per-tab callback wiring.
Wiring into CanaryTokens.tsx + the hook lands next.
Lift the parallel initial-load fetch and the deleteBlob mutation
off the page shell. Modal-driven optimistic merges (created
token, uploaded blob, drawer-revoked token) flow through narrow
setter helpers so the modals don't have to know how state is
shaped internally.
GET /canary/tokens
GET /canary/blobs (silent 403 -> empty list, viewer-friendly)
GET /deckies
GET /topologies/?status=active
DELETE /canary/blobs/:uuid
deleteBlob returns { ok, reason } so the page can branch the
toast/alert tone without seeing the axios error type. Wiring
into CanaryTokens.tsx lands in the next commit.
- New CanaryTokens/useCanaryTokens.ts
- useCanaryTokens.test.ts MSW-covers happy load, viewer 403 ->
empty blobs, deleteBlob ok + refused-with-detail paths, and the
markTokenRevoked optimistic write.
Verbatim move of the file-drop modal (~310 LOC) and its
localStorage glue (FILEDROP_LS_KEY, FileDropEntry type,
loadFileDrops, saveFileDrops) into one file. The list view that
shows these entries lives in the page; the persistence layer
travels with the writer.
- New CanaryTokens/FileDropModal.tsx (modal + LS helpers + entry type)
- FileDropModal.test.tsx covers loadFileDrops empty / round-trip /
200-row cap / malformed-JSON, plus modal title rendering, the
bypass-warning banner, and CANCEL -> onClose.
- CanaryTokens.tsx loses the inline modal + LS glue plus the
now-unused imports (useRef/X/AlertTriangle/useEscapeKey/
useFocusTrap, plus BTN_PRIMARY/BTN_GHOST/Field that only the
modals consumed).
Verbatim move of the artifact upload modal (~130 LOC) into its
own file. Drop-or-browse picker, server-side-injection warning
banner, and the multipart POST stay unchanged.
- New CanaryTokens/UploadModal.tsx
- UploadModal.test.tsx covers title rendering, empty drop-zone
hint, server-injection warning banner, UPLOAD-disabled-until-
file, and CANCEL -> onClose.
Verbatim move of the canary-token creation modal (~280 LOC) into
its own file. Renamed from CreateModal to CreateTokenModal so the
component name carries scope across the package boundary.
- New CanaryTokens/CreateTokenModal.tsx
- CreateTokenModal.test.tsx covers title rendering, CANCEL ->
onClose, empty-deckies hint, and the Operator-upload mode
switch revealing the no-blobs message. useFocusTrap is
vi.mock'd to avoid jsdom focus shenanigans.
- CanaryTokens.tsx loses the inline modal + its now-unused
imports (KNOWN_GENERATORS, KIND_OPTIONS, GeneratorName).
Foundation for the CanaryTokens split. Types, error/format helpers,
and the inline style + small primitives move out of the page so
the upcoming modal/list extractions can import without reaching
back through CanaryTokens.tsx.
- New CanaryTokens/types.ts (BlobRow, DeckyOption, TopologyOption,
Scope, KNOWN_GENERATORS / GeneratorName, KIND_OPTIONS, STATE_COLOR)
- New CanaryTokens/helpers.ts (extractError, fmt, fmtBytes)
- New CanaryTokens/ui.tsx (INPUT_STYLE, BTN_PRIMARY, BTN_GHOST,
Field, Stat)
- CanaryTokens.tsx loses ~110 LOC of inline definitions; behavior
unchanged.
Phase 2 lands. DeckyFleet.tsx dropped from 1,674 to 274 LOC; the
fleet page is now a thin composition of useDeckyFleet + 6
extracted children (DeckyInspectPanel, IntervalEditor, DeckyCard,
DeployWizard, DeckyFilters, DeckyGridEmpty), each with co-located
tests.
Lock the gain by bumping the threshold floor in vite.config.ts:
lines 7 -> 11
functions 6 -> 10
branches 5 -> 8
statements 7 -> 11
Phase 2 final scoreboard: 21 test files, 98 tests, all green.
Final integration step. The page shell is now a thin composition
of the hook + the previously-extracted children:
- DeckyFleet.tsx: 1,674 -> 274 LOC. Page owns only the
pure-UI state (filter, search, armed-confirm, modal visibility,
selected-card-for-inspect) and the toast-wrapping handlers that
translate hook results into toast tone. Polling, REST plumbing,
role lookup, and archetype catalog all moved to useDeckyFleet
in the prior commit.
- New DeckyFilters.tsx (header pill row + DEPLOY shortcut) +
DeckyGridEmpty.tsx (fleet-empty vs. filter-empty copy).
- DeckyFilters.test.tsx + DeckyGridEmpty.test.tsx cover count
rendering, filter-click callbacks, and admin-gated DEPLOY
visibility.
Two-step teardown arming logic stays in the page (it's pure UI).
Toast tone branching on { ok, reason } from useDeckyFleet
results moves the policy decision out of the data layer.
Lift every read- and write-side data flow off the page shell:
GET /system/deployment-mode (decides which list endpoint to hit)
GET /deckies | /swarm/deckies (mode-switched + shape-normalized)
GET /config (role -> isAdmin)
GET /topologies/archetypes (live catalog with bundled fallback)
POST /deckies/:name/mutate
PUT /deckies/:name/mutate-interval
POST /swarm/hosts/:uuid/teardown
10s polling loop refreshing mode + list
Operations return discriminated results ({ok:true} | {ok:false,
reason:...}) so the page can branch toast tone without seeing the
axios error type. Toasts, arm-confirm, and modal visibility stay
in the consuming page — the hook is pure data.
- New DeckyFleet/useDeckyFleet.ts
- useDeckyFleet.test.ts MSW-covers initial load, swarm-mode shape
normalization, mutate ok/error paths, teardown ok path, and
applyServicesChange optimistic write.
- DeckyFleet.tsx wiring lands in the next commit so the diff stays
reviewable.
Lift the multi-step deploy wizard (~520 LOC) plus its private
INI-builder helpers (PLACEHOLDER_LINES, b64encodeUtf8, buildIni,
PickMode type) into their own file. Verbatim move; the
underscore-prefixed helpers drop the leading underscore now that
they're file-local rather than competing with hoisted parent
constants.
- New DeckyFleet/DeployWizard.tsx
- DeployWizard.test.tsx covers the closed render guard, the
open-at-step-0 archetype list, NEXT-disabled-until-archetype,
and CANCEL -> onClose. ServiceConfigFields is vi.mock'd to a
stub since it pulls schemas via api.get() that are out of
scope for these tests.
- DeckyFleet.tsx loses the wizard plus the now-unused imports
(DEFAULT_SERVICES, Modal, PickIcon, ServiceConfigFields and
its type aliases).
Lift the per-decky tile (~430 LOC) into its own file. Tarpit
controls, live add/remove service flow, and the per-service config
toggle stay inside the card — those are tile-local UI concerns and
only ever rendered from this component anyway.
- New DeckyFleet/DeckyCard.tsx
- DeckyCard.test.tsx covers identity row + services rendering,
admin-gated FORCE MUTATE visibility, the FORCE MUTATE callback,
TEARDOWN -> CONFIRM toggle when armed matches, and card-body
click firing onInspect. AddServiceConfigModal +
ServiceConfigForm are vi.mock'd so we don't need MSW handlers
for their unrelated network fetches.
- DeckyFleet.tsx loses the inline component plus the now-unused
imports it dragged in (Network/PowerOff/RefreshCw/Plus/X icons,
ServiceConfigForm, AddServiceConfigModal, useCallback).
Verbatim move of the per-decky mutation-interval modal (~60 LOC)
into its own file. Saves null when the toggle is off, minutes
otherwise.
- New DeckyFleet/IntervalEditor.tsx
- IntervalEditor.test.tsx covers null-current disabled path,
numeric-current enabled path, and CANCEL not firing onSave.
- src/test/fixtures/decky.ts now derives DeckyFixture from the
canonical Decky type (the fixture's loose swarm shape was
missing host_address/host_status; aligning to Decky catches
that statically).
Lift the right-side inspect drawer (~115 LOC) into its own file.
This is a verbatim move — same JSX, same useEscapeKey + body
overflow lock, same swarm-section gating. Underscore-prefixed
helper calls (_dotFor, _stateColor) drop the leading underscore
since they're now imported from helpers.tsx.
- New DeckyFleet/DeckyInspectPanel.tsx
- DeckyInspectPanel.test.tsx covers identity-row rendering, the
SERVICES chip list, the conditional SWARM block, and the close
button callback.
- DeckyFleet.tsx loses the panel + the now-unused useEscapeKey
import.
Foundation for the DeckyFleet split. Types and helpers move to
their own files so the upcoming subcomponent extractions can
import without reaching back through the parent module.
- New DeckyFleet/types.ts (Decky, SwarmDeckyRaw, SwarmMeta,
Archetype, FilterKey, DeckyStatus). Names exported to match the
pattern set by AttackerDetail/types.ts.
- New DeckyFleet/helpers.tsx (archetypeIcon, PickIcon, dotFor,
hitsFor, stateColor). Underscore-prefixed call sites stay via
import-rename so this commit changes zero behavior.
- DeckyFleet.tsx loses ~110 LOC of inline definitions plus the
now-unused icon imports (Cpu / Database / Globe / Monitor /
Shield / Terminal).
Phase 1 of the UI refactor is in. AttackerDetail dropped from
2,579 LOC inline data + JSX to a 408-LOC shell composed of
extracted sections, each with co-located tests. Lock the gain by
bumping the threshold floor in vite.config.ts:
lines 0 -> 7
functions 0 -> 6
branches 0 -> 5
statements 0 -> 7
Future PRs raise these; never lower. Phase 1 final scoreboard:
9 test files, 45 tests, all green.
Lift STORED MAIL into its own section and pull the mail drawer
selection state along with it. Section signals admin-gating
through the section's own props (mailForbidden), since the data
hook already converts a 403 into that boolean.
- New AttackerDetail/sections/MailLogPanel.tsx
- MailLogPanel.test.tsx covers row rendering, mailForbidden empty
state, no-mail empty state, from_hdr/from_addr/mail_from
fallback, and drawer open/close. MailDrawer vi.mock'd same as
ArtifactDrawer.
- AttackerDetail.tsx loses the mail JSX block, mailItem state,
and now-unused Mail/MailDrawer imports.
Lift CAPTURED ARTIFACTS into its own section, taking the drawer
selection state with it (the parent shell no longer owns
artifact-modal state).
- New AttackerDetail/sections/ArtifactsPanel.tsx
Drawer is rendered as a sibling of the section so its z-index
and focus-trap behavior mirror the original.
- ArtifactsPanel.test.tsx covers row rendering with parsed SD
fields, empty state, missing stored_as (no OPEN button), and
the open/close cycle. ArtifactDrawer is vi.mock'd to a stub
so we don't need MSW handlers for its content fetch.
- AttackerDetail.tsx loses the artifact JSX block, the artifact
state, and now-unused Paperclip/Package/ArtifactDrawer imports.
Lift the COMMANDS collapsible — paginated table with header-bar
prev/next controls — into its own section. The page math
(cmdTotalPages = ceil(total/limit)) and conditional empty state
both live in the section now.
- New AttackerDetail/sections/CommandsViewer.tsx
- CommandsViewer.test.tsx covers title formatting (unfiltered vs.
filtered), empty state, single-page pagination hiding, and
prev/next button behavior
- AttackerDetail.tsx loses the IIFE-wrapped commands JSX block
plus now-unused ChevronLeft/ChevronRight/Terminal imports
Lift the SERVICES TARGETED collapsible — interactive two-tone badge
chips with click-to-filter — into its own section. The selection
state was already lifted into useAttackerDetail in the prior
commits, so the section just consumes serviceFilter /
setServiceFilter as props.
- New AttackerDetail/sections/ServicesTargeted.tsx
- ServicesTargeted.test.tsx covers badge rendering, empty state,
inactive-click-sets-filter, and active-click-clears-filter
- AttackerFixture grows ip_leaks/ip_leaks_total fields so the
TimelineSection rotation test (added in the prior commit) keeps
passing under the new factory shape
Lift the TIMELINE collapsible (timestamps, ASN, reverse DNS,
leaked-IPs row with rotation detection) into its own section.
LeakedIPsRow + the rotation/inline-limit constants come along
since they were only ever used here.
Also moves the shared `Section` collapsible primitive into
AttackerDetail/ui.tsx so the remaining sections can adopt the
template without re-importing through the parent module.
- New AttackerDetail/sections/TimelineSection.tsx (LeakedIPsRow
inline as a private helper)
- AttackerDetail/ui.tsx now exports both Tag and Section
- AttackerDetail.tsx loses LeakedIPsRow, the Section helper, the
Timeline JSX block, and now-unused imports (ChevronUp, ChevronDown,
AttackerData)
- TimelineSection.test.tsx covers timestamps, unknown-origin path,
rotation badge, empty leaks, collapse, and toggle callback
Lift the 5-up counter grid + the conditional scan-vs-interact row
into AttackerStats. The activity row's visibility predicate
collapses into a single boolean inside the section so the parent
no longer encodes UX rules.
- New AttackerDetail/sections/AttackerStats.tsx
- AttackerStats.test.tsx covers all-five counters, activity present,
activity empty, and service_activity undefined paths.
Lift the header (IP, country tag, traversal badge, identity badge)
into its own section component. Tag helper moves to a shared
AttackerDetail/ui.tsx so future sections can reuse it without
re-importing through AttackerDetail.tsx.
- New AttackerDetail/sections/AttackerHeader.tsx (~50 LOC)
- New AttackerDetail/ui.tsx for shared presentational helpers
- AttackerDetail.tsx imports both; local Tag definition deleted
- AttackerHeader.test.tsx covers country present/absent,
TRAVERSAL badge, IDENTITY click-through, identity null path
The AttackerDetail page body owned all 7 REST fetches plus 2 SSE
streams inline as 200+ lines of useEffect plumbing. Lift them into
a single hook so section components extracted in follow-up commits
consume typed values, not setState pairs.
- New ./AttackerDetail/types.ts holds the canonical AttackerData,
BehaviouralObservation, AttributionPrimitiveState plus newly-named
ArtifactLog / SessionLog / SmtpTargetRow / MailLog / CommandRow
(previously inline anonymous types).
- New ./AttackerDetail/useAttackerDetail.ts owns:
* GET /attackers/:id (404 -> ATTACKER NOT FOUND)
* GET /attackers/:id/attribution (silent-tolerant)
* GET /attackers/:id/commands paged with 422 alert preserved
* GET /attackers/:id/{artifacts,smtp-targets,mail,transcripts}
(mail surfaces a 403 boolean for the admin-gated viewer)
* useAttackerStream + useIdentityStream subscriptions, including
the live attribution-state-changed merge.
- AttackerDetail.tsx re-exports BehaviouralObservation /
AttributionPrimitiveState so AttackerDetail.behaviour_panel.test
and any future external importer keeps working unchanged.
- New useAttackerDetail.test.ts covers loading -> success, 404,
paged commands offset, serviceFilter resets cmdPage, and mail 403
via MSW handlers (the SSE hooks are vi.mock'd; jsdom can't host
EventSource).
No behavior change for the rendered page; all 37 tests green.
Phase 0 of the decnet_web refactor: stand up an MSW server, fixtures,
and a router-aware render helper so the upcoming god-component splits
(AttackerDetail first) can land with same-commit test coverage.
- msw devDep + setupServer wired into src/test/setup.ts
- src/test/server.ts re-exports server, http, HttpResponse, apiUrl()
- src/test/fixtures/{attacker,decky,canary,topology}.ts factories
- src/test/renderWithRouter.tsx wraps MemoryRouter + ToastProvider
- baseline coverage thresholds (0%) in vite.config.ts; raise per PR
- coverage/ added to decnet_web/.gitignore
Existing Orchestrator/AttackerDetail/ThemeLab tests stay on vi.mock
and continue to pass; new tests use MSW.
The global button:hover rule in index.css forces color: var(--bg)
+ matrix-glow on the lucide icon's currentColor stroke, making
the sun/moon icon disappear into the toggle button's tinted
background on hover. Pin color: var(--accent) and box-shadow:
none on .theme-toggle-btn:hover so the icon stays in its base
colour and the button doesn't pick up the wider button-hover
halo.
Even with fill: 'both', the new pseudo paints once at its default
style (no clip-path = full size) before the JS animation
registers — the brief open flash that survived the previous fix.
Pre-publish click coords as --reveal-x / --reveal-y on <html>
before calling startViewTransition. The static CSS rule on
::view-transition-new(root) now sets clip-path: circle(0px at
var(--reveal-x) var(--reveal-y)) as the pseudo's default, so
the very first paint is already fully clipped. The animation
then grows the circle outward from there.
ANTI prefers the new theme growing outward from the click point
(visually clearer cause-and-effect than the old theme burning
away). The original outward implementation flashed at the start
because the new pseudo defaulted to its computed style (no
clip-path = fully visible) for one frame before the JS animation
registered.
Switching the animation's fill from 'forwards' to 'both' enforces
the start keyframe (circle(0) at click point) before the first
paint, in addition to pinning the end keyframe through pseudo
teardown. New layer is invisible until the animation begins,
fully visible until cleanup. No flash either end.
Without fill: 'forwards' the clip-path keyframes release at
animation end and the pseudo reverts to its computed style
(no clip-path), so the old layer flashes back at full size for
a frame before View Transitions tears the pseudo-elements down.
Pinning the final keyframe with fill-forwards keeps the old
layer fully clipped through to teardown.
Growing the NEW theme layer from circle(0) outward leaves a
one-frame gap where the new pseudo is fully opaque at full size
(the default state) before the clip-path animation registers.
Result: a flash of the destination theme right before the
reveal starts.
Inverted the layering and animation direction:
- NEW theme snapshot sits on the bottom (z-index 0), static
- OLD theme snapshot sits on top (z-index 1), shrinks via
clip-path from circle(N) at click point down to circle(0)
The new layer is now hidden behind the old one until the old
shrinks away — no flash possible because the new layer was
never visible before the animation. Same 520ms duration, same
ease curve, same direction-of-travel from the user's POV
(circle expanding from cursor).
User-facing theme toggle ships now that the design system has
been audited end-to-end. A Sun/Moon button lives between the
threat indicator and the SYSTEM status pill in the topbar — same
slim 28x28 voice as the rest of the topbar controls, no chrome
shouting at the user.
Click coords drive a View Transitions API circle clip-path that
grows from the cursor to the farthest viewport corner over 520ms
with the project's standard --ease curve. Browsers without
startViewTransition (older Firefox, Safari < 18) fall through to
an unanimated swap — the hook returns instantly in that case.
Persistence is two-tier:
- localStorage decnet_theme — the user's saved preference, the
thing the topbar toggle writes. Survives reloads, applies
everywhere.
- sessionStorage decnet_theme_lab — dev-mode lab override (Task
3). Tab-scoped, wins on boot so devs can A/B without nuking
the saved preference.
App.tsx hydrates both on first mount in the right order so the
correct theme is on <html> before the first paint.
useThemeToggle is a small hook in lib/ rather than a Layout-only
helper so the same toggle can be reused later from a settings page
or hotkey.
--dim-color and --danger-color were referenced across drawers and
RemoteUpdates but never defined; --dim-color silently inherited
(defeating its purpose) and --danger-color fell back to literal
#f88 salmon (the 'ugly red' WifiOff icon next to UNREACHABLE
hosts). Added both as aliases in :root: --dim-color = var(--fg-3),
--danger-color = var(--alert).
--fg-2/3/4 alphas in light mode were tuned identical to dark
(0.78/0.55/0.35), but ink-on-cream needs more punch than
matrix-on-black at the same alpha — the deploy preview code
block (.code-block .comment / .key) and every dim caption
rendered too faint. Bumped to 0.88/0.70/0.50.
.maze-net-box.inactive applies opacity 0.42 + grayscale(0.7) for
the 'no traffic' signal. On cream that fades the LAN out of
visibility entirely. Override in light mode keeps the dotted
border as the dim-state cue and bumps opacity to 0.85 so the
header text stays legible.
Credentials drawer code-block labels (printable:, b64:) and a
dozen other violet wash/tint sites still carried bare rgba(238,
130, 238, *) literals — bright magenta in light mode where
--violet has resolved to charcoal-purple #2d1b4e. Mirrors the
prior matrix/alert/warn/info sweeps: by-alpha buckets land on
var(--violet-tint-10) or var(--violet).
Pre-this-commit, ~80 rgba() literals across 24 files were
hardcoding alert-red, warn-amber, info-cyan, panel-dark, and
white-text-with-alpha shades that bypassed the token cascade.
Net effect in light mode: the .eml/SESSREC drawers, AttackerDetail
verdict pills, MazeNET net-box headers, OPEN/REPLAY action
buttons, threat-intel cards, and all the dim 'whitish' overlays
stayed on their dark-mode hex values, producing the unreadable
panels in the screenshots.
Sweep maps each rgba colour family onto the existing token by
alpha bucket — rgba(13,17,23,*) -> var(--panel),
rgba(255,65,65,*) -> var(--alert)/-tint-10,
rgba(255,170,0,*) and rgba(224,160,64,*) -> var(--warn)/-tint-10,
rgba(0,200,255,*) -> var(--info)/-tint-10,
rgba(255,255,255,*) -> var(--fg-N)/var(--matrix-tint-N) by alpha.
VERDICT_TONE in AttackerDetail (MALICIOUS/SUSPICIOUS/BENIGN/
NO SIGNAL) was the worst offender — string literals
'#ff4d4d'/'#ffae42'/'#5fd07a'/rgba(255,255,255,0.4) baked into
inline JS styles. Now resolves at render time via var(--alert)/
var(--warn)/var(--ok)/var(--fg-4).
New tokens in :root:
- --bg-color (alias of --bg) — drawers used this name with
#0d1117 fallback that fired in every browser because nothing
defined --bg-color. Adding the alias makes drawers re-tone.
- --info / --info-tint-10 / --info-tint-30 — REPLAY buttons and
any future neutral-secondary use.
- --ok — semantic alias for 'verified good' (matrix in dark,
emerald in light) so BENIGN pills stay readable across themes.
Login.css left intentionally — pre-auth surface, not themed.
Sweeps four invariant violations that were leaking dark surfaces
into light mode and producing the unreadable / inverted areas:
1. Hardcoded `color: #000` in 14 :hover rules across 11 CSS
files swapped to `color: var(--bg)` — collapses to #000 in
dark mode (no-op), becomes cream in light. Fixes DEPLOY
DECKIES (button hover was rendering charcoal-purple text on
charcoal-purple background).
2. Hardcoded `background: #000` (3 sites) and `#0d1117`
(3 sites) replaced with `var(--bg)` / `var(--panel)`. Fixes
code blocks and modal panels staying dark on cream — the
deploy-wizard preview, topology-creation NAME input, and the
MazeNET canvas backdrop now follow the active theme.
3. `rgba(0,0,0,0.35)` and `rgba(0,0,0,0.5)` input/card
backgrounds (ServiceConfigForm, DeckyFleet .input)
swapped to `var(--panel)`. Fixes per-service config rows
in the deploy wizard rendering as dark slabs.
4. SVG arrow markers in MazeNET Canvas.tsx hardcoded
`fill="#00ff41"` / "#ee82ee" — replaced with currentColor +
style hook so they re-resolve on theme change.
New behaviour: light-mode hovers tint instead of inverting. The
dark-mode rules fully fill bg with --matrix/--violet/--alert and
flip text to --bg; that lands cream-on-near-ink in light mode
and reads as a jarring colour inversion every cursor move. Light
mode now layers a *-tint-10 background and keeps text in its
base colour. Single override block in index.css targets every
scoped `.X-btn`/`.btn`/`button:hover` via :is() + [class*="-btn"]
so we don't have to chase every component file.
37 bare rgba(0, 255, 65, ...) literals across 10 component CSS
files were forcing matrix-green to bleed into light mode no matter
what data-theme=light overrode in :root. They're now mapped onto
existing tokens by alpha bucket (0.025-0.05 -> --matrix-tint-5,
0.08-0.10 -> --matrix-tint-10, 0.18-0.30 -> --matrix-tint-30,
0.4 -> --fg-4, 0.5-0.6 -> --fg-3, 0.7-0.8 -> --fg-2).
Adds --warn (#e0a040), --amber (alias of --warn), --crit
(#e74c3c), and their tint-10 variants to :root, with
ink-friendly light-mode overrides. Sweeps bare #ffaa00 / #e0a040
/ #f59e0b / #ff4d4d / #e74c3c usages in the same files onto the
new tokens.
Files with var(--token, #fallback) patterns left alone — those
were already token-driven and the fallbacks just provide safety.
Login.css and inline TSX hex left for the per-page sweep.
Initial light-theme palette kept --matrix as a darker emerald
and --violet as a darker purple, which washed out badly on
warm cream — auth-helper chips, ACTIVE/PASSIVE/INACTIVE pills,
and CREDS/REUSE tabs all became unreadable because their tint
backgrounds + low-saturation text collapsed to sludge.
Light mode now collapses --matrix and --violet to near-ink
shades (#0d0d0d and #2d1b4e). --alert stays the one
saturated colour — the only element allowed to shout.
Dark mode is untouched; the matrix-vibe identity stays
exclusive to dark.
Also collapses the matrix/violet accent knob in light mode:
data-accent only flavours dark mode now, since two ink
shades are visually identical.
Adds html[data-theme="light"] block to index.css overriding the
core six tokens (bg, matrix, violet, panel, border, alert), the
matrix/violet/alert tints, and the foreground opacity ramp to a
cream-on-ink palette anchored on #dbdad6. Glows are no-op'd —
light mode trades neon haloes for hard 1px borders.
Lab page gets a Dark/Light toggle that flips
html.dataset.theme and persists to sessionStorage
(decnet_theme_lab) — intentionally tab-scoped, not user-facing.
App.tsx hydrates the same key on boot so a tab reload keeps the
dev's chosen theme. The user-facing localStorage toggle ships
later via Config.
Renders every primitive in the design system on the lab page so
theme-token edits can be evaluated against all states at once:
colour swatches with WCAG contrast vs --bg, the full type scale,
buttons (5 variants × default/hover/disabled), badges and status
pills, info/error banners, metric cards, table rows
(default/hover/selected/drop-target), form inputs, drawer panel
sample, and net-box compose states (internet/inactive/selected/
drop-target — independent classes layering, per memory).
Wrapper uses .fleet-root so global .btn/.btn.violet/etc resolve
identically to real pages. Lab-local CSS owns layout only — every
colour comes from index.css tokens.
Adds VITE_DECNET_DEVELOPER build-time gate: when unset, the
isDeveloperMode() helper collapses to a constant false and Vite
tree-shakes both the lazy import and the conditional <Route> out
of the prod bundle.
ThemeLab is currently a header stub; subsequent tasks fill it
with the design-system primitive zoo plus a Dark/Light toggle
for live token tuning. Route is intentionally absent from
ROUTE_LABELS / sidebar — direct URL only.
_detect_prompt_suffix accepted ANY line ending in $#%> as a PS1 prompt,
so a single `cat /var/log/dpkg.log` (195 lines closing in `<none>`)
flooded environmental.shell_type votes and flipped a plainly-bash
session to fish.
A prompt line now requires either a trailing space after the suffix
(default PS1 shape across bash/zsh/fish/PowerShell) or a PS1-shape
token (user@host, "PS " prefix, or a Windows drive-letter prefix).
Regression tests pin the dpkg.log false-positive and a $-terminated
prose line.
* decnet attribution — Typer command mirroring decnet reuse-correlate
(--multi-actor-tick, --daemon flags). Calls run_attribution_loop
with the dependency-injected repo.
* deploy/decnet-attribution.service.j2 — systemd unit mirroring
decnet-reuse-correlator.service.j2: ExecStart=decnet attribution,
same hardening posture (NoNewPrivileges, ProtectSystem=full,
ProtectHome=read-only, dedicated /var/log/decnet/decnet.attribution.log).
* worker_registry.KNOWN_WORKERS += "attribution" — heartbeat already
publishes as system.attribution.health from
attribution_worker._WORKER_NAME, so the Workers panel surfaces the
row the moment the unit is enabled.
* api_start_all_workers preferred-order list + "attribution" between
reuse-correlator and enrich so a fresh start-all brings it up
alongside its peers.
After this commit `systemctl enable --now decnet-attribution` (or
the dashboard's start-all) actually launches the engine.
Per-primitive state badge rendered next to each value in the
Behavioural Primitives panel. Five-state vocabulary, frozen, mirrors
decnet/correlation/attribution/aggregate.py:
* STABLE — green, low-key
* DRIFTING — amber, draws the eye
* CONFLICTED — red
* MULTI-ACTOR — purple, loudest (cross-primitive escalation lives
in attribution.multi_actor_suspected, not the
per-primitive badge)
* UNKNOWN — neutral border, no fill
Wiring:
* GET /api/v1/attackers/{id}/attribution on mount + on id change.
Failures swallowed silently (the worker may be off in dev).
* useAttackerStream gains attribution.state_changed +
attribution.multi_actor_suspected named events. The state-changed
handler merges by primitive and locks last_change_ts when the
state did not actually flip (defensive — backend already gates
these on transition, but a future relaxation shouldn't lie about
"stable since X" on the badge tooltip).
* multi_actor_suspected is wired but unused by the badges; the
per-primitive multi_actor signal already shows on each contributing
primitive. The handler is in place so a future "two operators
detected" banner has a live source.
Vitest: 4 new tests (badge renders only for mapped primitives, all
five states render with distinct labels, no badge when prop omitted)
on top of the existing 4. 7 of 7 pass; tsc + vite build clean.
Four synthetic operator-behaviour scenarios at the merger level
(aggregate_observations) that pin v0's calibration:
* Stable HUMAN over 7 sessions -> all primitives stable
* HUMAN switches to LLM mid-week -> primitives flip stable -> drifting
* Two operators alternating -> primitives flag multi_actor
(per-primitive; the cross-
primitive multi_actor_suspected
correlator is exercised by Phase 5)
* Single short session -> all primitives unknown
Plus a threshold-lockdown test that asserts every named constant in
_thresholds.py against its v0 ship value. Anyone adjusting a
threshold without updating the scenarios fails this file.
This closes DEBT-051 at v0 — the attribution engine has a calibrated,
test-locked answer to "is this attacker stable / drifting / showing
multiple operators?" without crossing the persona-attribution bright
line. v1 (cross-attacker clustering, KD simhash linkage signal) is
gated on this v0 surface being stable in production for >= 1 month.
GET /api/v1/attackers/{uuid}/attribution
Returns the merger output for an attacker's identity:
{
"identity_uuid": "abc..." | null,
"primitives": [
{primitive, current_value, state, confidence,
observation_count, last_change_ts, last_observation_ts},
...
]
}
Pre-attribution-worker: identity_uuid=null, primitives=[]. Surfacing
identity_uuid keeps the cross-attacker rollup story visible to the
frontend ahead of v1's clusterer landing.
api_events SSE relay also subscribes to attribution.> and forwards
to the AttackerDetail page filtered on payload.identity_uuid (the
identity is resolved at stream open from the URL's attacker_uuid;
attribution payloads are identity-keyed, not attacker-keyed). New
SSE event names: attribution.state_changed,
attribution.multi_actor_suspected.
Frontend (AttackerDetail.tsx badge rendering, useAttackerStream
consumer) deferred — there's already WIP on AttackerDetail.tsx in
the working tree; merging the badge logic is a separate commit
once that lands.
Tests: 4 endpoint scenarios — 401 unauth, 404 unknown attacker,
200 empty (no stub), 200 with primitive-ordered rows.
Add tick_multi_actor() — periodic walk of attribution_state firing
attribution.profile.multi_actor_suspected when an identity carries
>= MULTI_ACTOR_MIN_PRIMITIVES rows in multi_actor state.
* Repo's list_multi_actor_identities() already filters to >= 2
primitives; the correlator just dispatches.
* In-memory dedup keyed on identity_uuid -> frozenset(primitives):
same set as last fire -> no re-emit. Set grows -> re-emit.
Set shrinks below threshold -> evict so a future re-flap re-fires.
Restart-resets are honest because attribution_state persists; a
v1 multi_actor_suspect_log table can replace this if needed.
* run_attribution_loop() now supervises three concurrent tasks:
observation handler, multi_actor tick loop, health/control. Tick
interval comes from _thresholds.MULTI_ACTOR_TICK_SECS (60s) with
test override.
Tests: 6 scenarios — single-primitive doesn't fire, two-primitive
co-flag fires, dedup blocks unchanged set, set growth re-fires,
threshold drop re-arms, multiple identities fire independently.
attribution_worker.handle_observation_event now executes the full
end-to-end path:
* ensure stub identity (Phase 1)
* observations_for_identity_primitive() — new repo helper joining
observations through attackers.identity_id, so v1's clusterer
gets cross-attacker rollup for free
* aggregate_observations() with ValueKind dispatched off the BEHAVE
PRIMITIVE_REGISTRY; unknown primitives default to categorical
* upsert_attribution_state() — last_change_ts locked when state is
unchanged so the dashboard can render "stable since X"
* publish attribution.profile.state_changed only on transition;
idempotent re-runs over the same observation set fire nothing
(loop-prevention invariant matching ttp.tagged)
Tests:
* 5 end-to-end attribution scenarios over in-memory SQLite + FakeBus.
* test_base_repo's DummyRepo + coverage body now stub every abstract
surface BaseRepository declares — the 6 added by this branch plus
the 12 left un-stubbed by earlier work (BEHAVE Phase 1, TTP
rollups, iter helpers). The coverage test could not previously
even instantiate.
* test_aggregate_categorical's dispatcher rejection updated for the
Phase 3 + 4 contract — ValueError on unknown kinds, not
NotImplementedError.
aggregate_numeric(): EWMA + dispersion (CV) over numeric primitive
values. Stable when CV < 20% AND mean shift < 30%; drifting on >= 30%
mean shift; conflicted on CV > 100%. Confidence is 1 - min(CV, 1).
multi_actor is intentionally NOT a numeric state — bimodal
distributions belong to the categorical detector once the value space
is bucketed.
aggregate_hash(): counts distinct hash values within
HASH_DRIFT_WINDOW_SECS of the most recent observation. 0 rotations =
stable, 1..HASH_DRIFT_MAX = drifting, > HASH_DRIFT_MAX = conflicted.
Reads rotation events; never recomputes hashes (DEBT-032 already
produces them via decnet.correlation.fingerprint_rotation).
aggregate_observations() dispatcher now routes "categorical" |
"numeric" | "hash" | None and rejects unknown kinds with ValueError
(louder than NotImplementedError now that all three v0 mergers
exist). 17 synthetic-input tests cover both new mergers and the
dispatcher.
aggregate_categorical(): pure function over a per-(identity, primitive)
observation list. Five-state vocabulary, last-N=5 window comparison
with one-outlier-tolerant majority threshold:
* unknown — < 3 observations
* stable — recent 5 agree (≥ 4 of 5 share top value), older 5 same
* drifting — recent 5 stable but disagrees with older 5, or older
was conflicted and recent stabilised
* conflicted — recent 5 split, no two-value alternation pattern
* multi_actor — recent 5 split + alternation between exactly two
values (operator A↔B handoff). Confidence capped at 0.6 per
_thresholds.MULTI_ACTOR_MAX_CONFIDENCE; flapping primitives on
flaky networks would otherwise look like two operators.
aggregate_observations() dispatcher honours value_kind="categorical"
(or None) and raises NotImplementedError for "numeric" / "hash" so
Phase 3 lands cleanly. 14 synthetic-input tests cover every state
+ boundary condition.
v0 Phase 1 of ATTRIBUTION-ENGINE.md:
* AttributionStateRow SQLModel keyed on (identity_uuid, primitive)
per ANTI direction — re-keying state rows when the v1 clusterer
merges attackers is the migration debt v0 should not bake in.
ATTRIBUTION-ENGINE.md updated with the deviation note.
* AttributionMixin: ensure_stub_identity_for_attacker, idempotent
upsert_attribution_state, get_attribution_state[_for_identity],
list_multi_actor_identities (the Phase 5 correlator's read).
* attribution.profile.{state_changed,multi_actor_suspected} bus
topics + builder; wiki Service-Bus.md updated separately.
* attribution_worker.py: subscribes to attacker.observation.>,
ensures stub identity per event, logs and continues. No merger,
no state writes, no derived events — Phase 4 wires those.
* attribution/{aggregate.py,_thresholds.py} skeletons: Phase 2
fills _aggregate_categorical, Phase 3 adds numeric+hash+dispatcher.
Real-world bug surfaced on the first live decky run: sessrec.c's
json_escape (decnet/templates/_shared/sessrec/sessrec.c:111-141)
only escapes bytes < 0x20 + DEL — bytes >= 0x80 pass through raw.
An attacker pasting Latin-1 / GB18030 / any non-UTF-8 8-bit text
yields a shard line that chokes Python's default UTF-8 text-mode
read with 'utf-8 codec can't decode byte 0xac'.
Three changes:
1. _events_for_sid now opens with errors='surrogateescape', preserving
byte fidelity through the JSON parse. Surrogate-half chars
correctly fail isascii() / isalpha() so the typed-letter
histograms filter them out automatically. Tightening sessrec.c to
escape >= 0x80 is filed for v0.2 — that's the proper forensic-data
fix; the surrogateescape read makes the engine robust meanwhile.
2. Regression test
(test_handler_tolerates_non_utf8_bytes_in_shard) builds a shard
with raw 0xAC bytes inside a JSON 'data' string and asserts the
handler still persists observations.
3. Collector's _emit_session now logs at WARNING (was DEBUG) when
find_shard_with_sid returns None, citing the three usual causes
(ARTIFACTS_ROOT perms, _SERVICE_RE whitelist, sessrec/collector
race). Surfaces the silent-skip class of bug in seconds instead of
hours — the first live run hid a perm mismatch
(User=anti without SupplementaryGroups=decnet) for an entire
session window before the symptom was traced upstream.
Two-half deliverable per BEHAVE-INTEGRATION.md §587-594:
* scripts/behave_shell/replay_calibration.py — Python helper that
drives the production handler against one asciinema shard, mints
a temp SQLite repo + an Attacker per session, captures bus
emissions in-process. Exits non-zero on zero-observation sessions.
* scripts/behave_shell/smoke.sh — bash entry that replays all five
2026-05-02 calibration shards (HUMAN / YOU-sim / LW-sim /
CLAUDE-FF / CLAUDE-CL). Auto-activates .311 venv, forces
DECNET_DB_TYPE=sqlite, prints per-class summary. Suitable for CI.
* scripts/behave_shell/README.md — runbook covering both halves.
Pins the manual live-decky procedure (one SSH session per class
against a deployed smoke-decky, expected dominant primitives table,
SQL verification query, AttackerDetail panel check, pass criteria).
* BEHAVE-INTEGRATION.md — Phase 6 completion log appended with
current corpus results table (15 sessions, 424 observations across
the five classes) and a note that the v0 tag (drop -pre) is gated
on the manual live-decky round-trip and lands as a separate
commit.
Live-decky run is intentionally NOT scripted — the integration doc
calls for manual SSH sessions per class so an operator confirms the
bus / collector / disk-reach plumbing under real PTY conditions.
Four tests pin the panel surface:
* Empty-state placeholder renders when no observations.
* Day-one priority primitives sort to the top of their group:
motor.input_modality first in motor; the three cognitive priority
primitives in documented order at the top of cognitive.
* Each row renders primitive leaf, value, and confidence-percent
badge.
* Groups follow the canonical domain order
(motor / cognitive / temporal / operational / environmental /
emotional_valence); unknown domains alphabetise at the end.
Mirrors the Orchestrator.test.tsx harness shape (DEBT-043). Live
update path (useAttackerStream → setObservations) is exercised
indirectly via the static render — the hook is dumb glue and the
state mutation is React-side.
Adds the AttackerDetail.tsx panel that surfaces BEHAVE-SHELL
behavioural primitives. Hydrates from the existing
GET /api/v1/attackers/{uuid} response field 'observations',
live-updates via the new useAttackerStream hook (replace-by-primitive
on every 'observation' SSE event).
* New BehaviouralPrimitivesPanel component, exported for vitest.
* Day-one render priority per BEHAVE-INTEGRATION.md §441-454:
motor.input_modality, cognitive.feedback_loop_engagement,
cognitive.command_branch_diversity,
cognitive.inter_command_latency_class — these four sort to the top
of their respective groups; everything else alphabetises.
* Grouped by top-level domain (motor / cognitive / temporal /
operational / environmental / emotional_valence) with the canonical
domain order; unknown domains alphabetise at the end.
* AttackerData interface gains an 'observations' field.
* Empty-state placeholder when the panel has nothing yet.
* Section collapse state extends to 'behavioural', defaults open.
tsc --noEmit clean. Vitest coverage ships in P5.4.
Per-attacker SSE consumer hook. Mirrors useIdentityStream's shape:
* Connects to /api/v1/attackers/{uuid}/events with ?token= auth.
* Per-event-name dispatch via addEventListener for snapshot,
observation, fingerprint.rotated, attacker.scored.
* Reconnect-on-error backoff (3s).
* Callback refs so consumer rerenders don't tear down the connection.
The 'observation' event handler receives every primitive's update
through one event name; the primitive rides in payload.primitive
(matches the backend's _sse_name_for collapse decision).
Hook coverage rides on P5.4's panel test.
GET /api/v1/attackers/{uuid}/events streams behavioural events for
one attacker. Mirrors decnet/web/router/topology/api_events.py
end-to-end: ?token= auth, require_stream_viewer gate,
sse_connection_slot per-user cap, snapshot-on-connect, three bus
subscriptions (attacker.observation.>, attacker.fingerprint_rotated,
attacker.scored) merged through asyncio.Queue, 15s keepalive,
request.is_disconnected() exit, finally task cancellation.
Per-attacker filter keys on payload['attacker_uuid'] which the
profiler worker stamps onto every published payload (Phase 5 P5.0
amendment) — O(1) drop without a repo round-trip per event.
_sse_name_for derives SSE event names:
attacker.observation.<primitive> → observation.<primitive>
attacker.fingerprint_rotated → fingerprint.rotated
attacker.scored → attacker.scored
10 tests cover snapshot, live forward, per-attacker filter (drops
other attackers' events), fingerprint.rotated forward, 404, 401, and
the sse-name derivation across all four cases. Topology events
regression green.
The profiler worker's per-observation publish now re-merges
attacker_uuid into the bus payload alongside id/ts/v. Same shape as
the existing DECNET-side deviation from BEHAVE's wire-format
docstring (BEHAVE-INTEGRATION.md §339-366) — widens the deviation
by one DECNET denorm field.
Phase 5's per-attacker SSE route can now filter
attacker.observation.* events to one attacker in O(1) without a repo
round-trip per event. identity_ref stays None today (until the
attribution engine ships); attacker_uuid is independent.
Two test changes:
* test_happy_path_persists_and_publishes asserts attacker_uuid is in
every published payload.
* New test_attacker_uuid_in_payload_for_filter pins the field
explicitly and confirms it doesn't conflate with identity_ref.
The profiler worker now consumes attacker.session.ended on the bus
AND walks unprofiled session_recorded log rows on every tick. Both
paths converge on a single handler that:
1. Validates required payload fields (session_id, decky_id, service,
attacker_ip, shard_path).
2. Builds evidence_ref shard:{decky}/{service}/{shard_basename}#{sid}
and skips when has_observations_for_evidence is True (idempotent
re-runs).
3. Resolves attacker_uuid via get_attacker_uuid_by_ip; defers if the
profiler tick hasn't materialised the row yet.
4. Reads the asciinema shard, slices events for the sid, calls
extract_session, persists each Observation via upsert_observation
(per-row; batch transaction filed as follow-up), then publishes
each on the bus best-effort (fire-and-forget per DEBT-029 §6).
Architecture:
* Handler lives in decnet/profiler/behave_shell/_handler.py — pure
function, unit-tested in isolation.
* Worker.py adds _behave_pump (queue feed), _drain_behave_queue
(per-tick drain), _behave_poll_tick (cursor scan over
session_recorded logs), and _payload_from_log_row (Log → bus-shape
payload projection).
* Poll cursor uses a separate state key
(attacker_worker_session_cursor) so the correlation tick's cursor
doesn't conflate.
* has_observations_for_evidence promoted to BaseRepository abstract.
22 new tests across handler / drain / poll layers covering happy
path, all skip paths, isolation against handler exceptions,
idempotency on re-run, and cursor key separation. TTP worker bus
tests still green — payload field is purely additive.
Closes BEHAVE-INTEGRATION.md Phase 4.
Lock the BEHAVE library versions per BEHAVE-INTEGRATION.md
§Versioning. The profiler worker (Phase 4 wiring) imports
`Observation`/`Window` from `decnet_behave_core.spec.envelope` and
`event_topic_for`/`to_event_payload` from
`decnet_behave_shell.spec.event_adapter`; without the pin a broken
wheel or missing install would only show up on first publish.
Four-test smoke pins the public surface: envelope construction,
registry import non-empty, event-adapter topic shape, and the
adapter's id/ts/v exclusion contract.
The collector's _SessionAggregator now resolves the asciinema shard
via find_shard_with_sid and stamps it onto every emitted
attacker.session.ended payload as `shard_path`. None when the shard
isn't on disk yet (collector race with sessrec flush) — consumers
treat that as "skip until next tick".
Additive field; existing TTP worker consumes the same topic and
ignores unknown keys, so no payload-version bump needed. Two new
tests pin the shard-found and shard-missing cases.
Unblocks BEHAVE-INTEGRATION Phase 4: the profiler worker reads
shard_path directly from the payload instead of disk-reaching.
Move `_find_shard_with_sid`, `_resolve_shard`, `_validate_names`,
`_get_index`, and the index cache from
`decnet/web/router/transcripts/api_get_transcript.py` into
`decnet/artifacts/shards.py`. The shared module speaks
`ValueError`; the router keeps thin wrappers that translate to
`HTTPException(400)` so the route's error UX is unchanged.
This unblocks the BEHAVE-INTEGRATION Phase 4 worker wiring — the
profiler worker (and the collector's session aggregator) need to
disk-reach asciinema shards but must not import from a FastAPI
router.
11 new unit tests for the shared helper. Existing transcript router
tests pass (the shard fixture's monkeypatch points at the shared
module's ARTIFACTS_ROOT now).
decnet.profiler.behave_shell.__version__ = '0.1.0-pre'.
The -pre suffix is honest: the extractor is feature-complete (37/37
Tier-A primitives emit, calibration grid honest), but the engine
package — worker wiring, observations writes, AttackerDetail panel —
still rides BEHAVE-INTEGRATION.md Phase 4. The actual 0.1.0 tag
lands when Phase 4 lands.
The marker version-tracks the engine, not the spec library
(decnet-behave-shell already at 0.1.0); they version independently.
Run the five-class calibration grid (HUMAN / YOU-sim / LW-sim /
CLAUDE-FF / CLAUDE-CL) against the 2026-05-02 shards.
* Hard gate green for 27 primitives across all 5 shards.
* environmental.keyboard_layout moved from hard gate to
PHASE_F_CONDITIONAL_PRIMITIVES — short SSH-recon corpus maxes at
~90 typed letters per session, well below the LAYOUT_MIN_TYPED_LETTERS
(200) floor. The 200-floor stays per the per-phase "v0 ships when
honest" rule; longer-text corpora will surface the layout signal.
* Three primitives never fire on the 2026-05-02 corpus, all already
conditional and all expected:
- cognitive.error_resilience.frustration_typing
- environmental.locale
- environmental.keyboard_layout
No D / F / G threshold re-tunes needed; only the keyboard_layout
binding-set move. Phase H step log appended to BEHAVE-EXTRACTOR.md
with per-class observation counts.
Static assertion that every Tier-A primitive in PRIMITIVE_REGISTRY
has a slot in the calibration grid (hard gate or conditional set).
Excludes Tier B (8 cross-session primitives) and Tier C (toolchain.*)
by explicit allow-list and prefix filter.
Three checks:
* every Tier-A primitive is covered (forward direction)
* no extractor set drifts from the registry (reverse, catches typos)
* Tier-A count == 37 (design doc invariant)
CI now fails before a registry addition ships without a feature
function.
Widen calibration binding from PHASE_ABCDEF_PRIMITIVES (25) to
PHASE_ABCDEFG_PRIMITIVES (28 hard). Three Phase G primitives that
emit on any session-with-commands ride the hard gate:
* operational.opsec_discipline
* operational.cleanup_behavior
* emotional_valence.stress_response
The remaining five Phase G primitives ride a new
PHASE_G_CONDITIONAL_PRIMITIVES because their sample-size floors make
them legitimately absent from short shards:
* operational.objective (≥ 3 classified commands)
* operational.multi_actor_indicators (≥ 8 commands)
* emotional_valence.arousal (typing bursts)
* emotional_valence.valence (≥ 80 typed letters)
* emotional_valence.frustration_venting (≥ 30 typed letters)
Backwards-compat alias PHASE_ABCDEF_PRIMITIVES kept. Phase G
completion log + checkbox flips in BEHAVE-EXTRACTOR.md.
Tier-A corpus delta: all 37 Tier-A primitives now emit. Phase H
(full-corpus lockdown + v0 release) is next.
Compare median post-error intra-command IATs against baseline
(commands not immediately following an errored command):
* ratio ≥ STRESS_EUSTRESS_RATIO_MIN (1.20) → eustress_positive
* ratio ≤ 1/STRESS_DISTRESS_RATIO_MIN → distress_negative
* otherwise → none
Confidence hard-capped at 0.5; 0.30 below
STRESS_MIN_ERRORED_WITH_IATS (2).
high_agitated when any of:
* caps_run_max ≥ 5
* bang_run_max ≥ 3
* fastest typing burst median IAT < 0.06s with ≥ 30 IATs total
low_calm when slowest qualifying burst median IAT > 0.30s with ≥ 30
IATs. Else medium_engaged. Confidence hard-capped at 0.5; 0.30 below
AROUSAL_MIN_IATS.
Compare median intra-command IATs of the two temporal halves of the
session. ≥ MULTI_ACTOR_HALF_MIN_COMMANDS (4) per half required;
relative delta > MULTI_ACTOR_HANDOFF_DELTA (0.5) → handoff_detected.
team_coordinated is Tier B (cross-session); never emitted from a
single session. Confidence 0.55 with both halves ≥ 8 commands; 0.40
otherwise.
* careful — operator hits OPSEC_HISTORY_TOKENS AND tail-K commands
include _CLEANUP_TOKEN_HASHES (re-imported from temporal.py).
* learning — history hit without cleanup-tail follow-through.
* careless — no history-clearing vocabulary at all.
Confidence 0.45 (small lexicon, soft); 0.30 below
MIN_COMMANDS_FOR_FULL_CONFIDENCE.
Phase G shared infrastructure (no primitive yet emitted):
* New `_intent.py` — five precomputed first-token-hash sets (recon /
exfil / persistence / lateral / destructive) with documented
precedence, plus opsec-history and three lexeme sets (positive /
negative / obscenity) for the typed-text counter pass. Stop words
that collide with registry value vocabulary (`no`, `hell`, `ok`)
are deliberately excluded — the PII regression test catches such
collisions.
* `_typed_char_histograms()` extended with five integer counters
populated in the same single-pass walk: `obscenity_hits`,
`positive_lex_hits`, `negative_lex_hits`, `caps_run_max`,
`bang_run_max`. Longest-suffix match against bounded lexicon
(`LEXEME_MAX_LEN`); paste-class events excluded.
* `SessionContext` widened by the same five fields. Drives G.5
(valence), G.6 (arousal), G.8 (frustration_venting) without retaining
raw operator text.
* Bump twisted >= 26.4.0rc2 to clear CVE-2026-42304 (pre-existing,
caught by pre-commit pip-audit). Adjust ftp template type-ignore
code from attr-defined to misc to match the new Twisted typing.
PII discipline: same shape as F.4 — fixed-vocabulary integer counters
on ctx, never on observations.
Widens the binding calibration set from PHASE_ABCDE_PRIMITIVES (20)
to PHASE_ABCDEF_PRIMITIVES (25). The five new entries:
* environmental.shell_type (per-shard hard gate)
* environmental.terminal_multiplexer (per-shard hard gate)
* environmental.keyboard_layout (per-shard hard gate; PII boundary
lifted by ANTI; emits all 4 registry values)
* environmental.numpad_usage (per-shard hard gate)
* temporal.lifecycle_markers.exit_behavior (resolution of the E.4
hold; uses Command.followed_by_prompt from F.0)
environmental.locale joins a new PHASE_F_CONDITIONAL_PRIMITIVES set
(only fires on shards with an env / locale dump in the output).
Phase F completion log appended to BEHAVE-EXTRACTOR.md. The original
F.0 row hinted at D.0 subsumption; reversed in the log — D.0 is
enriched, not subsumed (regex catches errors when PS1 is suppressed).
Tier-A corpus delta: 25 of 37 primitives now emit. Phase G is next.
Resolves the E.4 hold from Phase E. F.0's Command.followed_by_prompt
gives us the exit-code proxy (prompt-after-last-command) we couldn't
get in Phase E.
Logic: last command without trailing prompt → abrupt; first_token_hash
in {exit, logout, quit, logoff} → graceful; any of the last K=3
commands' first_token_hash in {history, unset, rm, shred, clear, kill}
→ cleanup; else → graceful (clean Ctrl-D / window close).
Sliding-window scan over single-char digit input events. A run of
NUMPAD_RUN_MIN (4) consecutive digit events whose pairwise IATs are
all ≤ NUMPAD_FAST_IAT_S (50ms) → detected. Otherwise → not_detected.
Skips below NUMPAD_MIN_TYPED_CHARS (50) typed chars. Confidence cap
0.50 per the registry's weak-signal flag.
ANTI authorised dropping the PII boundary for this primitive. ctx
gains typed_unigram_counts / typed_bigram_counts / typed_letter_count
populated during the existing single-pass input walk (paste-class
events excluded).
Two-axis classifier:
* layout-artefact unigrams take priority — q rate above floor with
low English saturation → azerty; z above floor with y below → qwertz
* fallback to English-bigram saturation: ≥ floor → qwerty, else other
Sample-size floor 200 typed letters; bigram histogram capped at
top-64 to bound memory. Confidence cap stays moderate (0.40-0.55) —
heuristic discriminator.
Searches ANSI-stripped output for LANG / LC_ALL / LC_CTYPE envvar
substrings emitted by env / locale / printenv. Highest-priority key
wins (LC_ALL > LANG > LC_CTYPE); POSIX value normalised to BCP-47:
en_US.UTF-8 → en-US, pt_BR.UTF-8 → pt-BR, C/POSIX → und. Free-string
registry value emitted directly.
PII discipline: only the parsed locale value enters observations;
surrounding output is read once for matching and dropped.
Scans RAW output (multiplexer escapes are themselves ANSI; never
strip first) for tmux markers (DCS passthrough, focus-reporting,
window-title with tmux marker) and screen markers (DCS, screen-OSC).
Detected → tmux/screen at 0.85; otherwise → none at 0.55. Skips
emission entirely when no commands — silence on a pure-echo or
empty session, per the smoke gates.
When both detected (nested mux), prefer tmux.
Adds PromptLine dataclass + extract_prompt_lines() helper. PromptLine
carries ts, suffix_char ($/#/%/>), raw_line (ANSI-stripped, capped),
is_root flag. Populated during the existing single-pass output-window
walk; SessionContext gains prompt_lines, Command gains
followed_by_prompt.
PII trade-off (ANTI-authorised at Phase F): PS1 text retained on ctx
so F.1 / F.3 / E.4 can read it. Capped at PROMPT_LINE_MAX_CHARS=256.
Observations still only carry derived primitive values.
D.0's regex error helpers stay alongside (NOT subsumed) — they fire
even when PS1 echo is suppressed. F.0 enriches D.0 rather than
replacing it.
F.0's row in BEHAVE-EXTRACTOR.md was forward-only — readers landing
on Phase F couldn't tell that F.0 also has a backlog (E.4 held, D.0
subsumption). Add a 'Carry-overs F.0 must unblock' section to the
Phase F prelude and a back-reference on the F.0 checkbox in the
implementation order checklist.
Widens the binding calibration set from PHASE_ABCD_PRIMITIVES (17) to
PHASE_ABCDE_PRIMITIVES (20). The three shipped Phase E primitives
(session_duration, escalation_pattern, landing_ritual) join the
per-shard hard gate.
E.4 (temporal.lifecycle_markers.exit_behavior) is held at ANTI's
direction pending Phase F.0's prompt parser — abrupt-vs-cleanup
needs exit-code visibility to be honest, and first-token membership
alone over-fires on benign rm / clear mid-session. E.4 picks up at
the tail of Phase F.
Phase E completion log appended to BEHAVE-EXTRACTOR.md; E.1-E.3
checkboxes flipped, E.4 left unchecked with a held note.
Inspect the first N commands; if at least K of their first_token_hashes
match the recon-survey vocabulary (uname/id/whoami/pwd/hostname/w/who),
emit present, else absent. Hashes precomputed at module load; PII-safe.
v0.1 N=5, K=2.
Bin commands into non-overlapping windows of width
max(ESCALATION_WINDOW_MIN_S, duration_s / ESCALATION_WINDOW_TARGET).
CV of per-window counts + zero-window fraction classify bursty /
sustained / erratic. v0.1; corpus re-tune deferred.
Bucket ctx.duration_s against SESSION_DURATION_SHORT_MAX (60s) /
MEDIUM_MAX (600s) / LONG_MAX (3600s); else marathon. Direct
measurement, confidence 0.85. Skip emission only when no commands
and zero duration. New _features/temporal.py module opens Phase E.
Widens the binding calibration set from PHASE_ABC_PRIMITIVES (13) to
PHASE_ABCD_PRIMITIVES (17). The four unconditional Phase D primitives
(cognitive_load, exploration_style, planning_depth, tool_vocabulary)
join the per-shard hard gate. The three error_resilience.* primitives
are conditional on at least one errored command in the shard and
tracked in PHASE_D_CONDITIONAL_PRIMITIVES — excluded from the
per-shard required-emission set, included in the cross-class
discrimination check.
cognitive_load empirical re-tune deferred to the next
BEHAVE_CALIBRATION_DIR run; v0.1 thresholds ship.
Phase D completion log appended to BEHAVE-EXTRACTOR.md; Phase D
checkboxes flipped to [x].
For each errored command, check whether the next command's
first_token_hash is in {man, help, info} (precomputed at module
load). At least one match → present, else absent. The --help / -h
flag forms aren't first tokens; v0.2 will reconsider once arg-token
hashing is justified by corpus.
Compares median within-command IAT for commands following an errored
command vs commands following a successful one. Relative absolute delta
buckets to low / moderate / high. Skips when either group is empty
(no errors, or no clean baseline). v0.1; D.8 re-tunes.
Modal response across Command.errored=True commands:
* same first_token_hash on next command → rerun
* different first_token_hash → switch
* no next command → abort
Tiebreak in registry order. The fourth registry value 'modify'
requires within-command arg diffing (PII boundary); deferred to v0.2.
Distribution of inter-command IATs bucketed against IKI_THINK_MAX_S
(deep) and INTER_CMD_INSTANT_MAX (reactive); fall-through is shallow.
v0.1 thresholds; D.8 re-tunes.
Two-axis classification over the first_token_hash sequence:
repetition_rate (drilling) vs backtrack_rate (jumping among prior
tools). chaotic/targeted/methodical buckets. v0.1 thresholds; D.8
re-tunes.
Composite over three [0, 1]-clipped sub-signals (chunking variance,
error rate from D.0's Command.errored, pace variability), mean-aggregated
and bucketed against COGNITIVE_LOAD_LOW_MAX / COGNITIVE_LOAD_MEDIUM_MAX.
Components missing data drop out of the mean rather than zeroing it.
v0.1 thresholds; D.8 re-tunes once D.2-D.7 are stable. Confidence
held at 0.60 (composite over soft sub-signals) and halved below the
5-command sample-size floor.
Lifts the error-signal slice of F.0 forward as a D.0 prelude. ANSI
strip + canonical bash/sh error fingerprints classify each command's
post-execution output window; Command gains errored / output_bytes
fields. PII discipline preserved — only a bool and an int leave the
helper, the stripped output text is dropped on return.
Drives D.1 (cognitive_load error_rate term) and D.5–D.7 (error_resilience
family). Phase F.0 will subsume this with PS1 + exit-code parsing.
Tick the four Phase B checkboxes (B.1-B.4) and append a Phase B
completion log inline (per the "append phase logs to design docs"
memory rule). Captures per-primitive confidence ranges, source
signals, and the PII-discipline regression that all four
primitives uphold.
Phase A + Phase B = 10 primitives emitting on every shard;
PHASE_AB_PRIMITIVES is binding for every subsequent phase.
Phase C (motor.shell_mastery.*) lands next.
BEHAVE-EXTRACTOR.md Phase B Step B.3. Replaces the prototype's
two-line "0 vs >0 backspaces" placeholder with a backspace-timing
classifier that honours the registry's full vocabulary.
* SessionContext gains backspace_count, backspace_iats (IAT from
each backspace back to the preceding non-backspace input event),
and kill_line_count (^U / ^W). Built by _scan_correction_signals,
which retains only counts and timing aggregates — no character
data leaves the helper, in line with the BEHAVE PII discipline.
* _features/motor.py:error_correction(ctx) emits one Observation
in {immediate, deferred, absent, route_around}.
- 0 backspaces + ≥1 ^U/^W → route_around (rewrite, not correct)
- 0 backspaces + 0 kill-lines → absent
- backspaces with median IAT ≤ 500 ms → immediate
- slower → deferred
Confidence 0.65 / 0.65 / 0.55 / 0.55.
* < 3 inputs → skip emit.
* Calibration grid widened to include motor.error_correction;
green across all five shards.
Tests cover all four buckets, the < 3 inputs skip, and the PII
regression (raw command body never appears in the serialised
observation).
BEHAVE-EXTRACTOR.md Phase B Step B.2. First principled
implementation — the prototype doesn't ship this primitive at all.
* _features/motor.py:motor_stability(ctx) emits one Observation
in {steady, variable, tremor}. Reuses ctx.typing_bursts from B.1.
* Tremor proxy: fraction of within-burst IATs below
TREMOR_FAST_FLOOR_S (30 ms — humans can't sustain sub-50 ms IATs).
≥ TREMOR_RATE_MIN (10%) sub-floor → tremor (double-press / motor
twitch / stuck-key).
* Otherwise median burst CV decides: < CV_STEADY_MAX → steady,
else → variable. Confidence 0.70 / 0.60 / 0.65.
* No typing bursts or fewer than 5 within-burst IATs → skip emit.
* Calibration grid widened to include motor.motor_stability; green
across all five shards.
Tests cover all three buckets + skip paths.
BEHAVE-EXTRACTOR.md Phase B Step B.1.
* SessionContext gains typing_bursts: tuple[tuple[float, ...], ...]
built by _split_typing_bursts(iats) — splits at gaps > IKI_THINK_MAX_S
(1.5s) and drops bursts of fewer than 3 IATs. Mirrors prototype's
_split_into_bursts at BEHAVE/prototype_extractors/shell/extract.py:275.
* _features/motor.py:keystroke_cadence(ctx) emits one Observation
in {steady, bursty, hunt_and_peck, machine}. Median CV across
typing bursts; mean IKI < IKI_MACHINE_MAX_S paired with CV <
CV_MACHINE_MAX → machine. Confidence 0.85/0.70/0.65/0.60 per the
prototype's calibration history.
* < MIN_INPUTS_FOR_CADENCE inputs or zero typing bursts → skip
emission. v0.1 emits only the burst-CV variant; the prototype's
NAIVE session-CV variant is parked for v0.2.
* Calibration grid widened (PHASE_A_PRIMITIVES → PHASE_AB_PRIMITIVES)
to include motor.keystroke_cadence. Grid green across all five
shards.
Tests: too-few-inputs → no emit, all-think-pauses → no burst → no
emit, uniform IATs → steady, sub-5ms → machine, mixed-pace → bursty,
extreme bimodal → hunt_and_peck.
BEHAVE-EXTRACTOR.md Phase A Step 10. Closes the discriminative
floor: six primitives emit, the five-class calibration grid is the
binding regression test for every subsequent phase.
* Phase A checklist boxes (Steps 0-10) ticked in
development/BEHAVE-EXTRACTOR.md.
* Phase A completion log appended inline to the design doc per
the "append phase logs to design docs" memory rule — captures
per-primitive confidence ranges and the 2026-05-02 empirical
anchors that drove threshold calibration.
* Hard gate: tests/profiler/behave_shell/test_calibration_grid.py
parametrised over five class shards, all green; skips cleanly
on BEHAVE_CALIBRATION_DIR unset.
Phases B-G expand horizontally across the registry. Phase H is
the full-corpus lockdown + v0 release. Worker
(BEHAVE-INTEGRATION.md Phase 4) is unblocked at this milestone —
it can wire per-session production against the Phase A engine
without waiting for the rest of the Tier-A corpus.
BEHAVE-EXTRACTOR.md Phase A Step 9 — the gate. Runs the pure
engine against each of the five 2026-05-02 calibration shards and
pins the contract that all subsequent Phase B-G PRs must keep
green: every Phase A primitive (motor.input_modality,
motor.paste_burst_rate, cognitive.inter_command_latency_class,
cognitive.command_branch_diversity, cognitive.feedback_loop_engagement,
cognitive.inter_command_consistency) fires at least once per shard.
* tests/profiler/behave_shell/test_calibration_grid.py
parametrized over (shard_file, class_label) for HUMAN / YOU-sim /
LW-sim / CLAUDE-FF / CLAUDE-CL. Skips entirely when
BEHAVE_CALIBRATION_DIR is unset (CI provides the path; local dev
doesn't have to).
* Plus a discrimination-smoke check: at least one primitive
produces different majority values across present classes —
catches the "constant-output regression" failure mode where the
engine quietly degenerates to a stub.
Calibration tweak: BRANCH_DIVERSITY_LINEAR_MIN dropped from 0.80 to
0.70 to align with the prototype's empirical anchors (CLAUDE-CL ≈
0.55-0.60 adaptive; YOU-sim / CLAUDE-FF scripted recon ≈ 0.75+
linear). Test for the middle band re-pinned at the new boundary.
Per-class value pinning (e.g. HUMAN must emit
inter_command_consistency=bimodal) is intentionally NOT a hard gate
yet — v0.1 thresholds put real human sessions in "variable", and
true bimodal detection (Hartigan dip / two-peak) is registry-flagged
for v0.2. Tighter pinning lands as the corpus grows.
BEHAVE-EXTRACTOR.md Phase A Step 7. The orthogonal axis — does the
operator's pause-after-command correlate with bytes of output they
just saw? Splits HUMAN/CLAUDE-CL (closed_loop) from LW-sim/CLAUDE-FF
(fire_and_forget); cuts ACROSS the LLM/human axis.
* _features/cognitive.py:feedback_loop_engagement(ctx) emits one
Observation in {closed_loop, fire_and_forget, unknown}.
* Pearson correlation between ctx.output_per_cmd[i] and
ctx.inter_cmd_iats[i] (paired by construction in Step 4); via
statistics.correlation with constant-series fallback to "unknown".
* r > FEEDBACK_CORRELATION_MIN (0.30) → closed_loop; otherwise
(zero, negative, or undefined) → fire_and_forget.
* First primitive that depends on output events: zero output events
in the shard or fewer than FEEDBACK_MIN_PAIRS (5) pairs → emit
"unknown" at confidence 1.0 (the absence-of-data is itself a
high-confidence answer). Zero-command session skips entirely.
Tests: no-output → unknown, few-pairs → unknown, strong positive r
→ closed_loop, constant pace → fire_and_forget/unknown,
negative r → fire_and_forget.
BEHAVE-EXTRACTOR.md Phase A Step 6. Content-based playbook-vs-
adaptive split. Splits CLAUDE-FF (linear_playbook, ~10 distinct
tools) from CLAUDE-CL (adaptive_branching, 5-6 tools with curl
re-invoked) per the 2026-05-02 empirical anchor.
* _features/cognitive.py:command_branch_diversity(ctx) emits one
Observation in {linear_playbook, adaptive_branching, unknown}.
* unique_first_token_hashes / total_commands ratio. ≥ 0.80 →
linear_playbook, otherwise adaptive_branching (the doc instructs
bias-to-adaptive in the middle band — that's the discriminative
signal we actually want).
* < 5 commands → "unknown" at confidence 1.0 (the absence of data
is itself a high-confidence answer per the registry's allowed
vocabulary). Zero-command session skips emission entirely.
Tests cover unique-tokens → linear, repeated-tokens → adaptive,
middle band → adaptive (bias), under-floor → unknown @ 1.0, plus
PII regression: raw tokens never appear in the serialised
observation.
BEHAVE-EXTRACTOR.md Phase A Step 5. Classifies the operator's
thinking pace between commands. Splits LW-sim / CLAUDE-FF /
CLAUDE-CL.
* _features/cognitive.py:inter_command_latency_class(ctx) emits one
Observation in {instant, typing_speed, deliberate,
llm_lightweight, llm_heavyweight, long}, computed as the median
of ctx.inter_cmd_iats bucketed against the prototype thresholds
(v0.2 split: lightweight 2-8s, heavyweight 8-30s).
* Sample-size honesty: < 5 commands halves confidence (0.40 vs
0.80) per BEHAVE-EXTRACTOR.md.
* Threshold consts (INTER_CMD_*_MAX, MIN_COMMANDS_FOR_FULL_CONFIDENCE,
plus parked Step 6/7/8 thresholds for the next three commits)
added to _thresholds.py.
Tests cover all six buckets at empirically-anchored IATs (15s ≈
Claude Opus driving recon via tmux send-keys), plus the
single-command no-IAT and low-sample-count paths.
BEHAVE-EXTRACTOR.md Phase A Step 4. Pure refactor inside _ctx.py —
no new feature emits. Lays the shared utility for the three
cognitive primitives next in line (Steps 5-7).
* Command dataclass (frozen): start_ts, end_ts, first_token_hash.
PII-safe by construction — only the first whitespace-delimited
token of the command is retained, and only as a sha256 hash
(decnet/profiler/behave_shell/_parse.py:hash_token).
* _segment_commands walks input events char-by-char, splits on
\r / \n, hashes the first token, drops the rest.
* SessionContext gains commands, inter_cmd_iats, output_per_cmd.
output_per_cmd[i] counts bytes between commands[i].end_ts and
commands[i+1].start_ts — the natural pairing for Step 7
(feedback_loop_engagement).
Tests: empty / unterminated streams, single command (CR + LF
terminators), paste-with-newline, multi-command IAT pairing,
output-byte counting between boundaries, blank-line skip,
first-token-only PII discipline.
BEHAVE-EXTRACTOR.md Phase A Step 3. Same paste-event ratio as
motor.input_modality but coarser-bucketed: this is the *habit*
signal (does the operator reach for paste at all?), where
input_modality is the dominant-channel signal.
* _features/motor.py:paste_burst_rate(ctx) emits one Observation
per session in {none, occasional, habitual} with confidence
0.70 / 0.70 / 0.80.
* Thresholds: PASTE_RATE_OCCASIONAL_MIN=0.10,
PASTE_RATE_HABITUAL_MIN=0.50.
Splits YOU-sim from LW/CLAUDE-FF/CLAUDE-CL — LLM-driven sessions
paste habitually, real humans rarely paste.
Tests: pure-typed → none; 1-paste-in-10 → occasional;
paste-majority → habitual; output-only → no observation; habitual
confidence > occasional confidence.
BEHAVE-EXTRACTOR.md Phase A Step 2. The first primitive — picked
first because it has the highest discriminative value (HUMAN vs
everyone) and the simplest implementation (paste-event ratio over
total inputs).
* _features/motor.py:input_modality(ctx) emits one Observation
per session in {typed, pasted, mixed} with confidence 0.75 / 0.70.
* _features/_emit.py centralises the make_observation helper so
every feature module gets the same Window/source/evidence_ref
boilerplate without copy-paste.
* Thresholds inherited from the prototype's calibration history
(MODALITY_PASTED_MIN=0.40, MODALITY_TYPED_MAX=0.05).
* Zero-input session skips emission — registry doesn't admit
"unknown" here.
Tests: pure-typed → typed, pure-pasted → pasted, mixed → mixed,
output-only session → no observation, full envelope round-trip.
BEHAVE-EXTRACTOR.md Phase A Step 1. Lays the shared primitives that
Steps 2-3 (motor.input_modality, motor.paste_burst_rate) will
consume:
* parse_shard_line / parse_shard turn a shard JSONL line/file into
AsciinemaEvents, skipping headers and malformed records.
* PasteBurst dataclass + _detect_paste_bursts group consecutive
paste-class input events (len(d) >= 4 chars per the prototype's
empirical floor) into contiguous bursts, splitting on IAT gaps
larger than PASTE_BURST_MAX_IAT_S (200ms).
* SessionContext now carries iats and paste_bursts derivations.
* Threshold constants harvested from
BEHAVE/prototype_extractors/shell/extract.py — calibrated against
the five 2026-05-02 shards.
Tests cover pure-typed, pure-pasted, mixed streams; close vs far
paste events; typed events breaking a burst; PasteBurst immutability;
and the JSON parser's junk handling.
BEHAVE-EXTRACTOR.md Phase A Step 0. Lays the package skeleton
(__init__/extract/_parse/_ctx/_thresholds/_features) with empty
FEATURES = (), so the worker plumbing in BEHAVE-INTEGRATION Phase 4
has a stable import path before any primitive lands.
extract_session() builds a SessionContext once and fans the
registered feature functions across it; at Step 0 that fan-out is
empty and the function yields nothing. Step 1 (asciinema parser +
paste-burst detector) and Step 2 (motor.input_modality) land next.
Smoke suite asserts the empty contract: empty stream → no
observations, single event → t_start == t_end, multi-event → events
routed into input_events / output_events by kind, evidence_ref
defaults to "session:<sid>" or honours an explicit override.
Destructive half of BEHAVE-INTEGRATION.md Phase 1. SessionProfile +
its kd_* columns + the dialect ALTER TABLE migration helpers are
deleted outright; pre-v1, the table shipped empty, no migration
ceremony required (per the no-new-_migrate_-pre-v1 memory rule).
DEBT-036 closes via DEBT-050 supersedure. AttackerDetail's
``observations`` field is wired to the new ``observations`` table
and returns an empty list until the BEHAVE-SHELL extractor (DEBT-050
Phase 2) starts emitting.
decnet/web/db/models/attackers.py — SessionProfile class deleted
(~135 lines), KD_PAUSE_*/KD_START_OF_ACTION_IDLE_S module constants
deleted, module docstring updated to point at the observations
table. AttackerIdentity.kd_digraph_simhash is KEPT — it's the v2
federation centroid hook, not a SessionProfile field; docstring
repointed to the BEHAVE primitive that will populate it.
decnet/web/db/sqlmodel_repo/attackers/sessions.py — DELETED.
SessionProfilesMixin dropped from the AttackersMixin MRO.
decnet/web/db/repository.py — abstract upsert_session_profile +
get_session_profile removed.
decnet/web/db/sqlite/repository.py + mysql/repository.py —
_migrate_session_profile_table helpers and their initialize() calls
removed. mysql initialize() now goes attackers → column_types →
admin (no session_profile step).
decnet/web/db/models/__init__.py — SessionProfile re-export gone.
decnet/web/db/models/attacker_intel.py — docstring cross-reference
to SessionProfile.schema_version retargeted to AttackerIdentity.
decnet/web/router/attackers/api_get_attacker_detail.py — adds
``observations: []`` to the response by calling
``repo.latest_observation_per_primitive(uuid)`` and projecting to a
list sorted by primitive path. Empty until the extractor lands;
shape matches BEHAVE-INTEGRATION.md §"AttackerDetail consumer".
tests/profiler/test_session_profile.py — DELETED (56 lines).
tests/db/test_base_repo.py — DummyRepo loses upsert_session_profile
and get_session_profile overrides.
tests/db/mysql/test_mysql_migration.py — initialize-call-order
assertion updated; session_profile step removed from the expected
sequence; docstring records why.
tests/ttp/test_lifter_absence.py — docstring "no SessionProfile" →
"no ObservationRow".
Additive Phase 1 of BEHAVE-INTEGRATION.md. Lays the storage layer
the BEHAVE-SHELL extractor (DEBT-050) will write into. Nothing
breaks; SessionProfile coexists for now and is dropped in the
follow-up commit.
decnet/web/db/models/observations.py — new ObservationRow SQLModel
mirroring the BEHAVE Observation envelope field-for-field
(core/decnet_behave_core/spec/envelope.py). ``id`` is a hex-string
UUID (matching BEHAVE), not a typed UUID column. ``identity_ref``
is str | None — written by the future attribution engine, NULL
until then. ``attacker_uuid`` is the one DECNET-side
denormalisation; FK'd to attackers.uuid for cheap AttackerDetail
joins. ``evidence_ref`` is NOT NULL for DECNET emissions even
though the upstream envelope makes it optional — the worker's
"already profiled?" check keys on it. UniqueConstraint(evidence_ref,
primitive) enforces idempotency at the schema level so re-running
the extractor on the same shard+sid produces a DB-side conflict
the upsert path resolves deterministically. Class is named
``ObservationRow`` (not ``Observation``) to avoid colliding with
the BEHAVE Pydantic envelope at sites that import both.
decnet/web/db/sqlmodel_repo/observations.py — ObservationsMixin.
Three public methods backing the canonical queries from
BEHAVE-INTEGRATION.md §"Storage": ``upsert_observation`` (idempotent
on the natural key), ``latest_observation_per_primitive`` (per-
primitive MAX(ts) subquery, portable across SQLite and MySQL — no
DISTINCT ON), ``observations_time_series`` (asc-by-ts). Plus
``has_observations_for_evidence`` for the worker's session-already-
profiled check.
decnet/bus/topics.py — ATTACKER_OBSERVATION_PREFIX = "observation"
constant + ``attacker_observation(primitive)`` builder. Full topic
shape ``attacker.observation.<primitive>`` matches what BEHAVE's
spec.event_adapter.event_topic_for produces upstream. Documentation
+ pattern matching only — bus auth is socket file perms (DEBT-029
§2), not topic-level.
decnet/web/db/repository.py — abstract ``upsert_observation``,
``latest_observation_per_primitive``, ``observations_time_series``
on BaseRepository.
tests/db/test_observations.py — 11 tests covering upsert round-trip,
idempotency under the unique constraint, latest-per-primitive
ordering across multiple sessions, time-series asc-ordering, empty-
attacker contract, every BEHAVE ValueKind round-tripping through
the JSON column, and the has_observations_for_evidence check.
tests/db/test_base_repo.py — DummyRepo gains the three new abstract
overrides so its coverage suite still instantiates.
Three sibling design docs plus DEBT.md updates that supersede the
stale DEBT-036 with a BEHAVE-aligned plan.
development/BEHAVE-INTEGRATION.md — five-phase rollout: storage
(observations table mirroring the BEHAVE Observation envelope plus
one DECNET-side denorm; UniqueConstraint(evidence_ref, primitive)
enforcing idempotency); engine (in decnet/profiler/behave_shell/
sublibrary, no new daemon, not in BEHAVE — DECNET is the engine);
BEHAVE pin; worker wire; UI panel + per-attacker SSE route; live
smoke. Bus payload merges id/ts/v back in to preserve sensor
identifiers across the bus envelope.
development/BEHAVE-EXTRACTOR.md — engine route in eight phases
(A–H). Phase A locks the 6-primitive calibration grid; Phases B–G
expand horizontally; Phase H is the full Tier-A corpus + v0
release. v0 ships every shell-extractable primitive (37 of them);
Tier B is cross-session and lives in the attribution engine; Tier
C is network-domain (toolchain.*) and lives elsewhere.
development/ATTRIBUTION-ENGINE.md — sublibrary inside
decnet/correlation/ that consumes attacker.observation.* events
and emits attribution.profile.* derived state. Five-state machine
(unknown / stable / drifting / conflicted / multi_actor) with per-
ValueKind merge functions. v0 closes DEBT-051; v1 adds the real
clusterer; v2 federation gossip. The bright line forbidding
attribution to natural persons is lifted directly from BEHAVE's
envelope docstring.
development/DEBT.md — DEBT-036 marked STALE; DEBT-050 and
DEBT-051 entries added; summary table + open list updated.
New MalHashProvider sibling ABC (decnet/intel/base.py) since SHA-256
is a different keyspace from IntelProvider's IPs. MalwareBazaarProvider
mirrors FeodoProvider's bulk-feed shape: 24h refresh via _ensure_fresh
/ _refresh, in-memory set[str] of hex-lowercased hashes, set-membership
lookup. Auth-keyed via DECNET_MALWAREBAZAAR_AUTH_KEY; absent key
silent-no-ops the lane (single warning, no HTTP traffic).
Per-hash observations persist to a new observed_attachments table.
DECNET is a honeypot platform — every attachment hash an attacker
delivers is intel, regardless of whether anyone classified it. Verdict
is sticky: True never downgrades to False/None on subsequent
observations. Out of scope: API surface, federation export, retention.
Ingester _publish_email_received calls the provider for each attachment
sha256, sets mal_hash_match on the bus payload (omitted entirely when
the message had no attachments — keeps R0046's `is True` predicate
silent on hash-less mail, matching pre-paydown behavior), and upserts
the row regardless of provider availability.
New GET /api/v1/orchestrator/events/stats?since=1h&success=false&kind=...
backed by repo.count_orchestrator_failures(since_ts, kind), which
counts failed rows across both orchestrator_events and
orchestrator_emails since the cutoff.
Window parser accepts ^\d+[smhd]$, capped at 7d. Today only
success=false is accepted on this surface so the endpoint isn't
accidentally repurposed before the next consumer is properly
designed.
Orchestrator.tsx polls the endpoint on mount + every 30 s and
renders the authoritative DB-derived count instead of deriving from
the in-memory SSE buffer + one paginated page (which silently
excluded failures older than the local window).
Wire vitest 4 + jsdom + @testing-library/{react,jest-dom,user-event}
+ @vitest/coverage-v8 through vite.config.ts (defineConfig from
vitest/config). src/test/setup.ts registers jest-dom matchers and
RTL cleanup. tsconfig.app.json picks up vitest/globals types.
Seed suite Orchestrator.test.tsx covers the three regressions
called out in DEBT-043: empty-state render, kind-filter toggling
triggers a scoped refetch, mocked stream callback prepends a row.
When the prober observes a NEW hash for an
(attacker_uuid, port, probe_type) triple it has seen before — VPS
rotation, SSH server rebuild, TLS cert swap — emit a derived
attacker.fingerprint_rotated event carrying both old and new hash.
Detection is a small library (decnet.correlation.fingerprint_rotation)
called inline from the prober at each of the three emit sites
(JARM/HASSH/TCPFP). No new daemon. New AttackerFingerprintState table
holds per-triple last-hash state; Attacker.rotation_count and
Attacker.last_rotation_at are stamped on every diff. Library is sync,
fully unit-tested via injected publish_fn / syslog_fn callbacks.
All base images (debian:bookworm-slim, ubuntu:22.04, ubuntu:20.04,
rockylinux:9-minimal, centos:7, alpine:3.19, fedora:39,
kalilinux/kali-rolling, archlinux:latest, honeynet/conpot:latest)
now carry their resolved sha256 digest so 'docker pull' is
deterministic. :tag retained for human readability; @sha256 is what
Docker actually resolves. Refresh procedure documented at the top of
decnet/distros.py.
Empty directory tracked via .gitkeep so operators see it on first
clone; README documents the .eml/.json drop-in flow that the IMAP/POP3
compose fragments wire up by default.
When service_cfg["email_seed"] is absent, compose_fragment now falls
back to $PROJROOT/bait/ if that directory exists on the host. Lets
operators drop a deployment-wide bait corpus into one place without
threading email_seed through every decky's config. Missing dir keeps
old no-op behavior.
IMAP_EMAIL_SEED / POP3_EMAIL_SEED accept a directory (rglob *.eml +
*.json) or a single .json/.eml. Loaded entries CONCATENATE with the
hardcoded _BAIT_EMAILS — additive to the realism-engine emailgen
output rather than replacing it. JSON dicts require from_addr /
to_addr / subject / body; bare bodies are wrapped into RFC 5322 on
load. compose_fragment reads service_cfg["email_seed"] and bind-mounts
the host path read-only at /var/spool/decnet-emails/seed.
The TTP-tagging worker is now safe to run on agent hosts: EmailLifter
disk-reaches body-aware predicates from the local artifacts tree
(DEBT-035 unblocked filesystem access; DEBT-047 added the helper).
Drop `ttp` from MASTER_ONLY_COMMANDS in cli/gating.py and remove the
defence-in-depth `_require_master_mode("ttp")` call in cli/ttp.py.
`ttp-backfill` walks the master DB and stays master-only.
R0047 (BEC) and the encoded-payload predicate substring-match against
the email body. Shipping raw body text on the abstracted service bus
is the wrong privacy stance — the bus transport may swap from UNIX
socket to networked at any time, and "loopback today" is not a license
to put PII on the wire.
EmailLifter now opens the .eml lazily from
/var/lib/decnet/artifacts/{decky_id}/smtp/{stored_as} when a body-aware
predicate runs and parses the body in-process via stdlib email +
policy.default. The decoded body is memoized into the payload dict so
multiple body-aware predicates on the same event open the file once.
Bus envelope only carries the artifact pointer (decky_id + stored_as);
raw body bytes never cross the host disk boundary on the agent → master
hop. Filesystem access on agents is unblocked by DEBT-035 (setgid +
group-readable artifacts root, paid 2026-05-02).
The legacy inline body_text path is preserved — when the producer ships
body_text on the bus the helper short-circuits without opening the file.
Move artifact path validation + symlink-escape check out of the
admin-gated download endpoint into decnet/artifacts/paths.py so the
TTP EmailLifter can disk-reach .eml files at tag-time without
duplicating regex/root logic (DEBT-047).
The router now catches ArtifactPathError and re-raises HTTPException(400);
behavior is unchanged.
ANTI flagged two regressions in the existing command-event capture:
1. **Tell**: PROMPT_COMMAND lived in /root/.bashrc, the FIRST file
an attacker greps after landing root. The logger invocation
sitting there is plain-text honeypot signage.
2. **Bypass**: even when missed, `export PROMPT_COMMAND=""` silently
disables capture. ANTI personally bypasses this on engagements.
Reshape:
* Move the assignment to **/etc/environment** — read by pam_env at
session open (sshd via /etc/pam.d/sshd, telnet via
/etc/pam.d/login), before any shell rc file fires. Far less
obvious than .bashrc; a casual `cat .bashrc` no longer surfaces
the capture.
* Define the helper as a function `__bash_history_sync` in
**/etc/bash.bashrc** (system-wide bashrc, sourced by every
interactive bash). Function name reads as generic bash
housekeeping; no DECNET branding in the symbol.
* Pin both the function and PROMPT_COMMAND **readonly** so
`export PROMPT_COMMAND=""` fails with "readonly variable"
instead of silently winning. Mitigation, not airtight —
`bash --norc` still bypasses — but the passive `export`
bypass is closed.
The actual `logger --rfc5424 --msgid command ... CMD ...` invocation
is preserved exactly; only its location and the readonly guard
change. R0001–R0030 (command-rule pack) consume the same syslog
shape as before.
Three new tests assert: the value lands in /etc/environment, the
function body lives in /etc/bash.bashrc, no PROMPT_COMMAND line
remains in /root/.bashrc, and `readonly PROMPT_COMMAND` /
`readonly -f __bash_history_sync` are both present. Mirror
assertions added on the Telnet Dockerfile via
test_config_schema.py.
Real Linux deployments (especially Ubuntu cloud images) ship a non-
root admin user; honeypots that only accept root logins are a tell.
Add a second account on both SSH and Telnet decoys, configurable
via service_cfg keys `user` / `user_password`, defaulting to
`ubuntu` / `admin` so the lure is live on every fresh deploy.
* `decnet/services/{ssh,telnet}.py` — two new ServiceConfigFields
(`user` string, `user_password` secret) and matching env vars
(`SSH_USER` / `SSH_USER_PASSWORD`, mirror for telnet) propagated
via the compose fragment.
* `decnet/templates/ssh/entrypoint.sh` — runtime `useradd -m -s
/usr/libexec/login-session -G sudo "$SSH_USER"` so the new user
inherits the same sessrec pty-recording shell as root and lands
in the sudo group. Privesc attempts (`sudo`) flow through the
existing sudo-log capture; network-enum from the user's shell
rides the recorded transcript.
* `decnet/templates/telnet/entrypoint.sh` — same useradd pattern
(no sudo group — busybox+login telnet image has no sudo
package; privesc rides `su -` which itself flows through the
existing PAM auth-helper at /etc/pam.d/login).
* New tests for default + custom user / password + independence
from root password. Updated the schema-keys assertion to match
the four-field shape.
The new account is ALSO the natural home for the body-aware
predicates that were previously gated on root-only sessions —
attackers who land on `ubuntu@host` and run network-recon /
privesc commands now generate the same structured TTP-rule
events as root sessions did, captured via the same auth-helper
+ sessrec + sudo-log pipes.
DEBT-035 (artifacts written as the container uid, not the API's) is
resolved by the two preceding commits:
* 39a298f6 — persists DECNET-service api-user/api-group as names in
decnet.ini for any future composer / worker that wants to resolve
the local uid via pwd.getpwnam.
* b2733216 — creates /var/lib/decnet/artifacts at init time with mode
0o2775 (setgid + group-write) owned by the DECNET-service
user:group.
The setgid bit is the load-bearing fix: Linux mkdir(2) propagates a
parent's group AND its setgid bit to every new subdirectory. Docker
auto-creates the per-decoy / per-service subtree as bind-mounts fire,
so those subdirs come up with group=decnet and setgid set; container
file writes (default umask 0o022 → mode 0o644) inherit the decnet
group; the API process and the local TTP worker (both running as the
DECNET-service user, primary group decnet) read via group-read.
The original recommendation of compose `user:` injection turned out
infeasible for SSH and Telnet — PAM's setuid(2) during login
fundamentally cannot run from a non-root container. Setgid covers
both root-internal and unprivileged-internal templates uniformly
without requiring per-template carve-outs.
DEBT-047 (R0047 BEC disk-reach) was gated on DEBT-035 for filesystem
access. That blocker is lifted — `decnet ttp` running on agents as
the local DECNET-service user can now read .eml files written by
the SMTP decoy. The remaining DEBT-047 work is the master-only gate
flip in decnet/cli/gating.py and the EmailLifter disk-reach helper
itself (factor _resolve_artifact_path out of the artifacts API
endpoint into a shared module).
Soft-fail paths in api_get_transcript.py and api_get_artifact.py
stay as defence-in-depth — option 2 should make them never fire on
a healthy install but a misconfigured deploy must not 500 the API.
DEBT-035 step 2. Today the artifacts subtree is auto-created by
Docker as root when a decoy container's bind-mount fires for the
first time. The resulting permissions are root:root 0o755 — the API
process (running as the decnet user) hits PermissionError trying to
read transcripts written by the container, and the soft-fail 404
path gets exercised on every fresh deploy.
Add `/var/lib/decnet/artifacts` to init's dirs list with mode 0o2775:
* 0o2000 — setgid bit. New files inherit the directory's group
(decnet), regardless of which uid created them. This is the load-
bearing bit for cross-container reads.
* 0o0775 — owner+group rwx, world rx. Group-write lets the API
process and the local TTP worker read each other's outputs
without a manual chown.
`_ensure_dir` already respects the full mode word via `os.chmod`,
no helper change needed.
Test asserts the resulting directory carries exactly 0o2775 after
a fresh `decnet init --prefix`. Defence-in-depth: this works even
if the per-decoy compose `user:` directive (next commit) misses a
template — files still land in the decnet group.
DEBT-035 step 1. The composer needs to know which uid/gid to inject
into each compose fragment's `user:` directive at deploy time. Today
the resolved `--user` / `--group` values reach systemd unit
rendering (init.py:349–354) but are not persisted anywhere the
composer can read them.
Persist as **names** (not numeric ids) under `[decnet] api-user` /
`api-group` in the rendered decnet.ini placeholder. Resolution to
uid/gid happens at deploy time on whichever host runs the deploy,
via `pwd.getpwnam(...)` / `grp.getgrnam(...)` — so the same user
name can have different uids on master vs agents (heterogeneous
/etc/passwd) without breaking artifact ownership. The existing
config_ini auto-translates kebab→DECNET_API_USER / DECNET_API_GROUP
at load time; no domain-map changes needed.
Two new tests: one asserting the rendered ini carries the
`api-user` / `api-group` keys for the values passed to `--user` /
`--group`; one round-tripping through `load_ini_config` to confirm
the env vars land in `os.environ` for the composer to pick up.
A previous agent (and several of my own commits) wrote to a top-level
DEBT.md without seeing the existing development/DEBT.md — the
canonical register since DEBT-001. Resulted in two parallel files,
inconsistent numbering schemes, and references that resolved to the
wrong place.
Migrate the six entries that landed in the rogue file into the
canonical register as DEBT-044 through DEBT-049, preserving their
status (resolved / partial / open) and cross-references. The
TTP_TAGGING.md references to "DEBT.md" already resolve to
development/DEBT.md by virtue of being in the same directory; only
the comment in decnet/ttp/impl/intel_lifter.py needed disambiguation
to "development/DEBT.md DEBT-048".
* DEBT-044 — `attacker.email.received` producer wiring (✅ RESOLVED 2026-05-02)
* DEBT-045 — EmailLifter heavyweight feature extraction (PARTIAL PAID 2026-05-02)
* DEBT-046 — EmailLifter mal-hash feed integration (open)
* DEBT-047 — EmailLifter R0047 BEC unblock (open, gated on DEBT-035)
* DEBT-048 — TTP intel provider mapping review (recurring quarterly)
* DEBT-049 — TTP Sigma adapter — post-v1 (open)
Summary table extended; "Remaining open" line updated; root file
removed. The DEBT-047 entry now explicitly cross-references DEBT-035
as the gating dependency for the R0047 BEC unblock.
Mark the EmailLifter heavyweight follow-up as PARTIAL PAID — R0042 /
R0046 (macro / password / smuggling lanes) / R0048 fire end-to-end
after commits 291b78c1 (decky extractors) and the ingester producer
projection that follows.
Two narrower DEBT entries replace the lanes that remain gated:
* "EmailLifter mal-hash feed integration" — R0046's mal_hash_match
lane needs a curated bad-hash feed (MalwareBazaar SHA-256 dump as
the v0 candidate, mirroring the FeodoProvider bulk-feed pattern at
decnet/intel/feodo.py). Feed integration, not extraction. Lifter
predicate already reads `payload.get("mal_hash_match")` — silent
today only because the field is absent.
* "EmailLifter R0047 BEC — unblock when artifact disk-reach lands"
cross-references the agent UID/GID DEBT entry that blocks
`decnet ttp` from reading artifacts written by deckies on the
same host. Disk-reach is the intended solution; raw body_text on
the bus is rejected because the bus transport is abstracted (the
UNIX-socket implementation may swap to networked at any time, and
privacy decisions must hold regardless of transport).
Append to TTP_TAGGING.md §"Producer wiring": the email.received
producer pointer (was "none — DEBT"), the full per-message payload
shape with the new heavyweight fields, and an explanatory block on
why the bus is body-text-free + how R0047 / R0048 each handle their
body dependency (R0048 via the precomputed scalar; R0047 deferred).
The decky's Layer-2 extension (commit 291b78c1) emits body_simhash /
body_base64_bytes / html_smuggling on the message_stored log and adds
macro_indicator / encrypted booleans to each attachments_json
manifest entry. Lift them all onto the email.received bus payload:
* body_simhash — passes through as-is (16 hex chars or "")
* body_base64_bytes — coerced to int (0 on absent / malformed)
* attachment_macros / attachment_password_protected — OR-reduced
across the per-attachment manifest booleans; matches R0046's
matched_trigger semantics where a single positive lane fires the
rule
* html_smuggling — coerced bool from the decky's 0/1 int
Pre-Layer-2 message_stored events (older deckies, malformed log
rows) project to safe defaults: empty simhash, zero base64-bytes,
all booleans False — the EmailLifter then stays silent, never
fires a false positive on missing data.
R0042 (mass-phish) / R0046 macro / R0046 password / R0046 smuggling
/ R0048 (encoded payload) all fire end-to-end after this commit.
R0046 mal_hash_match and R0047 BEC remain deferred per their
respective DEBT entries (filed in the next commit).
Heavyweight Layer-2 extractors land alongside the cheap projections
shipped in commit e9324aca, so the EmailLifter R0042 / R0046 (macros
/ password / smuggling lanes) / R0048 fire from the bus payload
without the lifter having to reach back to disk.
Extractors:
* body_simhash — inlined 64-bit Charikar simhash (md5-keyed,
frequency-weighted) over word tokens of the union of text/* body
parts. Inlined rather than pulling the `simhash` PyPI dep, which
transitively brings numpy ~50 MB into a slim decky container; the
algorithm is ~15 lines and identical in extraction quality.
* body_base64_bytes — largest decoded base64 chunk's byte count,
scanning text body parts with the same `_BASE64_RE` the lifter's
`_p_encoded_payload` fallback uses. R0048 fires from this scalar
alone; the lifter's body_text fallback becomes dead in normal
operation.
* attachment_macro_indicator — stdlib zipfile sniff for
`vbaProject.bin` inside OOXML containers. Catches modern .docm /
.xlsm / .pptm and macro-injected .docx; legacy .xls (CFBF) is a
follow-up.
* attachment_encrypted — flag_bits & 0x01 on any ZIP / OOXML entry's
central directory; magic-byte match for 7z / RAR / CFBF (encrypted
Office wrap).
* html_smuggling — structural lxml parse first: fires when an `<a
download>` element coexists with a `<script>` referencing
`Blob` / `Uint8Array` / `URL.createObjectURL`. Regex pair-check
fallback on lxml parse failure (real-world phish HTML is often
malformed). Cuts the FP rate that pure-regex would produce on
legitimate "click to download" links.
Add `python3-lxml` (~5 MB Debian package, C-extension, no transitive
Python deps) to the SMTP decky's Dockerfile. simhash stays inline.
Per the dependency rule: lxml earns its weight by cutting R0046's
OR-combined FP rate; a heavier macro-detection lib (oletools ~5 MB
pure-python with msoffcrypto) would not measurably improve the
boolean signal we need, so stdlib stays for that lane.
Wires the EmailLifter (R0041–R0048) producer that DEBT.md item #3
deferred. After the existing add_bounty() call in _extract_bounty
(line 615), call _publish_email_received() which:
* resolves the attacker_uuid via repo.get_attacker_uuid_by_ip; drops
the publish if unresolved (the TTP worker can't anchor orphan
events)
* projects the message_stored fields onto the EmailLifter wire
contract: from_domain / mail_from_domain / return_path_domain
parsed via _domain_of, rcpt_count + rcpt_domains via
_rcpt_projection, attachment_sha256s + attachment_extensions
derived from the existing attachments_json manifest, urls from
urls_json, dkim_signed/spf_pass coerced from 0/1 ints to bool
* mirrors _publish_probe_pending's bus-per-call pattern and
swallows all exceptions (the bus is the notification layer, not
the source of truth)
Fires for both relay and non-relay SMTP services. R0041 / R0043 /
R0044 / R0045 are now live end-to-end; R0046 partial (extension
lane). Heavyweight predicates (R0042 simhash, R0046-deep, R0047 /
R0048 body_text) stay deferred per the EmailLifter heavyweight
DEBT entry.
The EmailLifter (R0041–R0048) keys on header-derived signals that the
v0 _summarize_message did not extract. Add cheap Layer 2 projections
inside the existing single-pass parse:
* return_path / x_mailer — direct header reads, decoded RFC 2047
* dkim_signed / spf_pass — booleans derived from any
Authentication-Results header (multiple lines tolerated; positive
verdict on any line wins)
* urls — http(s) URLs lifted from text/* body parts via a tight
regex, deduplicated first-seen-wins, capped at 64 in the wire
payload to bound the syslog SD value
Heavyweight extraction (body simhash, office-macro detection,
HTML-smuggling, password-protected archives, mal-hash-match,
body_text projection) stays deferred per the EmailLifter heavyweight
DEBT entry — those rules need privacy / extractor decisions before
they ship.
The 2026-05-02 paydown wires the producer at ingester.py after
add_bounty(), with the cheap projections (domains, rcpt_count,
attachment_count, x_mailer, dkim/spf, attachment shas + extensions,
URLs). R0041 / R0043 / R0044 / R0045 fire end-to-end after this PR;
R0046 partial.
The remaining lanes (R0042 body_simhash, R0046 macro / smuggling /
password / mal_hash, R0047 / R0048 body_text projection) are filed
as a new entry "EmailLifter heavyweight feature extraction" with the
field map and the privacy-vs-completeness fork on body_text called
out for the next maintainer to pick a side.
Appendix A.10 corrected to match the post-2026-05-02-audit reality:
AbuseIPDB cat 7/13/16/17 land on their canonical AbuseIPDB names
(Phishing / VPN IP / SQL Injection / Spoofing); cats 4 and 10 carry
explicit "drop" annotations so the next reviewer sees the intent
rather than guessing. ThreatFox table re-keys on `threat_type` (the
canonical taxonomy field) and adds the `payload` and `cc_skimming`
rows. GreyNoise table promotes bare-malicious to a half-multiplier
emission of T1071.
§"Hard parts §9 Intel provider drift" replaces the prose handwave
with a runnable check: provider URLs, the ThreatFox curl invocation
that needs DECNET_THREATFOX_API_KEY, the rule_version + emits +
attack_catalog co-evolution rules, and the full chain of files to
exercise. Adds a "Ship-time audit log" subsection so future quarterly
runs have a known-good baseline to diff against.
DEBT.md item #1 records LAST_REVIEWED: 2026-05-02 / NEXT_REVIEW:
2026-08-02 and points at §9 for the runbook. DEBT.md item #3 (the
attacker.email.received producer) flags its gating premise as
potentially stale — ANTI noted SMTP honeypots already persist
received messages, contradicting the "no source row" claim that
deferred the wiring.
The IntelLifter's _emit_filtered fans out only the rule.emits entries
whose technique_id appears in the predicate's decision set. v1's emits
lists were narrow supersets of the common case, silently dropping the
rest of the predicate's possible emissions:
R0054 dropped: T1046 (cat 14), T1078 (cat 20), T1090 (cats 9/13),
T1496 (cat 11), T1595 (cats 14/19)
R0055 dropped: T1090 (tor_exit_node), T1110 (ssh_bruteforcer),
T1588 (the second emit of every C2-framework tag)
R0057 dropped: T1105 (payload_delivery, download_url)
Bump rule_version 1->2 on R0054/R0055/R0057, expand emits to cover
every technique the predicate produces. R0056 (Feodo) and R0058
(aggregate bump) carry no enum and stay at v1.
All five YAMLs gain `last_reviewed: "2026-05-02"` and
`next_review: "2026-08-02"` markers; the rule YAML is now the
canonical record of when the mapping was last reconciled against
upstream, with DEBT.md as the calendar reminder.
Three bug classes uncovered by the 2026-05-02 ship-time audit:
* AbuseIPDB code/name mismatch in v1: cat 10 was treated as DDoS (it's
Web Spam — DDoS is cat 4, intentionally unmapped per A.10) and cat 17
as VPN IP (it's Spoofing — VPN IP is cat 13). Both typos mirrored in
code AND the design doc Appendix A.10. Code now matches the AbuseIPDB
taxonomy exactly; cat 17 retargets to T1566 (email-spoofing as a
phishing precursor), and cats 7 (Phishing) and 16 (SQL Injection)
pick up T1566 / T1190 emissions that v1 didn't cover.
* ThreatFox dispatch keyed on `ioc_type` in v1, but `ioc_type` is the
indicator format (url / domain / hash variants) and carries no ATT&CK
signal. The canonical taxonomy field per ThreatFox's API is
`threat_type` (botnet_cc / payload_delivery / payload / cc_skimming).
Repoint dispatch through the new `threatfox_threat_types` payload
field; `ioc_type` rides as evidence only. Also adds the missing
cc_skimming -> T1056 (Input Capture) mapping and registers T1056 in
attack_catalog.py.
* GreyNoise bare-malicious lane: a `classification == "malicious"` row
with no recognised tag used to emit nothing. Now lights T1071 at a
half multiplier, suppressed when a tag already fires T1071 to avoid
double-stamping at conflicting confidence levels.
The TTP worker forwards the bus payload verbatim to the IntelLifter as
TaggerEvent.payload. The pre-audit publish payload only carried
{attacker_uuid, attacker_ip, aggregate_verdict, providers}, so even with
the new AttackerIntel taxonomy columns populated the lifter still saw
nothing. Lift the relevant fields (categories / tags / threat_types /
malware family / score / classification) into the bus event and decode
JSON-string list columns back to native lists at the boundary.
The 2026-05-02 ship-time audit of the R0054-R0058 intel rule pack found
that AbuseIPDB / GreyNoise / ThreatFox stored only the aggregate verdict
(score / classification / listed-bool) plus the raw response blob. The
TTP IntelLifter expects per-provider taxonomy fields (categories, tags,
threat_types) that were never populated, so R0054 / R0055 / R0057
emitted zero tags in production despite passing unit tests.
Add typed columns: abuseipdb_categories, greynoise_tags, greynoise_name,
feodo_malware_family, threatfox_threat_types, threatfox_ioc_types,
threatfox_malware_families. Each provider now parses the relevant
taxonomy out of the upstream response and writes it through
column_updates. JSON-list columns ride as TEXT with default "[]" to
keep the SQLite/MySQL backend split honest, deserialised back to native
lists by the repo on read.
The inspector was dumping the whole `CMD uid=0 user=root src=… pwd=…
cmd=nmap -p- 192.168.1.0/24` syslog body into a single ``command_text``
blob. ANTI: "I'd like to separate the fields." Done — three layers
work together:
1. Collector session aggregator: new `_parse_cmd_msg` splits the bash
PROMPT_COMMAND msg into `{uid, user, src, pwd, command}`. The
session-ended envelope's per-command dict now carries the
structured fields, with `command_text` set to just the cmd= value
(preserving embedded whitespace — `nmap -p- 1.2.3.0/24` etc.).
2. Rule engine: per-source_kind auxiliary evidence list
(`_AUX_EVIDENCE_FIELDS`). For `command` events the engine
automatically promotes uid/user/src/pwd into the persisted
`evidence` dict on top of the rule's explicit `evidence_fields`.
Engine-controlled, not per-rule — adding a new aux field is one
line here, not a 30-rule YAML sweep, and rule authors can't
accidentally drop it.
3. TTPInspector frontend: evidence renders as a structured
`kvs` grid (UID / USER / SRC / PWD / CMD rows) instead of
pretty-printed JSON. Primary-order list keeps shell fields at
the top; everything else falls below alphabetically so unfamiliar
evidence shapes still surface predictably.
Tests:
- session_aggregator pins the structured-fields emit (uid/user/src/
pwd/command_text without "CMD" prefix, embedded whitespace
preserved).
- rule_engine_tagger pins the aux-field auto-promotion + the
no-`None`-leakage path when payload doesn't carry an aux key.
"T1595" alone is opaque; "T1595 — Active Scanning" tells you the
story at a glance. The names come from a backend-side static catalogue
pinned to the same ATT&CK release as the rule engine
(_ATTACK_RELEASE = "v15.1") — names are the canonical MITRE labels,
not author-supplied strings on rules, so a rule author can't typo a
name and the entire fleet sees the typo.
- New `decnet/ttp/attack_catalog.py` with `TECHNIQUE_NAMES` covering
every technique_id + sub_technique_id emitted by `rules/ttp/`
(R0001..R0058 → 69 IDs in the v0 pack).
- `IdentityTechniqueRow` / `TechniqueRollupRow` / `CampaignTechniqueRow`
/ `TTPTagDetailRow` gain optional `technique_name` /
`sub_technique_name` fields. Repo + router populate them from the
catalogue at row-construction time. None when an ID isn't in the
catalogue — UI falls back to the bare ID.
- Coverage test (`tests/ttp/test_attack_catalog.py`) walks every
YAML rule and asserts every emitted ID has a catalogue entry, so
a future rule author who forgets to update the catalogue gets a
loud failure rather than a silent UI fallback.
Frontend:
- `TTPsObservedSection` shows "T1595.002 — Active Scanning:
Vulnerability Scanning" instead of just the ID, with overflow
ellipsis + tooltip for narrow viewports. Inspector header /
TECHNIQUE row also surface the names.
The TTPsObservedSection rollup tells the operator "we saw T1059" but
not why. Click any technique row → side drawer opens listing every
ttp_tag row in scope with the persisted evidence JSON, firing
rule_id / rule_version, source_kind / source_id, confidence, and
created_at. Mirrors the CredentialReuseInspector / BountyInspector
pattern (drawer-backdrop + bd-head/bd-body + kvs grid).
Backend:
- New `GET /api/v1/ttp/tags/by-{scope}/{uuid}/{technique_id}`
(`scope ∈ {identity, attacker, session}`, optional
`?sub_technique_id=`, `?limit=` capped to 1000). Returns raw
TTPTag rows newest-first.
- New `TTPTagDetailRow` Pydantic model + re-export.
- New repo method `list_tags_by_scope_and_technique` on
TTPMixin (+ abstract on BaseRepository) — single query branched
on scope; identity scope projects through `Attacker.identity_id`
the same way `list_techniques_by_identity` does.
- Tests: evidence round-trips, sub_technique filter, JWT-required,
empty scope, unknown scope rejected.
Frontend:
- New `TTPInspector.tsx` + `TTPInspector.css` (violet accent, slide
animation, focus-trapped panel matching the existing inspector
family).
- `TTPsObservedSection`'s TechniqueBar is now click+keyboard
activatable; clicking opens the inspector for that
(technique, sub_technique) tuple.
mypy clean. 532 passed in the targeted sweep.
The collector's `attacker.session.ended` envelope carries
`attacker_uuid: null` and `attacker_ip: <ip>` because the collector
doesn't talk to the DB. The TTP worker passed that null straight
through, and `TTPTag.__init__` raised the documented invariant:
ValueError: ttp_tag requires at least one of attacker_uuid /
identity_uuid; both NULL is not a valid anchor.
The worker now resolves `attacker_uuid` from `attacker_ip` via
`BaseRepository.get_attacker_uuid_by_ip` before fanning out the
event. When the IP isn't in the DB yet (profiler hasn't ingested
the row), the event is dropped with one log line — better than
exploding mid-tag.
- New `get_attacker_uuid_by_ip(ip) -> str | None` on the repo
(BaseRepository abstract + AttackersCoreMixin impl).
- `_resolve_attacker_uuid` helper in `decnet/ttp/worker.py` runs
before `_build_events`. Short-circuits when the payload already
has either anchor; drops the event when neither anchor is
resolvable.
- Tests pin: short-circuit on existing uuid/identity, repo lookup,
drop on unknown IP, drop on "Unknown" sentinel, drop on
no-anchor payload, drop on repo failure.
Add a "Producer wiring" subsection under TTP_TAGGING.md §"Bus
topics" mapping every topic the TTP worker subscribes to onto the
file:line that publishes it. Calls out the gap (`email.received`
has no producer today) and the new `attacker.session.ended`
payload shape from the collector aggregator.
Also lists the four producer regression tests added in this series
so a future contributor sees the safety net before staring at the
silent rule engine.
DEBT.md gets the `attacker.email.received` follow-up entry — wire
the producer when SMTP-receive persistence lands, since today the
honeypot relay path doesn't store received emails anywhere a
publisher could read from.
Three producer-side regression guards. Each drives the worker's run
loop with a fake bus + stubbed repo and asserts the documented topic
fires when the producer has data:
- reuse correlator → credential.reuse.detected (one finding row)
- clusterer → identity.formed + identity.merged (one ClusterResult)
- intel worker → attacker.intel.enriched (one unenriched attacker
+ a fake provider returning a "malicious" verdict)
These complement commit 1's attacker.session.ended producer test —
together the four cover every TTP-relevant publisher in the tree
(modulo email.received, which has no producer yet; tracked in
DEBT.md).
The TTP worker subscribes to attacker.session.ended but no upstream
component published it — the rule pack (R0001–R0030) therefore never
fired on live SSH traffic even after the consume-side wiring landed
in E.3.18a/b/c.
The collector now hosts a per-attacker_ip command index
(_SessionAggregator) that watches the same parsed-event stream as
_publish_log. Shell `command` events are appended to a per-IP list;
on `session_recorded` the aggregator slices the list to commands
inside the [ended_at - duration_s, ended_at] window and publishes
attacker.session.ended with the session metadata + commands list.
The TTP worker's _build_events fan-out (E.3.18b) turns each command
into a source_kind="command" TaggerEvent that the RuleEngineTagger
(E.3.18c) matches against R0001–R0030.
Memory bound: per-IP entries TTL-evict at DECNET_COLLECTOR_SESSION_AGG_TTL_SEC
(default 3600 s). Publish failures are swallowed in the aggregator —
a misbehaving bus cannot stall the per-container stream threads.
Honeypot SSH containers run `PROMPT_COMMAND` that calls
`logger --rfc5424 --msgid command -t bash "CMD …"`. The Docker-stdout
reader prepends an outer RFC5424 envelope (HOSTNAME=<decky>,
APP-NAME=1, MSGID=NIL) around that inner syslog line. Both the
collector parser (`parse_rfc5424`) and the correlation parser
(`parse_line`) saw the outer NIL MSGID and emitted `event_type="-"`
for every shell command — which:
- kept `Attacker.commands` rows missing `command_text`
- left R0001–R0030 (the pattern rule pack that matches shell
commands) with no haystack
- made `decnet.collector.log` show `event written … type=-`
for the very lines that should be `type=command`
Both parsers now detect the inner-RFC5424 shape (`<TS> <HOST> <APP>
<PROCID> <MSGID> <rest>`) when the outer MSGID is NIL and the SD-arm
is also NIL, and re-extract HOSTNAME / APP-NAME / MSGID / remainder
from the body. The collector parser also recovers the post-SD msg
tail when the SD block isn't `relay@55555` (the bash CMD line carries
a `[timeQuality …]` block) so the kv-fallback can find `src_ip`.
Mirroring tests in tests/collector and tests/correlation pin both
the unwrap and the regression guard for non-double-wrapped lines.
The endpoint was a contract-phase stub returning `[]` even though the
RuleStore loaded all 58 YAML rules at worker startup. UI saw an empty
table; operators couldn't tell whether anything was wired up.
- `api_list_rules` now calls `get_rule_store().load_compiled()` and
serializes each CompiledRule + its operational state into a
RuleCatalogueRow. Sorted by rule_id for stable golden snapshots.
- Add `description: str` to RuleSchema (pydantic) and CompiledRule
(NamedTuple, defaulted) + propagate through `_compile_one` so the
catalogue surfaces the human-readable YAML description, not just
the slug-style `name`.
- Update `tests/ttp/test_rule_engine.py` _fields assertion for the
new column; new `tests/api/ttp/test_rules_catalogue.py` pins the
catalogue contents (R0001/R0014 presence, row shape, sort order).
Worker behaviour is unchanged: it was already loading rules
correctly. This is purely a read-side wiring fix on the operator API.
Closes the CDD design phase. Records:
- §E.1 contract inventory (every file exists, compileall clean).
- Targeted pytest pass: 604 passed, 1 skipped, 10 xfailed
(all xfails are `xfail(strict=True)` with reason= pointing to the
impl step that flips them; carry-overs, not flakes).
- Strict mypy over decnet/ttp + decnet/cli/ttp.py +
decnet/web/router/ttp + decnet/web/db/sqlmodel_repo/ttp.py: clean.
- Stranger-readability spot check on tests/ttp/: no doc bugs.
Notes the three pre-E.4 wiring fixes (E.3.18a/b/c) and the E.4
backfill CLI / DEBT entries that landed in this series.
Quarterly TTP provider mapping review for AbuseIPDB / GreyNoise /
abuse.ch (Feodo Tracker, ThreatFox) catalogue drift against
`rules/ttp/R0054..R0058`, and the post-v1 trigger for the Sigma rule
adapter. Both items reference TTP_TAGGING.md sections so the
rationale stays linked to the design doc.
The TTP worker entry moved out of decnet/cli/workers.py into its own
module so the TTP CLI surface (worker + admin verbs) is colocated,
mirroring decnet/cli/canary.py / webhook.py / swarm.py.
- New `decnet/cli/ttp.py` with `decnet ttp` (worker, ExecStart-stable
for decnet-ttp.service) and `decnet ttp-backfill --since-days N`.
- `decnet ttp-backfill` walks Attacker.commands and CanaryTrigger
history, dispatches each row through the live CompositeTagger,
persists tags via repo.insert_tags (idempotent INSERT OR IGNORE).
--dry-run / --source command|canary|all / --batch-size supported.
- Backfill deliberately bypasses bus publish — historical replay
must not re-trigger SIEM/webhook fan-out per TTP_TAGGING.md
§"Bus topics" loop-prevention invariant.
- Added `iter_attacker_commands_since` / `iter_canary_triggers_since`
read-only iterators on TTPMixin + abstract bindings on
BaseRepository.
- Master-only via gating; both `ttp` and `ttp-backfill` listed in
MASTER_ONLY_COMMANDS.
The canonical rule-based engine from §"Tagging engines, layered §1"
of TTP_TAGGING.md was fully implemented but never instantiated as a
composite child — pure pattern rules (R0014/R0017/R0023/... 23 rules
total) had no tagger to dispatch them.
- Add `RuleEngineTagger(Tagger)` adapter in rule_engine.py wrapping
`RuleEngine.evaluate()`. `HANDLES = {command, http_request,
auth_attempt, payload}` — the source kinds whose rules typically
live outside any per-source lifter.
- Adapter's `watch_store()` filters via `_is_engine_owned` so the
engine's dispatch index excludes lifter-claimed rules
(`match.kind: lifter:*`) and stays disjoint from per-lifter ownership.
- Prepend `RuleEngineTagger` to the `CompositeTagger` lifter list so
generic pattern rules dispatch before per-source cross-event logic.
- Composes with E.3.18a (worker hydrates `watch_store`) and E.3.18b
(worker fans session payloads into per-`command` events) — together
these three commits make R0001–R0030 actually fire at runtime.
R0001–R0030 declare `applies_to: [command]` and match per command, not
per session. The worker now translates one `attacker.session.ended`
payload carrying a `commands: list` into:
- one source_kind="session" event (behavioral / cross-event lifters)
- one source_kind="command" event per command (RuleEngineTagger)
Both string and dict command shapes are accepted; dicts contribute
their `id` / `uuid` / `command_id` as the per-command source_id so
the deterministic `compute_tag_uuid` keeps replays idempotent. Tags
from session + per-command dispatch are aggregated into a single
`ttp.tagged` envelope per upstream session.
Each per-source lifter holds its own RuleIndex and exposes an
`async watch_store()` that loads the corpus and drains store change
events forever. Until this commit nothing called `watch_store()` in
production — every dispatch index stayed empty and no rule fired.
- Add `WatchableTagger` runtime-checkable Protocol in `decnet.ttp.base`.
- `CompositeTagger.iter_watchables()` yields lifters that satisfy it.
- `run_ttp_worker_loop` fans out one task per watchable, cancelled
and awaited alongside pump/heartbeat/control in the existing finally.
- Watch failures log and exit the watch task without taking the
worker down — mirrors the pump-task tolerance contract.
Wires decnet-ttp as a first-class worker:
* `decnet ttp` CLI command (master-only via MASTER_ONLY_COMMANDS)
* deploy/decnet-ttp.service.j2 systemd unit (After= identity / intel
/ reuse-correlator workers; ProtectHome=read-only since
FilesystemRuleStore only reads ./rules/ttp/)
* deploy/decnet.target Wants= chain extended with decnet-ttp.service
* `ttp` was already in web/worker_registry.KNOWN_WORKERS
tests/api/test_schemathesis_ttp.py: TTP-routes-only schemathesis
suite, filtered via the OpenAPI tags=["TTP Tagging"] annotation
shared by the eight TTP routes. Reuses the live uvicorn subprocess
the wider test_schemathesis spawns; max_examples=400 keeps the
focused gate fast for E.3.13–E.3.16 iteration.
wiki-checkout/Service-Bus.md committed in its own repo: ttp.tagged
and ttp.rule.fired.<id> flipped from "reserved (TTP worker)" to
"decnet.ttp.worker" now that the worker publishes them.
TTPsObservedSection.tsx: shared analyst-facing rollup. scope=
identity drives /ttp/by-identity/{uuid} (primary, with Navigator
export download); scope=attacker drives /ttp/by-attacker/{uuid}
(per-IP slice). Tactic → technique tree in fixed UKC-aligned order,
counts and confidence-weighted bars. Literal "NO TECHNIQUES
OBSERVED YET" empty state per TTP_TAGGING.md §"UI surface — Empty
state": no spinner, no fallback list.
RuleStateControls.tsx: admin-only rule operational state panel
backed by POST/DELETE /ttp/rules/{rule_id}/state. Server-gated by
require_admin AND client-gated on /config?.role so a non-admin
never sees the controls (per feedback_serverside_ui.md the client
gate is UX, not security — the server returns 403 either way).
Wired into Config.tsx as a new "TTP RULES" admin tab.
Wired TTPsObservedSection into IdentityDetail (above fingerprints)
and AttackerDetail (above TIMELINE). DeckyFleet/PersonaGeneration
vocabulary throughout (logs-section / section-header / btn /
matrix-text / dim-chip).
tsc --noEmit and vite build clean.
The dev-server browser smoke is deferred per the "can't reliably
exercise UI from this harness" reality — typecheck + build is the
correctness gate, not feature verification.
Add BaseRepository.list_ttp_decky_phases(identity_uuid) returning
per-decky tag observations as (decky_id, tactic, created_at_ts) rows
ordered by creation time. Rewrite from_identity_row() to project
tactic → UKCPhase via tactic_to_ukc_phase and populate the four
phase-handoff maps (first/last_phase_per_decky,
first/last_seen_per_decky) so combined_campaign_weight finally lights
up on real DB rows — not just synthetic fixtures.
ConnectedComponentsCampaignClusterer.tick() pulls each active
identity's per-decky phase observations before projecting features.
Repo failures are non-fatal: a partial repo falls back to the empty
phase-handoff signal (legacy behavior) so the worker stays up.
tests/clustering/test_ttp_phase_handoff.py pins the production-row
pair clearing CAMPAIGN_EDGE_THRESHOLD on a C2 → DISCOVERY hand-off —
the trip-wire that says the whole project paid off.
commands_by_phase_on_decky itself stays empty on the production path:
it is consumed only by the synthetic-fixture similarity surface, and
the phase-handoff edge does not use it. Synthetic fixtures still
populate it directly via from_synthetic_identity.
Inner loop drains a per-process asyncio.Queue populated by one pump
task per topic in _TOPICS, dispatches each event through
CompositeTagger, persists via repo.insert_tags(), and publishes
ttp.tagged + per-technique ttp.rule.fired.<id> only when the insert
returned a non-zero rowcount.
CompositeTagger seeded with all six lifters (Behavioral, Intel,
CanaryFingerprint, Email, Identity, Credential).
Loop-prevention invariant from TTP_TAGGING.md §"Bus topics" enforced:
N replays of the same upstream event publish exactly one ttp.tagged
event. test_worker_bus covers both the direct invocation path and
the idempotency replay path.
Intel catch-up via attacker.session.ended is intentionally deferred
to E.3.14b — needs a session→intel join the repo doesn't expose yet.
IdentityLifter owns lifter:identity_* — currently R0003 (password
spraying). CredentialLifter owns lifter:credential_* — R0001 generic
auth brute, R0002 password guessing, R0004 credential reuse, R0005
valid-account use, R0006 default credentials.
YAMLs R0001/R0002/R0003/R0005/R0006 had their match.kind normalised
to fit the lifter prefix scheme — the design doc's promised "YAMLs
normalised in a separate refactor commit" lands here.
Identity-rollup tags null out attacker_uuid on emit so the worked-
example invariant holds (the tag belongs to the Identity, never to
one member IP).
Tests: test_identity_lifter.py + test_credential_lifter.py cover
each predicate's positive/negative path, state modulation
(disabled/clipped/expired), source-kind gating, and idempotent
replay. test_lifter_absence and test_lifters updated for the new
ctor signature.
Records the RuleIndex extraction prerequisite, the lifter:<owner>_
prefix routing convention, per-provider technique fan-out logic for
intel rules, the canary identity-merge guard rail, and the email PII
allowlist + R0042 simhash requirement.
SMTP message-level technique tagger per Appendix A.6: open relay abuse
(rcpt_count + foreign From), mass phishing (rcpt_count + body simhash),
phishing-kit X-Mailer, IDN/punycode URL, sender masquerade composite
(From/Return-Path/DKIM/SPF), malicious attachment (macro/.lnk/.iso/.img/
hash match), BEC subject+body composite, encoded payload in body.
PII discipline (TTP_TAGGING.md §'Hard parts §6') is enforced at the
lifter layer via _filter_evidence(): emitted TTPTag.evidence is
restricted to the EmailEvidence-allowed allowlist (body_sha256,
matched_headers — names only, rcpt_domain_set — domains only,
attachment_sha256s, rcpt_count) plus PII-safe match discriminators
(matched_kit, matched_trigger, matched_url_host, etc). Raw addresses,
raw body bytes, full URLs, and decoded base64 previews NEVER appear in
evidence — defense-in-depth over the YAML evidence_fields hint.
Tests: tests/ttp/test_email_lifter.py per-rule positive + negative +
PII allowlist guard + state modulation. tests/ttp/rule_precision/
test_email_rules.py xfail flipped to real precision (R0041-R0048
H-band ≥95%). Corpus rows updated to acknowledge that R0045 (masquerade)
co-fires with R0041 / R0047 when the sender-masquerade signals are
present alongside open-relay or BEC patterns — overlap is by design,
not a precision bug.
Browser-payload derivations per Appendix A.9: navigator.webdriver flag,
canvas/audio/WebGL automation hash matches (Puppeteer/Playwright/
Selenium/curl-impersonate), WebRTC IP leak, TZ/language vs source-IP
geo mismatch, navigator.platform vs userAgent vs WebGL renderer
inconsistency.
Evidence shape pinned to CanaryFingerprintEvidence (metric +
matched_signature) — raw fingerprint blobs (canvas hashes, full UAs,
navigator.platform values) explicitly NOT carried into TTPTag.evidence
per TTP_TAGGING.md §'Hard parts §7' (enrichment vs tag boundary). The
identity-merge guard rail is preserved: composite fp.id matches across
IPs are NOT a TTP, so no rule fires on the bare hash.
Tests: tests/ttp/test_canary_fingerprint_lifter.py per-rule positive +
negative + evidence-shape guard + state modulation.
tests/ttp/rule_precision/test_canary_rules.py xfail flipped to real
precision (R0049/R0050/R0051/R0053 H-band ≥95%; R0052 M-band ≥80%).
Per-provider verdict translator for AbuseIPDB, GreyNoise, Feodo Tracker,
and ThreatFox per Appendix A.10. Each rule's predicate inspects payload
fields produced by the enrich worker (no DB I/O, no decnet.intel.*
imports — E.2.7 decoupling guard preserved). AbuseIPDB confidence is
scaled by abuse_confidence_score / 100; categories drive per-technique
fan-out. R0058 aggregate-bump is a no-op in v0 (cross-tag bump deferred
to E.3.14 worker bootstrap).
Per-provider null tolerance is the steady state — a missing provider
column produces zero tags from that rule, never an error.
Tests:
- tests/ttp/test_intel_lifter.py — per-provider positive + negative +
state modulation + decoupling source-import guard.
- tests/ttp/rule_precision/test_intel_rules.py — xfail flipped, real
precision driven over seed_intel.jsonl (R0054-R0057 H-band ≥95%;
R0058 skipped as bump-only).
- tests/ttp/test_lifter_absence.py — IntelLifter all-populated test
flipped from xfail-strict to real assertion with realistic payload.
- tests/ttp/test_lifters.py — partial-null xfail flipped to real
assertion.
Reads pre-shaped session aggregates from TaggerEvent.payload and emits
techniques per Appendix A behavior tables. Per-rule predicates dispatch
on match.kind (lifter:behavioral_<name>); the lifter holds its own
RuleIndex watching the same RuleStore as the engine, so disable / clip /
TTL state reaches lifter-bound rules through the same atomic-swap path.
R0032/R0036/R0037/R0040 YAMLs had over-escaped regex strings (\\
instead of \\) — fixed in place.
Factory wired so default get_tagger() returns CompositeTagger with
BehavioralLifter shipped; remaining three lifters (E.3.10-E.3.12) land
in subsequent commits.
E.2.6 contract preserved via TolerantTagger: empty payload steady-state
yields [] with zero ERROR records. Disabled / clipped / expired state
verified.
E.3.9.1 prerequisite. Rules R0031-R0040 now use lifter:behavioral_*,
R0041 (open_relay) uses lifter:email_open_relay; the rest of the email,
canary, and intel cohorts already conformed. Each lifter at E.3.9-E.3.12
will claim its rules via str.startswith('lifter:<owner>_'), keeping the
ownership routing explicit and trivially extensible.
R0001-R0006 / R0030 lifter:* rules are E.3.13 (Identity/Credential)
territory and stay as-is.
E.3.9.0 prerequisite for the per-source lifters (E.3.9-E.3.13). The
dispatch index, install/evict/apply_change atomic-swap protocol, and
state-modulation helpers (is_active / apply_ceiling) move out of
rule_engine.py into _rule_index.py and _state.py. RuleEngine wraps a
RuleIndex; back-compat shims preserve _by_kind / _by_rule / _install
attribute access for tests poking at the dispatch internals.
Lifters in E.3.9-E.3.12 will each hold their own RuleIndex, watching
the same RuleStore via subscribe_changes() fan-out. Hot-reload
semantics (disable / clip / TTL via set_state API) now reach
lifter-bound rules through the same atomic-swap path the engine uses,
not a future composite-rebuild compromise.
5 YAMLs for the intel-verdict cohort per Appendix B / A.10:
AbuseIPDB category mapping, GreyNoise classification, Feodo
Tracker hit, ThreatFox IOC type, aggregate-malicious bump-only.
IntelLifter (E.3.10) consumes by rule_id and tolerates absence
silently (null provider column → no tag).
R0058 is the meta bump-only rule — emits a single confidence=0.0
sentinel so it validates and surfaces in the catalogue, but the
repository's sub-0.3 drop ensures no fresh tag persists if the
fanout fires accidentally. test_intel_rules.py pins that
zero-confidence invariant.
Marks E.3.8 done in development/TTP_TAGGING.md with the cohort-
split summary.
5 YAMLs for the canary-fingerprint cohort per Appendix B / A.9:
navigator.webdriver flag, automation canvas/audio/WebGL hash match,
WebRTC IP leak, TZ/lang vs geo mismatch, platform inconsistency.
CanaryFingerprintLifter (E.3.11) consumes by rule_id.
test_canary_rules.py: YAML-present + inert-in-v0 + xfail(strict)
gated on E.3.11.
10 YAMLs for the behavioral / cross-event cohort per Appendix B:
beaconing, data destruction, ransom note, web exfil, DB mass-read,
credentials-in-files, k8s SA token harvest, Docker host escape,
LLMNR poisoning, TFTP router-config retrieval.
Every rule is lifter-bound (BehavioralLifter / IdentityLifter) —
the v0 RuleEngine cannot count, aggregate, or compose cross-event
signals, so these YAMLs declare the technique mappings the lifter
will consume by rule_id at E.3.9. Their match specs use a
'kind: lifter:*' shape inert to the regex matcher.
test_behavioral_rules.py asserts each YAML compiles, none fire
from the v0 engine (FP regression guard against a YAML drifting
into a regex), and an xfail(strict=True, reason='impl phase E.3.9')
precision case that will flip green when the lifter lands.
30 YAMLs for the shell/command rule cohort per Appendix B (rules/ttp/).
Splits into engine-active (R0007-R0029, regex on command_text /
raw_url / user_agent) and lifter-bound (R0001-R0006, R0030 — the
v0 RuleEngine cannot count auth attempts, do identity rollups, or
parse fingerprint blobs; the BehavioralLifter / IdentityLifter /
CredentialLifter consume them by rule_id at E.3.9 / E.3.13).
test_command_rules.py asserts:
- every R000N has a YAML that compiles
- lifter-bound rules NEVER fire from the v0 engine (regression
guard against a YAML drifting into a regex match.spec)
- engine-active rules meet their Appendix-C precision target
against the seed corpus (≥0.95 high-conf, ≥0.80 medium)
Conftest fixes: precision_engine moved to module-scope so module-
scope precomputed dispatch fixture (fired_by_label) can request it;
_RULES_DIR path bumped from parents[2] to parents[3] so the loader
resolves the project root regardless of pytest cwd; make_event
synthesizes attacker_uuid so TTPTag's anchor invariant is satisfied.
Seed corpus broadened: positive examples for every regex rule plus
6 negative examples across innocuous shell verbs (ls, echo, cd, ps,
df, free) so FPs surface in precision rather than passing vacuously.
Sub-step preceding the rule-pack commits per TTP_TAGGING.md:2967.
Adds the per-rule precision suite scaffolding under
tests/ttp/rule_precision/:
- conftest.py: precision_engine fixture (RuleEngine populated from
./rules/ttp/), corpus_loader (real → seed → empty fallback),
precision_for() helper for TP/FP accounting.
- _build_corpus.py: extractor for a real prod corpus pull. Mandatory
--exclude-ip / DECNET_TTP_CORPUS_EXCLUDE_IPS — operator IPs never
end up in the committed exclusion list. Pulls both 'command' and
'unknown_command' event types.
- corpus/seed_*.jsonl: synthetic seed rows for each cohort so the
harness exercises in clean checkouts.
- corpus/*.jsonl (operator-built) is gitignored.
- test_corpus_loads.py: sentinel that every seed file parses.
Implements the rule engine body left empty at contract phase: evaluate()
dispatches by source_kind through self._by_kind, runs the rule's match
spec against event.payload, and emits one TTPTag per emits entry.
watch_store() loads the initial corpus from RuleStore.load_compiled,
then drains subscribe_changes, applying definition changes via
single-statement dict assignment (atomic swap, GIL-atomic to readers)
and state changes via NamedTuple._replace on the existing CompiledRule.
Why: with the FS + DB stores in place (E.3.5/E.3.6), the engine is the
last piece of the rule plane. Lifters (E.3.9–E.3.13) consume the
engine; the worker bootstrap (E.3.14) wires watch_store into the
asyncio event loop. After this commit a CompositeTagger constructed
with a RuleEngine + a populated rules dir will produce real tags.
Notes:
- CompiledRule.emits extended to 4-tuple
(technique_id, sub_technique_id, tactic, confidence). Tactic + confidence
ride per-emit so a single rule can carry multiple precision targets
(the "one event maps to many techniques" property). Compile helpers in
both backends extract them from the YAML emits dict; missing tactic
or confidence is a deploy-time error.
- v0 match operator is "pattern" (regex). The field defaults per
source_kind (command_text / raw_url / subject / verdict / …) and is
overridable via match.field. Future ops (contains, equals, in_set)
extend _match_event without touching the engine surface.
- Confidence model: rules with state="clipped" + confidence_max set
cap the per-emit confidence downward; clipped is a soft suppress, not
a hard skip. Disabled rules are skipped wholly; expires_at past is
re-checked at evaluate as defense-in-depth (the store auto-reverts,
but a racing read between expiry and revert must not fire the rule).
- _span(name, **attrs) helper in engine + both stores short-circuits on
decnet.telemetry._ENABLED — matches the project's @traced /
wrap_repository zero-overhead-when-disabled pattern instead of relying
solely on the no-op tracer indirection.
- Late-bound tracer (telemetry.get_tracer called per-span, not at
module load) so test_tracing's monkeypatch reaches the production
code path.
xfails flipped: tests/ttp/test_rule_engine.py multi-emit fan-out +
rule_version-collision-via-engine; tests/ttp/test_multi_mapping.py
N×M engine fan-out + idempotent replay; tests/ttp/test_tracing.py
ttp.eval span hierarchy + ttp.rule.fire span attributes.
Tests: 214 passed, 19 xfailed (gated on E.3.8 lifters / rule pack /
worker bootstrap).
mypy: clean on prod code; pre-existing test-stub arg-type warnings
unchanged.
Implements the DB-backed rule store body left empty at contract phase:
load_compiled reads from ttp_rule + ttp_rule_state; get_state /
set_state hit ttp_rule_state with the same expires_at auto-revert and
bus-event semantics as the FS backend; subscribe_changes returns a
per-subscriber queue. State persists across process restarts — the
swarm property the FS backend deliberately doesn't have.
Also lands two swarm-mode helpers:
- sync_from_filesystem(fs_store) — master-side, subscribes to a
FilesystemRuleStore and projects each RuleChange onto a ttp_rule
upsert/delete.
- tail_db(poll_interval) — worker-side, watermark poll over
ttp_rule.updated_at; emits RuleChange("definition", ...) for each
row that moved.
Why: swarm mode needs rule definitions and operator state to
propagate across hosts. The filesystem backend (E.3.5) was the
single-host-dev variant; this one survives restart and serves N
workers from a shared DB.
Notes:
- DatabaseRuleStore() with no args lazy-inits an in-memory SQLite
repo so the conformance fixture works without test plumbing. In
production the worker bootstrap (E.3.14) passes an explicit repo.
- The conftest.py rule_store fixture became async (pytest_asyncio),
per-backend creates/initializes a SQLite repo for the DB run.
- Adds a `seed_rule(store, rule_id, yaml)` helper to bridge backend
semantics: drop a YAML file (FS) vs insert a ttp_rule row (DB).
Used by the parametrized load_compiled conformance test.
- Late-bound _tracer() in both backends (was module-level get_tracer
binding) so test_tracing's monkeypatch of decnet.telemetry.get_tracer
actually affects span output.
xfails flipped: tests/ttp/store/test_database.py set_state-writes-to-
ttp_rule_state + filesystem-to-DB sync; tests/ttp/store/test_conformance.py
DB-side load_compiled / set_state isolation / round-trip / per-rule
fan-out / expired-state revert / set_state failure / get_state default
(was xfail-only-on-DB); tests/ttp/test_tracing.py set_state span
hierarchy.
Tests: 208 passed, 25 xfailed (gated on E.3.7 + lifters).
mypy: clean on all touched files.
Implements the filesystem-backed rule store body left empty at contract
phase: YAML parse + Pydantic validation, asyncinotify watch over
./rules/ttp/, in-process state cache with auto-revert on expires_at,
and a subscribe_changes() async iterator yielding one RuleChange per
per-rule edit. Bus topic builders ttp_rule_reloaded / ttp_rule_state
ship alongside.
Why: the rule plane needed a store before the engine (E.3.7) could
consume RuleChange events and atomically swap compiled rules into its
dispatch index.
Notes:
- Linux-only by construction (asyncinotify wheel gated by sys_platform
marker; FilesystemRuleStore.__init__ raises on non-Linux).
- Filename allowlist is the FIRST check on every inotify event.
- Content-hash dedup so a single write firing IN_CREATE + IN_CLOSE_WRITE
produces exactly one RuleChange.
- All compile work serializes on a single asyncio.Lock.
- Subscribers register their queue eagerly so events fired between
subscribe_changes() and the first __anext__() are buffered.
xfails flipped: per-save-style + filter-ordering + atomic-swap in
test_filesystem.py; load_compiled / set_state isolation / round-trip /
per-rule fan-out / expired-state revert / set_state failure semantics
in test_conformance.py (FS side; DB side stays xfail until E.3.6);
malformed-YAML compile-time check in test_rule_engine.py.
Tests: 197 passed, 35 xfailed (gated on E.3.6 / E.3.7 / lifters).
mypy + bandit: clean on all touched files.
Wiki update for the per-rule reload + state-change topics lands in a
matching wiki-checkout/Service-Bus.md edit (separate repo).
Five GET rollup endpoints (techniques, by-identity, by-attacker,
by-campaign, by-session) and the Navigator export (fleet +
per-identity) now call into the TTPMixin methods. Rule catalogue
endpoint still returns [] — backed by the RuleStore which lands
at E.3.5/E.3.6.
Dialect-split: portable rollup queries on TTPMixin; bulk insert with
ON CONFLICT DO NOTHING / INSERT IGNORE in the per-dialect repos.
Confidence-floor (< 0.3) drop applied at mixin layer before the
dialect hook. BaseRepository now declares the six TTP methods abstract.
Tests in tests/web/db/test_ttp_repo.py flipped from pytest.fail stubs
to real dual-backend behavioral tests; tests/ttp/test_confidence.py
drop-below-floor xfail removed.
Each section gets a Status: ✅ done block summarising what's GREEN
today vs xfail-gated and noting any divergence from the doc's
original wording (E.2.9 lossy observable phases; E.2.13 db_backends
fixture landed alongside; E.2.14a Jaeger-skip + tracing-enabled
plumbing; E.2.14b NamedTuple AttributeError vs FrozenInstanceError).
Session-scoped autouse fixture in tests/ttp/conftest.py sets
DECNET_DEVELOPER_TRACING=true and forces decnet.telemetry._ENABLED
so the no-op tracer doesn't silently swallow emitted spans. The
span_exporter fixture also monkeypatches decnet.telemetry.get_tracer
so production code under test lands spans in the in-memory
exporter. Tracing tests skip when DECNET_OTEL_ENDPOINT (default
localhost:4317) isn't reachable so the dev loop stays green
without lying about coverage.
In-memory span exporter fixture wired to a per-test TracerProvider
(OTEL global is locked once set, so each test gets its own).
ttp.eval / ttp.lifter.{name} / ttp.rule.fire / ttp.rule.state.change
hierarchy + no-PII canary battery xfail-gated behind E.3.5–E.3.13.
Hypothesis property: N rule_ids × M technique_ids on one event yield
N×M distinct tag UUIDs. Worked example pinned: one rule emitting
(T1110, None) and (T1078, None) → two distinct UUIDs. Engine-level
fan-out + replay xfail-gated behind E.3.7.
Pre-target phases (RECONNAISSANCE/RESOURCE_DEVELOPMENT/WEAPONIZATION/
SOCIAL_ENGINEERING) and observable-but-unmappable phases (EXPLOITATION/
PIVOTING/OBJECTIVES, UKC-only concepts ATT&CK lacks tactics for) are
pinned as lossy via _LOSSY_INVERSE_REFERENCE so a future contributor
cannot 'fix' the asymmetry without tripping the suite.
Adds decnet/ttp/store/ subpackage:
- base.py: RuleState frozen dataclass, RuleChange NamedTuple, RuleStore ABC
- factory.py: get_rule_store() reading DECNET_TTP_RULE_STORE_TYPE
- impl/filesystem.py: FilesystemRuleStore with sys.platform=='linux'
fail-fast guard, allowlist filename regex, raw inotify mask bits
(lib import deferred to E.3 so contract phase compiles without the
asyncinotify dep installed)
- impl/database.py: DatabaseRuleStore stub (no platform guard)
TTPRule + TTPRuleState SQLModels were already shipped at E.1.1; this
commit closes the type-only TYPE_CHECKING forward-ref in
rule_engine.py via real runtime imports through the new package.
Empty NotImplementedError bodies; the SQL lands at E.3 implementation.
Mixin composed onto SQLModelRepository alongside the existing domain
mixins. Dialect-specific INSERT-OR-IGNORE syntax overrides land in
the per-backend subclasses at E.3 per the dual-DB-backend convention.
Mounts /api/v1/ttp/* with empty-list / empty-Navigator responses.
GET endpoints viewer-gated; POST/DELETE /rules/{rule_id}/state
admin-gated server-side. POST parses JSON manually so a malformed
body returns the documented 400 (per feedback_schemathesis_400).
Drops xfail-strict markers from E.2.8 tests now that the router is
mounted; 26 tests pass against the contract handlers.
Third and fourth TTP-tagging contract commits, plus a scoped subset
of the E.2.4 conformance tests covering the contract surface shipped
here (full hypothesis-fuzz suite still lands with E.2.4).
E.1.3 — decnet/ttp/base.py
- TaggerEvent NamedTuple: source_kind, source_id, attacker_uuid,
identity_uuid, session_id, decky_id, opaque payload.
- Tagger(ABC) with abstract async tag(); class-level name and
HANDLES: frozenset[str] (default empty so a misconfigured subclass
is loudly idle, not loudly noisy).
- TolerantTagger(Tagger): concrete tag() wraps abstract _tag_impl()
in try/except Exception (deliberately not BaseException — so
KeyboardInterrupt / SystemExit / asyncio.CancelledError propagate
and the worker can shut down cleanly). Swallowed exceptions log
at WARNING with exc_info, never ERROR — absence is the steady
state, not a bug. Subclasses override _tag_impl, never tag — the
tolerance contract is enforced in the base class, not on trust.
- KNOWN_SOURCE_KINDS: Final[frozenset[str]] enumerating every
source_kind a producer is allowed to emit. Closed-by-enumeration
at the runtime layer; the composite tagger keys its WARNING/INFO
bridge off this constant to surface the silent-drop trap from
the design doc (lines 160–195).
E.1.4 — decnet/ttp/factory.py
- get_tagger() reads DECNET_TTP_TAGGER_TYPE (default 'composite');
unknown values raise ValueError with the known-list. Mirrors
decnet.intel.factory and decnet.clustering.factory.
- _KNOWN = ('composite',). Per-lifter classes (E.1.6) are children
of the composite, not standalone tagger types.
- CompositeTagger(Tagger): pre-computes a dict[str, list[Tagger]]
dispatch index from each lifter's HANDLES; fans events out
concurrently with asyncio.gather and concatenates results.
Empty lifters=[] is the legal contract-phase state — E.1.6
wires the real lifters in.
- Unhandled-event observability: source_kind in KNOWN_SOURCE_KINDS
but no lifter claims it -> WARNING once per kind per process
(missed E.1.6 update). Unknown kind -> INFO once per kind per
process (future-feature telemetry, by design). Per-process dedup
via plain set; E.1.6 may swap in a proper rate-limiter once
production traffic shapes are known.
Tests — tests/ttp/test_base.py, tests/ttp/test_factory.py
- Tagger / TolerantTagger abstractness, missing-tag-impl rejection,
WARNING-not-ERROR log level, propagation of KeyboardInterrupt /
SystemExit / asyncio.CancelledError.
- Factory env-var routing, unknown-name ValueError, dispatch-index
correctness, only-claiming-lifter invocation, WARNING-once for
known-but-unclaimed kinds, INFO-once for unknown kinds, result
concatenation across lifters.
Mypy clean under .311/bin/mypy --ignore-missing-imports.
Second TTP-tagging contract commit. Constants only — no publishers,
no subscribers, no tests. (E.2.3 ships the bus-topic naming tests.)
- New roots: EMAIL, TTP.
- New leaves: EMAIL_RECEIVED ('received', single-token under EMAIL),
TTP_TAGGED ('tagged'), TTP_RULE_FIRED ('rule.fired'),
TTP_RULE_SUPPRESSED ('rule.suppressed'). Per-rule reload + state
topics ship with the RuleStore (E.1.11) — co-located with
producer.
- New builders: email_topic(event_type), ttp(event_type),
ttp_rule_fired(technique_id). The ttp_rule_fired builder validates
technique_id as a single segment so sub-techniques like T1110.001
are rejected at construction; topic key is the parent technique,
sub_technique lives in the payload.
- email_topic is named with the _topic suffix to avoid shadowing the
Python email stdlib at import sites that pull both.
- TTP_TAGGING.md E.1.2 entry corrected: the spec referenced
'ATTACKER_ENRICHED' but the actual constant is
ATTACKER_INTEL_ENRICHED ('intel.enriched'). The existing constant
covers the design intent (TTP intel_lifter wakes on
attacker.intel.enriched). No rename — would break every existing
subscriber.
Wiki update for the four new topics ships in a sibling commit in
wiki-checkout (separate repo per project layout).
First contract commit of TTP tagging. Shapes only — no behavior.
- TTPTag SQLModel: deterministic UUIDv5 PK; (source_kind, source_id)
discriminated provenance; nullable attacker_uuid + identity_uuid
with ON DELETE CASCADE; native sqlalchemy.JSON evidence column;
required attack_release; CheckConstraint('attacker_uuid IS NOT
NULL OR identity_uuid IS NOT NULL'); composite indexes for the
primary query patterns (identity_uuid+technique_id,
attacker_uuid+technique_id, technique_id+created_at); __init__
guard raising ValueError with both anchor names in the message
(belt-and-braces for MySQL <8.0.16 where CHECK is silent).
- compute_tag_uuid(): RFC-4122 UUIDv5 over the six tag-identity
fields under a fixed _TTP_TAG_NS. Pure, deterministic, replay-safe.
- Per-source_kind evidence TypedDicts (CommandEvidence,
IntelEvidence, EmailEvidence, CanaryFingerprintEvidence) — PII
rule lives in the type: EmailEvidence has no field for raw rcpt
addresses or body bytes.
- TTPRule + TTPRuleState tables for the DatabaseRuleStore (E.1.11).
- All symbols re-exported from decnet.web.db.models per the
package's existing convention.
Tests for invariants (CHECK behavior, evidence round-trip across
SQLite+MySQL, idempotency property, init-guard ordering) land in
E.2.1/E.2.2 with xfail-strict markers per Appendix E discipline.
Pre-implementation spec for the TTP-tagging worker. Defines the
ATT&CK-canonical vocabulary, schema (ttp_tag + ttp_rule[_state]),
bus topics, worker shape, lifter layering (rule-based v0,
behavioral/intel/email v0.5, sigma/biometric later), confidence
model, API surface, UI surface, observability, performance targets,
and a CDD plan (Appendix E) that splits contracts from tests with
xfail discipline so CI stays green between steps.
- Add _MixinBase abstract class to _helpers.py: declares _session(),
_deserialize_attacker(), _assert_pending(), _check_and_bump_version(),
and list_running_topology_deckies() so mypy can see cross-mixin contracts
- Add _require(val, msg) helper for narrowing T | None → T
- Inherit _MixinBase in all 26 leaf mixin classes
- Wrap SQLAlchemy column method calls (.is_(), .like(), .notin_(), .in_(),
.contains()) with col() from sqlmodel — fixes attr-defined false positives
caused by pydantic plugin typing class-level fields as Python value types
- Wrap select(Model.field) with select(col(Model.field)) for column projections
- Add pyproject.toml [[tool.mypy.overrides]] to disable arg-type in
sqlmodel_repo.*: pydantic plugin resolves .where(Model.field == v) as
where(bool), a false positive; call-arg still catches real argument errors
- Remove 9 stale # type: ignore comments (logging, helpers, credentials)
- Fix telemetry.py traced() overload no-redef + misc
- Fix logs.py datetime/str operator and nullable PK comparison with col()
- sqlmodel_repo/ now has 0 mypy errors
syslog_bridge.py: base64.binascii is not a public mypy-visible attribute;
import binascii directly and reference binascii.Error at the except clause.
Propagated to all 26 template subdirectory copies (all were drift-free).
ntlmssp.py: `principal = username or None` widened the type to str | None
for no runtime reason — _decode_str() always returns str. Drop the `or None`.
Propagated to smb/ and rdp/ copies.
762 → 722 mypy errors (-40).
Replace repo: BaseRepository with a structural TopologyRepository protocol
in persistence.py and allocator.py. All read methods now return typed DTOs
(TopologySummary, LANRow, DeckyRow, EdgeRow) instead of raw dicts, eliminating
silent field-shape regressions across the topology subsystem.
TopologySummary gains email_personas and language_default so api_personas.py
can continue reading those fields via attribute access. hydrate() converts
DTOs to dicts before passing to _backfill_decky_configs, keeping the mutable
working-state function dict-based at its boundary. All production callers
(router handlers, mutator, CLI, heartbeat) migrated from dict/get access to
attribute access. 134 tests pass.
MutationRow.op was str despite _MUTATION_OPS existing; Topology.mode/status,
TopologyDecky.state, TopologyMutation.op/state carried valid values only in
comments; deferred json import had no justification.
- Promote _MUTATION_OPS before table classes so table fields can reference it
- Add sa_column=Column(String) on each Literal-annotated table field to satisfy
SQLModel 0.0.38 column-type inference
- Move import json to module top; remove deferred import inside _decode_json_payload
- MutationRow.op: str -> _MUTATION_OPS
[swarm] swarmctl-host → DECNET_SWARMCTL_HOST so operators set the bind
address once in decnet.ini; `decnet swarmctl` and the systemd unit both
resolve it via envvar — no --host/--port pinned on ExecStart.
ApiError: defined once in utils/api.ts, replaces 9 ad-hoc anonymous casts
across MazeNET, Inspector, DeckyFleet, SwarmHosts, Webhooks, PersonaGeneration,
ServiceConfigFields, CanaryTokens.
hex4 renamed to tempIdSuffix — the name now matches the comment that already
explained its purpose.
NET_GRID_{W,H,GAP,COLS} extracted from inline magic numbers to module-level
constants in MazeNET.tsx.
onPaletteDrop (130-line useCallback) split into three module-level handlers
(_dropNetwork, _dropArchetype, _dropService); the callback becomes a 10-line
router.
The 17-line block comment at _RULES was prose covering for a design wart.
The explanation belongs on the function itself — moved there and condensed.
_RULES now has a 2-line pointer instead of an essay.
topology_id[:8] appeared twice with no explanation. 8 chars is the
git short-SHA convention; collision-safe within a single deployment's
network namespace.
fix(generator): correct service pool count in _SVC_MIN/_SVC_MAX comment
BLE001 is not in ruff.toml select (F/ANN/RUF/E/W only); the suppressions
were whispering apologies to a linter that wasn't listening. Generator
comment now cites the actual ~28-entry non-singleton service pool.
apply() was an 85-line function handling hash verification, validation,
superseding teardown, bridge/compose provisioning, and store persistence.
Extracted _check_hash_and_validate(), _teardown_superseded(), and _materialise()
so each step is independently readable and testable.
_take_ip and _new_decky were closures capturing outer-scope state. Promoted to
module-level with explicit parameters. seen_service_pairs name makes the intent
clear — it prevents the same service frozenset from being assigned repeatedly.
The original except Exception silently disabled port collision detection for
any runtime error — not just a missing package. Now only ImportError degrades
gracefully; real psutil failures propagate.
_host_set is computed once in __init__ — reserve() and is_free() were rebuilding
the full host frozenset on every call. BaseRepository already existed; the Any
annotations were just never updated.
await inside a threading.Lock yields to the event loop while the OS
thread still holds the lock — potential deadlock under FastAPI thread
pool dispatch. asyncio.Lock is the correct primitive for async
critical sections. Also fixed stale diurnal.py docstring that had the
delegation direction backwards.
_parse_weights was silently dropping content_class values that don't
belong on their target list with no operator feedback. Changed it to
return (weights, dropped), apply_payload to collect and return all
dropped names, and put_config to include dropped_entries in the
response when non-empty.
The initial stat and read happened without a lock between them. A file
change mid-window stored the mtime of the pre-change stat against the
post-change content, suppressing the next reload. Re-stat after
read_text; fall back to the pre-read stat only on OSError.
The persona arg was typed Any to avoid a circular import. Added a
TYPE_CHECKING guard to import EmailPersona annotation-only so mypy
has the type without a runtime import cycle.
get_config was calling planner.apply_payload on every GET request, racing
concurrent reads on module-level globals. Added a _hydrated flag + lock
so DB hydration runs at most once per process lifetime; put_config marks
it done too. Test fixture resets the flag between tests.
Concurrent PUT requests could observe a half-updated planner between
the four sequential global assignments. Added _planner_lock so the
rebind is atomic; same lock wraps reset_to_defaults.
personas.in_active_hours was discarding the minute component of the
active-hours window, making "09:30-17:45" behave as "09:00-17:00".
Rewrote it to delegate to diurnal.in_work_hours (which uses full
minute arithmetic) and updated the scheduler caller to pass the full
datetime instead of now_dt.hour.
ServiceNotFoundError (→ 404) and ServiceConflictError (→ 409) replace the
"not found" / "already on" / "not on" substring checks in _map_mutation_error;
base ServiceMutationError still maps to 422. Fixes three pre-existing test
status-code assertions (201 vs 200 on POST endpoints).
Pure tarball construction (_build_tarball, _render_*, _iter_included,
_SYSTEMD_UNITS) moved to decnet/swarm/bundle_builder.py — no FastAPI
dependency, independently testable. EnrollBundleRequest/Response moved
to decnet/web/db/models/swarm.py alongside the other swarm DTOs.
Router drops from 504 to 260 lines; keeps only the in-memory token
registry, sweeper, and endpoints.
MailDrawer was reading fields.date / from_addr / message_id —
all wrong; actual log field names are date_hdr, from_hdr,
message_id_hdr, to_hdr. The mail table in AttackerDetail
showed only DECNET capture time and used from_addr instead
of from_hdr. Add a DATE (attacker) column so the attacker-
supplied Date header (including timezone) is visible at a
glance — useful for correlating campaigns like the Tiscali
run where IPs used distinct TZs (+0800 vs -0700).
Relay-test scripts send minimal DATA with no headers. Without a From:
header the mail client falls back to displaying the envelope sender
(upstream_sender). Inject From: <attacker MAIL FROM> before forwarding
when the message has no existing From: header.
bus.subscribe() is sync and returns an async iterator, not a coroutine.
Awaiting it caused an immediate crash at startup; bus.next_message() does
not exist either. Rewrote _run_smtp_probe_listener to use the standard
pattern: sub = bus.subscribe(...) / async with sub / async for event in sub.
SERVICE_NAME was hardcoded to 'smtp' in server.py; the ingester's probe
publish guard checked service == 'smtp_relay' and never matched.
Read SMTP_SERVICE_NAME from env (default 'smtp'); smtp_relay compose
fragment sets it to 'smtp_relay' so the two services are distinguishable.
The bind-mounted quarantine dir is owned by the host decnet user; the
logrelay process had no write access because the Dockerfile USER directive
pre-applied before the entrypoint could fix permissions.
Run entrypoint as root, chmod 0777 the quarantine dir, then exec the
server under logrelay via su.
Attacker probe emails are now forwarded by the master (realism worker)
rather than inside the MACVLAN container, which has no internet gateway.
- New smtp.probe.pending bus topic: ingester publishes when smtp_relay
message_stored fires; worker subscribes and does the actual delivery
- decnet/orchestrator/drivers/smtp_relay.py: pure-sync forward_probe()
reads the .eml from disk and sends via smtplib on a thread executor
- worker.py: _run_smtp_probe_listener + _handle_probe_pending subtask;
limit enforced via count_probe_relays() (DB-backed, restart-safe)
- bounties.py: count_probe_relays() query on probe_relay bounty type
- fleet.py: get_fleet_decky_by_name() to pull service config from DB
- services/smtp_relay.py: upstream_* and probe_limit fields defined in
config_schema but NOT injected into container env (credentials stay
out of docker env vars)
- ingester.py: stripped of smtplib; publishes probe.pending and exits
- tests: assert upstream keys absent from container environment
forwarded=0 was silent — now fwd_error carries the exception string so
you can see exactly why the upstream refused (auth failure, connection
refused, timeout, etc).
Docker Compose tracks the previous container by internal ID. When that
container was already removed or renamed, --force-recreate fails with
"No such container". Remove by name first so Compose always starts clean.
Override the envelope MAIL FROM with a domain we own when talking to the
upstream relay. SPF passes at the recipient; the attacker's From: header
inside the message body is untouched so they see their own address in their
inbox and believe the relay is real.
Adds probe_forwarded to meaningful event kinds and stores it in the
bounty table as bounty_type=probe_relay with forwarded=true/false, so
the dashboard shows whether the upstream actually accepted the test email.
First SMTP_PROBE_LIMIT messages per source IP are forwarded via a real
upstream relay (SMTP_UPSTREAM_HOST/PORT/USER/PASS) so the attacker's
test email actually lands in their inbox. All subsequent messages from
the same IP get 250 Ok but only hit the quarantine — campaign content
captured, nothing delivered.
The Dockerfile PROMPT_COMMAND logger uses --msgid command, so the MSGID
field arrives as 'command' not '-'. The CMD rewrite block was guarded by
event_type == '-' so it never fired, leaving fields['command'] unpopulated
and cmd_text=None for every SSH session command.
Broaden the guard to also match event_type == 'command' with no existing
'command' field, which covers both the intended (MSGID=NIL) and actual
(MSGID=command) wire formats.
.content-viewport is overflow-y: auto so flex:1 on dash-grid grew to
content height. Fix: dashboard uses height:100% instead of min-height,
and :has(>.dashboard) disables content-viewport scroll only on that
route — all other pages keep their normal scroll.
Sticky thead was floating mid-content when the container auto-scrolled
as new log entries arrived. Pinning scrollTop to 0 on each logs update
keeps the thead at position 0 where it belongs.
Use flex: 1 on dash-grid instead of height: 480px so the panels
consume all remaining space below the stat cards; dash-side uses
height: 100% to fill its grid cell
- Use height: 480px on .dash-grid so both columns are the same height;
side panels split that height via flex instead of their own max-height
- Add LayoutDashboard icon to the DASHBOARD page header
- Dashboard: fix invisible bar at bottom of LIVE FEED by constraining
max-height on the section instead of the inner container; same fix
for side panels
- Page icons: add violet-accent icon beside h1 on all 9 missing pages
(CanaryTokens, RealismConfig, SyntheticFiles, PersonaGeneration,
Attackers, Webhooks, LiveLogs, Topologies, DecoyFleet)
- Attackers filter chips: replace ad-hoc chip buttons with seg-group
tabs (ALL / ACTIVE N / PASSIVE N / INACTIVE N) matching Credential
Vault style; country chips use same seg-group treatment
- Credential Vault: add sortable headers to REUSE tab (LAST SEEN,
PRINCIPAL, KIND, TARGETS, ATTEMPTS); reuses same SortTh pattern
- Bounty: remove CREDENTIALS and PAYLOADS tabs; keep ALL, ARTIFACTS,
FINGERPRINTS; add EMAIL (artifact subtype, filtered client-side)
- DeckyFleet: card click opens inspect side-drawer instead of
auto-filtering (localSearch filter behavior removed)
- Dashboard: LIVE FEED / DECKIES UNDER SIEGE / TOP ATTACKERS panels
now have fixed max-height with overflow scroll instead of growing
- parseEventBody: defensive RFC 5424 header strip so raw syslog lines
from the collector render as k=v pills instead of raw text
- Attackers: search placeholder updated; activity (Active/Passive/
Inactive) and country chip filters added on top of existing IP search
- Credentials + Bounty: sortable column headers (click to asc/desc/clear)
- SwarmHosts + RemoteUpdates: icon extracted from <h1> into flex div
with violet-accent class, matching site-wide Identities pattern
- Swarm.css: fix --panel-border undefined variable → --border so the
title border-bottom line is visible on SwarmHosts and RemoteUpdates
Monkeypatching services_live.add_service had no effect because api_services
already held a local reference to the name. Patch api_services.add_service
and update fake stubs to accept the config kwarg added to the real signature.
Once a fingerprint canary's HTTP beacon passes all 4 validation layers
and the trigger row lands, the token is immediately set to state=revoked
and canary.<id>.revoked is published on the bus. The slug lookup is
tightened to only return planted tokens, so subsequent requests to the
same URL silently return the transparent GIF without persisting anything
(stealth posture preserved). Plain http/dns canaries with no
fingerprint_nonce are not affected.
Changes:
- sqlmodel_repo/canary.py: add state == "planted" filter to
get_canary_token_by_slug so revoked slugs resolve to None
- worker.py: after record_canary_trigger, if parsed_fp survived all
layers and token has a fingerprint_nonce, call
update_canary_token_state("revoked") + publish CANARY_REVOKED; errors
are best-effort (trigger row already landed)
- test_worker_http.py: assert state=revoked in test_fp_valid_nonce_persists;
new test_fp_deregisters_slug_after_valid_hit (second hit records nothing);
new test_plain_http_canary_not_deregistered (env_file stays planted)
Adds per-mint nonce gating, structural shape validation, mint UUID
consistency checks, and a per-(token, IP) rate limiter to the canary
worker so attackers who extract a canary from a decky filesystem cannot
poison fingerprint forensics by replaying or forging ?d= submissions.
Changes:
base.py
fingerprint_nonce: Optional[str] added to CanaryArtifact so generators
can surface the nonce to the cultivator without coupling the generator
directly to DB code.
obfuscator.py
nonce_for(callback_token, mint_uuid): HMAC-SHA256 keyed on
DECNET_CANARY_FINGERPRINT_SECRET, truncated to 16 hex chars.
FingerprintSecretMissing raised at mint time if env var is unset.
render_fingerprint_js() now accepts nonce= and substitutes MINT_NONCE.
fingerprint_payload.js
New MINT_NONCE placeholder. Appended as &k= on all beacon URLs (bare-open,
single-shot, chunked). Using &k= avoids colliding with &n= (chunk total).
fingerprint_html.py / fingerprint_svg.py
Derive nonce via nonce_for() and pass to render_fingerprint_js(). Set
artifact.fingerprint_nonce so the cultivator can persist it.
cultivator.py
Passes fingerprint_nonce into create_canary_token() when present on the
artifact; NULL for all non-fingerprint generators.
canary.py (model)
fingerprint_nonce: Optional[str] = Field(default=None, max_length=16)
added to CanaryToken. None for non-fingerprint tokens.
worker.py
_extract_fingerprint now returns (meta_dict, parsed_fp) tuple.
_record_hit accepts parsed_fp + raw_nonce and runs 4 layers after
token lookup: nonce match, shape check, mint UUID consistency, rate limit.
Each failure sets _fp_invalid_* flag and drops structured _fp.
Trigger row always lands regardless.
tests/canary/conftest.py
Session-scoped autouse fixture sets DECNET_CANARY_FINGERPRINT_SECRET so
fingerprint generator and worker tests work offline.
tests
5 new worker HTTP tests and 2 new generator tests covering each
validation layer.
The fingerprint canaries' obfuscator shells out to a Node helper that
require()s javascript-obfuscator. Without this commit, a fresh
pip install decnet would land the .py modules but not the .js helper /
package.json, and there'd be no documented way to provision Node side.
* pyproject.toml - extend tool.setuptools.package-data to ship
canary/_obfuscate_helper.js, canary/fingerprint_payload.js, and
canary/package.json with the wheel.
* decnet/cli/canary.py - new "decnet canary-install-toolchain"
subcommand. Resolves decnet.canary.__file__'s dir, runs
npm install --omit=dev there, exits non-zero with a clear message
if npm is missing or install fails. Idempotent - safe to call
every API service start.
* deploy/decnet-api.service.j2 - non-fatal ExecStartPre that calls
the new subcommand. Leading '-' so a missing Node toolchain only
degrades fingerprint canaries (loud at mint time) without keeping
the API from booting.
* tests/canary/test_cli.py - registration smoke test, missing-npm
exit path, and a mocked-subprocess test asserting the right argv
and cwd land on npm.
Realism cultivator already has a broad except Exception around
cultivate() in scheduler.py:195-211, so a missing toolchain on a
host running the realism tick degrades to an inert noise file with
no extra plumbing.
The two new fingerprint canary generators existed at the API level
since f64e78f but weren't visible to the realism engine or the
operator-facing dashboard. Threads them through every place that
enumerates canary content classes.
Backend:
* realism/taxonomy.py - two new ContentClass members
(CANARY_FINGERPRINT_HTML, CANARY_FINGERPRINT_SVG); enum is
wire-visible (synthetic_files.content_class column + bus discrim)
so we add at the bottom, never reorder.
* canary/cultivator.py - class-to-generator dispatch, kind mapping
(both http), and default placement paths
(~/Documents/asset_directory.html and network_topology.svg).
* realism/naming.py + bodies.py - _name_canary / _body_canary entries.
* realism/planner.py - added to _DEFAULT_CANARY_CLASS_WEIGHTS and
the _CANARY_CLASSES classification set.
Frontend:
* decnet_web/src/realism/labels.ts - display labels.
* decnet_web/src/components/RealismConfig/RealismConfig.tsx - default
canary weight rows so operators see them in the realism config UI.
* decnet_web/src/components/SyntheticFiles/SyntheticFiles.tsx - added
to the CONTENT_CLASSES allow-list so filter dropdowns show them.
Also: re-applied the nosec B404/B603 markers on canary/obfuscator.py;
the first commit's pre-commit autoformatter stripped them.
Tests: extended tests/realism/test_taxonomy.py's stability assertion
to include the two new values. Full canary + realism suites pass
(362 / 2 skipped).
The fingerprint payload beacons fingerprint data as base64url JSON in
GET query params: ?o=1 for the bare-open beacon, ?d=<blob> for a
single-shot dump, or ?s/i/n/d=<chunk> for chunked dumps. Until now
those params were buried inside request_path; consumers had to parse
the URL themselves.
Worker now extracts them in _extract_fingerprint and merges into
raw_headers under reserved _fp* keys:
* _fp_open — bare-open marker
* _fp — decoded fingerprint dict (single-shot path)
* _fp_sid/idx/total/chunk — chunked metadata + raw base64 (reassembly
is a downstream concern, not the worker's job)
* _fp_decode_error / _fp_oversize — failure markers for trash dumps
Per-chunk size capped at 8KB so an attacker spamming /c/<known_slug>
can't inflate trigger rows indefinitely. Decode failures degrade
gracefully — the trigger row still records the hit, just with a
_fp_decode_error flag instead of structured fingerprint data.
Tests cover the single-shot decode, bare-open flag, chunked metadata,
malformed input, and oversize drop paths.
Two new synthesised-artifact generators that bake the obfuscated
fingerprint payload into plausible-looking decoy files:
* fingerprint_html — a mundane "Internal Asset Directory" page with a
small table of fake hosts; the obfuscated payload is inlined at the
bottom of <body>. Visible content (row pool slice, sync timestamp)
also varies per mint via SHA-256-derived stable ints, so two
extracted canaries don't diff to zero even on the rendered surface.
* fingerprint_svg — standalone SVG with an embedded <script> CDATA
block. SVG <script> only fires for top-level loads / <object> /
<iframe>; <img>-referenced renders are safely inert.
Both derive the mint UUID via uuid.uuid5 from the callback token, so
re-mints are byte-identical (preserving the generator determinism
contract) AND the same token produces the same mint UUID across HTML
and SVG variants — the worker can correlate beacons across artifact
shapes.
Wired into the factory + KNOWN_GENERATORS, default placement paths
under ~/Documents/asset_directory.html and ~/Documents/network_topology.svg
for both linux and windows personas. Tests cover determinism, per-token
divergence, structural validity (DOCTYPE/SVG headers), and that the
beacon URL stays inside the obfuscated string array (not in plaintext).
The two new entries skip in test_generators.py when Node toolchain is
absent so bare CI checkouts still pass.
Adds the load-bearing primitives for obfuscated browser-fingerprinting
canaries. Step 3 (HTML/SVG generators) and step 4 (worker-side
fingerprint ingestion) build on top of these.
* decnet/canary/obfuscator.py - javascript-obfuscator wrapper. Seed
and polymorphic config bits both derive from the callback token, so
output is byte-identical for the same mint (preserving the generator
determinism contract from base.py) and structurally distinct across
mints.
* decnet/canary/fingerprint_payload.js - port of canary-self-test.html
with the rendering UI stripped. Two placeholders (BEACON_URL,
MINT_UUID) substituted before obfuscation. MVP beacon strategy:
bare-open GET pixel first, then base64url-encoded fingerprint as
query params on subsequent GETs (chunked above ~6KB) so the existing
worker records hits before step-4 lands.
* decnet/canary/_obfuscate_helper.js - Node subprocess helper that
reads code+options JSON from stdin and writes obfuscated JS to
stdout. Vendored javascript-obfuscator under decnet/canary/.
* tests/canary/test_obfuscator.py - determinism, per-mint divergence,
template substitution, Node syntax check, error path.
Add/remove/update_config on a fleet decky living on a swarm worker — and on
an agent-pinned topology — used to run the master's local docker-compose only,
which has no containers for the remote decky. The mutation persisted on master
and silently no-op'd on the worker.
- Fleet swarm: lookup DeckyShard.host_uuid; if found, rebuild a single-host
shard from master state and call dispatch_decnet_config — same proven path
as POST /swarm/deploy. Skip local _compose (no containers to touch).
- Topology agent-pinned: call decnet.engine.deployer.resync_agent_topology
(existing helper) to push the latest hydrated blob to the worker.
- Local-only deckies: behaviour unchanged.
- Tests: 5 new in tests/engine/test_services_live_swarm.py covering all
three mutations on a swarm fleet decky (no local _compose, dispatch fires
with the right host's deckies), plus apply=False save-only path (no
dispatch), plus regression that local-only fleet add still runs local compose.
Bus signal `decky.{name}.service_config_changed` keeps publishing as an
audit trail; it is not the propagation trigger.
- DeckyServiceAddRequest gains an optional `config: dict` field, validated
against the service's config_schema before any state mutation (400 on
bad type, no half-written rows).
- Engine: add_service threads `config` into _add_topology_service /
_add_fleet_service, persisting validated cfg to decky_config.service_config
BEFORE compose regen so the first `up -d --build` materialises the env on
the new container. No follow-up apply needed.
- Frontend: shared AddServiceConfigModal — same wizard accordion shape, used by:
* DeckyCard's ADD SERVICE picker (Fleet & MazeNET inspectors via shared component)
* MazeNET Inspector's ADD SERVICE picker
* MazeNET palette drag-drop onto a deployed decky
Empty-schema services short-circuit to a one-click add (no modal flash).
Operator can cancel; errors surface in the modal.
- Tests: add_service config plumbing — persist, drop unknown keys, 400-equivalent
on bad types, back-compat empty-config.
- Drive-by: fix stale repo-method names in test_services_live.py
(create_topology_decky → add_topology_decky, get_topology_decky → list+pick helper,
service.added → service_added topic).
- Declarative config_schema on RDP, Telnet, MySQL, Redis, SMTP, SMTP_Relay
matching the keys each service already reads at compose time.
- TODO marker on the 19 services that accept service_cfg but never read it,
so future contributors know where to plug schemas in.
- Wizard base64-wraps all textarea values at INI emit (DeckyFleet
buildIni); validate_cfg detects the b64: sentinel and decodes back to
UTF-8. Plain raw strings still pass through for direct API submitters.
- HTTPS image entrypoint accepts PEM content or path in TLS_CERT/TLS_KEY:
detects a BEGIN header, writes content to /opt/tls/, and re-exports
the on-disk path so server.py keeps reading paths.
- Tests cover schema/compose alignment for each new service plus
textarea base64 round-trip (incl. UTF-8) and HTTPS PEM end-to-end.
Setting a password, banner or TLS material AFTER deployment forces a
container recreate on every change. The deploy wizard now lets the
operator set service config up-front so the initial build has the
right env from the start.
Mechanics:
- Extracted the schema-driven field rendering out of ServiceConfigForm
into a standalone ServiceConfigFields component (no API/buttons,
just inputs + onChange). ServiceConfigForm now delegates to it.
- Wizard step 2 (CONFIGURATION) renders one accordion block per
selected service; clicking a service reveals its schema-driven
inputs and a 'N set' badge tracks how many overrides are populated.
Removing a service (back to step 1) drops its config so the INI
doesn't carry orphans.
- _buildIni emits one [<prefix>.<svc>] group subsection per service
with at least one override. The INI loader's prefix-matcher
applies it to every ${prefix}-NN decky in the batch, so one block
covers all clones.
- Multi-line string values (PEM textareas etc.) are escaped as \n
on the way into INI; downstream consumers re-expand.
PersonaGeneration.css scopes .info-banner under .persona-gen-root,
which doesn't match elements rendered inside the Modal portal —
so the wizard's CONFIGURATION-step banner I just added rendered
as plain text.
Add a page-unscoped .info-banner rule in DeckyFleet.css with the
same visual treatment (faint bg, violet left rule) so any modal
context picks it up.
The systemd unit grants AmbientCapabilities=CAP_NET_ADMIN so the API
service can program host-side macvlan/ipvlan interfaces without
running as root, but setup_host_macvlan/_ipvlan rejected with euid!=0
before even trying — making web-driven 'decnet deploy' impossible
under the privilege model the unit advertises.
Replace _require_root with _require_net_admin, which reads CapEff
from /proc/self/status and accepts the cap (bit 12) as well as
euid==0. No libcap dep — pure /proc parse.
The CONFIGURATION step had a stale disabled placeholder textarea
("per-service overrides") from before the schema-driven Inspector
landed. Replaced with a one-line info banner pointing at the Inspector,
which is now where per-service config actually lives.
The DEPLOY step's CLI preview was rendering '--archetype custom' when
pickMode==='services', but no such archetype is registered — only the
preset archetypes plus 'services' (free-form list). Drop the
--archetype line entirely in the services-mode preview so the rendered
command reflects what the API actually receives.
Previous CSS lived in DeckyFleet.css only, so when the form rendered
inside MazeNET Inspector the inputs fell back to browser defaults
(white-on-white, oversized labels, mismatched buttons).
New ServiceConfigForm.css ships with the component itself: small
uppercase tracking-1 labels at 0.6rem (matches kvs .k), dark
transparent inputs with violet focus, matrix-green text inside
inputs, custom select chevron, dedicated svc-cfg-btn that visually
mirrors maze-btn.small, password reveal toggle, and a 96px label
column so labels never wrap into the input. Help text drops to
0.58rem dim under the input. Works identically in both surfaces.
The schema useEffect depended on currentConfig, which the parent
passes as a fresh `{}` literal on every render — referentially new
each time, so the effect re-ran and the GET /services/.../schema
hammered the server.
Schema fetch now only depends on serviceSlug; form seeding from
currentConfig moved to a separate effect keyed on JSON-stringified
config so a real change reseeds but referential churn doesn't.
ServiceConfigForm.tsx fetches /topologies/services/{slug}/schema and renders
typed inputs (string/password/int/bool/textarea/enum) with reveal toggles for
secrets. SAVE persists via PUT (no restart); APPLY persists + force-recreates
the service container after a confirm dialog (matches the forwards_l3 pattern).
Mounts:
- DeckyFleet DeckyCard: clicking a service tag toggles the form below the
EXPOSED row, gated on liveServicesEnabled (admin + non-swarm).
- MazeNET Inspector: renders the form above REMOVE SERVICE when a service
is selected on a non-observed decky.
UI test plan is manual — no jsdom test infra in decnet_web yet.
- GET /topologies/services/{name}/schema serves the declared ServiceConfigField
metadata so the Inspector can auto-render forms.
- PUT /(topologies/{id}/)deckies/{decky}/services/{svc}/config persists the
validated dict (DB + compose); container untouched (Save).
- POST /(topologies/{id}/)deckies/{decky}/services/{svc}/apply persists then
force-recreates <decky>-<svc> so the new env takes effect (Apply, destructive).
- New engine helper update_service_config wires both fleet and topology paths
through the existing _persist_fleet_change / _rerender_topology_compose
machinery; emits decky.<name>.service_config_changed on the bus.
ServiceConfigField dataclass + BaseService.validate_cfg coerce/drop submitted
service_cfg dicts against per-service typed schemas. SSH/HTTP/HTTPS now declare
the keys they already read in compose_fragment, so the upcoming Inspector form
has metadata to render from instead of hardcoded inputs per service.
Dashboard's ACTIVE DECKIES (active_deckies in get_stats_summary) counts
TopologyDecky rows where state='running'. No code path was flipping
that state away from the default 'pending', so the count read 0/N
even when every container was running fine — the dashboard was lying.
Two complementary fixes:
1. deploy_topology — after the post-deploy compose ps verification,
reconcile each TopologyDecky.state from the corresponding base
container's docker state. running → 'running'; anything else →
'failed'. Reuses the ps_rows already gathered for the
ACTIVE-vs-DEGRADED status decision; no extra docker hit.
2. apply_add_decky — _materialise_decky_spawn now returns True/False;
on True the row is updated to state='running' before
_assert_valid_after. Catches the case where a decky added via the
live mutator queue stays at 'pending' indefinitely (the deployer's
reconcile only runs on a fresh deploy_topology pass).
Existing topology deckies in active topologies will still read as
'pending' until the next deploy_topology runs, since this is
forward-only. An operator-side fix is to teardown + redeploy or run
the (forthcoming) reconcile-on-startup pass.
apply_add_decky's compose-up was hard-failing whenever the operator's
~/.docker/buildx/activity/ landed on a read-only mount — the wedge
detection in _compose_with_retry correctly refuses to retry (would
just leak more mounts), but for live materialisation we don't want a
wedged buildx state to abort an admin's mutation. ANTI hit it on
adding decky-a977: 'failed to update builder last activity time: ...
read-only file system → buildx wedge detected → returned non-zero'.
_compose_up_with_buildkit_fallback wraps _compose_with_retry: on a
CalledProcessError whose stderr matches both wedge signatures
(_BUILDX_WEDGE_SIGNATURE + _BUILDX_EROFS_SIGNATURE), it logs a
warning with the manual recovery steps + retries once with
DOCKER_BUILDKIT=0 set. The legacy non-buildx builder doesn't use
the activity dir and isn't affected.
Wired into the two paths that pass --build:
* _materialise_decky_spawn (apply_add_decky)
* _materialise_decky_services_diff (apply_update_decky service add)
_materialise_decky_recreate_base doesn't build — it just recreates a
container from an existing image — so it's not affected.
Operator-facing log message points at the manual fix
(rm -rf ~/.docker/buildx/activity + docker buildx create) so they
can recover at their leisure; we don't ATTEMPT the recovery because
the activity dir might be RO for a reason (zfs/btrfs snapshot, etc.)
that an automated rm would be wrong to fight.
apply_update_decky's flip path now refuses to promote a decky to
gateway unless its home LAN is a DMZ. The compose generator publishes
host ports for forwards_l3=True; a non-DMZ gateway would shadow the
host's port space without anything legitimately able to reach the
service. Same posture as the existing 'forwards_l3 flip on live
requires force=true' guard — refused before any DB write so a bad
mutation leaves zero side-effects.
The check is intentionally NOT a standing _RULES invariant — the
codebase uses forwards_l3 for two semantics:
1. Generic L3 forwarding (internal bridge deckies routing between
their multi-home LANs). The generator writes this on internal
bridges via bridge_forward_probability; legitimately non-DMZ.
2. DMZ gateway (host-port publisher). Only meaningful on DMZ.
Standing validation can't enforce DMZ-homing without breaking case 1.
The guard fires only on the explicit user-driven flip path where the
operator's intent is unambiguously case 2. Generator output and
internal-bridge attachments bypass the check.
check_gateway_homed_in_dmz lives in validate.py for callers that want
the explicit form (and for the test surface), but is not a standing
rule — comment in _RULES explains the asymmetry.
W5's apply_update_decky now accepts a forwards_l3 flip on a live
topology only when payload['force'] is true (the unforced flip raises
MutationError to keep half-thinking operators from killing
in-container state). Until this commit there was no UI surface that
could even submit such a flip.
Inspector grows a 'PROMOTE TO GATEWAY' / 'DEMOTE GATEWAY' button when
a (non-observed) decky is selected. The handler:
* On pending topologies → submits via editor.updateDecky immediately.
No confirm dialog; no live containers to disturb.
* On active/degraded topologies → window.confirm() explaining the
destructive base recreate ('In-container state is lost; active
sessions to it drop'), then submits with extras.force=true.
useTopologyEditor.updateDecky grows an optional extras arg that
threads force: true into the queued mutation payload. The pending
CRUD path ignores it (no force needed when no containers exist).
MazeNET.tsx wires a toggleGateway callback that handles the
optimistic local state update, surfaces an enqueue toast on the
active path, and lets the SSE forwarder reconcile when
mutation.applied lands.
Two related fixes that came out of running the W5 tests locally:
1. tests/__init__.py — empty file, makes 'tests/' a package so pytest
stops inserting it into sys.path. Without it, 'tests/docker/'
(the docker-image test category) shadowed the installed docker SDK
on every engine-touching test in the repo:
module 'docker' has no attribute 'DockerClient'
Pytest's default --import-mode=prepend was the culprit; making
tests/ a package is the cheapest fix and doesn't change
--import-mode for the whole tree.
2. delete_topology_decky / delete_topology_edge / delete_lan grow an
'enforce_pending: bool = True' kwarg. Default preserves the HTTP
CRUD guard (api_decky_crud / api_edge_crud / api_lan_crud get the
409 for free). apply_remove_decky / apply_detach_decky /
apply_remove_lan now pass enforce_pending=False — the mutator
queue is the live-editing surface and has its own active-topology
gating; the repo's pending-only guard was for design-time CRUD
that mustn't bypass it. Without this, apply_remove_decky was
silently broken on active topologies pre-W5; W5's new test
surfaced it on first run.
10/10 new W5 tests pass; 58/58 across mutator + topology suites.
apply_update_decky now discriminates three sub-cases:
* services list changed → diff old vs new and call
_materialise_decky_services_diff (compose up -d for added,
stop + rm -f for removed). Mirrors services_live's pattern but
doesn't import it — mutator-routed mutations carry a different bus
surface (mutation.applied) than the direct API path
(decky.<name>.service_added).
* forwards_l3 flipped → port publishing changes, which docker can
only apply at container-create time. Gated on payload['force'] is
true; default raises MutationError so a half-thinking operator
can't stomp a live decky. When force=true,
_materialise_decky_recreate_base does compose up -d --no-deps
--force-recreate. Pre-checked BEFORE the DB write so a refused
mutation leaves zero side-effects.
* coord-only (x/y) → DB only, no docker work.
Ships tests/mutator/test_ops_materialisation.py with focused coverage
for every new helper: add_decky/remove_decky/attach_decky/
detach_decky/update_decky/update_lan paths against an active
topology, with compose primitives + docker SDK mocked at the source
modules so the helpers' lazy imports pick up the stubs. Also covers
the pending-topology skip and the force-flag gating.
Symmetric to apply_attach_decky — after deleting the multi-home edge
from the DB, calls the docker SDK to drop the base container's
interface in the now-detached LAN. Service containers lose
visibility automatically (they share the base's netns).
Idempotency: 'not connected' / 'no such' APIError is logged at info
and treated as success.
After the DB writes that record the multi-home edge, calls the docker
SDK directly to add an interface to the base container's netns:
client.networks.get(<topology bridge>).connect(<base>, ipv4_address=ip)
Non-destructive — the base keeps running, no recreate. Service
containers automatically see the new interface because they share
the base's netns via network_mode: service:<base>.
Idempotency: docker APIError with 'already' / 'endpoint exists' is
logged at info and treated as success. Other errors log + leave the
DB row in place; an operator retry will hit the same path.
Captures the decky's name and services list before delete_topology_decky
runs (the helper needs both as compose targets even though the DB row
is gone), then calls _materialise_decky_remove which stops + rm -f's
the base + per-service containers via 'docker compose stop / rm -f'.
Re-renders the per-topology compose AFTER the stop/rm so a future
'compose up -d' on the file doesn't try to bring the decky back.
Adds _materialise_decky_{spawn,remove,connect,disconnect,services_diff,recreate_base}
helpers alongside the existing _materialise_lan_change. Each follows
the same skip rules: bail when topology is not active/degraded, when
agent-pinned, or when docker calls fail (logged, not re-raised — DB
remains source of truth).
apply_add_decky now calls _materialise_decky_spawn after the DB writes.
The helper:
* re-renders the per-topology compose so it lists the new decky;
* runs 'compose up -d --no-deps --build <decky_base> <decky>-<svc>...'
in a worker thread (matches engine/services_live's pattern).
Service container targets are filtered through get_service() so
fleet_singleton services are skipped — they don't have per-decky
compose entries. Gateway (forwards_l3=True) deckies need no
special-case here; the compose generator already emits the host
'ports:' block for them.
Subsequent commits wire the other apply_* ops to the matching
helpers. Tests for the full set ship in the workstream's last
commit.
subnet and is_dmz are pinned at deploy time — live deckies bind to
the bridge with IPs allocated from the old subnet, and is_dmz flips
the docker network's internal flag which can't be changed while
containers are attached. Today the op happily wrote the new value
into the DB and left docker on the old one, drifting the two surfaces.
apply_update_lan now raises MutationError when topology status is
active or degraded and the patch touches subnet or is_dmz. Coord
(x/y) and rename updates still pass through; renames don't currently
have a live caller and the bridge's docker name keys off the lan name
in the renderer, so the next deploy will reconcile.
This matches the posture taken by _materialise_lan_change for live
LAN add/remove (commit 472c84b).
list_topology_edges has no ORDER BY, so SQL row order is undefined.
After apply_attach_decky added a bridge edge to a second LAN, on
refetch the bridge edge could come back first — firstLanFor then
picked it as the decky's home and the visualization 'teleported' the
decky into the other LAN (the bug ANTI saw immediately after
connecting two deckies across LANs).
Hydration now prefers the non-bridge edge (is_bridge=false) as home.
apply_add_decky writes is_bridge=false for the original edge;
apply_attach_decky writes is_bridge=true for subsequent multi-homing
edges. Picking the non-bridge edge is stable across row reordering.
Two-pass implementation: pass 1 sets pinned homes (DMZ for gateways,
non-bridge for others); pass 2 fills any gap with the first edge
(legacy rows where is_bridge was never written).
apply_add_lan and apply_remove_lan were DB-only — they wrote/deleted
the topology_lans row but never created or destroyed the docker bridge
network. Adding a LAN to a deployed topology silently did nothing on
the substrate side; any decky later attached to it had nowhere to bind.
Both ops now call a shared _materialise_lan_change helper after the DB
write. When the topology is active/degraded and not pinned to a swarm
agent, the helper:
* creates / removes the docker bridge network (internal=True for
non-DMZ LANs, mirroring engine/deployer.deploy_topology),
* re-renders the per-topology compose file so future redeploys reflect
the change.
Failures are logged, not re-raised — the DB row stays as source of
truth so an operator can retry without leaking inconsistent state.
Agent-pinned topologies are skipped; the next agent push reconciles.
apply_add_decky / apply_attach_decky have the same gap and are not
fixed here — multi-homing a running container needs careful
recreate-vs-network-connect handling and is its own commit. Without
those, dropping a decky into a freshly-added LAN still won't spawn a
container; only the LAN itself is now live.
Bus topic segments are NATS-style tokens and the validator at
bus/topics.py:402 rejects '.', '*', '>', whitespace. My W3 constants
'service.added' / 'service.removed' tripped this on every live
add/remove call:
ValueError: topic segment 'service.added' may not contain '.', ...
Renamed both to underscore form: DECKY_SERVICE_ADDED = 'service_added'.
Aligned the SSE forwarder's name mapping (decky.<name>.service_added →
SSE event 'decky.service_added') and the frontend's
useTopologyStream listener + MazeNET.tsx event handler. Also updated
the wiki entry with a note about the underscore.
deploy_topology was flipping to ACTIVE the moment 'compose up -d'
returned 0, but compose returns 0 as soon as containers are *started*.
A service that crashes on boot (port bind failure, bad image, missing
entrypoint) left the topology row sitting at ACTIVE indefinitely while
half the substrate was dead.
After compose returns, we now run 'compose ps --all --format json',
parse the newline-delimited per-container rows, and downgrade to
DEGRADED with a reason listing the first eight unhealthy containers if
anything isn't in state='running'. Operators see real state on the
topology page instead of an optimistic flag.
_compose_ps swallows compose-level errors (returns []) so an unrelated
docker hiccup doesn't gate the success path — the existing in-flight
exception path still catches genuine deploy failures with FAILED.
When topoStatus is active/degraded, editor.updateDecky enqueues into
the mutator queue and returns {kind:'enqueued'}. The palette-drop
handler then short-circuits on that and never updates local state, so
a service dragged onto a deployed decky just vanishes — what ANTI saw
as 'no way to APPLY'.
Same gap on the design-time 'REMOVE SERVICE' button in the Inspector's
service detail panel: enqueue + no local update = chip stays.
Both now route through liveAddService / liveRemoveService when the
topology is active, hitting POST/DELETE /topologies/{id}/deckies/{name}/services
directly and patching local state from the response. Pending
topologies still queue through the mutator (correct: no live
containers to mutate).
Hoisted serviceRegistry / liveAddService / liveRemoveService above
the palette-drop callback so the deps array doesn't trip the const
TDZ at render time.
FastAPI's redirect_slashes=True 307s /topologies → /topologies/, and
the browser drops Authorization on the redirected URL — the topology
picker in the canary create modal was landing as 401 even for admins.
Hit the canonical (trailing-slash) path so the request resolves on the
first hop.
The /topologies/{id}/events SSE proxy now subscribes to two bus
patterns concurrently and merges them through a bounded asyncio.Queue:
* topology.{id}.> — lifecycle (status, mutation.*) — unchanged.
* decky.> — per-decky events, filtered by payload.topology_id
so a fleet decky sharing a name with a topology
decky doesn't leak across.
_sse_name_for routes 'decky.<name>.service.added' to the SSE event
name 'decky.service.added' (kept the prefix so the frontend doesn't
collide with topology lifecycle events that share leaf names like
'status').
useTopologyStream surfaces the two new event names; MazeNET.tsx's
onStreamEvent optimistically patches the matching node's services
list so a second tab reflects shape changes without a refetch.
DeckyCard grows the same per-chip × + dashed '+ ADD' affordances we
just shipped on the MazeNET Inspector. Wired to POST/DELETE
/api/v1/deckies/{name}/services{,/svc}; the response's services list
flows back through onServicesChanged to update the parent's deckies
state without a refetch.
Gated on isAdmin && !decky.swarm — swarm deckies live on a remote
agent and the W3 endpoint runs docker compose locally, same gap as
the canary planter has for agent-pinned topologies. Out of scope
here; flagged as a known limitation.
stopPropagation on the inline buttons + add-row container keeps the
card-level click (which selects the decky for inspection) from firing
on intra-row interactions.
ObservedNode.services is the literal tuple ['*']; narrowing inside the
.filter() callback was tripping TS2345. We already gate the live
controls on node.kind !== 'observed', so casting to readonly string[]
inside the filter is safe and keeps the discriminated union strict
elsewhere.
When the topology is active/degraded the Inspector switches services
chips into live controls: each chip gets a × button that DELETEs to
the W3 endpoint, and a dashed '+ ADD' chip opens a typeahead picker
fed by useServiceRegistry().perDecky.
Pending topologies still use the existing design-time path
(onRemoveService → editor.updateDecky); the Inspector picks based on
topologyStatus, so an operator never accidentally hits a live API
call against a topology that isn't deployed yet.
The mutation handlers in MazeNET.tsx hit POST/DELETE
/api/v1/topologies/{id}/deckies/{name}/services{,/svc} and
optimistically apply the response's services list to local state.
Cross-tab reconciliation rides on the SSE forwarder shipped in the
follow-up commit.
Adds a fleet_singletons array to ServiceCatalogResponse so per-decky
add UIs can filter out services like LLMNR that run once fleet-wide
(and would 422 server-side at the live add endpoint).
The existing 'services: list[str]' field is unchanged for back-compat
with MazeNET/useMazeApi.ts:257; the new field is additive.
decnet_web/src/hooks/useServiceRegistry.ts wraps the endpoint with a
module-scoped cache (registry only changes on BYOS install / plugin
drop, neither of which happens mid-session) and exposes a precomputed
.perDecky list so consumers don't need to re-derive the diff.
CanaryTokens.tsx grows a third tab — File drops — alongside Tokens
and Blobs. The page now covers every 'admin landed bytes on a decky'
operation in one place.
FileDropModal mirrors the canary CreateModal's shape: Fleet/MazeNET
toggle, topology+decky picker, absolute-path validation matching the
backend (DeckyFileDropRequest rejects relative + ..-traversal), mode
+ mtime offset inputs, and a -1w preset for backdating. FileReader →
data URL → strip prefix → POST /api/v1/deckies/files.
The list is local-only (localStorage, capped at 200 entries). W2's
backend doesn't persist drops by design — the endpoint is for staging
payloads, not as an audit trail. CLEAR LIST button on the tab; no
DELETE button on rows since the local entry doesn't track whether the
file is still there (an attacker may have moved it).
Alt+D shortcut joins Alt+C; alt-key only per the Linux-meta-key rule.
CanaryTokens.tsx grows a Fleet/MazeNET toggle in the create modal. In
topology mode we hydrate /topologies?status=active for the topology
picker, then GET /topologies/{id} on selection to repopulate the decky
picker — topology deckies have a different shape than fleet's /deckies
endpoint.
The tokens table gains a SCOPE column (chip: 'fleet' / 'topology'),
and a third filter dropdown alongside state. The drawer's metadata
section shows a Scope row with a clickable jump-link back to the
MazeNET view at the right topology.
CanaryTokenRow grows a topology_id field so the drawer/list can
discriminate without re-fetching.
decnet.engine.services_live exposes add_service / remove_service for
both fleet and topology decky scopes. The host's _compose() wrapper
already supported per-service targeting (up --no-deps -d <svc>,
stop, rm -f); what was missing was the orchestration around it:
* add: validate against decnet.services.registry (rejects unknown +
fleet_singleton); persist the new services list; re-render the
per-scope compose file (so future redeploys reflect the change);
run docker compose up -d --no-deps --build <decky>-<svc>.
* remove: stop + rm -f the service container; persist; re-render
compose so a future up -d doesn't bring it back.
Both publish decky.<name>.service.added / .removed on the bus, with
the post-mutation services list. Topic constants added to
decnet.bus.topics; the matching wiki entry in wiki-checkout/Service-Bus.md
ships in a separate commit on the wiki repo (wiki-checkout/ is gitignored).
Four new admin endpoints:
* POST/DELETE /api/v1/deckies/{name}/services{,/svc}
* POST/DELETE /api/v1/topologies/{id}/deckies/{name}/services{,/svc}
ServiceMutationError messages are mapped at the API boundary to 404
(decky/topology missing), 409 (idempotency violation), 422 (unknown
or fleet_singleton service).
Extracts the docker-exec-with-base64-stdin pattern out of canary/planter
and orchestrator/drivers/ssh into a shared decnet.decky_io package.
Both consumers now delegate; the canary planter test still proves the
contract end-to-end.
Adds POST/DELETE /api/v1/deckies/files for arbitrary file drops.
Container resolution is shared with the canary path: topology_id absent
means fleet (<name>-ssh), present routes through resolve_decky_container
which picks <name>-ssh when the topology decky exposes ssh, else the
topology base container decnet_t_<id8>_<name>.
Path validation rejects relative paths and '..' traversal at the request
model layer. Bad base64 → 400; unknown topology → 404; decky not in
topology → 422; docker exec failure → 409.
POST /api/v1/canary/tokens grows an optional topology_id field. When
present, the server hydrates the topology, validates the named decky is
in it, and resolves the docker container via
planter.resolve_topology_container — <name>-ssh if the decky exposes ssh,
else the topology base container. Absent ⇒ fleet semantics, unchanged.
The token row gets a nullable topology_id column (no migration helper
per pre-v1 policy). GET /api/v1/canary/tokens accepts ?topology_id= as
a filter. DELETE re-resolves the container at revoke time so a
redeployed topology is still reachable.
422 when the named decky isn't in the topology; 404 when the topology
itself doesn't exist.
Topology deploys now plant the configured canary baseline set on every
decky in the topology, mirroring the fleet-deploy hook. Containers are
resolved via resolve_topology_container — <decky>-ssh when the decky
exposes an ssh service, else the topology base container
decnet_t_<id8>_<decky>.
The planter's plant/revoke/seed_baseline grow an optional container=
kwarg; default preserves the fleet <name>-ssh resolution.
The Vault page already shows file drops and stored mail (e3ddeb0) but
the inspector drawer had no download button — only the live-feed
ArtifactDrawer/MailDrawer offered raw byte retrieval. Add a DOWNLOAD
RAW action to BountyInspector that fires when bounty_type=artifact,
hitting /artifacts/{decky}/{stored_as}?service=<svc> with the bounty's
own service field (ssh or smtp). Mirrors ArtifactDrawer's blob handling
and 400/403/404 error mapping.
Also widen the icon/label vocabulary: artifact bounties get FileText
(file drops) or Mail (message_stored) instead of the generic Package,
and the inspector header chip mirrors the change.
The Bounty Vault page only read from the Bounty table, but
inotifywait-captured file drops (event_type=file_captured) and SMTP
quarantined messages (event_type=message_stored) were only landing in
the Logs table. AttackerDetail's tabs queried logs directly, so they
showed up per-attacker but were invisible on the global Vault page.
Mirror both events into Bounty as bounty_type=artifact with
payload.kind ∈ {file, mail} so the existing dedup
(bounty_type, attacker_ip, payload) collapses repeats by sha256. Add an
ARTIFACTS segment to the Vault filter row, plus dedicated render
branches: file drops show orig_path + size + writer attribution; mail
shows subject + From + attachment count + size, with the Mail icon
distinguishing them from FileText for file drops.
Forward-only — existing logs stay where they are. A backfill pass would
be straightforward (read Log WHERE event_type IN ('file_captured',
'message_stored') and feed each row through _extract_bounty) but is out
of scope here.
sshd, pam_unix, sudo, CRON, systemd, kernel, rsyslogd, and dbus-daemon
all share the SSH/telnet decky containers and write to the same syslog
socket as DECNET's own emitters. Their output was being parsed and
ingested into the JSON stream, the dashboard, and the profiler — pure
noise: sshd's "Failed password for root from X" duplicates the
auth-helper's structured auth_attempt event, pam_unix repeats it again,
CRON/systemd say nothing about attacker behavior.
Drop these APP-NAMEs in _should_ingest before the JSON write and bus
publish. Raw .log file still captures everything for forensics. The
denylist is overridable with DECNET_COLLECTOR_DROP_APPS so operators
can extend it without code changes.
Add --rfc5424 --msgid command to the logger invocation in SSH and telnet
decky bashrc. MSGID arrives as "command" instead of NIL, which is what
the profiler's _COMMAND_EVENT_TYPES filter expects. The parser heuristic
shipped in d4591b3 stays as a safety net for any future emitter that
forgets the flags or for inflight pre-rebuild containers.
SSH/telnet decky containers emit shell commands via `logger -t bash "CMD …"`
which produces RFC 5424 lines with MSGID=NIL. Both parsers were leaving
event_type="-", so the behavioral profiler's `_COMMAND_EVENT_TYPES` filter
silently dropped them — the IP profile existed but no command transcripts
or artifacts. Confirmed in the wild: 44/48 events from one attacker were
event_type="-".
Rewrite event_type to "command" in both parsers when MSGID=NIL and the
msg starts with "CMD ". Correlation parser also extracts the cmd= payload
into fields["command"] so the profiler can build the transcript; collector
parser leaves fields={} to avoid duplicate pills in the dashboard.
Splits the 459-line credentials.py into two submixins plus a composing
CredentialsMixin in credentials/__init__.py:
_core.py (~190) Credential capture: upsert, list, filters,
per-attacker / per-secret reads, attacker_uuid
backfill
reuse.py (~270) CredentialReuse correlation: upsert, candidate
mining, list/get + the _enrich_with_secret helper
that lifts the printable/b64 from underlying rows
_merge_unique stays with reuse.py (its only caller).
_enrich_with_secret stays with reuse.py — it's an internal helper of
list_credential_reuses / get_credential_reuse_by_id, never called
from the capture path.
Moves the 31 MazeNET topology methods (topologies CRUD, LANs, deckies,
edges, status events, mutation queue) into sqlmodel_repo/topology.py.
Includes _assert_pending and _check_and_bump_version concurrency
guards.
This is the last domain extraction; sqlmodel_repo/__init__.py is now
~165 lines: lifecycle (initialize/reinitialize/migrations), the admin
self-heal seed, get_state/set_state, and the mixin composition.
Splits the AttackerIdentity and Campaign clustering reads/writes into
sqlmodel_repo/identities.py and sqlmodel_repo/campaigns.py.
Both call _deserialize_attacker (identities only) which resolves
through AttackersMixin via MRO.
Moves the 19 attacker-domain methods (core CRUD, behavior, sessions,
smtp targets, log-derived activity views) plus the _deserialize_attacker
and _deserialize_behavior helpers into sqlmodel_repo/attackers.py.
Moves the 8 log methods (incl. get_stats_summary aggregator) into
sqlmodel_repo/logs.py. get_log_histogram remains an abstract dialect
override point; sqlite/mysql subclasses still override it via MRO.
Moves the 6 fleet-decky methods (incl. cross-source list_running_deckies
aggregator) into sqlmodel_repo/fleet.py. _serialize_json_fields and
_deserialize_json_fields move to _helpers.py since they're shared
across fleet, topology, and canary.
Moves the 7 user CRUD methods into sqlmodel_repo/auth.py.
_ensure_admin_user stays in __init__.py so DECNET_ADMIN_PASSWORD
remains addressable at the module path tests already monkeypatch.
Moves upsert_attacker_intel, get_attacker_intel_by_uuid,
and get_unenriched_attackers into sqlmodel_repo/attacker_intel.py.
Composed onto SQLModelRepository via mixin inheritance.
Pure rename — the old monolithic 3505-line file becomes
decnet/web/db/sqlmodel_repo/__init__.py. No code changes.
Subsequent commits will extract per-domain mixins out of __init__.py
to mirror the topical layout used by decnet/web/db/models/.
DummyRepo couldn't instantiate — TLS-cert fingerprint rollup added a new
abstract method without a stub here. Add the override and a call site so
the abstract pass body is hit.
Three independent issues conspired to make stress tests record 0 requests:
1. Every virtual user did /auth/login in on_start. With 1000 users in a
spike window, bcrypt-bound logins never finished and on_start failed
for all users — aggregated requests stayed at 0. Pre-fetch a single
admin token in the fixture (cached per-host) and pass it via
DECNET_STRESS_TOKEN so locust users skip the login storm.
2. Locust exits non-zero on any request failure by default, causing
run_locust to throw away an otherwise valid stats CSV. Pass
--exit-code-on-error 0 so per-test assertions are the only fail gate.
3. test_stress_sustained ran two locust subprocesses against the same
uvicorn. Phase 1's keep-alive connections wedged phase 2 into 0
recorded requests ~2/3 of the time. Refactored stress_server into a
start_stress_server() context manager and gave each phase its own
uvicorn.
Stable 3/3 on full suite, 3/3 on test_stress_sustained alone.
Brings the federation-gossip columns on AttackerIdentity to life —
ja3_hashes, hassh_hashes, and the new tls_cert_sha256 — by projecting
the union of every member observation's fingerprints JSON onto the
identity at clusterer create / link / merge time.
- decnet/profiler/identity_rollup.py: pure extract_fp_summaries()
reads the production bounty shape (payload.fingerprint_type +
payload.{ja3,hash,cert_sha256}) and returns deduped+sorted JSON
list[str] per family, or None when a family has no signal so the
column stays NULL instead of '[]'.
- BaseRepository.update_identity_fingerprints + SQLModel impl: one
idempotent write that overwrites the three summary columns and
bumps updated_at.
- ConnectedComponentsClusterer: after every per-component
reconciliation (fresh-create OR existing-merge+link), recomputes
and writes the rollup for the target identity. Wrapped in a
best-effort helper so a write failure logs but never breaks the
tick.
- Tests: extract_fp_summaries unit (dedup, sort determinism,
unknown types ignored, malformed JSON, nested-stringified
payloads, non-string values); end-to-end clusterer ticks
populate the columns on create + on later observation links;
no-fingerprint clusters keep the columns NULL.
- FpCertificate renders the new cert_sha256 field (truncated, with
full hash on hover) and a FROM line carrying the prober-side
target_ip/port so the source is visible.
- tls_certificate payloads split on target_ip presence: prober certs
land under ACTIVE PROBES, sniffer certs under PASSIVE FINGERPRINTS.
Two synthetic fpType keys (tls_certificate_active /
tls_certificate_passive) drive the bucketing without disturbing
the on-the-wire fingerprint_type.
JARM probes are crafted ClientHellos with weird ciphers — they never
complete a real handshake, so the peer cert isn't reachable from
those sockets. After a non-empty JARM hash proves the port speaks
TLS, do a separate ssl.wrap_socket() against the same (ip, port) to
fetch and parse the leaf cert.
- decnet/prober/tlscert.py: fetch + parse via cryptography lib;
swallows all connect/handshake/parse failures (returns None).
- decnet/prober/worker.py::_capture_tls_cert: emits a tls_certificate
event with subject_cn / issuer / SANs / validity / SHA-256 +
publishes on the bus. Wired from _jarm_phase only when JARM
succeeds, so non-TLS ports never trigger a second connect.
- Tests cover happy path, cert-fetch failure, defense-in-depth crash,
empty-JARM skip, publish_fn, and parser edge cases (garbage DER,
empty bytes, missing SAN extension, non-self-signed).
Adds storage for TLS certificate details collected from attacker-run
servers by the active prober (sibling to the existing JARM probe).
- AttackerIdentity.tls_cert_sha256 / Campaign.tls_cert_sha256:
JSON list[str] columns mirroring ja3_hashes / hassh_hashes for
federation gossip.
- ingester clause 9b: emits a 'tls_certificate' fingerprint bounty
when a prober event carries subject_cn (disjoint from the existing
sniffer-gated clause).
- Prober-side capture (ssl.wrap_socket follow-up after JARM) and
profiler rollup land in sibling commits.
The check expects 405 for any HTTP method not declared on a path.
DECNET's topology router has a static `/topologies/services` (GET only)
sibling to a parameterized `/topologies/{topology_id}` (DELETE), so a
DELETE on the static path falls through to the parameterized route and
hits auth, which returns 401 — by design. Leaking 405-vs-401 would let
unauthenticated callers enumerate valid topology UUIDs.
The same shape applies to other static/dynamic sibling pairs across
the API. The check is fundamentally incompatible with that routing
strategy; document the omission inline.
Schemathesis fires up to 3000 examples per endpoint. POST /auth/login
caps at 10/5min per IP, so the second example onward returns 429 and
the positive_data_acceptance check flags it as RejectedPositiveData
(its allowed-status list is hardcoded in schemathesis to
2xx/401/403/404/409/5xx, so OpenAPI tweaks can't fix it).
DECNET_LIMITER_ENABLED=false exists for exactly this case (see
limiter.py docstring on stress/load testing).
Reverts the custom_openapi shim from 5d88346 / 9b1168c — the endpoint
already declares 429 in its responses= map (api_login.py:38), and the
shim turned out to address a problem that wasn't there. Drop the
companion test along with it.
Previous commit advertised 429 on every operation. Only routes
decorated with @limiter.limit can actually return slowapi's 429 —
currently just POST /api/v1/auth/login. Documenting it elsewhere is
dishonest and would mislead clients into expecting a response the
server cannot produce.
Walk slowapi's _route_limits / _dynamic_route_limits registries to
identify decorated endpoints, match them to FastAPI routes by
{module}.{name}, and only inject 429 on those.
Existing per-route 429 declarations (e.g. SSE connection-cap on
events streams via sse_limits) are untouched.
SlowAPI middleware can short-circuit any request with 429 once a
per-route or per-IP rate limit fires (e.g. POST /api/v1/auth/login is
capped at 10/5min). The OpenAPI spec did not declare 429 on any
operation, so schemathesis flagged legitimate rate-limit responses as
RejectedPositiveData / status-code-nonconformance failures.
Override app.openapi to inject a generic 429 response object on every
HTTP operation in the generated schema. Add a contract test that fails
if any operation drops the 429 advertisement.
- swarm/test_swarm_api, swarm/test_heartbeat: replace deprecated
asyncio.get_event_loop().run_until_complete() with asyncio.run();
the former raises in 3.11 once another test has set+closed a loop on
the main thread.
- prober/test_prober_bus, prober/test_prober_worker: extend tcp_fingerprint
mocks with tos/dscp/ecn/server_isn so the worker doesn't KeyError into
the prober_error branch.
- services/test_service_isolation: collector now retries on event-stream
errors instead of exiting; assert it stays running and cancel cleanly.
- live/test_imap_live, live/test_pop3_live: log format emits
outcome="failure", not "failed".
- live/test_service_isolation_live: is_service_container accepts label
OR state-name; rewrite the empty-state test against a synthetic
unlabeled container instead of the host's real fleet.
OpenSSH's native syslog ("Failed password", "Connection from",
"Connection closed by …") and the pam_unix lines emitted from sshd's
PAM stack add no signal beyond what auth-helper already captures as
structured login_attempt events. They cluttered the dashboard and
arrived without an SD wrapper, forcing prose-IP heuristics in the
collector.
Add a `:programname, isequal, "sshd" stop` rule above the forwarding
actions in /etc/rsyslog.d/50-journal-forward.conf. pam_unix lines from
sshd inherit programname=sshd so the same rule covers both. sudo /
login / su pam_unix lines keep flowing (different programname), so
post-login privilege escalation telemetry is preserved.
Native sshd and pam_unix lines route through rsyslog without the
relay@55555 SD wrapper and without key=value pairs, so attacker_ip
fell through to "Unknown". Add a prose-IP fallback to both parsers:
anchored patterns (from/rhost/client/src) win first so we never pick
the local listener in "Connection from X port Y on Z port 22", with
a bare-IPv4 scan as the last resort.
The prober writes events with hostname=decnet-prober and target_ip=
<the attacker being fingerprinted>. The parser pulls target_ip into
attacker_ip (it's one of _IP_FIELDS), which is correct for indexing
fingerprints under the attacker — but it had a side effect: every
fingerprinted attacker had two distinct deckies on file (the real
decoy they touched + decnet-prober) and the correlation engine's
traversals() classified that as lateral movement. Live dashboard
showed bogus "dmz-gateway -> decnet-prober" paths and TRAVERSAL
badges on attackers who'd done nothing but knock on the front door.
The prober is internal infrastructure, not a hop. Filter the
"decnet-" namespace out of distinct-decky counts and hop paths in
the engine. Fingerprints stay attached to the attacker profile via
the existing per-IP event index — just no longer as traversal.
Hit live on first VPS deploy: a window between the initial
client.containers.list() snapshot and the client.events() start-event
stream let topology service containers slip through, requiring an
operator restart for them to be picked up.
Two fixes:
* `_watch_events` now wraps the events() call in a retry loop with
exponential backoff (1s -> 30s cap). A docker.errors.APIError, daemon
reload, or SDK stream-decode hiccup used to make the executor task
return cleanly, leaving the collector "running" with no event
subscription. Future container starts were silently dropped until
the unit was restarted.
* New `_reconcile_loop` async task ticks every
DECNET_COLLECTOR_RECONCILE_S (default 30s), re-scans
client.containers.list(), and calls _spawn for any service container
not already in `active`. Belt to the event watcher's suspenders:
even if a start event is dropped during a reconnect window, the
reconciler picks it up within one cycle. Also prunes finished
futures from `active` so the dict's bounded by current container
count rather than agent lifetime churn.
The module docstring teaches inline comments — `mode = master # or
"agent"` is the canonical example for the [decnet] section. Python's
configparser ignores those by default unless inline_comment_prefixes
is set explicitly, so the comment became part of the value and
downstream validators rejected it ("mode must be 'agent' or 'master',
got 'master # or \"agent\"'").
Hit live on first VPS deploy: every CLI invocation crashed at import
time with a stack trace that didn't make it obvious the docstring's
example was the trigger. Now the parser does what the docs promise.
Decky service containers join their base via `network_mode:
container:<base>` and Docker binds that share at service start time. If
`docker compose up` recreates a base (e.g. ports: changes after a
forwards_l3 toggle) but decides services are unchanged, services keep
a stale FD into the destroyed namespace and end up with only `lo` — so
external traffic hits a closed port on the live base and gets RST.
Hit live on the first VPS deploy: external SSH to the dmz-gateway was
refused while sshd was listening, because base and service netns
inodes had drifted apart. `--always-recreate-deps` makes compose
rebuild every dependent whenever its base is recreated, removing the
race entirely.
The dashboard's /api/* proxy hardcoded 127.0.0.1 as the target host.
That works when the API binds to a wildcard or to loopback, but
breaks the moment an operator binds the API to a specific address —
e.g. a Tailscale IP for tailnet-only deploys: the API stops listening
on loopback entirely and the proxy gets ECONNREFUSED on every request.
The web command now reads DECNET_API_HOST and falls back to loopback
only when the API is on a wildcard (0.0.0.0 / :: / unset). A new
--api-host flag overrides at the CLI level.
Without rotation, the syslog listener and per-host collector grow
/var/log/decnet/ without bound — a noisy attacker (or an active
probe storm) fills the disk in hours on a small VPS. New
deploy/logrotate.d/decnet caps at 7 daily rotations or 100 MiB,
whichever comes first, and uses copytruncate because the ingester
and forwarder hold the files open via Python and won't reopen on
a rename rotation.
Wire install / remove into `decnet init` and `decnet init --deinit`
alongside the existing tmpfiles.d / polkit handling.
Refuse to start decnet.web.api when DECNET_MODE=agent (unless the
operator explicitly opts into dual-role with DECNET_DISALLOW_MASTER=
false). The Typer CLI already hides master-only commands on agents,
but a misconfigured systemd unit or a direct uvicorn invocation
would bypass that — now the lifespan itself refuses, before any
worker, DB or bus comes up.
Resolve DECNET_JWT_SECRET eagerly at startup so a missing or known-
bad value fails at boot rather than on the first auth-gated request.
The lazy-load shape stays useful for non-master CLIs.
Add validate_public_binding() called from the master API lifespan: when
DECNET_API_HOST is non-loopback, refuse to start if DECNET_CORS_ORIGINS
still contains a loopback origin (catches the "operator flipped to
0.0.0.0 to make it work and forgot to update CORS" footgun) or if
DECNET_CANARY_HTTP_BASE is plaintext http:// to a non-loopback host.
Log CRITICAL when DECNET_LIMITER_ENABLED=false on a public binding.
The validator no-ops under pytest so unrelated suites don't trip on it.
Add DECNET_VERIFY_HOSTNAME env knob; AgentClient and UpdaterClient
consult it when verify_hostname is None, giving production deploys
TLS hostname verification on top of the existing CA + fingerprint pin.
Default off so dev enrollments with mismatched SANs keep working.
Reject symlinks, hardlinks, device nodes and FIFOs in update tarballs;
validate each member's resolved path stays under dest after symlink
resolution; cap uncompressed size at 256 MiB to bound gzip-bomb damage;
strip setuid/setgid bits from extracted modes.
Add an optional sha256 form field to /update and /update-self; the
master client computes and sends it on every push, the executor
refuses to extract on mismatch. mTLS already authenticates the
master, so this is defence-in-depth against in-transit corruption
and gives operators a way to pin "exactly these bytes" for vetted
releases.
Both pages now layer on DeckyFleet.css + PersonaGeneration.css and use
the project's house vocabulary — fleet-root shell, page-header with
title-group + actions, btn / btn.violet / btn.ghost, info-banner with
the violet left rule, and the dim/matrix/alert text accents.
RealismConfig: inputs are flush-styled weight-input fields with a
violet focus ring; section heads carry a TOTAL badge; canary rows get
the project's amber accent; canary probability lives in a panel-bordered
slider row.
SyntheticFiles: the inline-styled table is now a styled .files-table
with the standard hover affordance, the filter-row uses tweak-group
label+select pairs, the drawer carries .drawer-eyebrow / .drawer-title
/ .meta-grid in the same style as the canary token drawer, and pager
buttons share the .btn.ghost.small treatment.
No behavioural change.
Single source of truth in decnet_web/src/realism/labels.ts: maps each
ContentClass enum value to a friendly display name ("Note",
"Cron Log", "Canary · AWS Credentials", …). Used by RealismConfig
(weight tables + class filter dropdown) and SyntheticFiles (table row
+ drawer detail).
Canary classes get a subtle amber accent so the dashboard's read of
"this row is callback-bearing" doesn't depend on prefix-spotting in
mono text. Raw enum value still appears in dim mono next to the label
so an operator copy/pasting from logs or grepping the codebase still
finds it.
No backend change: the wire shape is still the snake_case enum; the
beautification is render-time only.
New /realism-config page sits next to Persona Generation and
Synthetic Files under the Automation nav. Editable weight tables for
user / system / canary content classes (with live percent share),
plus a slider for canary_probability.
Wires GET/PUT /api/v1/realism/config — viewer can read; admin
required to save. Validation errors from the API are surfaced inline
rather than swallowed; the SAVE button refreshes from the server's
canonical snapshot so the operator sees exactly what landed (matters
because cross-list entries are silently dropped server-side).
New realism_config table (uuid PK + unique key) + two repo methods
(get/set) backs an admin-only GET/PUT /api/v1/realism/config surface.
The planner now exposes apply_payload(payload) / current_payload() /
reset_to_defaults() and reads its weights through mutable module
globals; pick() resolves the live values each call. Validation
catches negative weights, zero totals, out-of-range canary_probability,
unknown content_class names, and silently drops cross-list entries
(canary class on the user list, etc).
The orchestrator worker calls _refresh_realism_config(repo) on
startup and every 5 ticks (~5min at 60s interval). Operator changes
land within one refresh window with no bus signal — the simpler path
for a knob whose latency tolerance is minutes.
The (decky_uuid VARCHAR(64), path VARCHAR(1024)) UNIQUE constraint
generated a 4352-byte composite key under utf8mb4 (4 bytes/char),
busting MySQL's 3072-byte cap and crashing decnet api on init with:
Specified key was too long; max key length is 3072 bytes
Tighten path to VARCHAR(512) — (64+512)*4 = 2304 bytes, well under
the cap. Real realism + canary placement paths are short
(/home/<persona>/Documents/<file>, ~70 chars); 512 keeps headroom
without the index hassle. Pre-v1, no migration helper.
Adds a regression test pinning the (decky_uuid + path) byte budget so
a future widening fails loudly in CI rather than at MySQL deploy
time.
Surfaces realism subsystem state on the existing worker heartbeat
extra hook (system.orchestrator.health) — no new bus topic. Payload
carries {llm_enabled, llm_backend, llm_model, llm_breaker_state}, so
the dashboard's worker panel renders a live LLM badge with a colored
breaker-state dot:
closed (green) — LLM healthy
half_open (amber) — cooldown elapsed; next call is a probe
open (red) — short-circuiting to deterministic templates
Heartbeat is the canonical worker self-report channel; piggybacking on
extra(...) avoids a new topic family while keeping the snapshot
recomputed each beat (30s).
New /synthetic-files page sits next to Persona Generation and Canary
Tokens under the Automation nav group. Operators get a paginated
inventory of files realism has grown across the fleet (decky, path,
persona, content_class, last_modified, edit_count, hash) with filters
on decky / persona / content_class.
Decky filter is a dropdown sourced from /deckies — never free text.
Row click opens a drawer with the body preview; the drawer surfaces a
TRUNCATED chip when the stored body is at the 64KB cap.
Adds GET /api/v1/realism/synthetic-files (paginated list, filters by
decky_uuid, persona, content_class) and
GET /api/v1/realism/synthetic-files/{uuid} (single row with last_body
and a truncated:bool flag set when the stored body is at the 64KB cap).
Repo gains count_synthetic_files() and get_synthetic_file(uuid). The
list view drops last_body to keep the wire payload bounded; the detail
endpoint is the only path that returns it. Read-only — orchestrator
remains the sole writer.
FileAction and EditAction both write kind="file" — the discriminator
is action="file:create" vs "file:edit". The dashboard timeline used
to render both identically; now an EDIT sub-chip surfaces edits without
widening the kind enum (which doubles as the bus topic family).
No schema or API change. Polish only.
decnet/canary/cultivator wrote kind="http" for every cultivated
token, even DNS-trip ones (ssh_key, mysql_dump) and passive bait
(aws_creds). The canary worker uses kind to route attacker callbacks
to the right token; a misaligned kind means a real DNS resolution of
ssh_key or mysql_dump never attributes to the planted slug.
Add _GENERATOR_TO_KIND aligned with CanaryKind in models/canary.py
and look it up at create_canary_token time.
decnet/realism/naming._home and decnet/canary/cultivator._persona_login
both normalised "John Smith"→"johnsmith" with identical logic. Lift
to decnet.realism.personas.login_for(persona) and have both consumers
import it. Drift between the two would have left canary placement and
realism path naming using different login derivations.
The orchestrator worker clipped last_body at write time, but the repo
didn't enforce. A future caller that forgot the clip would write the
full body. Move the clip to record_synthetic_file and
update_synthetic_file via SYNTHETIC_FILE_BODY_LIMIT in
decnet/web/db/models/realism.py. Worker now passes the full body and
trusts the repo. Tests retargeted to assert repo enforcement.
Four gaps from the realism migration plan, plus one flaky test
fixed.
Added:
- tests/deploy/test_orchestrator_unit.py — replaces the dead
test_emailgen_unit.py. Asserts:
* decnet-orchestrator.service.j2 carries the DECNET_REALISM_*
env block (LLM, MODEL, TIMEOUT, PERSONAS) so per-host tuning
works without editing the .j2.
* Legacy DECNET_EMAILGEN_* vars are NOT referenced — clean break
contract from stage 5.
* decnet.target wants orchestrator + canary, does NOT want
decnet-emailgen.service. Anti-regression for service-collapse.
* deploy/decnet-emailgen.service.j2 stays deleted.
- tests/orchestrator/test_worker_integration.py — new
test_one_tick_email_branch_records_orchestrator_email. Pins the
action-roll to email, seeds a topology with an IMAP mail decky +
two personas, stubs LLM + docker-exec write paths, verifies an
orchestrator_emails row + bus event land. Restores end-to-end
email coverage that was lost when the pre-collapse
test_worker_integration.py was deleted.
- tests/realism/test_synthetic_files_truncation.py — pins the 64KB
last_body cap on create + edit, and documents the consequence:
edit candidates carry a truncated snapshot of files that exceeded
the cap. If a future change lifts the cap, _LIMIT in the test
must lift with it.
Fixed flaky:
- tests/orchestrator/test_scheduler.py — two pick_file tests
pinned to random.Random(1). Without a seed, the 3% canary gate
(stage 7) and 10% leave-alone roll occasionally flaked the
assertions because the _FakeRepo doesn't carry a
create_canary_token method.
Note: the existing
test_realism_subprocess_import_personas_rejects_in_agent_mode
already covers agent-mode rejection of decnet realism
import-personas; no new gating test needed.
Stage 7 — final stage of the realism migration. Canary plants are
now scheduled by the same realism planner that handles inert content,
keeping the orchestrator as the single decision point and avoiding
duplicate diurnal / persona / rate-limit logic in the canary
subsystem.
New surface:
- decnet/canary/cultivator.py: cultivate(plan, repo) builds a
CanaryContext, calls the right generator (canary_aws_creds ->
aws_creds, canary_mysql_dump -> mysql_dump, …), persists the
canary_tokens row before plant so the canary worker can attribute
callbacks even on plant-time previews. Resolves canary placements
to credible operator paths (~/.aws/credentials, ~/.ssh/id_rsa,
/var/backups/db_backup.sql).
- realism/planner.py adds 8 canary content_classes uniformly weighted
inside a 3% probability gate. Hard-capped: each tick at most one
canary; create branch falls through to inert otherwise.
- scheduler.pick_file dispatches canary content_class to the
cultivator; FileAction grows an optional content_bytes field so
binary canary artifacts (DOCX/PDF/honeydoc) survive the wire
intact instead of being utf-8 round-tripped.
- SSHDriver._run_file uses content_bytes when set, falls back to
encoding the str content otherwise.
Stealth (per feedback_stealth.md): cultivator does not introduce
any DECNET literal; the underlying generators are already
stealth-clean and the test suite asserts the contract holds.
Tests cover round-tripping every canary class through the cultivator,
verifying placement-path conventions, persona-login normalisation
("John Smith" -> /home/johnsmith/.aws/credentials), and the
no-DECNET-leak invariant.
Stage 6 of the realism migration. User-class file bodies (note,
todo, draft, script) optionally get LLM-authored content; system
classes (cron / daemon logs, /tmp caches) stay template-only because
formulaic *is* the right look for them.
New surface:
- realism.llm.circuit.LLMCircuitBreaker — process-local sliding-window
breaker. 3 consecutive failures trip open; 60s cooldown to half-open;
half-open success closes, failure re-opens. Protects the orchestrator
tick from sustained Ollama wedges (per-call timeout already covers
one-shot hangs).
- realism.prompts._style — em-dash suppression lifted from the
email prompt. Persona.uses_llms_heavily opts out per the
feedback_em_dash_llm_tell.md memory. Includes strip_em_dashes
belt-and-braces sub for output that slipped past the prompt rule.
- realism.prompts.filebody — class-conditioned prompts (note / todo
/ draft / script) with persona context, language pinning, output
shape rule.
- realism.bodies.make_body_with_llm — async wrapper around make_body
that calls the LLM when one is provided AND the breaker allows.
Falls back to template on timeout / error / empty / system-class.
Wiring:
- scheduler.pick_file accepts optional llm + llm_breaker + llm_timeout.
When the planner picks a create action and the content_class is a
user-class, the body_hint is replaced with the LLM-authored body
(or falls back to the deterministic body_hint).
- orchestrator.worker constructs get_llm() at startup gated by
DECNET_REALISM_LLM env var (any non-empty value enables; empty /
"off" / "none" / "0" disables). Passes llm + breaker through every
tick.
- decnet orchestrate gains --llm/--no-llm flag overriding the env var.
Stage 3b of the realism migration. A TODO.md planted on Monday gets a
checkbox flipped on Tuesday; a notes file grows a follow-up line; a
cron log gets a fresh entry tacked on. The synthetic_files row's
edit_count, last_modified, and content_hash advance.
New surface:
- EditAction dataclass (peer of FileAction in scheduler.py): carries
decky, path, persona, content_class, previous_body, mtime, and
synthetic_file_uuid for the worker's update path.
- realism.bodies.next_iteration(cls, persona, prev, rng): per-class
deterministic mutators. TODO flips an unchecked box and/or appends;
notes/drafts/scripts append; logs are append-only (mirroring real
log behaviour). Canary, cache_tmp, email raise KeyError —
unsupported.
- realism.planner.pick gains an edit branch: 60% create, 30% edit
(when an edit_candidate is supplied), 10% leave-alone. Returns
None on leave-alone — quiet ticks are realism too.
- scheduler.pick_file pre-fetches a single edit candidate via
repo.pick_random_synthetic_file_for_edit ~50% of ticks; the
planner decides whether to use it.
- SSHDriver._run_edit: turns next_iteration output into a
plant_file call (mtime-bumped, mode 0o644). Stashes new_body in
result.payload so the worker can hash it for synthetic_files.
- worker._bump_synthetic_file_after_edit: patches edit_count + 1,
last_modified=now, content_hash, last_body for the row UUID.
No-op when the row was pruned mid-flight.
- events.to_row / topic_for / event_type_for now recognise
EditAction (kind="file", action="file:edit").
Stage 3 of the realism migration. Replaces orchestrator/scheduler.py's
hardcoded _FILE_TEMPLATES/_USERS (3 templates emitting epoch-suffixed
filenames like notes-1777315854.txt with identical bodies per
template) with a persona-driven realism engine.
New surface:
- SyntheticFile SQLModel (synthetic_files table, UNIQUE on
decky_uuid+path) — per-(decky, path) state for the future
edit-in-place flow. Pre-v1, no _migrate_* helper.
- BaseRepository methods: record_synthetic_file,
update_synthetic_file, list_synthetic_files,
pick_random_synthetic_file_for_edit (used by stage 3b).
- realism/naming.py: per-content-class filename templates,
persona-conditioned. /var/log/cron.log + logrotate skeleton for
system-class; /home/<persona>/TODO.md, scratch.md, etc. for
user-class. Anti-regression test pins "no 8+ digit decimals in
basenames" (the realism failure today).
- realism/bodies.py: deterministic body templates per content_class.
TODO body uses checkbox markdown, script body has a shebang, cron
body matches syslog cron shape ("CRON[PID]: (user) CMD (...)").
- realism/planner.py: pick(deckies, now, rng) returns a Plan.
Diurnal-gated, weighted user/system content split (70/30 user
bias). Create-only in stage 3; edit branch lands in stage 3b.
Scheduler split:
- scheduler.pick is now traffic-only (sync).
- scheduler.pick_file is async, takes a repo, resolves personas
(Topology.email_personas for topology-source deckies; global
realism.personas_pool otherwise), and maps Plan -> FileAction.
- FileAction gains persona/content_class/mtime fields.
Worker:
- _one_tick rolls 50/50 between traffic and file each tick. After a
successful FileAction plant, _record_synthetic_file persists or
patches the synthetic_files row (catching the unique-constraint
collision on re-plant of the same path).
- SSHDriver._run_file passes action.mtime through to plant_file so
files don't all stamp at wall-clock-now.
Stage 4 of the realism migration. Lifts the driver Protocol into a
proper ABC with default plant_file/read_file methods (raise
NotImplementedError), and adds get_driver_for(action) so the
orchestrator worker can dispatch by action shape without isinstance
chains.
SSHDriver now inherits ActivityDriver and implements:
- plant_file: streams base64 via stdin (ARG_MAX-safe, mirrors
decnet.canary.planter; commit c17b9e0). Honours mtime via touch -d
so realism-planned files don't all stamp at wall-clock-now.
- read_file: docker exec cat with FileNotFoundError on rc=1, used by
the upcoming EditAction (stage 3b).
EmailDriver inherits ActivityDriver. Driver alias kept for back-compat
during the migration; removed once realism stages 5-7 land.
Empty subpackage skeleton for the realism migration: ContentClass enum
(file/email/canary content categories), Plan dataclass (frozen, with
edit-action invariant), in_work_hours window check (wrap-around
supported, fail-open on parse error), and sample_mtime for backdated
file timestamps that snap into a persona's active hours.
Stage 1 of the orchestrator+canary realism unification — no
production caller wired yet; planner.pick is a stub returning None
until stage 3.
Mirrors the Canarytokens.org trick: a base64-wrapped CHANGE REPLICATION
SOURCE TO + START REPLICA block in the dump trailer. Importing the
file into MySQL resolves <slug>.<dns_zone> (DNS trip) and opens a 3306
replica handshake whose SOURCE_USER smuggles @@hostname and
@@lc_time_names of the victim DB.
DNS lookup alone is sufficient for detection via the existing canary
dns_server; capturing the smuggled metadata via a 3306 handshake
responder is a follow-up.
honeydoc previously emitted HTML only — operators picking 'Document'
out of the dropdown got a .html file dropped at /Documents/
quarterly_report.docx, which any attacker would clock the moment they
ran 'file' on it.
Two new generators that emit the real artifact format:
- honeydoc_docx: stdlib zipfile only. Builds a minimal but valid
Office Open XML zip with the same Q3 review body as the HTML
flavor and an external-image relationship pointing at the
callback URL — same trick the operator-upload DOCX instrumenter
uses, fetched on document open by Word and LibreOffice. Reuses
_drawing() and _next_rid() from instrumenters/docx.py to keep
the body/relationships shape identical between synthesised and
instrumented files.
- honeydoc_pdf: pikepdf-backed. One-page PDF in the 14 base fonts
(Helvetica, no font embedding), realistic body, /OpenAction /URI
on the catalog so most viewers fire the callback on document
open. Falls back to a clear error if pikepdf is missing so the
operator can switch to honeydoc / honeydoc_docx.
Default placement paths now reflect each generator's true extension
(.html / .docx / .pdf) so the UI suggests something sensible. Both
generators surfaced in the New Token modal's generator dropdown.
Real-world plant() crashed with OSError [Errno 7] Argument list too
long when an artifact (honeydoc HTML / DOCX / PDF) base64-encoded
into the sh -c script body exceeded the kernel's argv limit (typically
128KB-2MB depending on the host).
Fix: keep the script trivial ('mkdir -p ... && base64 -d > path && ...')
and stream the encoded bytes through 'docker exec -i ... sh -c'
stdin instead. _run() grew an optional stdin_bytes parameter that's
piped into proc.communicate(input=...). The stdin path covers
arbitrarily large artifacts.
Tests updated:
- test_plant_argv_and_base64_round_trip now asserts the docker -i
flag is present and the base64 payload reaches stdin (and notably
is NOT in the script body).
- _FakeProc.communicate accepts input=None across the board so the
patched fast path no longer trips on the new kwarg.
Fetches GET /deckies on page load and feeds the running fleet into
the create modal as a <select>. Falls back to an empty-state hint
('No deckies running. Deploy a fleet first.') when the list is
empty so the operator isn't staring at an unusable form. Default
selection is the first decky returned.
Switches the page header to the standard .fleet-root .page-header /
.page-title-group / h1 / .page-sub / .actions pattern used by every
other top-level page. Drops the redundant AUTOMATION supertitle (the
sidebar group already labels that) and the inline Target icon next
to the title. Action buttons use the project's btn / btn violet
classes for visual parity with ADD PERSONA / BULK UPLOAD.
Worker unit mirrors decnet-webhook.service shape: simple type, runs
as the decnet user/group, append-style log file, full security
hardening (NoNewPrivileges/ProtectSystem/ProtectHome/PrivateTmp/
LockPersonality + the rest). Added /var/lib/decnet to ReadWritePaths
because the API process persists operator-uploaded canary blobs there.
CAP_NET_BIND_SERVICE granted (ambient + bounded) so an operator who
overrides DECNET_CANARY_DNS_PORT to 53 or HTTP_PORT to 80/443 in
.env.local doesn't need to fight systemd. The defaults stay
unprivileged (5353 / 8088).
Added decnet-canary.service to decnet.target so 'systemctl start
decnet.target' brings it up alongside the rest of the workers.
decnet init auto-discovers deploy/decnet-*.service.j2 files (per
decnet/cli/init.py:_install_units) so no further wiring needed —
running 'decnet init' on a fresh host installs the new unit.
Static tests confirm the unit references decnet canary, depends on
the bus, carries the standard security directives, and is listed
in the master target.
Hooks decnet.canary.planter.seed_baseline into the deploy() flow's
fleet-mirror step. After upserting a FleetDecky as 'running' we seed
the configured baseline canary set on the freshly-deployed decky.
Persona detection: read d.nmap_os (Windows -> windows path-mapping,
otherwise linux). Failures are logged and surface as state=failed
rows in the UI; the deploy itself MUST NOT abort (resilience
principle in CLAUDE.md).
Tests confirm:
- seed_baseline produces one row per configured generator per decky;
- the deployer source wires seed_baseline inside a try/except so a
failure can't abort the deploy.
Two sub-routers under /api/v1/canary:
blobs (operator-uploaded artifacts, deduped by sha256):
- POST /blobs (multipart upload; admin)
- GET /blobs (list with token_count; admin)
- DELETE /blobs/{uuid} (refcount-aware; 409 when referenced; admin)
tokens (per-decky planted artifacts):
- POST /tokens (generate or instrument + plant; admin)
- GET /tokens?decky_name=&kind=&state= (filter; viewer)
- GET /tokens/{uuid} (detail; viewer)
- GET /tokens/{uuid}/preview (instrumented bytes; admin)
- GET /tokens/{uuid}/triggers (paged callback log; viewer)
- DELETE /tokens/{uuid} (revoke + bus event; admin)
XOR validation: exactly one of blob_uuid / generator must be set.
Path validation rejects relative/NUL/newlines/.. segments. Every
body-bearing route documents 400 plus 401/403/404 as applicable.
Stdlib MIME sniffer (no python-magic dep) covers PNG/JPEG/GIF/PDF/
HTML/XML/DOCX/XLSX/JSON/YAML/TOML/text/plain; everything else falls
through to passthrough.
Tests run end-to-end through the live FastAPI app (planter docker
exec is patched); 17 cases covering dedup, refcount, lifecycle,
XOR validation, path validation, and 404 paths.
decnet canary launches the HTTP + DNS callback receiver via
decnet.canary.worker.run. Mirrors the shape of decnet webhook
(typer command with --daemon flag, asyncio.run in the foreground).
Deliberately NOT added to MASTER_ONLY_COMMANDS — every host that
hosts deckies runs its own canary worker, and the bus events stay
local to that host (per-host webhook fanout handles SIEM egress).
decnet canary worker hosts both callback surfaces in one process:
- HTTP: a tiny FastAPI app on its own port (default 8088). The only
meaningful route is GET /c/{slug} which looks up the slug, persists
a CanaryTrigger, publishes canary.<id>.triggered, and returns a 1x1
transparent GIF. Unknown slugs return the same response (stealth);
no decnet strings leak in headers/banners; docs/openapi/redoc are
disabled. X-Forwarded-For is honored.
- DNS: an authoritative UDP server for *.<canary_zone> using
asyncio.DatagramProtocol with stdlib-only DNS wire-format parsing
(no dnslib dep). Same lookup -> persist -> publish flow, plus a
sinkhole A record (192.0.2.1) so the attacker's resolver doesn't
loop on NXDOMAIN. Single-label slugs only; multi-label probes
return NXDOMAIN. Pointer loops in malformed queries are caught
(10-hop cap) so an adversarial packet can't wedge the parser.
Tests cover both surfaces without privileged sockets:
- HTTP via Starlette TestClient: known/unknown slug, headers, XFF,
stealth-string assertions.
- DNS via direct DatagramProtocol drive: known slug -> ANSWER,
unknown -> NXDOMAIN, pointer-loop -> ValueError, malformed
packet -> silent drop.
Plant / revoke / seed_baseline using the same docker-exec-with-sh-c
pattern proven by decnet/orchestrator/drivers/ssh.py:_run_file.
Each plant call composes a single sh script:
mkdir -p <dirname> && printf %s <base64> | base64 -d > <path> &&
chmod <mode> <path> && touch -d @<mtime> <path>
Base64-on-the-host / decode-in-the-container keeps binary artifacts
(DOCX/PDF/PNG) safe across the argv boundary; the placement_path,
mode, and mtime are shlex-quoted.
State transitions hit the repo: planted -> failed on docker error
with stderr captured into last_error. Bus events fire on success
(canary.<id>.placed) and on revoke (canary.<id>.revoked) — wrapped
in try/except so a downed bus never blocks a placement.
seed_baseline(decky_name, repo) is the deploy-hook entry point —
reads DECNET_CANARY_BASELINE (default git_config,env_file,honeydoc,
aws_creds), persists one row per generator, plants each. Failed
placements are logged but do NOT abort; the deployer hook treats
the return list as informational.
Seven instrumenters that mutate operator-supplied artifacts to
embed the callback URL:
- passthrough — bytes unchanged; only DNS-callback tokens trip
detection, with the slug embedded in the placement path
- plain — substitutes {{CANARY_URL}}/{{CANARY_HOST}} placeholders;
falls back to appending a comment line whose prefix adapts to the
apparent file syntax (#, //, ;)
- html — injects a 1x1 tracking pixel before </body>, appends
if the close tag is missing
- docx — direct zipfile manipulation (no python-docx dep):
inserts an external-image Relationship into word/_rels/document.xml.rels
and a matching <w:drawing> element before </w:body>
- xlsx — sibling of docx; injects an external-image relationship
into xl/_rels/workbook.xml.rels (orphan rels are still fetched on
open by most viewers)
- pdf — uses pikepdf to install /OpenAction /URI on the catalog;
rejects with a clear message when pikepdf isn't installed
- image — uses Pillow to embed slug + URL in PNG tEXt / JPEG
comment; rejects with a clear message when Pillow isn't installed
DOCX and XLSX share the rId allocator + relationship injector via
the docx module; both work on stdlib zipfile only.
Tests synthesise minimal real DOCX/XLSX fixtures inline, round-trip
each instrumenter, and assert the callback URL ends up in the
mutated bytes while the file still parses.
Five built-in generators that produce deterministic fake artifacts
keyed by the token slug:
- aws_creds — passive [default]/[prod] credentials block, no
callback wiring (AWS-key tokens require an external
trap, which is post-v1)
- git_config — .git/config with origin url = http_base/c/<slug>/repo.git
- env_file — .env with API_BASE_URL + WEBHOOK_NOTIFY_URL embedding
the callback URL plus inert realism filler
- ssh_key — PEM-shaped fake private key whose host comment carries
<slug>.<dns_zone> when DNS is deployed, else the
http_base host
- honeydoc — minimal HTML report with a 1x1 tracking-pixel <img>
whose src is the callback URL; fallback for the
deploy-time baseline before the operator uploads a
real DOCX/PDF
Tests assert byte-stability (same ctx -> same bytes), slug presence
in the embedded fields, that aws_creds is intentionally URL-free,
and that every artifact carries operator-facing notes for the
preview endpoint.
Mirrors the decnet.intel layout (base + factory + lazy concrete
imports). Defines:
- CanaryArtifact / CanaryContext dataclasses + the generator and
instrumenter ABCs they share
- factory dispatch for generators (git_config/env_file/ssh_key/
aws_creds/honeydoc) and instrumenters (docx/xlsx/pdf/html/image/
plain/passthrough), plus pick_instrumenter_for_mime() for MIME-driven
dispatch on operator uploads
- persona-aware default placement paths (Linux vs. Windows-shaped)
and absolute-path validation that the API will use to validate
operator-supplied placement_path values
- on-disk blob store: sha256-keyed two-level fan-out, idempotent
writes, refcount-aware unlink (the DB row is the source of truth)
Also covers prior commits' tests (bus topics, models, repo CRUD)
under tests/canary/. 79 tests, all pass.
Adds the abstract surface on BaseRepository and the SQLModel-backed
implementation (shared by SQLite and MySQL) for:
- canary blobs (upsert-by-sha256, list-with-refcount, refcount-aware delete)
- canary tokens (create, slug lookup, list with filters, state update)
- canary triggers (record+bump-counters atomically, list, attribute)
The triggers path is a single session that inserts the row and bumps the
parent token's counters together, so a subscriber that reads the token
right after the bus event sees the updated count. Blob delete refuses
while any token (including revoked) still references the blob; pre-v1
revoked tokens stick around for forensic value.
Three new tables for the canary tokens feature:
- canary_blobs — operator-uploaded source artifacts, deduped by sha256
- canary_tokens — one planted artifact in one decky; carries the
callback slug, generator/instrumenter, and lifecycle
- canary_triggers — append-only log of every callback hit; attacker_id
back-filled by the correlator
Pydantic request/response shapes live in the same file per the
single-source-of-truth convention. No migrations file — pre-v1
SQLModel.metadata.create_all() covers it.
Reserved topic family for the upcoming canary-tokens feature so the
correlator and webhook fanout can subscribe to canary.> from day one.
No producers yet; planter, decnet canary worker, and API will publish
in subsequent commits.
* Dashboard / Layout / index CSS — flexbox cleanup so the sidebar
scrolls independently and dashboard panels fill available height
without overflowing the viewport (min-height: 0 on the flex
ancestors that were collapsing).
* pyproject.toml — add sqlite_vec runtime dep (groundwork for an
embeddings-backed feature ANTI is wiring up separately).
* decnet/templates/{rdp,smb}/ntlmssp.py — minimal Type 3 (Authenticate)
parser shared between the SMB and RDP-NLA templates. Lands NTLM
creds in the universal Credential table with secret_kind=ntlmssp_v1
/ ntlmssp_v2 and secret_b64 = base64 of the NtChallengeResponse so
the bounty pipeline can feed the right hashcat mode.
* scripts/decnet-init.sh — convenience wrapper around `sudo decnet init
--force` that targets the current working directory; saves operators
retyping the install paths during dev iterations.
New dashboard surface for editing the global emailgen persona pool —
the JSON file fleet (MACVLAN/IPVLAN) and SWARM-shard mail deckies pull
from. MazeNET topology personas are out of scope here; they're
configured per-topology in the topology editor.
Backend:
* GET/PUT /api/v1/emailgen/personas — admin-write, viewer-read. PUT
validates with the same Pydantic schema the worker uses
(parse_personas), drops invalid entries with a warning, returns 400
only when the entire payload fails. Path is operator-discoverable
on every response so a CLI-driven backup workflow stays visible.
Frontend:
* PersonaGeneration.tsx + .css — table + add/edit modal with the full
EmailPersona schema (name, email, role, tone, mannerisms list,
language, signature, active hours, reply latency, uses_llms_heavily).
Local edits are batched; explicit "SAVE CHANGES" writes back, with a
dirty-indicator pill and a "DISCARD" reset. Email uniqueness is
enforced client-side so the scheduler never picks the same persona
as both sender + recipient.
* Sidebar AUTOMATION group gains a "Persona Generation" entry next to
Orchestrator; route registered at /persona-generation.
The worker reads the same on-disk file the API writes — see
decnet.orchestrator.emailgen.global_pool. The API resets the
in-process cache on every read/write so the worker picks up dashboard
edits within its next tick rather than waiting on mtime.
The SSE pipe at /orchestrator/events/stream was already streaming
'orchestrator.email.{decky_uuid}' events (the subscription is for the
'orchestrator.>' wildcard), but the consumer side dropped them on the
floor. Three fixes to close the loop:
* useOrchestratorStream.ts now registers an 'email' SSE listener — the
EventSource silently ignores frames whose event name has no listener,
so missing this entry meant every email frame was dropped before
reaching the page's onEvent handler.
* /api/v1/orchestrator/events accepts kind=email and dispatches to
list_orchestrator_emails, adapting rows to the existing wire shape:
subject -> action, sender_email -> src_decky_uuid, recipient_email
-> dst_decky_uuid, plus email-specific extras (thread_id, language,
mail_decky_uuid, message_id, in_reply_to) ride along as top-level
keys.
* Orchestrator.tsx gains an 'email' tab in the kind filter and a
branch in the row renderer / inspector that:
- shows full sender / recipient (no UUID truncation),
- chips the language code next to the subject,
- relabels ACTION as SUBJECT in the inspector and surfaces
thread / in-reply-to / mail-decky details.
The 'all' tab continues to show traffic+file only (today's behavior);
operators see emails by switching to the email tab. A union view at
the API layer is the obvious follow-up but not necessary for now.
Plug emailgen into the systemd-supervised fleet:
- New deploy/decnet-emailgen.service.j2 mirroring decnet-orchestrator's
shape: simple service, restart-on-failure, docker supplementary group
(driver shells `docker exec` to drop EMLs into the spool), the same
hardening directives as the rest of the fleet.
- decnet.target now Wants both decnet-emailgen.service and
decnet-orchestrator.service. Orchestrator's absence from the target
was a historical oversight — fixing it here while the file is open.
`decnet init` already globs deploy/decnet-*.service.j2 so the new unit
ships automatically; no init-side change needed. Emailgen-specific env
knobs (DECNET_EMAILGEN_LLM, _MODEL, _PERSONAS, _TIMEOUT) are documented
in the unit and operator-tunable via /opt/decnet/.env.local.
Two-layer gating per CLAUDE.md:
- registration-time: emailgen added to MASTER_ONLY_GROUPS so agents
don't see the sub-app in 'decnet --help' at all.
- body-guard: _require_master_mode('emailgen ...') at the top of every
sub-command body so a direct callable import (third-party tooling)
still bails on agent hosts.
Matches the convention used for 'swarm', 'topology', 'geoip'. SWARM
agents push their generated mail through the master's emailgen worker
(or none at all); cross-agent emailgen federation stays out of scope.
Lift the Ollama subprocess shell-out out of EmailDriver and into a
proper provider subpackage shape:
decnet/orchestrator/emailgen/llm/
base.py — LLMBackend Protocol + LLMResult + LLMTimeout
factory.py — get_llm() reads DECNET_EMAILGEN_LLM
impl/ollama.py — current 'ollama run' subprocess path
impl/fake.py — canned-output backend used by tests
Driver now takes an LLMBackend on construction (or inherits the
factory default). Tests inject FakeBackend instead of monkeypatching
the subprocess layer, which is cleaner and ~10x faster. Swapping
Ollama for the Anthropic API / vLLM / llama.cpp is now a third branch
in factory.py; no driver rewrite needed.
Mirrors the convention used by decnet.web.db.factory + decnet.bus.factory
per the provider-subpackages-from-day-one rule in memory.
Two changes that unwind earlier MazeNET-only assumptions and fix a
realism tell:
1. Persona resolution is now per-decky-source, not topology-only. The
scheduler walks the union view (list_running_deckies, including
fleet MACVLAN/IPVLAN + SWARM shards) and picks the right persona
list for each source:
* topology decky -> Topology.email_personas (per-topology richness
preserved)
* fleet / shard -> a single host-wide pool loaded from disk
(DECNET_EMAILGEN_PERSONAS, /etc/decnet/email_personas.json, or
~/.decnet/email_personas.json)
Operators install the global pool via 'decnet emailgen
import-personas <file>' which validates with the same Pydantic
schema the worker uses.
2. The driver now runs 'touch -d <Date>' inside the docker exec right
after the EML write so file mtime matches the email's RFC 2822
Date: header. Without this an attacker 'ls -lt'ing the spool sees
every email clustered inside the worker's tick window — the
cluster itself was a stylometric tell.
CLI now exposes 'decnet emailgen' as a sub-app with 'run' (default,
backwards-compatible with bare 'decnet emailgen') and 'import-personas'.
list_running_deckies carries topology_id through so consumers can resolve
the parent topology without a second round-trip.
When IMAP_EMAIL_SEED / POP3_EMAIL_SEED points at a directory of .eml
files (the orchestrator emailgen worker's drop path,
/var/spool/decnet-emails/ by convention), the bait mailbox is replaced
with those LLM-generated, persona-driven, threaded messages. Empty /
missing dir keeps the hardcoded fallback so a fresh deployment is never
silent. Cached with mtime invalidation + a short TTL so a hot mailbox
doesn't pay the parse cost on every IMAP/POP3 command.
Replaces the DEBT-026 stub on both templates that named the env var but
never wired it through.
Second orchestrator worker (decnet emailgen) that drips persona-driven,
threaded, multi-language fake emails into running mail deckies. Personas
live on Topology.email_personas; topology-wide language_default falls
through to any persona that doesn't pin its own. Em-dashes are
suppressed at the prompt layer by default and only lifted for personas
explicitly marked uses_llms_heavily — em-dashes are an LLM tell and a
flat corpus of em-dashed mail is a giveaway.
EML delivery writes into /var/spool/decnet-emails/<thread>/<msg>.eml on
the mail decky via docker exec; wiring the IMAP/POP3 templates to read
from that spool (replacing the hardcoded _BAIT_EMAILS) is the next step.
Mirrors the CredentialsInspector pattern: clicking a row opens a
right-edge drawer with the full event payload pretty-printed and
copyable. The table view truncates the src/dst id to 8 chars; the
drawer shows the full identifier plus a SOURCE chip
(TOPOLOGY / FLEET / SHARD) so operators can tell at a glance whether
the orchestrator hit a MazeNET decky, a unihost fleet decky, or a
SWARM shard.
Source detection is purely client-side based on id shape — bare UUID
→ topology, "local:*" → fleet, "<host>:*" → shard. The server
already returns a normalized id from list_running_deckies; this
inspector just labels it.
Backdrop click closes via target===currentTarget guard (per the
React stop-propagation memory: never use stopPropagation on drawer
panels — it breaks native event delegation).
Live (in-flight stream) events use synthetic uuids prefixed "live-";
the drawer hides the EVENT UUID row and shows "LIVE EVENT" in the
header for those, since the server-side id won't exist until the
backend persists the row.
Once the orchestrator started seeing fleet + SWARM shard sources via
list_running_deckies (a844148), every event row landing on a fleet decky
broke the FK to topology_deckies — the column now carries opaque ids
("local:omega-decky" for fleet, "host_uuid:decky_name" for shards) that
will never match topology_deckies.uuid.
Symptom on the operator's mothership:
IntegrityError 1452 — orchestrator_events_ibfk_2 FK violated on every
tick once the reconciler populated fleet_deckies.
Index on dst_decky_uuid is preserved (the dashboard reads
"events for this decky" frequently); only the FK is removed. Keeps
data integrity loose by design — events are append-only history that
should outlive the deckies they reference.
Existing MySQL deployments need the FK dropped manually:
ALTER TABLE orchestrator_events
DROP FOREIGN KEY orchestrator_events_ibfk_2,
DROP FOREIGN KEY orchestrator_events_ibfk_1;
SQLite users are unaffected — SQLite doesn't enforce FKs by default.
The Workers panel (Config → Workers tab) hardcodes its row list in
KNOWN_WORKERS — by design, so a rogue publisher can't inject UI rows.
Three heartbeat-emitting workers were missing:
* clusterer — behavioral clustering (decnet/clustering/)
* campaign-clusterer — campaign assembly (decnet/clustering/campaign/)
* reconciler — host-local fleet convergence (added in 430262e)
Each already publishes on system.<name>.health via run_health_heartbeat,
so they show up live the moment they're added to the registry — no
frontend or subscriber wiring needed (Config.tsx renders whatever
/workers returns).
Also added to _PREFERRED_ORDER in start-all so START ALL WORKERS brings
them up in dependency-friendly order: data-plane → reconciler → intel
→ clustering → output → orchestrator.
Three deployable units (listener, web, swarmctl) intentionally remain
absent from KNOWN_WORKERS — they don't emit heartbeats (CLI / static
server / one-shot tooling), so they'd permanently render as UNKNOWN
and confuse operators. Adding them is a separate decision that needs
a "synthesize installed-but-silent rows" pass on the registry.
Two pieces, one PR because they share a deployment surface:
1. systemd. decnet-reconciler.service.j2 mirrors the orchestrator unit
shape (docker group, hardened sandbox, append-logs). Read-only
/var/lib/decnet so it can read decnet-state.json without write
access. Auto-discovered by `decnet init` via the existing
decnet-*.service.j2 glob — no init.py change needed. Added to
decnet.target so `systemctl start decnet.target` brings it up
alongside collector / sniffer / mutator / etc. Also added to the
agent reaper script so self-destruct cleans it up on workers.
2. Bus signal. reconcile_once now publishes
`decky.<host_uuid:name>.state` on every insert / delete /
state-changed transition. Reuses the existing DECKY_STATE topic
family (no bus/topics.py change → no wiki update needed per the
bus-signals doc rule). Composite host_uuid:name segment keeps
fleet rows distinguishable from MazeNET TopologyDecky rows whose
ids are bare UUIDs. Quiet ticks publish nothing — convergence
means silence.
Bus is plumbed through the worker, defaults to None for unit-test
callers. publish_safely keeps the source-of-truth contract: DB write
is authoritative, the publish is best-effort notification.
Captures previous_state into a local before update_fleet_decky_state
runs — a fake repo that mutates rows in-place would otherwise see the
post-update state and report previous == current. Real repos don't
have this concern but the fix is cheap and makes the function less
order-dependent.
Switches _one_tick from list_running_topology_deckies to
list_running_deckies (the union view added in 095500a). Resolves the
permanent "no actionable deckies (running+ssh count=0)" log on hosts
running only unihost MACVLAN / IPVLAN decoys — the orchestrator now
sees fleet_deckies rows alongside MazeNET topology rows and SWARM
DeckyShard rows.
Also fixes the misleading log message: the old "running+ssh count=N"
reported the *pre-filter* total (count of all running deckies, not
the SSH-eligible subset that scheduler.pick actually evaluates). New
line breaks down running, ssh_eligible, and per-source counts so
debugging "why isn't it picking?" no longer requires reading
scheduler internals.
Regression test: orchestrator integration suite now seeds fleet_deckies
rows (not just topology_deckies) and verifies a tick picks them and
records an event with dst="local:fleet-*" — proving the original bug
on the operator's mothership is fixed.
Adds decnet.fleet.reconciler — a pure async function plus a long-lived
worker — that periodically reconciles the three sources of truth on a
DECNET host:
1. decnet-state.json (CLI-canonical fleet record)
2. fleet_deckies table (DB mirror, written by engine.deployer)
3. docker inspect (actual per-container runtime state)
Drift handling:
* JSON has X, DB doesn't → INSERT (deploy ran with DB offline)
* DB has X (this host), JSON doesn't → DELETE (teardown ran with DB offline)
* Both have X, docker disagrees → flip state to running/failed/degraded
* Docker socket unreachable → leave existing state alone (don't
torch every row to torn_down)
Cross-host safety: deletions are scoped to host_uuid for the local host;
a master that runs both a local fleet and swarm workers will never
clobber a peer's slice.
CLI:
decnet reconcile --once # one-shot, prints counts
decnet reconcile [--interval N] # long-lived worker, mirrors
# orchestrator's lifecycle (control
# listener + heartbeat + tick loop)
Promotes decnet/fleet.py → decnet/fleet/ package so the reconciler can
live alongside it without name collision (build_deckies_from_ini and
all_service_names re-exported unchanged via __init__.py).
14 new tests cover state aggregation rules, all four drift directions,
host_uuid scoping, docker-unreachable safety, and worker shutdown via
the bus control event.
The unihost API path delegates to engine.deployer.deploy(), which now
writes both decnet-state.json (existing) and the fleet_deckies DB
table (added in 646aeec). Comment makes the single-sink design
explicit so future maintainers don't add a parallel save_state /
upsert_fleet_decky call here.
No behavioral change — every fleet-creation path on every host (CLI
deploy, this unihost API path, and per-worker SWARM agent deploys)
already routes through the engine.deployer single sink.
CLI deploy now writes both surfaces: decnet-state.json (existing,
canonical for offline / no-API hosts) and the new fleet_deckies DB
table (visible to orchestrator, web dashboard, REST API).
Best-effort: a DB outage logs a warning and returns. The JSON file
remains the source of truth for `decnet status`, `decnet teardown`,
sniffer, and collector — operators on a CLI-only host keep working.
_run_async helper bridges sync deploy() into the async repository.
Always uses a fresh thread because the API handler at
web.router.fleet.api_deploy_deckies invokes deploy() from inside a
FastAPI event loop, which would otherwise break asyncio.run.
Verified end-to-end against MySQL: deploy mirror inserts rows, union
view (list_running_deckies) returns them with source="fleet",
teardown mirror removes them. Works from both sync (CLI) and async
(API handler) call sites.
Adds a fleet_deckies table so DB-only consumers (orchestrator, web
dashboard, REST API) can see unihost / MACVLAN / IPVLAN deckies
without reading the JSON state file. Mirrors DeckyShard field-for-field.
Composite PK (host_uuid, name) future-proofs for a mothership that
runs both a local fleet and acts as a swarm master. host_uuid defaults
to the "local" sentinel — no FK to swarm_hosts because the local
mothership isn't enrolled as a worker.
Repo additions: upsert_fleet_decky, delete_fleet_decky,
list_fleet_deckies, list_running_fleet_deckies,
update_fleet_decky_state, plus list_running_deckies which unions
topology + fleet + shard sources for the orchestrator.
Smoke-tested round-trip against MySQL: upsert, list_running, union
view (source="fleet"), delete.
TTL extraction was already wired in the active prober and passive sniffer
plus profiler rollup; the checkbox was just stale. TCP/IP stack now
includes ToS/DSCP/ECN, IP-ID sequence classification, and ISN sequence
classification as of the previous three commits.
Mirrors the IP-ID classifier for TCP ISN values: per-source-IP rolling
deque (maxlen=8) populated from each inbound SYN's tcp.seq, classified
on every emission. A 'random' verdict is the modern norm; 'incremental',
'zero', or 'constant' indicates legacy stacks or hand-rolled raw-socket
tooling — a strong fingerprint signal.
Active prober now also captures server_isn (single sample, not classified
in-flight; downstream consumers correlating multi-probe results can apply
seq_class.classify_sequence themselves).
Profiler rollup carries the latest non-'unknown' label into
attacker.tcp_fingerprint. Dedup key already covers isn_class from
the previous commit, so transitions emit cleanly.
UI surfaces ISN class as a colour-coded tag with a ⚠ glyph for
non-random verdicts, since they're the genuinely interesting case.
Adds a per-source-IP rolling sample buffer (deque, maxlen=8) for IP-ID
values seen on attacker SYNs and a stdlib-only classifier in
decnet/sniffer/seq_class.py. Each new SYN appends ip.id and re-classifies
the buffer; the result is logged on tcp_syn_fingerprint events alongside
sample count.
The dedup key now folds in ipid_class so a transition from 'unknown' to
a definitive verdict emits exactly one fresh event instead of being
suppressed by the old (os|options) key. Profiler rollup carries the
latest non-'unknown' label into attacker.tcp_fingerprint.
UI surfaces it as a colour-coded tag in the TCP STACK panel: random
neutral, incremental amber, zero/constant green (the strong signal).
Active prober now reads ip.tos from the SYN-ACK and emits tos/dscp/ecn
alongside the existing TTL/window/options fields. dscp is folded into the
fingerprint hash so different DSCP markings produce distinct signatures.
Passive sniffer logs the same three fields on tcp_syn_fingerprint events;
profiler rollup carries them into the attacker tcp_fingerprint snapshot;
AttackerDetail's TCP STACK panel now surfaces DSCP and ECN cells.
Replaces inline styles + .bounty-root reuse with a dedicated
.orchestrator-root scope. Adds animated status pill (live/connecting/
error), bordered seg-group kind filter that matches DeckyFleet's
fleet-filter-group, dedicated kind chips (matrix-green for traffic,
violet for file), failure-row tint, and a brief 'fresh' tint for
just-prepended live rows that fades after 5s.
DEBT-042 — orchestrator failure-count badge is computed from the
in-memory SSE window; remediation is a dedicated stats endpoint.
DEBT-043 — no frontend test framework configured; the planned
Orchestrator.tsx component test couldn't be written without first
adding vitest + RTL.
New /orchestrator route. Paginated read-only event list with kind
filter (all|traffic|file), pause-stream toggle, in-window failure
badge ('X failures / 1h'), and an SSE-driven 'live' status pill.
Streamed rows prepend on top up to a 500-row in-memory cap.
Sidebar gains an AUTOMATION nav group; Orchestrator is the first
child. Future workers (mutator/prober activity) plug in as siblings.
Every 100 ticks, trim per-dst_decky_uuid history down to 10000 rows
(oldest first). Keeps the events table bounded on long-running fleets
without paying the cost on every write.
GET /api/v1/orchestrator/events — paginated list with optional
kind=traffic|file filter. GET /api/v1/orchestrator/events/stream —
SSE: snapshot on connect, live forward of orchestrator.> bus events
mapped to 'traffic' / 'file' SSE event names.
Repo gains list_orchestrator_events(limit, offset, kind?, since_ts?),
count_orchestrator_events(kind?), and prune_orchestrator_events
(per_dst_cap=10000) for periodic worker-side trimming.
Aligns the bus token with the DB column value; OrchestratorEvent.kind
is 'traffic'/'file' but the topic was 'activity'/'file'. The asymmetry
made consumer code (UI filter, SSE event names) need a translation
layer. No external subscribers existed yet.
Adds a new decnet orchestrate worker whose job is to keep the honeypot
ecosystem from looking suspiciously static — a frozen LAN with no
inter-host traffic and no filesystem aging is its own honeypot tell.
MVP scope:
- New OrchestratorEvent table + repo methods (purpose-built sibling
to Log so synthetic events stay separable from attacker-driven ones).
- New orchestrator.{activity,file}.<decky_id> bus topics +
system.orchestrator.health heartbeat.
- SSH-only driver. Traffic action runs python3 inside src container
to TCP-connect dst:22 and read the SSH banner — real on-the-wire
SSH-protocol traffic without shipping creds. File action drops or
refreshes a small file via docker exec on the destination.
- Random scheduler (50/50 traffic/file when >=2 SSH-capable deckies
are running). Diurnal shaping, role-aware pairing, and session-aware
backoff are explicit non-goals for MVP.
- CLI registration, systemd unit (SupplementaryGroups=docker),
worker-registry entry so the dashboard shows orchestrator health.
- 11 tests: scheduler policy, driver argv shape + injection-safety,
end-to-end one-tick integration with FakeBus + SQLite.
Adds proper /identities and /campaigns list pages following the
Bounty/Attackers convention (page-header + page-title-group +
controls-row + logs-section + logs-table + EmptyState). Both pages
live-update via the existing identity / campaign SSE streams.
Sidebar: Attackers, Identities, Campaigns now group under a
THREAT DATA NavGroup, matching the SWARM grouping pattern.
CampaignDetail and IdentityDetail rewritten to use the house class
system (page-header / logs-section / chip / dim-chip) instead of
inline styles. The campaign chip on IdentityDetail navigates to
/campaigns/:uuid; both pages share a small fp-group helper for
fingerprint listings (added to Dashboard.css).
decnet-clusterer.service.j2 ships the identity clusterer that
landed last session (was overlooked) — bus-woken on attacker.>,
publishes identity.> events.
decnet-campaign-clusterer.service.j2 ships the campaign clusterer
from this session — bus-woken on identity.>, publishes campaign.>
events plus the cross-family identity.campaign.assigned. After=
decnet-clusterer.service so the identity layer is up before the
campaign layer reads its rows.
decnet.target Wants both new units. Both follow the same security
hardening profile as enrich + reuse-correlator.
API: /api/v1/campaigns (paginated list), /api/v1/campaigns/{uuid}
(soft-merge chain follow), /api/v1/campaigns/{uuid}/identities
(member identities), and /api/v1/campaigns/events (SSE under
campaign.> + JWT-via-?token=, snapshot-on-connect). Mirror of the
identity router; same auth, same shape, same OpenAPI tags pattern.
Frontend: CampaignDetail.tsx page (same visual vocabulary as
IdentityDetail), useCampaignStream hook (mirror of
useIdentityStream), /campaigns/:id route, IdentityDetail's
CAMPAIGN badge becomes clickable and navigates to the campaign.
useIdentityStream now listens for identity.campaign.assigned so
the badge appears live without a manual refresh.
Runs the chained identity + campaign clustering pipeline against all
seven fixtures via from_synthetic / from_synthetic_identity adapters
and ratchets every YAML floor to 1.0 — the production clusterer
(and the reference clusterers used in the per-fixture tests) all
score perfectly across ARI / homogeneity / completeness /
singleton_recall on each fixture.
Three substrate fixes surfaced by the ratchet:
- Tuning: shared_infra now Jaccards payload+C2 only; decky_set moved
into cohort_weight to prevent fleet-scarcity false-merges (F1's
shared_wordlist failure mode). Tier weight raised to 1.0 so
shared payload+C2 alone crosses threshold (F5's intended pass).
- Adapter: from_synthetic_identity now reads SyntheticSession
started_at + duration_s for session_windows and per-decky
timestamps (the production-row adapter still uses start_ts/end_ts
when available).
- Fixture data: paused_campaign.yaml's JA3 collided exactly with
vpn_hopping.yaml's (same TLS extension list). The collision
fused two unrelated campaigns under the chained identity layer
in the noise_floor composite. Made paused's JA3 distinct.
Also wires Campaign / CampaignsResponse into models/__init__.py's
__all__ that was missed in the schema commit.
The campaign clusterer worker mirrors the identity-side worker shell
(bus connect, heartbeat, control listener, slow-tick fallback) but
wakes on identity.> instead of attacker.> — campaign-level work is
gated on identity-layer changes, not raw observations.
The connected-components implementation reads identities via
list_identities_for_clustering, projects them with from_identity_row,
runs union-find over combined_campaign_weight, writes campaigns rows,
sets attacker_identities.campaign_id, and runs the same revocable-
merge pass as the identity layer (a merged-out campaign whose
identities no longer co-cluster with the winner gets revoked).
Bus: adds campaign.> family (formed / identity.assigned / merged /
unmerged) plus the cross-family identity.campaign.assigned so
existing identity-stream subscribers see the badge update without
having to subscribe to campaign.>. Wiki Service-Bus.md updated in
wiki-checkout in the same wave per the project's bus-signals
discipline.
CLI: decnet campaign-clusterer registered as master-only via
MASTER_ONLY_COMMANDS; --poll-interval / --daemon mirror the identity
clusterer command surface.
The signal taxonomy for the campaign clusterer (next commit). Mirror
of the identity-layer module but with edge families that don't
translate 1:1: phase-handoff (load-bearing for F5 multi_operator —
the signal the identity-side fingerprint-disagreement veto deliberately
isn't), shared-infra (vetoed at identity level, primary positive
signal here), temporal-overlap (pairwise-relative — F7 invariance
preserved), cohort (weak supporting weight only).
Tier weights tuned so phase-handoff alone crosses threshold (F5),
shared-infra + temporal-overlap together cross (canonical co-op
pattern), and shared-infra + cohort together do NOT (F1
shared_wordlist's failure mode). The F7 time-shift invariant is
explicitly tested on every time-bearing edge and on the combined
weight.
Adds the campaigns table and the BaseRepository / SQLModelRepository
methods that the campaign-clusterer worker (next commit) needs to
populate it. Mirrors the AttackerIdentity layer: schema_version from
day one for federation gossip, soft-merge via merged_into_uuid with a
chain-walking get_campaign_by_uuid, list_campaigns excluding merged-
out rows while list_all_campaigns returns the unfiltered set for the
revoke pass. attacker_identities.campaign_id gets a real FK now that
the target table exists.
useIdentityStream hook mirrors useTopologyStream — opens an
EventSource against /api/v1/identities/events with the JWT in
?token=, dispatches the five named events (snapshot, formed,
observation.linked, merged, unmerged) to the consumer, reconnects
3s after any error.
AttackerDetail subscribes whenever it has an attacker id loaded.
On any event whose payload references this observation's uuid OR
the attacker's current identity_id, refetch /attackers/{id} so the
IDENTITY badge appears (or follows through merges / unmerges) live
without a tab refocus.
IdentityDetail subscribes whenever it has an identity id loaded.
On any event whose payload references this identity_id (formed for
it, merge winner / loser, unmerge resurrected / former-winner), it
refetches both the identity row and its observations list.
Both consumers filter inside onEvent — the hook itself is dumb glue
and stays unaware of which uuids any given component cares about.
Mirrors GET /api/v1/topologies/{id}/events: subscribes to identity.>
on the bus for the duration of the request and forwards each event as
a named SSE frame (formed / observation.linked / merged / unmerged).
The endpoint is broadly scoped (every identity event, not per-uuid)
because both AttackerDetail and IdentityDetail need the same
firehose: AttackerDetail watches for an identity.formed that finally
binds its identity_id; IdentityDetail watches for
observation.linked / merged / unmerged against its current row. A
per-uuid filter would force the client to know its identity before
subscribing, which it doesn't always.
JWT via ?token= (EventSource can't set headers), require_stream_viewer
gate, sse_connection_slot per-user cap, snapshot-on-connect with
the first 50 identities so the client buffer renders without a
separate REST call.
Bus-disabled / unreachable path keeps the connection alive on
keepalives so the client doesn't reconnect-storm; it can re-poll
the REST API on its own timer.
Reworks the clusterer's tick to handle multi-identity components and
re-evaluate prior merges. Two passes per tick:
Pass 1 — per-component reconciliation:
* Fresh component → mint identity (commit 4 path).
* Single-identity component → link unassigned observations.
* Multi-identity component → soft-merge: pick the smallest-uuid
winner deterministically, set merged_into_uuid on each loser,
link unassigned observations to the winner. Observations stay
FK'd to their original identity row — the merge is a soft
pointer, not a re-point. Audit trail preserved; cached
subscribers resolve through the chain.
Pass 2 — revocable-merge undo:
* For each merged-out identity, check whether its observations
still cluster with its winner's. If not, the merge is
contradicted by new evidence — clear merged_into_uuid and emit
identities_unmerged. The resurrected identity keeps its original
uuid, so subscribers that cached it during the merged interval
re-attach without a new lookup.
A pre-built merge-chain dict feeds Pass 1 so the effective-identity
lookup is O(1) per observation. The chain has a hop cap (paranoia
against accidental cycles in the underlying state).
Repo additions on BaseRepository + SQLModelRepository:
* list_all_identities() — includes merged-out rows.
* update_identity_merged_into(uuid, winner_or_None) — single
setter for both merge and unmerge.
DummyRepo coverage stub updated.
Tests:
* Two distinct identities bridged by a new observation merge with
the smaller uuid as winner.
* A pre-seeded soft-merge whose underlying observations diverge
gets revoked; resurrected uuid emerges with merged_into_uuid
cleared.
* Tick is idempotent under no state changes.
Two targeted invariants instead of a wholesale YAML-bounds re-use,
because the existing F6 bounds were tuned for the reference
composite_signals_clusterer (fingerprint OR C2). The production
clusterer trades that aggregation for tier discipline + the
fingerprint-disagreement veto, so its score profile differs even
when its judgments are correct — multi_operator stays as 2 truth
identities, paused_campaign's two DSL actors remain a single cluster
because they share fingerprints, etc. Wholesale bounds re-use would
fight the design.
The two production-side ratchets:
1. singleton_recall ≥ 0.95 at campaign-level scoring — truth-
singleton noise scanners must not be absorbed into real campaigns.
This is the F6 failure mode that motivates the fixture.
2. Intra-campaign recovery under cross-corpus interference:
* vpn_hopping's 5 rotations consolidate to one cluster.
* shared_wordlist A and B stay in disjoint clusters despite
sharing credentials with each other (and with the noise floor).
A future commit can revisit when the production clusterer's identity-
level truth alignment improves (e.g. when paused_campaign's DSL is
extended to mark its two actors as one truth identity).
Fixture 7 ratchet: one campaign across 3 multi-week operational
windows with stable JA3 + HASSH + C2. The production clusterer must
fold all 3 into one cluster despite multi-week silence between
windows; completeness = 1.0.
Time-shift invariance test: applying a +90 day delta to every
session start (and the per-attacker first/last seen) must produce
the same cluster membership as the baseline. This is the runtime
counterpart of the static no-time-fields check on Observation. If
either check ever fails, the clusterer has accidentally grown a
recency-aware edge — fixture 7's whole reason for existing.
Pins down the tier-discipline contract end-to-end:
- Credentials-only overlap doesn't fuse observations (F1 in
miniature).
- ASN-only overlap doesn't fuse observations (F2 in miniature).
- All three weak tiers (medium + low + very-low) stacked still
don't fuse — only a high-tier signal does.
- F1 (shared_wordlist) at identity-level: no false merges, every
row is its own predicted cluster, homogeneity = 1.0.
- F2 (vpn_hopping): 5 distinct ASNs collapse into 1 predicted
cluster, proving JA3 / HASSH dominate ASN as the design
requires.
The combination math itself was wired in commit 5; this commit is
the failure-mode regression suite that gates future tuning of the
tier weights.
Two operators cooperating on one campaign can share C2 endpoints +
stage-1 payloads while running distinct tooling — fixture 5
(multi_operator) is the canonical demonstration. The identity
clusterer must NOT fuse them: shared infra is a campaign-level
signal, not an identity-level one. The campaign clusterer (downstream
work) handles that grouping over identities.
Mechanism: when two observations have non-null fingerprints AND the
fingerprints fully disagree, the high-weight tier drops the payload
and C2 contributions to zero. JA3 / HASSH agreement still returns
1.0 directly — no veto applies when something agrees. Partial
agreement (one slot agrees, another disagrees) is treated as
agreement, since stable-tool partial overlap is more consistent
with one identity than two.
The veto only triggers when there is actual disagreement evidence —
two un-fingerprinted observations sharing a C2 still cluster, since
the absence of fingerprints is not the same as disagreement on them.
Fixture 5 production-clusterer assertion added at identity level:
ARI = 1.0, homogeneity = 1.0, exactly 2 predicted clusters from
2 truth identities. Phase-handoff edges (from the TODO) belong to
the downstream campaign clusterer, not this identity clusterer.
The clusterer now drops a single high-tier function call in favor of
a tier-weighted sum. Tier multipliers (high=1.0, medium=0.6, low=0.2,
very_low=0.05) are tuned so the threshold (1.0) admits high-tier
agreement alone while leaving every weaker tier — and every
combination of weaker tiers — under threshold.
Per-tier discipline tested:
- high alone clusters
- medium alone does NOT cluster (supporting signal only)
- low alone does NOT cluster (fixture 1's failure mode)
- very-low alone does NOT cluster (fixture 2's failure mode)
- all three weak tiers stacked still don't reach threshold
- high + medium clusters (high already saturates)
The combination is forward-compatible: low + very-low contributions
are computed today but always project to 0.0 because the production
adapter doesn't populate credentials / ASN-edge inputs into the
fixture path yet. Their contribution becomes load-bearing in commit 7
when the low-tier landing tightens the F1 / F2 bounds.
Fixture 4 (paused_campaign) ratchet added: high-tier signal carries
the multi-day-silence campaign into one identity. Time-agnostic
invariant — silence is irrelevant to the edge weight.
The connected-components clusterer now writes attacker_identities
rows + sets attackers.identity_id when high-weight signals (JA3 /
HASSH / payload-hash / C2-endpoint exact match) agree across
observations. Singletons stay un-fingerprinted and un-clustered.
Algorithm split:
- cluster_observations(observations) — pure union-find over the
high-weight edge function. Same code path for fixture validation
and production tick.
- from_attacker_row(row) — production-row adapter; recovers JA3 +
HASSH from Attacker.fingerprints JSON. Payload + C2 join from
logs in later commits; the function shape doesn't change.
Repo additions on BaseRepository + SQLModelRepository:
- list_attackers_for_clustering(limit=None)
- create_attacker_identity(row)
- set_attacker_identity_id(attacker_uuid, identity_uuid)
DummyRepo coverage stub updated.
v1 behavior is conservative: only assigns identities to observations
whose identity_id is currently NULL. Multi-identity components are
skipped this pass — merge / re-assign lands in commit 10 with
revocable merges.
Fixture bounds tightened against the production clusterer:
- lone_wolf (F3) — singletons stay singletons
- shared_wordlist (F1) — credential-only overlap doesn't cluster
(high-weight tier doesn't include credentials)
- vpn_hopping (F2, identity-level) — 5 rotated IPs with stable JA3
+ HASSH fold into one identity, ARI = 1.0, completeness = 1.0
Adds the four weight-tier edge functions as pure, time-agnostic
scoring primitives over an Observation projection. Each returns a
score in [0, 1]; the connected-components impl will combine + threshold
in subsequent commits.
Tier semantics (from IDENTITY_RESOLUTION.md):
- high — JA3/HASSH/payload-hash/C2-endpoint exact match
- medium — phase-bucketed command-sequence Jaccard
- low — credential-attempt-set Jaccard (defeated alone by F1)
- very low — ASN equality (defeated alone by F2)
Time-agnostic invariant is a static test: Observation has no time
fields, so no edge function can silently start using them. Fixture 7
forbids recency-decay clustering on multi-month APT campaigns.
A from_synthetic() adapter projects SyntheticAttacker corpora into
Observation; the production-row adapter lands when the clusterer
starts reading the attackers table.
Revocable merges (a contradiction-driven undo of identity.merged) ship
in the clusterer work; this reserves the topic up-front so identity.>
subscribers receive it day one without a re-subscribe.
The clusterer worker's ClusterResult fan-out now publishes on
identity.unmerged when populated. The skeleton clusterer never
populates it; the revocable-merge commit will.
Wiki update lives in wiki-checkout/Service-Bus.md (separate repo).
Adds the decnet clusterer master-only command + provider-subpackage
shape (base.py + factory.py + impl/connected_components.py) so
subsequent commits can land similarity-graph features without
churning callers.
The skeleton ConnectedComponentsClusterer.tick is a no-op; the
worker shell is fully wired (bus consumer on attacker.observed +
attacker.scored, slow-tick fallback, health heartbeat, control
listener, ClusterResult fan-out to identity.formed/observation.linked
/merged). Subscribers on identity.> see no events from this clusterer
until edge functions land, but the lifecycle is in place.
Multi-month APT campaign modeling real APT operational tempo: recon
over weeks, exploitation later, action-on-objectives later still.
The unique signal this fixture stresses is TIME-AGNOSTIC IDENTITY
across multi-week silences — a clusterer that silently expires old
edges fragments any campaign that operates over months.
Three DSL actors represent the operator's three operational windows
(week 2, month 2, month 3 of a 90-day campaign), all sharing JA3 +
HASSH + payload + C2 callback. Campaign-level fixture only — the
three actors mint distinct truth_identity_id rows by design (same
modeling caveat as fixtures 4 and 5).
The fixture's narrative mirrors how an APT works a deep nested
topology (DECNET MazeNET mode): map decoy networks for weeks, only
then commit to exploitation. Slow-and-low pacing is the signal.
recency_decay_clusterer added to fixture_harness — same edge
construction as composite_signals_clusterer, but each edge weighted
by exp(-time_distance / half_life_days) and dropped below a
threshold. Adversarial reference for slow_burn: with 14-day half-
life and 0.5 threshold, edges between operational windows (24+ days
apart) decay below threshold and drop. The campaign fragments into
three clusters; completeness collapses.
This is the canonical production failure mode for graph clusterers
that bound memory or bias toward "what's hot" by silently expiring
old edges. Catching it in synthetic data is what fixture 7 exists
for; the replay tier will surface real-world drift / dwell patterns
that calibrate the half-life threshold the real algorithm should
tolerate.
Four tests: corpus shape (window-isolated sessions, stable
fingerprint), pipeline pass via composite_signals_clusterer (time-
agnostic — folds all three windows), adversarial fragmentation
(3 clusters at 14-day half-life), long-half-life sanity (gentle
decay unions everything; confirms behavior depends on the half-life
parameter, not on something unrelated).
Bundles all five prior fixtures' campaigns into one corpus alongside
10 fresh Delivery-only noise scanners (on top of lone_wolf's 8
inherited). The fixture covers cross-corpus interference — signal
collisions across fixtures' JA3/HASSH/C2 strings, factory ID re-use,
clusterer ambiguity that only manifests when multiple campaigns
score together. Each constituent fixture already ships its own
in-fixture adversarial test; this one is the control for the class
of failures that single-corpus fixtures cannot catch.
Composition is declared via a fixture-6-specific include_fixtures
block in noise_floor.yaml. The test file's loader expands it into
a full corpus.campaigns spec at runtime so the factory itself stays
unaware — no factory primitive added for what only this fixture
needs. The 8 noise scanners declared by lone_wolf flow through
naturally; the extra_noise_scanners count adds 10 more.
composite_signals_clusterer (added in the fixture-5 commit) is the
pass clusterer — union-find combining (ja3, hassh) match OR
overlapping C2 callback. Approximates the planned similarity graph
well enough that every campaign resolves and every singleton stays
singleton in the merged corpus.
Three tests: corpus integrity (every campaign id present, 12
campaign-driven attackers + 18 noise = 30 total), pipeline pass
against the global bounds, and an explicit singleton-recall
assertion (21 truth-singletons — 1 lone wolf, 18 noise, 2
shared_wordlist actors whose campaigns are size 1 — all kept
singleton by the composite clusterer). Singleton recall is the
load-bearing metric here: noise absorption is the failure mode
that makes campaign attribution useless in practice.
Three new reference clusterers in fixture_harness:
* c2_callback_clusterer — union-find on overlapping C2 callback
sets across an attacker's sessions. Pass-clusterer for fixture 5
where two operators with distinct tooling share a C2 endpoint as
the campaign signal.
* shift_clusterer — deliberately-bad reference that buckets
attackers by majority session-start hour into night/day/swing.
Adversarial reference for fixture 5; proves operational schedule
is NOT a campaign signal.
* composite_signals_clusterer — union-find combining (ja3, hassh)
match OR overlapping C2 callback. Will serve as the pass-
clusterer for fixture 6 (noise_floor) where multiple campaigns
with heterogeneous signal types are scored together.
Also factored a small _union_find helper for the new clusterers
(existing time_window/credential_jaccard left untouched to avoid
mixing refactor with feature work).
Fixture 5 (multi_operator): one campaign, two operators with
distinct UKC roles. Actor A (broker, night shift): Delivery →
Exploitation → Persistence → C2. Actor B (post-ex, day shift):
Discovery → Lateral Movement → Collection → Exfiltration.
Distinct JA3/HASSH/ASN/IPs; shared C2 + payload hash.
Four tests: corpus shape (distinct fingerprints, shared C2,
disjoint shifts), pipeline pass via c2_callback_clusterer,
explicit harness sanity that fingerprint_clusterer cannot resolve
this fixture (documents which signal carries the campaign), and
adversarial shift_clusterer fragmentation.
Phase-handoff edges (the real load-bearing signal per the design
doc) wait for the production clusterer; this fixture will prove
they're needed when it ships.
Adds the actor.active_days primitive to the campaign factory so a
DSL actor can be bound to specific day indexes. Falls back to the
non-paused day pool when absent (existing fixtures unchanged).
Intersects with pause_windows so the campaign-wide silence still
wins if both are set.
Adds time_window_clusterer reference to fixture_harness — union-find
over attackers, edge if their session time-ranges are within
gap_days of each other. Deliberately-bad reference for fixture 4:
multi-day silent stretches fragment a single campaign because the
clusterer has no signal that bridges the gap.
Fixture 4 (paused_campaign): one campaign modeled as two DSL actors
representing the operator's two operational windows (active days
1-2 and 6-7), separated by a silent stretch (days 3-5). Both share
JA3 + HASSH + payload + C2 callback; only their active_days differ.
Five tests: corpus shape (rows in their windows, shared signals),
pipeline pass via fingerprint_clusterer at level=campaign,
adversarial fragmentation via time_window_clusterer (1-day union
threshold cannot bridge the 4-day silence → completeness collapses),
huge-gap sanity (gap_days=10 unions both halves), silent-stretch
invariant (no session leaks into the configured pause window).
Identity-level scoring is fixture 2's job; this fixture is
campaign-level only — modeling caveat documented in the YAML.
One campaign, one DSL actor, ip_pool: rotating + rotation_count: 5
across 5 synthetic private-use ASNs (RFC 6996 64512-64516). Stable
JA3, HASSH, and payload_hash across every rotation — these are the
"signals the attacker can't cheaply rotate" per IDENTITY_RESOLUTION.md
and the load-bearing reason all 5 observation rows must resolve to
one identity / one campaign.
Two new reference clusterers in fixture_harness.py:
* fingerprint_clusterer — groups by (ja3, hassh). Un-fingerprinted
rows stay singleton so it doesn't trivially fuse all noise into one
mega-cluster. Approximates the stable-signal arm of the planned
similarity graph.
* asn_clusterer — deliberately-bad reference for fixture 2's
adversarial test. Group-by-ASN shatters the campaign into 5
singletons; completeness collapses to 0.
Four tests in test_vpn_hopping_fixture.py: corpus shape (5 rows, 1
identity, 1 campaign, 5 distinct ASNs/IPs, stable fingerprints),
pass at campaign level, pass at identity level (asserts ARI exactly
1.0), asn_clusterer breaches the completeness floor.
Open Question 1 (merge revocability): adopted. The clusterer will
clear merged_into_uuid on contradicting evidence and publish a new
identity.unmerged topic alongside the existing three identity.* topics
so subscribers on identity.> get it from day one.
Open Question 2 (AttackerDetail UX on identity_id change): adopted
SSE over refresh-on-focus. New endpoint will mirror the existing
topology mutator SSE (bus subscription on identity.>, JWT via ?token=,
snapshot-on-connect + live forward).
Risk 2 (API URL stability for soft-merged loser UUIDs): struck —
already shipped in commit dc3d08d (read-only API follows
merged_into_uuid and surfaces the canonical winner).
Fifth and final commit of the identity-resolution substrate. Unblocks
fixture 2 (vpn_hopping) by making the synthetic factory match
production shape: an actor rotating across N IPs produces N
SyntheticAttacker rows that share fingerprints + truth_identity_id but
differ on ip / asn — exactly the shape the future clusterer needs to
recover via JA3/HASSH match.
Factory:
* SyntheticSession + SyntheticAttacker gain truth_identity_id field.
* DSL: ip_pool: rotating + rotation_count: N produces N observation
rows per actor. Optional rotation_asns: [...] cycles ASN per row;
defaults to the actor's primary asn.
* Sessions distribute round-robin across the actor's rotated rows.
* Noise scanners get truth_identity_id == truth_actor_id ==
truth_campaign_id (each is its own singleton at every level).
* GeneratedCorpus.truth_labels(level=) accepts "campaign" (default,
back-compat), "identity", or "actor" — picks the oracle the
metric harness scores against.
Harness:
* assert_fixture_bounds gains truth_level kwarg (default "campaign")
so identity-resolution fixtures can score against truth_identity_id
without churning the campaign-clustering test files.
Tests: 9 new (rotation_count emits N rows, shared identity +
fingerprints, distinct IPs, rotation_asns distribution + cycling,
round-robin session distribution, identity-level truth labels,
sticky default unchanged, sessions inherit identity label).
598 tests green across clustering / factories / db / web / bus /
profiler / correlation.
Fourth of the five-step identity-resolution substrate. Constants and
builder ship now; no publishers exist yet — they land with the
clusterer worker. Subscribers (webhook worker, dashboard SSE relay)
can register against identity.> from day one.
* decnet/bus/topics.py — IDENTITY root + IDENTITY_FORMED /
IDENTITY_OBSERVATION_LINKED / IDENTITY_MERGED leaves; identity()
builder mirroring the attacker() / system() helpers. Module
docstring topic-tree updated.
* tests/bus/test_topics.py — assert builder produces the expected
three topic strings + rejects empty event_type.
Wiki Service-Bus.md and a new Identity-Resolution.md page land in the
companion wiki-checkout commit.
Third of the five-step identity-resolution substrate. Frontend hooks
into the empty /api/v1/identities/* surface from commit 2; renders
nothing visible when identity_id is null (which is the universal state
until the clusterer ships).
* decnet_web/src/components/IdentityDetail.tsx — new page. Header with
uuid + optional CAMPAIGN / MERGED-INTO badges, stats row
(observations / JA3 / HASSH / payloads / C2), fingerprint tag lists
parsed from the JSON-in-TEXT columns, observations table that links
back to AttackerDetail, conditional analyst-notes panel.
* decnet_web/src/components/AttackerDetail.tsx — IDENTITY badge
inserted in the header row alongside TRAVERSAL. Clicking navigates
to /identities/<uuid>. AttackerData interface gains the optional
identity_id field.
* decnet_web/src/App.tsx — /identities/:id route + lazy-loaded chunk.
Verified by `tsc --noEmit` (clean) and `vite build` (clean — produces
IdentityDetail-*.js as its own lazy chunk). The repo has no JS test
harness; build + type-check are the gate.
Second of the five-step identity-resolution substrate. Ships the API
surface against the empty AttackerIdentity table from commit 1 — every
endpoint returns empty/404 cleanly until the clusterer populates rows.
Routes (auth-gated, viewer role):
* GET /api/v1/identities — paginated list, excludes merged-out rows
* GET /api/v1/identities/{uuid} — detail; transparently follows
merged_into_uuid to surface the canonical winner
* GET /api/v1/identities/{uuid}/observations — Attacker rows FK'd
to the (resolved) identity uuid
Repository (BaseRepository abstract + SQLModelRepository concrete):
* get_identity_by_uuid (with merge-chain following, hop-bounded)
* list_identities / count_identities (excluding merged-out)
* list_observations_for_identity / count_observations_for_identity
Tests: 12 new (empty-table behavior, seeded data, merge-chain
resolution, repo-level smoke against real SQLite). Also fixes the
pre-existing test_base_repo_coverage failure (DEBT-041 added abstract
methods without updating the DummyRepo stub) — included here because
this PR adds 5 more abstract methods, fixing it as a bonus.
474 db/web/profiler/correlation tests green.
Schema-only commit, first of the five-step substrate for identity
resolution. The clusterer that populates identities lands later; this
ships the table empty and the FK uniformly NULL on existing rows.
* decnet/web/db/models/attackers.py — new AttackerIdentity SQLModel
(uuid PK, schema_version, fingerprint summary lists, kd_digraph_simhash,
merged_into_uuid self-FK, all clusterer-populated fields nullable).
Attacker grows a nullable indexed identity_id FK + docstring marking
it as the per-IP observation row.
* decnet/web/db/models/__init__.py — re-exports AttackerIdentity.
* tests/db/test_identity_schema.py — 9 schema invariants: table exists,
identity_id nullable + indexed, FK targets attacker_identities.uuid,
schema_version defaults to 1, attacker rows inserted with NULL
identity_id, FK constraint blocks orphans.
463 unrelated db/web/profiler/correlation tests still green. See
development/IDENTITY_RESOLUTION.md for the full design.
Pre-implementation design for the observation/identity/campaign
three-level hierarchy. Sibling-add approach (not rename) — keep the
attackers table name, add attacker_identities as a sibling, nullable
attackers.identity_id FK. Documents the rationale, schema, bus
topics, API surface, and the 5-commit implementation sequence.
Companion to development/CAMPAIGN_CLUSTERING.md. Substrate for the
clusterer worker designed there; ships empty so the campaign
clustering fixtures can encode honest multi-row-per-actor scenarios.
Two campaigns sharing a credential wordlist; everything else (ASN, IPs,
JA3, HASSH, active hours) divergent. Pass condition: clusterer must NOT
merge. Protects against the "credential overlap is identity" failure
mode that commodity wordlists invite.
* tests/clustering/fixture_harness.py — shared assert_fixture_bounds
helper + identity_clusterer (placeholder, trivially correct on
all-singleton fixtures) + credential_jaccard_clusterer (deliberately-
bad reference used to PROVE the fixture catches what it should).
* tests/clustering/test_shared_wordlist_fixture.py — bounds pass with
identity, bounds FAIL (homogeneity → 0) with the bad credential
clusterer. The latter is the proof the fixture earns its keep.
* tests/fixtures/campaigns/shared_wordlist.{yaml,expected.yaml}.
* tests/clustering/test_lone_wolf_fixture.py — refactored onto the
shared harness. No behavior change.
Pre-implementation scaffolding for campaign clustering. The simulator is
the spec — algorithm code follows once fixtures + metrics are stable.
* decnet/clustering/ukc.py — UKCPhase enum (19 phases across In/Through/Out
stages), OBSERVABLE_PHASES set, stage_of() helper. Vocabulary aligns
with future MITRE ATT&CK tagging so synthetic data and runtime phase
inference don't need renaming when TTP-tagging lands.
* tests/factories/campaign_factory.py — YAML DSL parser + deterministic
generator emitting truth-labeled SyntheticAttacker / SyntheticSession
records. Validates phase names, warns on unobservable phases, supports
multi-campaign + noise corpora.
* tests/clustering/metrics.py — pure-Python ARI / homogeneity /
completeness / singleton_recall (no sklearn dep). Decided before any
algorithm exists, on purpose.
* tests/fixtures/campaigns/lone_wolf.{yaml,expected.yaml} — fixture 3
from the design doc; simplest of the six, exercises the full pipeline
with an identity-clusterer placeholder.
* development/CAMPAIGN_CLUSTERING.md — design spec for the feature.
* development/DEVELOPMENT_V2.md — note on DSL evolution path
(concurrent phases, multi-actor per phase) deferred post-v1.
The threat-intel surface was IP-keyed on day one as an expedient — the
worker is woken by IP-bearing bus events. ANTI's call: don't carry that
debt. NO IPs as primary keys anywhere on the attacker-intel surface.
Schema:
- attacker_uuid is now the canonical key — UNIQUE + FK to attackers.uuid.
- attacker_ip stays as a denormalised, indexed, NON-UNIQUE value column.
Updated on every upsert; useful for SIEM payloads and audit lookups,
but explicitly NOT a key. Model docstring says so.
- Pre-v1, no Alembic migration needed. SQLModel.metadata.create_all()
builds the new shape on fresh DBs.
Repo:
- upsert_attacker_intel now keys on attacker_uuid.
- get_attacker_intel_by_ip → get_attacker_intel_by_uuid.
- get_unenriched_attacker_ips → get_unenriched_attackers, returning
[{uuid, ip}] tuples so the worker writes by UUID and dispatches
provider calls by IP without a second round-trip.
Worker:
- _enrich_one(uuid, ip, ...) — UUID lands on the row, IP rides for
provider egress.
- attacker.intel.enriched bus payload gains attacker_uuid alongside
attacker_ip — webhook → SIEM consumers benefit; no removal.
API:
- GET /api/v1/attackers/{ip}/intel deleted outright (rip-and-replace,
never deployed beyond dev).
- GET /api/v1/attackers/{uuid}/intel is the only public route, matching
every other /attackers/* route.
Frontend:
- <IntelPanel uuid={id!} /> uses the URL param directly, fetches in
parallel with the rest of AttackerDetail rather than waiting on
attacker.ip.
Tests: re-keyed in place, 39 passed (same coverage as before the
refactor). Provider-impl tests untouched.
DEBT-041: closed in DEBT.md (entry preserved as historical rationale,
summary table flipped to ✅, remaining-open list shortened by one).
Read-only IP-keyed intel surface on the attacker detail page. Renders
the aggregate verdict (color-coded MALICIOUS/SUSPICIOUS/BENIGN/NO SIGNAL)
plus a per-provider row with verdict, queried-at timestamp, and
provider-specific detail (GreyNoise classification, AbuseIPDB
0-100 score, Feodo C2 listing + malware family, ThreatFox IOC match
+ malware family). 404 from the API renders as 'NO INTEL CACHED YET'
with a hint that decnet enrich will populate it on the next pass —
TTL drives the refresh, no manual button.
DEBT-041 documents the API/UI IP-keying as a v1 expedient that will
need a UUID-keyed sibling endpoint before federation lands. NAT
collisions, attacker.uuid consistency across attacker routes, and the
sequential-fetch UX are all callouts on that ticket; the migration
sketch is laid out so the v1.x followup is unambiguous.
Frontend build: clean (55.58 kB AttackerDetail bundle, +~5kB for the
panel). Note: not browser-tested in this session — recommend a manual
smoke against a deployed master before tagging.
Mirrors decnet-reuse-correlator.service.j2: same hardening posture
(NoNewPrivileges, ProtectSystem=full, etc.), same restart policy, same
log file convention. The decnet init renderer picks it up automatically
via the decnet-*.service.j2 glob.
Also reconciles a naming inconsistency I shipped earlier: the heartbeat
name was 'intel' (the package) but the CLI command and unit are 'enrich'
(the action). Renamed the heartbeat to 'enrich' so the workers panel
displays the same string the operator types and the same string in the
systemd unit file. Convention across the project: heartbeat name =
registry key = unit basename = CLI command name.
Registers 'enrich' in worker_registry.KNOWN_WORKERS and in the
start-all preferred order. The decnet.target Wants= list also picks
up the new unit so 'systemctl start decnet.target' brings everything
up together.
CLI command mirrors the reuse-correlate shape (--poll-interval, --ttl-hours,
--daemon). Run it under systemd as a sibling worker.
The API endpoint returns the most recent cached row for an attacker IP
or 404. Auth-gated via require_viewer like every other attacker route.
Also extends the worker test with a real FakeBus so the
attacker.intel.enriched publish path is exercised end-to-end (no longer
a no-op against NullBus).
Four concrete IntelProvider impls — three per-IP queries plus one bulk
feed:
* GreyNoiseProvider — community endpoint, optional API key for higher
rate limit. 404 = unknown (cache the absence so we don't re-query).
* AbuseIPDBProvider — score threshold mapping (>=75 malicious, >=25
suspicious, else benign). Self-disables with a clear error when no
API key is configured rather than burning quota.
* FeodoProvider — fetches the bulk botnet C2 IP feed once per refresh
window and answers every lookup from an in-memory set. Listed = C2.
* ThreatFoxProvider — POST /api/v1/ search_ioc query, optional Auth-Key
header. Match in data[] = malicious; no_result = absence-not-benign.
Every provider routes through decnet.net.http.stealth_client so the
egress UA never leaks 'DECNET'.
run_intel_loop fans out across configured providers per IP, writes the
aggregate row, and publishes attacker.intel.enriched. Mirrors the
correlation/reuse_worker.py wake-on pattern: subscribes to
attacker.observed and attacker.scored for sub-second latency, falls back
to a 60s poll when the bus is unavailable. Heartbeat + control-listener
wired so the workers panel sees it like every other supervised worker.
Aggregate verdict picks the strongest provider tier (malicious >
suspicious > benign > unknown). Provider-level errors land in
IntelResult.error and are logged without poisoning the row — partial
success is the expected case for free-tier providers under their daily
caps.
Concrete provider impls land in follow-up commits; the worker is fully
exercised here against fake providers so the framing is locked in.
Outbound calls to 3rd-party services (threat-intel providers, future TI
lookups) MUST NOT advertise 'DECNET' in their user-agent — operators
running honeypots want their reconnaissance dependencies to look like
generic infra. New decnet.net.http.stealth_client() returns a fresh
httpx.AsyncClient with a curl-shaped UA (pinned to a single constant so
future siblings — browser-shaped, Go-shaped — sit next to it cleanly).
Internal egress (webhook → operator's own SIEM, swarm worker → master)
keeps its DECNET-tagged UA; the docstring is explicit about not routing
those through this client.
IntelProvider is async-first (every concrete provider does HTTP), bounded
by a per-provider asyncio.Semaphore, and contractually never raises —
errors land in IntelResult.error so a single provider's outage doesn't
poison the worker pass for an entire IP.
Factory returns a list (not a singleton like geoip) because intel
enrichment fans out across all enabled providers per IP, with row-level
partial-success handling. Lazy imports keep the module dependency-free
when intel is disabled.
Concrete providers (greynoise/abuseipdb/feodo/threatfox) land in
follow-up commits — factory references them via lazy import so tests
covering the disabled and unknown-name paths pass on their own.
New TTL-cached threat-intel row keyed by attacker IP, with per-provider
verdict/raw/queried_at columns for GreyNoise, AbuseIPDB, abuse.ch Feodo
Tracker and ThreatFox. Carries schema_version from day one (federation
wire-format precedent set by SessionProfile). Repo gains
upsert_attacker_intel, get_attacker_intel_by_ip, and a
get_unenriched_attacker_ips backfill primitive that picks fresh + stale
rows for the forthcoming 'decnet enrich' worker.
Also documents the open-source intel-source backlog in DEVELOPMENT_V2.
The CredentialReuse table only stores the sha256+kind hash of the
secret; the printable + b64 forms live on the underlying Credential
rows. The dashboard drawer was therefore showing only the hash, which
defeats most of the value of having a reuse view in the first place.
Repo helpers list_credential_reuses + get_credential_reuse_by_id now
issue one batched SELECT against credentials keyed on the sha256s in
the result page and graft secret_printable + secret_b64 onto each row
before returning. The drawer renders the same printable/b64 code-block
the credentials inspector uses.
Adds the systemd template for the credential-reuse correlator daemon
and wires it into decnet.target so `decnet init` installs it
automatically (the unit installer globs decnet-*.service.j2). Mirrors
the mutator template: bus-woken Type=simple service with the standard
hardening + on-failure restart.
Also registers `reuse-correlator` in the in-process worker registry
(so the dashboard panel surfaces its heartbeat instead of dropping it
as unknown) and slots it into the start-all preferred order between
mutator and webhook.
Library shape (decnet/correlation/) consumed by profiler + reuse
correlator is the right model. The `decnet correlate` CLI helper has
been removed in the previous commit.
The CLI was a day-one debug helper that read a log file or stdin and
printed a traversal table. It hadn't been wired to the live data path
since the engine moved into the profiler worker (DEBT.md:218). No
deploy unit, no caller, no doc relied on it. Removed the command and
its two tests; `decnet/correlation/` stays as a library consumed by
the profiler and the reuse correlator.
Adds a CREDS/REUSE tab segment on the Credential Vault page. The REUSE
tab lists CredentialReuse rows (paginated 25 per page) ordered by
target_count desc; row-click opens a drawer mirroring the credentials
inspector with a deckies x services grid, attacker links, and a
PROFILING PENDING placeholder when attacker_uuids has not been
backfilled yet.
The CREDS tab gains a REUSE column showing a clickable target-count
badge for credentials whose (sha256, kind, principal) tuple matches a
reuse row; clicking the badge fetches and opens that row's drawer.
Section header gains a manual refresh button (no SSE/polling).
Ticks the credential-reuse line in DEVELOPMENT.md and notes the
vectorstore scaffold.
Read-only routes for the credential-reuse findings produced by the
correlator. Mirrors the /credentials route shape: JWT-gated via
require_viewer, paginated with optional secret_kind /
min_target_count filters, and a 404-on-missing detail route.
No POST/PUT/PATCH (and no body parsing) so no 400 contract is
documented.
Adds CorrelationEngine.correlate_credential_reuse + the
`decnet reuse-correlate` long-running worker. The worker mirrors the
mutator's bus-wake + slow-tick pattern: wakes on credential.captured
and attacker.observed for sub-second latency, falls back to a 60s
poll if the bus is unavailable, and publishes
credential.reuse.detected once per new or grown CredentialReuse row
(group-deduped so a 5-cred reuse doesn't emit 5 partial events).
The web ingester now publishes credential.captured after every
successful Credential upsert; bus + new repo helper
find_credential_reuse_candidates feed the engine pass.
Credential capture runs before the profiler mints an Attacker, so
Credential.attacker_uuid is nullable on write. The profiler now
backfills the FK after each successful upsert_attacker. Soft-fail
posture matches the surrounding behavior + smtp rollups so a backfill
error never blocks the next attacker.
Lays the storage and bus substrate for the "credential reuse patterns"
task in DEVELOPMENT.md and scaffolds decnet/vectorstore/ as the future
substrate for statistical attacker re-identification over behavioral
fingerprints. No correlator, profiler, API, or dashboard wiring in
this commit — see TODO.md for the handoff.
Schema:
- Credential.attacker_uuid (nullable FK to attackers.uuid),
backfilled by the profiler post-write to avoid coupling the
capture path to the profiler's ordering.
- CredentialReuse table — UUID PK, JSON list columns for the
accumulating attacker_uuids/ips/deckies/services, target_count
(the discriminative scalar), confidence reserved for a future
fuzzy-credential pass.
Repo:
- upsert_credential_reuse / list_credential_reuses /
get_credential_reuse_by_id / update_credential_attacker_uuid.
- Renamed pre-existing get_credential_reuse(secret_sha256) to
get_credential_attempts_for_secret(secret_sha256) — the new
findings table needs the cleaner name.
Bus topics:
- credential.captured (one per Credential upsert)
- credential.reuse.detected (correlator-emitted on insert/grow)
Vectorstore subpackage (decnet/vectorstore/, flat layout mirroring
decnet/bus/):
- BaseVectorStore ABC keyed by (kind, id) — kind discriminator
means new feature families are additive, no schema migration.
- FakeVectorStore (in-memory L2 KNN), NullVectorStore (no-op for
DECNET_VECTORSTORE_ENABLED=false), SqliteVecVectorStore (lazy
sqlite_vec extension load, one vec0 virtual table per kind).
- get_vectorstore() env-driven dispatch with graceful fallback
to FakeVectorStore when the sqlite-vec extension isn't on the
host, so workers don't crash on a missing optional dep.
Tests: 26 new (11 cred-reuse repo, 15 vectorstore). Existing
credentials and base-repo tests updated for the rename. Total: 34
passing on the touched files.
The events watcher's start-event filter previously called
_load_service_container_names(), which reads decnet-state.json on
every event. decnet deploy writes that state file out-of-band
with docker compose up, so a container's start event could
arrive before the state was committed — the watcher then dropped
the event silently and never tailed the container's stdout. The
visible symptom was an empty Credentials view (and Logs/Bounty)
after a fresh deploy until the collector was manually restarted.
Fix: stamp decnet.fleet.{service,decky,service_name} labels on
every fleet service container at compose-time, and let the
collector recognize either the fleet or topology label without
touching the state file. The state-file name match remains as a
fallback for legacy containers that predate the new labels.
New /credentials page mirroring the Bounty Vault pattern: list view
with search, dynamic service segment chips, plaintext vs hashed
secret rendering, and an inspector drawer with copyable SHA-256 +
service-fields JSON. Sidebar entry uses the Lock icon to keep
Bounty's Archive/Key visual identity distinct.
Surfaces the Credential table (deduped attacker auth attempts) via
a new /api/v1/credentials route. Mirrors the Bounty cache pattern
(5s TTL on the unfiltered default page) and reuses the existing
get_credentials / get_total_credentials repo methods + the already
defined CredentialsResponse DTO. Filters: search, service, attacker_ip.
When RDP_ENABLE_NLA=true (service_cfg.nla=true on the topology side),
confirm PROTOCOL_HYBRID on the X.224 Connection Confirm, upgrade the
socket to TLS using a self-signed cert generated at first start by
the entrypoint, then drive a tiny CredSSP loop:
- Read inbound TSRequest DER (bounded to MAX_TSREQUEST_LEN).
- Scan for the NTLMSSP signature, dispatch on message type:
Type 1 -> respond with a hand-built TSRequest carrying our Type 2
challenge. Type 3 -> parse_type3() and emit auth_attempt with the
universal credential SD shape (secret_kind = ntlmssp_v2).
- Hand-built DER: no pyasn1 dependency.
Also folds in a small fix-up to commit 1: SMB SERVER_CHALLENGE was
hardcoded to 0x11..0x88 across the fleet, which would let a scanner
fingerprint every DECNET decky by its NTLM challenge. Both SMB and
RDP now derive the 8-byte challenge from
instance_seed.random_bytes(8, "ntlm_challenge"), giving each decky a
deterministic-but-distinct value. SMB Dockerfile gets the
instance_seed.py copy too (was synced into the build context but not
COPYed into the image).
- decnet/services/rdp.py: optional service_cfg.nla bool flips
RDP_ENABLE_NLA in the compose env.
- decnet/templates/rdp/Dockerfile + entrypoint.sh: openssl install +
per-decky cert generation gated on RDP_ENABLE_NLA.
- 9 NLA unit tests cover the DER reader/builder, _handle_nla round-
trip with Type 1 / Type 3, oversized-DER rejection, and per-
NODE_NAME challenge divergence.
- DEBT.md: DEBT-040 closed; full TS_INFO_PACKET capture documented as
a follow-up if attacker telemetry justifies it.
Replace Twisted-based connection logger with an asyncio handler that
parses the X.224 Connection Request, extracts the mstshash routing
cookie (universal across mstsc / FreeRDP / Hydra / ncrack / MSF
rdp_login), records the rdpNegRequest.requestedProtocols flags, and
answers with a well-formed X.224 Connection Confirm selecting
PROTOCOL_RDP.
Scope-down vs. the original DEBT-040 plan: full TS_INFO_PACKET
extraction would require either Standard-RDP-Security RC4 stream-
cipher implementation (with our own RSA pair + MS-RDPBCGR signing) or
a complete MCS+GCC ASN.1/BER stack for the SSL path — both far
exceed the 150 LoC budget the DEBT cited. The mstshash cookie is the
only piece of credential information that flows in plaintext on the
wire when the attacker speaks RDP, so capturing it is the highest-
value-per-byte signal available without going down either rabbit
hole. Phase 3 (CredSSP/NLA, next commit) is where actual NTLMv2
hashes land.
- Drops Twisted dependency from rdp/Dockerfile; adds ntlmssp.py copy
ahead of the NLA path that consumes it.
- 7 unit tests cover cookie capture, requestedProtocols recording,
CC framing, no-cookie path, and oversized/non-TPKT drops.
Replace impacket's SimpleSMBServer with a hand-rolled asyncio SMB2
framer that walks Negotiate -> SessionSetup(Type1) -> SessionSetup(Type3)
just deep enough to extract the inner NTLMSSP Type 3 via the shared
parse_type3() parser. Always returns STATUS_LOGON_FAILURE; the
attacker's hash lands in the Credential table, the attacker doesn't
land on the host.
- decnet/engine/deployer.py: _sync_ntlmssp_sources() mirrors the
auth-helper / sessrec sync pattern, copies _shared/ntlmssp.py into
smb/ and rdp/ build contexts before docker compose up.
- Dockerfile: drop impacket dep, copy ntlmssp.py.
- 7 unit tests drive the asyncio handler in-process via
StreamReader.feed_data; assert dialect, MORE_PROCESSING_REQUIRED on
first SessionSetup, NTLMSSP Type 2 carriage in SPNEGO, credential
capture with universal SD shape, STATUS_LOGON_FAILURE on Type 3,
oversized-NBSS / SMB1 / short-PDU drops.
Ships the load-bearing primitive both Phase 5 (SMB) and Phase 7
(RDP NLA) need: a standalone NTLMSSP Type 3 (AUTHENTICATE_MESSAGE)
parser per MS-NLMP §2.2.1.3.
Surface:
parse_type3(blob) -> dict | None
find_ntlmssp(buf) -> int # locate NTLMSSP\\0 inside SPNEGO outer
Returns the universal Credential SD shape:
username + domain (decoded UTF-16-LE or ASCII per NEGOTIATE_UNICODE)
principal = "DOMAIN\\\\username"
secret_kind = "ntlmssp_v1" (24-byte fixed) or "ntlmssp_v2" (variable)
secret_b64 = base64 of NtChallengeResponse — canonical hashcat input
(-m 5500 v1, -m 5600 v2)
Bounds-checked for untrusted-input safety. Anonymous binds (empty NT
response) return None — no credential to record.
7 unit tests cover NTLMv1/v2 distinction, ASCII vs Unicode strings,
empty-domain shape, malformed signature/type rejection, and SPNEGO-
wrapped find_ntlmssp() lookup.
DEBT-040 opens to track the three remaining protocol framers that
will consume this parser:
- SMB: hand-rolled SMB2 + Session Setup framer (~200 LoC) replacing
Impacket's opaque SimpleSMBServer
- RDP basic auth: TPKT/X.224/MCS framer for legacy plaintext path
(~150 LoC)
- RDP NLA: TLS upgrade + CredSSP TSRequest parser, reuses parse_type3
via the SPNEGO inner blob (~250 LoC)
These are substantial protocol implementations each — landing them
inline with Phase 1-3+6's cred coverage rollout would have inflated
the session beyond reasonable scope. Cred-reuse analytics already work
across the 12 services covered in this session; the deferred three
just round out the fleet.
Plugs the cred-coverage gap for MongoDB. The template previously
parsed only the wire opcode + length and discarded the BSON body
entirely, so SCRAM-SHA-{1,256} client-proofs flowed straight through
without ever landing in the Credential table.
Adds an inline minimal BSON walker (~100 LoC) covering the 7 type
codes auth commands actually use: string, doc, array, binary, bool,
int32, int64. Hand-rolled rather than pulling pymongo as a runtime
dep — the parser is bounds-checked for untrusted-input safety
(won't loop on malformed length fields).
Wire flow MongoDB clients use for auth:
- OP_MSG body section (kind=0) → BSON doc with `saslStart` field
carrying mechanism + payload (SCRAM client-first-message:
"n,,n=<user>,r=<nonce>"). Username extracted, pinned to the
per-connection _sasl_username + _sasl_mechanism state.
- Subsequent OP_MSG with `saslContinue` → SCRAM client-final-message
("c=biws,r=<combined>,p=<base64 client-proof>"). The `p=` value is
the credential — emitted as secret_kind=scram_sha256 (or _sha1 /
_unknown depending on the prior saslStart's mechanism), principal
= the pinned username, secret_b64 = base64 of the decoded proof.
Reuse semantics: same client-proof across two auth attempts only
matches when both server salt and password were identical (proofs
include the salt). So cross-session reuse correlates only on
credential reuse against the same MongoDB account on the same decky
— honest, non-misleading signal.
680 tests pass across services, service_testing, db, web/ingester,
and core/fingerprinting (the broader scope my recent commits
touched). Phases 4, 5, 7 still pending (RDP basic-auth, SMB
NTLMSSP, RDP NLA).
Login forms (wp-login.php, phpMyAdmin, Joomla, etc.) ship a
`Content-Type: application/x-www-form-urlencoded` body with field
names like username/user/email/log/pwd/password. The HTTP/HTTPS
templates already captured the body as opaque bytes; now they parse
common login-form shapes into the universal credential SD shape.
Adds canonical templates/syslog_bridge.py:
extract_form_credentials(body, content_type) -> dict | None.
Field-name matching is case-insensitive and covers:
Principal: username, user, email, login, userid, account, log,
user_login (WordPress), uname / pma_username (phpMyAdmin)
Secret: password, pass, pwd, passwd, passwort, mot_de_passe,
user_password (WordPress), pma_password (phpMyAdmin)
The HTTP/HTTPS log_request handlers now call:
cred = classify_authorization(...) or extract_form_credentials(...)
— Authorization wins when present (current session credential beats
a follow-up form change), but POSTs to /wp-login.php with no Auth
header still surface their cleartext creds.
Secret-without-principal is intentional: a reset-confirm or auto-
fill abuse may carry a password without any field that maps to our
principal list. The cred row writes with principal=None — the
sha256 still correlates across services for reuse analytics.
The body capture cap bumped from 512 → 4096 chars so reasonable
form bodies aren't truncated before the cred extractor sees them;
the body stored in fields.body stays at 512 chars (display-friendly).
36 helper + emitter tests pass. Phases 4-7 still pending.
Closes the cred-coverage gap for two database services that had been
capturing only the username:
- MySQL — extends _handle_packet to read the auth-response after the
null-terminated username. mysql_native_password puts a 1-byte
length followed by 20 bytes: SHA1(password) XOR SHA1(salt +
SHA1(SHA1(password))). Plaintext irrecoverable, lands as
secret_kind="mysql_native_password" with the 20 hash bytes in
secret_b64. Hash is canonical for "hashcat -m 11200" if an operator
ever wants to crack offline.
- MSSQL — fixes a pre-existing bug AND adds password capture. The
prior _parse_login7_username read offsets 36/38, which is actually
ibHostName/cchHostName in the Login7 layout — username sat at
40/42 and was never touched. Replaced with _parse_login7_creds()
reading the correct offsets (40 username, 44 password). Login7
password is XOR-then-nibble-swap obfuscated against 0xa5;
_deobfuscate_login7_password reverses it. Plaintext-recoverable,
lands as secret_kind="plaintext".
The pre-existing test_login7_auth_logged_and_closes only verified the
error response ships and the connection closes; it didn't validate
the parsed username, so the hostname-as-username bug was silent. New
tests cover both the deobfuscation algorithm directly and the full
ingester round-trip for both services.
Sync: copies the canonical syslog_bridge.py into mysql/ and mssql/
template build contexts so service_testing tests load the version
with classify_authorization + encode_secret available.
37 tests pass in the touched scope. Phases 3-7 still pending.
Closes the cred-coverage gap for 7 services that already had the data
on the wire but never landed it in the Credential table:
- SNMP — community string lands as secret_kind="snmp_community",
principal=None (v1/v2c has no per-user identity, the community IS
the auth).
- SIP — Digest response hash, previously buried in the auth= header
dump, now classify_authorization()-extracted.
- HTTP / HTTPS — Authorization header was in the headers JSON but
never extracted. Now Basic decodes to plaintext, Bearer →
http_bearer (principal=None), Digest → http_digest_md5.
- K8s — already extracted Authorization but didn't normalize. Service-
account JWTs flow through as Bearer.
- Docker API — headers absent entirely. Adds the headers JSON dump
and runs Authorization through the classifier.
- Elasticsearch — five distinct request handlers; each gains a
per-handler _cred_fields() helper.
Adds canonical templates/syslog_bridge.py:classify_authorization().
Recognised: Basic / Bearer / Token / Digest. Unknown schemes (NTLM,
AWS4-HMAC, Negotiate) return None; the header still rides in the
ambient SD-block but isn't normalized as a credential. The SD shape
on the wire collapses sip_digest_md5 into http_digest_md5 — same
algorithm, so cross-protocol reuse correlates correctly when (rare)
nonce collisions allow.
Drive-by repair of tests/core/test_fingerprinting.py:
- The pre-existing `test_http_useragent_extracted` asserted both that
add_bounty was called exactly once AND that the UA payload carried
`path` and `method` fields. Both wrong since this session opened:
the http_quirks fingerprint added later fires too, and the UA
payload never actually included path/method despite the assertion.
- Adds `path`/`method` to the UA fingerprint payload (real operator
value: "Nikto hit /admin" beats "Nikto seen on this decky").
- Replaces `assert_awaited_once` with a `_find_ua_bounty()` helper
that filters add_bounty calls by `fingerprint_type`. New fingerprint
families landing later won't retroactively break old tests.
- Updates the two credential-bearing tests to use the post-DEBT-039
native shape (`secret_b64` / `principal`) and `upsert_credential`,
not the deleted legacy `username+password` adapter.
Also rebuilds the per-service fake `syslog_bridge` modules in
tests/service_testing/{conftest,test_imap,test_pop3,test_snmp,test_mqtt,test_smtp}.py
to expose `encode_secret` + `classify_authorization`. Service templates
that import either now no longer fail at test collection.
173 tests pass in the touched scope. Phases 2-7 still pending.
Honest correction to the "every cred-emitting service" claim. Audit
of templates/* found three gaps:
1. MQTT — was working through the legacy adapter, silently dropped
when Phase 3 (e696c2b) deleted it. Now migrated to encode_secret()
alongside the others.
2. Postgres — `auth, pw_hash=…` event captures the MD5
challenge-response the attacker sent. Plaintext irrecoverable, so
it never fit the (principal, secret_b64=raw_bytes) shape. Lands
in Credential as secret_kind="postgres_md5_challenge".
3. VNC — `auth_response, response=…hex` event captures the 16-byte
DES-encrypted challenge. Same situation as Postgres: plaintext
irrecoverable. Lands as secret_kind="vnc_des_response".
Adds a `secret_kind` discriminator column to Credential (default
"plaintext", indexed). The dedup tuple gains secret_kind so two
credentials with the same sha256 but different kinds are
fundamentally different rows — different challenges produce
different bytes for the same plaintext password, so cross-kind
reuse matches are meaningless and would only confuse analytics.
The model now genuinely covers every cred-emitting service in the
fleet:
plaintext SSH, Telnet, FTP, POP3, IMAP, SMTP, Redis, LDAP,
MQTT
postgres_md5_* Postgres
vnc_des_response VNC
Username-only services (MySQL/MSSQL — TDS pre-encryption captures
the user but never sees the password byte) intentionally don't feed
Credential — they're recon signals, not cred attempts.
40 tests pass in the touched scope. New cases: secret_kind dedups
independently in the repo; Postgres MD5 + VNC DES emitters thread
through; MQTT round-trips through the native branch.
Phase 3/3 of DEBT-039. Now that all six cred-emitting services
(SSH, Telnet, FTP, POP3, IMAP, SMTP, Redis, LDAP) emit the universal
`secret_b64`-bearing SD shape, the ingester's legacy fork has no
live emitters to handle. Deletes:
- `_ingest_credential_legacy()` — synthesized native fields from
username+password
- The `elif _fields.get("username") and _fields.get("password")`
branch in `_extract_bounty`
- `_printable_filter()` — only the legacy adapter called it; the
native branch trusts the emitter (encode_secret() in Python or
sd_escape() in C) to have already sanitized
- The legacy-adapter test cases in tests/web/test_ingester.py;
their coverage moved to tests/services/test_cred_emitters.py
per-service in Phase 2
The cred path is now single-shape end-to-end. A pre-migration log
row carrying only username+password silently produces no Credential
write — by design, since no current emitter writes that shape and
keeping a code path alive for theoretical legacy data risks masking
emitter regressions. Pre-v1: any historical Bounty cred rows from
before commit 2f47f67 stay untouched.
DEBT-039 marked resolved with summary of the three commits and the
silent-loss bug fix for Redis + LDAP that fell out of execution.
Phase 2/3 of DEBT-039. Switches FTP, POP3, IMAP, SMTP, Redis, and
LDAP from the legacy `username=` + `password=` SD-block shape to the
universal credential shape (`principal=` + `secret_printable=` +
`secret_b64=`) the new Credential storage model expects.
Pattern is uniform across all six services:
_log("auth_attempt", username=u, principal=u, **encode_secret(pw))
Each service emits the canonical SD keys. The ingester's native-shape
branch (introduced in 2f47f67) now writes their cred attempts
directly without going through the legacy adapter. Once Phase 3
removes the adapter the contract becomes single-shape.
Per-service notes:
- POP3 / IMAP — `status="success"|"failed"` renamed to
`outcome="success"|"failure"` to match Credential.outcome's
vocabulary; the ingester reads outcome directly.
- SMTP — AUTH path migrated; in addition the existing mail_from
event now exposes a parsed `domain=` field alongside the original
`value=` so future "what domains do attackers spoof from" analytics
have an indexed field. Not stored in Credential — regular Log row.
- Redis — was silently dropped by the legacy adapter (no `username`
field). Native branch handles `principal=None` correctly. BONUS
FIX: the Redis 6+ ACL syntax `AUTH <user> <pw>` now captures the
ACL username as principal (was previously discarded).
- LDAP — was silently dropped by the legacy adapter (no `password`
recognition for the `bind` event). Now lands as
`principal=<dn>`. BONUS FIX.
Tests (tests/services/test_cred_emitters.py, 9 cases):
- per-service native-shape ingest path produces correct Credential
rows; outcome maps for POP3/IMAP; principal=None for legacy Redis
AUTH; principal=dn for LDAP.
- mail_from event does NOT trigger a credential write (it's a
Log-only observation, not auth).
- 0xff/NUL/ANSI bytes in passwords survive losslessly through
secret_b64 even when secret_printable is sanitized.
Phase 3 deletes the legacy adapter once all migrations land — the
adapter has no live emitters to handle anymore.
Phase 1/3 of DEBT-039. Adds the Python emitter-side counterpart to
auth-helper.c's sd_escape + base64 logic so service templates can
emit the universal credential SD shape with a single spread:
_log("auth_attempt", principal=user, **encode_secret(password))
secret_printable mirrors the C helper's [0x20, 0x7f) → '?' contract;
secret_b64 preserves the ORIGINAL utf-8 bytes losslessly so non-ASCII
or control-byte payloads survive as fingerprinting signal even when
the printable form sanitizes them.
The canonical syslog_bridge.py is what _sync_logging_helper()
propagates into per-template build contexts at deploy time, so any
service that imports its local syslog_bridge picks this up
automatically on next rebuild.
Phase 2 migrates the six cred-emitting service templates (FTP, POP3,
IMAP, SMTP, Redis, LDAP) onto this helper. Phase 3 deletes the
ingester's legacy adapter once nothing emits the old shape.
Replaces the opaque Bounty.bounty_type='credential' path with a
dedicated `credentials` table whose schema is forward-compatible
across every auth-bearing service in the fleet. Hoisted indexed
columns (secret_sha256, principal, service, attacker_ip) carry the
universal reuse-analytics signal; service-specific JSON keys ride
in `fields`. Cross-service reuse queries become an indexed lookup
on secret_sha256 instead of JSON_EXTRACT scans.
Schema decisions baked in (per ANTI):
- New `Credential` table, not extension to Bounty
- Hoisted `principal` column for cross-service principal-reuse
- Standardized JSON keys: every payload carries secret_b64 +
secret_printable + principal universally; service-specific extras
(user, domain, dn, mech, …) ride alongside
The auth-helper SD-block emits the new shape natively. The ingester
forks at _extract_bounty:
- Native shape (SSH/Telnet, future emitters): secret_b64 present →
direct upsert_credential
- Legacy shape (FTP/POP3/IMAP/SMTP today): username + password →
adapter synthesizes secret_{b64,sha256,printable} on the fly,
upserts into the same Credential table. Tracked as DEBT-039;
one-shot bridge until those service templates migrate.
Defense-in-depth across five layers (input validation):
- C helper: bytes outside [0x20, 0x7f) collapse to '?', RFC 5424
escape rules for \\, ", ]; b64 preserves exact bytes
- Ingester native branch: rejects malformed secret_b64 (regex), drops
the credential row but keeps the underlying Log
- Ingester legacy adapter: same printable-ASCII filter as the C
code; sha256 + b64 over the original utf-8 bytes (lossless, even
when secret_printable is sanitized)
- DB column caps with truncation warning; sha256 always over the
full pre-truncation bytes so reuse queries match across truncation
- JSON serialized with ensure_ascii=True so utf8mb4 columns stay
safe even with non-ASCII service-specific keys
Bounty.bounty_type='credential' is no longer written. Pre-v1: no
historical backfill; existing rows stay untouched but unused.
595 tests pass; new tests cover the model + repo (upsert dedup,
null-principal independence, cross-service reuse, filters), both
ingester branches, b64 validation, sanitization preserving the
fingerprinting signal in b64.
Promotes auth-helper.c to decnet/templates/_shared/auth-helper/ and
adds _sync_auth_helper_sources() — mirrors the existing sessrec sync
pattern that keeps shared sources in step with per-template build
contexts.
Telnet's image grows the same multi-stage musl build, COPY of the
static helper into /usr/sbin/auth-helper, and prepended pam_exec line
in /etc/pam.d/login. Pulls in the `login` package (real Debian
PAM-aware /bin/login, replacing busybox's PAM-less applet) and
libpam-modules transitively for pam_exec.so.
Verified inside the rebuilt telnet image:
- /bin/login is the real 53KB Debian binary (PAM-aware)
- /etc/pam.d/login top line is the auth-helper hook
- pam_exec.so present at /usr/lib/x86_64-linux-gnu/security/pam_exec.so
- helper smoke-run emits correct RFC 5424 line for `telnetpw` →
password_b64="dGVsbmV0cHc="
SSH Dockerfile updated to read auth-helper.c from auth-helper/
subdirectory so both templates use the synced layout. The canonical
source lives in _shared/; per-template copies are tracked in git AND
synced at deploy time so a drift on either side rebases on the next
deploy.
Closes the telnet half of DEBT-038's #5 follow-up.
Real OpenSSH doesn't log attempted passwords — only success/failure
with username — leaving SSH the sole auth-bearing service in the
fleet that contributes nothing to the cred corpus FTP/MySQL/RDP/
VNC/etc. populate. Closes that gap with a tiny pam_exec shim.
A static C helper (~80 LoC, musl, ~38KB stripped) is wired into
/etc/pam.d/sshd as `auth optional pam_exec.so expose_authtok stdout
/usr/sbin/auth-helper`. pam_exec writes the attempted password to
the helper's stdin NUL-terminated; the helper formats an RFC 5424
line in the exact shape templates/syslog_bridge.py produces
(facility local0, PEN 55555, MSGID auth_attempt — same MSGID FTP
uses) and writes it to /proc/1/fd/1 so the existing collector
stdout-reader pipeline picks it up.
Two password fields ride in the SD-block:
- password= RFC 5424 escaped, ASCII-printable only, ? for non-
printables. FTP-compatible — existing dashboard
rendering picks up SSH attempts unchanged.
- password_b64= base64 of the exact PAM_AUTHTOK bytes. Preserves
NUL/0xff/control-byte fingerprinting signal that the
plain field necessarily drops.
Fail-open by design: the PAM line is `optional` so a malfunctioning
helper never blocks sshd auth. Better to miss a cred than break the
honeypot.
Verified end-to-end inside the rebuilt image:
- 38KB static ELF, runs without a dynamic linker
- correct RFC 5424 line for `hunter2` → b64 `aHVudGVyMg==`
- NUL truncation matches pam_exec's contract
- 0xff bytes survive losslessly through password_b64
- empty password produces a well-formed line (e.g. pubkey auth path)
Attacker list cards gain an AS<number> chip with the AS description
on hover. Attacker detail page adds an AS row beside ORIGIN — same
shape as the existing country/source pair so operators can read
"this attacker is in DE on AS24940 Hetzner" at a glance instead of
having to grep the IP into a separate tool.
Both fields collapse to "unknown" when the IP isn't BGP-announced
(CGNAT, dark space, RFC1918), matching the existing pattern for
country resolution.
Adds asn (int), as_name (varchar 128), asn_source (varchar 16) to
the Attacker SQLModel — direct columns, no _migrate_* helper per
feedback_no_new_migrations_prev1.
Profiler worker now calls decnet.asn.enrich_ip alongside the existing
geoip enrich_ip; both feed the upsert payload. Failure is total — if
either lookup throws or the IP is private/unannounced, the field stays
None and the row still writes.
Both lookups are independent: a CGNAT address can have a country (RIR
allocation) but no ASN (no BGP origin), and vice-versa for unrouted
RIR-allocated space. Storing them separately preserves that signal.
Mirrors decnet/geoip/ end-to-end: paths/base/factory/lookup at the
package level, iptoasn/ subpackage holds the data-source-specific
fetch+parse+provider. AsnLookup is bisect-indexed over (start, end,
AsnInfo) ranges with a pickled cache invalidated on raw-file mtime
bump.
Why iptoasn (and not bgp.tools / Team Cymru): public-domain dump,
zero attribution, no UA mandate, daily refresh — keeps DECNET stealth
intact (the geoip/rir module's "never identify as DECNET" comment
applies the same way here). bgp.tools' ToS would have required an
identifying UA, conflicting with feedback_stealth.
Public surface: decnet.asn.enrich_ip(ip) -> (asn, name, source) or
all-None on miss/disabled. Same shape as decnet.geoip.enrich_ip so
the profiler can compose them in one call site.
Renders the swarm host (or "master") that a topology is deployed to,
both as a meta line on each topology list card and in the war-map
header. Operators can now distinguish master-local from agent-targeted
topologies at a glance — previously the only signal was the abstract
"mode: agent" label, with no hint of which agent.
Adds useSwarmHosts() hook for the uuid → host lookup. Falls back to a
short uuid prefix when the hosts list is unavailable so the UI never
hard-fails on a missing /swarm/hosts response.
TopologySummary gains target_host_uuid in the frontend type so the
field actually narrows when checked.
Adds resolve_lan_host(lan, topology) and partition_lans_by_host(h)
in topology.persistence — the single source of truth every per-host
caller (deployer, mutator, validator) consults to decide where a LAN
belongs. Resolution: lan.host_uuid → topology.target_host_uuid →
None (master).
Adds validator rule BRIDGE_HOST_SPLIT: a multi-homed (bridge) decky
attached to LANs that resolve to different hosts is rejected at
deploy-time. A bridge decky is one container with NICs into multiple
LANs; under the co-locate constraint (no overlay network), all those
LANs must share a host.
Adds nullable LAN.host_uuid (FK swarm_hosts.uuid). Resolution order
when deploying a LAN: lan.host_uuid → topology.target_host_uuid →
master. A LAN is one Docker bridge so the bridge cannot span hosts;
this pin forces every decky in the LAN onto the named host.
LANCreateRequest / LANUpdateRequest accept host_uuid; both validate
that the host exists, returning 400 on unknown UUIDs. PATCH still
gated by the existing pending-only guard, so reassignment of a live
LAN is not yet possible (deferred to mutator support).
LANRow surfaces the field so the frontend can render per-host badges.
AgentClient now verifies the worker's TLS cert fingerprint against
SwarmHost.client_cert_fingerprint at __aenter__ time, on top of CA
validation. Required before fanning master-orchestrated topology
deploys out across multiple swarm hosts: CA pinning alone allows any
cert signed by the master CA, which is too coarse once a single
deploy can target N hosts.
Mismatch raises FingerprintMismatchError so callers can distinguish
"wrong worker on the wire" from a transport hiccup.
Pre: optimistic placeholders for enqueued LAN-add mutations were
indistinguishable from regular not-yet-deployed nets — same dim
mono chrome, same dotted border. User couldn't tell whether a drop
had been queued or had silently failed and re-stacked over an
existing LAN.
Tag the placeholder with `pending: true`, render it in the same
amber the REAP button uses (var(--warn, #e0a040)) with a 'PENDING'
chip-mini in the head. Visual is loud enough that there is no
chance of confusion with INACTIVE (dimmed) or regular pending-state
LANs (mono).
Reconciliation is the existing refetch pumping setNets(h.nets) on
SSE — no extra plumbing needed; placeholders disappear naturally
when the mutator's applied event lands and the canvas re-hydrates
from the server.
Two bugs sharing the same root cause: Net only carried a label
string, set to lan.name.toUpperCase() everywhere. Backend mutator
ops look up LANs by canonical lowercase name, so passing the
uppercase label through attachEdge / detachEdge / addDeckyToLan /
deleteLan failed with 'LAN \\'SUBNET-XXXX\\' not found'.
Add Net.name (canonical, lowercase) alongside Net.label (display).
Every backend call site now passes name; toasts and drag ghosts
keep label.
Second bug — new LANs stacking on top of each other on live
topologies — fell out of the same UX path: createLan returns
'enqueued' when the topology is active/degraded, the existing
early-return skipped local-state insertion, so the next drop
recomputed the same grid index. Now we drop a placeholder Net
with id 'pending-lan-<name>' immediately on enqueue. Grid index
advances and the user gets a visual ack right away; SSE replaces
the placeholder by canonical id when the mutator applies it.
MySQL ERROR 1093 forbids referencing the UPDATE target inside a
subquery; the existing UPDATE ... WHERE id = (SELECT id FROM
topology_mutations ...) form blew up on every mutation claim under
the MySQL backend, so no mutation ever progressed past pending.
Wrap the inner SELECT in a derived table (SELECT id FROM (...) AS
_next). MySQL materialises the derived rowset before applying the
UPDATE, sidestepping 1093. SQLite accepts both forms, so the
single-statement atomic claim semantics are preserved on both
backends — racing watchers still serialise correctly.
deploy_topology and teardown_topology are async, but every
_compose_with_retry / _compose call inside them was running in the
main event loop via subprocess.run — which means a multi-minute
docker compose --build froze the entire API: other endpoints,
mutator events, SSE streams, status polls. The user noticed when a
2-decky deploy blocked everything else for the duration of the build.
Wrap both calls in anyio.to_thread.run_sync. Same pattern the
mutator engine has been using at engine.py:104 since forever.
Per-LAN bridge create/remove docker SDK calls are still synchronous
in the loop — they're individually fast (~50-200ms per LAN) and
the loops are bounded by topology size, so they don't dominate.
Worth revisiting if a 200-LAN deploy turns out to stall noticeably.
The api unit's ProtectHome=read-only made the user's HOME read-only
inside the unit's namespace. docker compose --build then tried to
write ~/.docker/buildx/activity/* and got EROFS — which we'd been
misdiagnosing as a buildx wedge for the last few iterations.
Real fix: set DOCKER_CONFIG and BUILDX_CONFIG in the unit's
Environment= to a path inside ReadWritePaths. Hardening stays on,
docker CLI writes to install_dir/.docker instead of /home/<user>/.docker.
The wedge classifier now detects this case (count==0 + /home/ in
the stderr path) and emits a recipe pointing at the env-var fix
instead of the driver-rebuild path. Test added.
Wiki gets the new branch first since it's the most common cause
on systemd-managed installs.
'docker buildx create --name default' errors with 'default is a
reserved name and cannot be used to identify builder instance'.
The bundled builder always exists under that name; the recipe
should switch to it (buildx use default), not try to recreate it.
For the count==0 driver-rebuild branch, the new builder needs a
non-reserved name — using 'decnet-builder' as the example.
The hint was one-size-fits-all and pointed at prune+restart even
when zero mounts were leaked — a false positive caused by matching
any stderr containing the activity-dir path.
Two changes:
1. Tighten the wedge classifier. Both the buildx-specific phrase
('failed to update builder last activity time') AND the EROFS
marker ('read-only file system') must appear in stderr. Either
alone is now treated as a normal transient error and retried.
2. Branch the recipe on _count_leaked_buildkit_mounts():
* count > 0 → unmount loop + daemon stop + umount -l
(prune+restart alone doesn't evict held mounts)
* count == 0 → rebuild the buildx driver (rm builder state,
buildx create --use, inspect --bootstrap)
Original compose stderr is now preserved in the hint as
'Original error: ...' so the user sees both the recipe and what
compose actually said.
Tests cover both branches plus a negative case (unrelated EROFS).
str(CalledProcessError) is just 'Command ... returned non-zero exit
status N' — the stderr (where the buildx recovery hint lives) was
being silently dropped from both the deploy log line and the
persisted 'failed' status reason.
New _format_subprocess_error helper appends .stderr when the
exception is a CalledProcessError. Applied to transition_status
reason and the background-deploy log message so operators and the
UI see the real failure, not just the exit code.
This is what makes the buildx preflight hint from 86b9dec actually
reach the user.
When Docker's buildx leaks bind-mounts from a failed build it starts
reporting 'read-only file system' on its own activity file, even
though nothing is actually read-only. The user's host had 20+
leaked mounts before we noticed — each retry compounds the leak.
_compose_with_retry now:
* Pre-flight counts /var/lib/docker/tmp/buildkit-mount* entries in
/proc/self/mounts; if >= 10 and the command is a build, refuses
to start and returns a clean recovery recipe instead of retrying.
* On mid-build failures that match the wedge signature
('failed to update builder last activity time' or the activity-dir
path in stderr), short-circuits the retry loop with the same
recipe. The first occurrence no longer needs a pre-flight; the
pre-flight catches repeat attempts.
Recipe points at 'docker buildx prune -af && sudo systemctl restart
docker', which is what actually clears the leaked mounts.
Tests cover all three paths: wedge preflight blocks builds, non-build
commands (down/stop) ignore the preflight, mid-build signature
detection kills the retry loop. A new autouse fixture stubs the
wedge-detector to 0 so dev-host state doesn't poison the mocked
subprocess tests.
Wiki companion commit adds Troubleshooting → 'Buildx leaked mounts'.
Port-to-port edges previously lived only in the editor's local state
— the backend's edge model is decky<->LAN membership, so the deploy
validator still saw cross-LAN pairs as orphans. Drawing a line from
dmz-gateway to a decky in subnet-d6b2 did nothing that a later
DMZ_ORPHAN check could see.
Now onAddEdge inspects endpoints: same-LAN stays visual (no bridge
to create), cross-LAN calls attachEdge with the source decky and
the target LAN, multi-homing the decky so the validator's LAN
adjacency scan threads through it. The viz edge stores the returned
backendEdgeId; removeEdge detaches that membership before dropping
the local edge. Observed entities (attacker-pool) are read-only and
never bridge.
A toast ("BRIDGED <decky> -> <lan>") surfaces the backend-persistent
side of the gesture so the user knows it's not just a cosmetic line.
POST /topologies raised a 500 with a raw SQLAlchemy IntegrityError
traceback when the name collided with an existing topology. Catch
the error at the router, verify it's the ix_topologies_name
constraint (so unrelated integrity failures still surface as 500s
with their real traceback), and return 409 with a helpful detail.
Test covers the create-then-duplicate-create flow.
The .maze-edge-dash CSS animation invalidates each path's bounding
box every frame. Inter-LAN paths span the viewport so invalidations
overlap, and past ~60 edges the compositor spends every frame
repainting — the dominant cost on the 12+ LAN screenshot, even
dwarfing pan-drag overhead.
Drop the animation class when edges.length > 60. Edges stay fully
visible and traffic-tinted, just static. A MOTION: OFF segment in
the status bar surfaces the auto-disable so it doesn't look like a
broken animation.
Threshold is a constant in Canvas.tsx; if it needs to become a
user toggle later, lift it to state + localStorage in one place.
A 30-LAN generate request already fits in 172.20.0.0/16, but trees
with depth/branching that multiply past 256 (e.g. depth=6,
branching=4 ≈ 5k LANs) hit AllocatorExhausted before the first
write.
SubnetAllocator now accepts a full CIDR base ("172.16.0.0/12" →
4096 /24s) in addition to the legacy two-octet shorthand ("172.20",
auto-lifted to /16). The parent must be ≤/24; a /24 base yields
exactly one slot. Iteration order is preserved for /16 bases so
existing topologies keep their third-octet sweep; /12 adds a
second-octet dimension underneath.
Defaults bumped to 172.16.0.0/12: TopologyConfig.subnet_base_prefix,
/next-subnet query param, and the mutator's add-LAN fallback. The
field pattern widens to accept CIDR. create-blank and manual LAN
CRUD still use "10.0" (lifts to /16) — one DMZ LAN per topology,
256 is plenty.
Pan/zoom previously drove a full Canvas re-render on every mousemove
via setPan() — at 30 LANs that's ~1000 SVG paths and div cards
re-evaluating 60 times a second while you drag. The browser screamed.
Three fixes, one surgical pass:
1. Pan drag writes the translate/scale transform directly to the
pan-layer DOM ref inside requestAnimationFrame; setPan is deferred
to mouseup. Grid pattern attributes (x/y/width/height) get the
same treatment so the backdrop stays glued to the canvas content.
Wheel zoom, resetPan, and zoomBy also sync refs + fire a write so
React-driven changes land in one frame.
2. Edge rendering swaps the nodes.find() inside .map() for a
Map<id, node> built once per render — O(E) instead of O(E·N).
NetBox + NodeCard are now wrapped in React.memo; Canvas hoists
the setSelection closures into useCallback so memo can actually
short-circuit instead of seeing a fresh prop every render.
3. Drag-a-single-node still mutates state and re-renders, but now
only the moved node rerenders — the other 89 skip via memo.
Everything that reads panRef.current (toWorld, context menu, drop
targeting) still sees the live value during drag because we mutate
the ref synchronously on each mousemove; only React state is lazy.
Route all lucide-react icon usage through a single src/icons.ts
re-export that imports each icon from its own per-icon module
(lucide-react/dist/esm/icons/<name>) instead of the barrel.
Bundle-size impact: none (29kB icons chunk unchanged — tree-shaking
was already effective with sideEffects:false). Dev-experience win:
Vite transforms 247 modules instead of 1848 because the dep
optimiser no longer pre-bundles the full lucide barrel — faster
cold start and HMR.
Ambient d.ts declares the wildcard module so TS accepts per-icon
imports; lucide ships .d.ts only for the barrel.
Seven icons were renamed upstream and still work through the barrel
via aliases (AlertTriangle -> triangle-alert, BarChart3 -> chart-column,
CheckCircle -> circle-check-big, Filter -> funnel, PlusCircle ->
circle-plus, Sliders -> sliders-vertical, UploadCloud -> cloud-upload,
Fingerprint -> fingerprint-pattern). Component call sites stay on
the legacy names; the renames live only in icons.ts.
Switch all navigable route components to React.lazy() and wrap
<Routes> in <Suspense>. Dashboard/Login/Layout stay eager since
they're the shell.
Initial index bundle drops 246kB -> 34.67kB (gzip 10.5kB). Each
route becomes its own 8-51kB chunk, loaded on demand.
Nav hover/focus triggers prefetchRoute(path) which fires the same
dynamic import() specifier the bundler dedups against React.lazy,
so the chunk is warm by the time the user clicks. Avoids the
Suspense flicker that would otherwise show on every first nav.
Single-bundle build was tripping vite's 500 kB warning per chunk and
forcing every user to re-download the entire app on every deploy.
Manual chunks split the bundle along natural library boundaries so:
- Rarely-changing vendor libs (react-dom, react-router, lucide-react,
asciinema-player) cache across deploys.
- App code lives in its own `index-*.js` that's the only chunk that
changes when we ship feature work.
Split shape (manualChunks fn in vite.config.ts):
- charts — recharts + d3-*
- player — asciinema-player
- icons — lucide-react
- router — react-router / react-router-dom
- react-dom, react
- vendor — everything else in node_modules
Resulting bundle sizes (gzip):
index (app): 246 kB (gz 63)
react-dom: 182 kB (gz 57)
player: 176 kB (gz 65)
router: 42 kB (gz 15)
vendor: 36 kB (gz 14)
icons: 29 kB (gz 10)
Every chunk under the 600 kB ceiling we now set explicitly. The old
~705 kB single-chunk deploy is gone. No code changes — config only.
Five was still too loud on AttackerDetail when rotation is in play.
One inline is enough to read at a glance; everything else goes
behind the expand button. Rotation tag keeps carrying the count so
no signal is lost.
`for i in $(seq 1 100); do curl -H "X-Forwarded-For: 191.100.20.$i" ...`
was dumping 100 distinct IPs into AttackerDetail's LEAKED IPs row,
drowning the rest of the ORIGIN section. The 100-IP wall is itself a
signal (WAF-bypass-list probing) that deserves a short badge, not a
flood.
Backend:
- get_attacker_ip_leaks gains `limit: int = 10` parameter — caller
only ever needs a sample, not the full set.
- New count_attacker_ip_leaks() returns the unbounded COUNT(*) via
one cheap SQL aggregate.
- Detail endpoint returns {ip_leaks: [first 10], ip_leaks_total: N}
so the UI can render a rotation badge independent of list length.
UI:
- New LeakedIPsRow component. First 5 distinct IPs rendered inline
with hover tooltips (unchanged). When > 5, a `+ N more` expand
button reveals the rest of the sample; when total exceeds the
10-row cap, a subtle `(+M beyond sample)` note appears.
- When total ≥ 20, a red `ROTATION · N` tag renders leading the
row with a tooltip explaining the semantic: "almost certainly
XFF-rotation / WAF-bypass probing, not a real attribution leak."
DB churn is deliberately not capped — 100k rows × ~500 B is tolerable.
If it becomes a problem we can add an ingester-side count-and-skip;
for now the UX fix is the whole story.
Added test_ip_leaks_total_reported_separately_from_list asserting
the endpoint shape matches what the UI consumes.
Every http_useragent bounty now carries a `category` label plus an
optional tool name and a signals list. The main analytic win is the
`nonstandard` bucket — UAs like "FUCKYOU/1.0" or custom one-off
scanner labels that don't match any known pattern, which today
silently blend into the generic fingerprint list.
Buckets (priority order):
- scanner: nmap, nuclei, sqlmap, gobuster, nikto, masscan, zgrab,
ffuf, wpscan, katana, burp, acunetix, nessus, openvas, arachni,
whatweb, wappalyzer, etc.
- cli: curl, wget, httpie, xh, fetch.
- library: python-requests, aiohttp, httpx, urllib, Go stdlib, Java,
okhttp, Apache HttpClient, axios, node-fetch, got, undici, PHP,
Guzzle, Ruby stdlib, Faraday, .NET, PostmanRuntime, Insomnia, etc.
- bot: anything containing bot / crawler / spider / slurp / monitor
(catches Googlebot, bingbot, Baiduspider — many of which ship a
Mozilla/5.0 prefix, so the bot check runs BEFORE the browser
regex).
- browser: Mozilla/5.0-prefixed UAs that aren't bots.
- nonstandard: anything else. The interesting bucket.
- empty: literal empty User-Agent header.
Side signals computed regardless of category: suspicious_short (<8
chars), suspicious_long (>512 chars), nonprintable (control chars),
injection_like (SQLi / XSS / path-traversal / Log4Shell markers).
A sqlmap UA with a literal SQL-injection payload embedded fires
category=scanner + injection_like — the combination tells the
analyst the tool is being operated manually vs. on default config.
Classification is deterministic (same UA string → same tuple) so
add_bounty's payload-hash dedup continues to collapse repeat rows.
UI renderer upgraded from FpGeneric to a dedicated FpUserAgent that
colours the category tag by risk (scanner=alert-red,
nonstandard=warn-yellow, browser=accent-green, etc.) and renders
each signal as its own chip. Makes the interesting rows pop in the
fingerprints panel.
Also fixed: the ingester was using `_headers.get("User-Agent") or
_headers.get("user-agent")`, which short-circuits away empty-string
UAs. An explicit empty UA is itself a signal (real clients always
send something) — now captured.
An attacker hitting /admin with `X-Forwarded-For: 127.0.0.1` was
previously flagged as an IP leak. It isn't — that's the classic
IP-allowlist / WAF-bypass payload ("treat me as localhost and skip
your auth checks"). Misclassifying it as "LEAKED IPs" in the UI
confuses analysts and burns trust in the signal.
Split by claim category. After pulling the left-most claimed IP
from the proxy header, classify:
- public (routable) → bounty_type=ip_leak (real attribution leak;
the attacker's upstream proxy forwarded their real IP).
- loopback / private / link-local / multicast / reserved /
unspecified → bounty_type=fingerprint, fingerprint_type=
spoofed_source (WAF-bypass / allowlist-probing attempt; the
attacker is telling us they know what XFF does).
- unparseable → dropped.
Same extraction pipeline; diverges only at the last step. A new
shared _classify_proxy_header_claim returns (kind, payload);
_detect_ip_leak keeps its public-only contract for backward-
compat; _detect_spoofed_source is the new sibling.
UI renderer FpSpoofedSource shows the claimed IP in warn color with
the claim_category tag (LOOPBACK / PRIVATE / ...) and a WAF-BYPASS
ATTEMPT badge — distinct visual from the "LEAKED IPs" row which
stays reserved for genuine public-IP leaks.
Test addresses updated: RFC 5737 doc ranges (198.51.100.0/24,
203.0.113.0/24) are flagged `is_reserved` in Python's ipaddress
module, so they now correctly belong to the spoof bucket — tests
that meant to exercise real public IPs now use 8.8.8.8 / 1.1.1.1 /
Cloudflare DNS. Added eleven new tests locking the classifier +
the two detectors' mutual exclusion.
add_bounty dedups on (attacker_ip, bounty_type, full payload JSON).
Three fingerprint-family bounties (http_useragent, ip_leak,
http_quirks) were including method/path / header_count in their
payloads — fields that vary per request — so a scanner hitting 100
paths produced 100 rows instead of 1, which is what was swelling
AttackerDetail.
Payloads now carry identity-only fields:
- http_useragent: {fingerprint_type, value}. UA + path combinations
no longer collide; one row per distinct User-Agent string.
- ip_leak: {source_ip, real_ip_claim, source_header, headers_seen}.
One row per distinct (proxy source, leaked IP, leaking header)
triple; repeat hits with the same header on different paths dedup.
- http_quirks: {fingerprint_type, order_hash, order, casing_hash,
casing_category, stable_count, tool_guess}. No more header_count
(included volatile headers; Cookie-presence variance broke dedup).
Per-request context (path, method, etc.) was never load-bearing for
analysts — the logs table already answers "when + where" at
per-event resolution. The bounty table is for stable identity.
UI:
- FpHttpQuirks renderer drops the method/path footer line and the
header_count/duplicates tags; shows stable_count instead.
- LEAKED-IPs tooltip on AttackerDetail swaps "X on GET /path" for
"Leaked via X; source 203.0.113.42" — same information, stable.
Tests add a "payload stable across paths and methods" assertion on
http_quirks — locks the contract so a future regression that sneaks
a per-request field back in fails loudly.
Existing duplicate bounty rows don't retroactively collapse.
Dev: `decnet db-reset --i-know-what-im-doing drop-tables` and
restart. Prod: one SQL pass to dedup by (attacker_ip, bounty_type,
payload) — trivial but not automated.
Per-request HTTP fingerprint derived from the header dict we already
log. Captures:
- order_hash: SHA-256 prefix (16 hex) over the lowercased header-name
sequence, minus volatile/per-request headers (Content-Length,
Cookie, Authorization, XFF family, trace IDs). Stable identity for
a given client stack regardless of which target / path is hit.
- casing_hash: same shape but over the per-header casing category
(Title-Case / lower / UPPER / mixed). Attackers frequently spoof
User-Agent but forget their stack sends `user-agent` while browsers
send `User-Agent`.
- tool_guess: prefix match against curl / python-requests /
Go-http-client / nmap-nse signatures. Cheap, best-effort — the
hash is the hard signal.
- duplicates: reserved for when the HTTP template switches from
dict(request.headers) to a list form; today it always fires empty
because dict() collapses duplicates.
Payload is a fingerprint bounty (bounty_type="fingerprint",
fingerprint_type="http_quirks"). Bounty dedup collapses identical
hashes per attacker — one row per distinct fingerprint — so a chatty
scanner doesn't spam the vault, but a tool-chain change from the
same IP surfaces as a new row.
UI renderer (FpHttpQuirks) shows the two hashes, tool guess badge in
violet, casing/count tags, and a collapsible header-order list.
Added to the passiveTypes group so it nests with JA3/JA4L/etc. in
the AttackerDetail fingerprints panel.
One library note: the naive "title-case" classifier failed on tokens
like `X-Forwarded-For` because Python's "".islower() returns False
so `p[1:].islower()` rejects single-letter tokens like the `X`.
Fix: explicitly accept single-char tokens when uppercase.
Attackers routinely front their scanners with VPNs/proxies, so the
TCP source we log is the proxy egress, not the real host. But a
surprising number of attacker setups are misconfigured: the proxy
forwards the real IP in an X-Forwarded-For (or Forwarded / X-Real-IP
/ CDN-variant) header. From our side that's a free attribution leak.
New _detect_ip_leak extractor in decnet/web/ingester.py fires at
ingest time per HTTP request. Logic:
1. Require service=http, source_ip present, headers present.
2. If source_ip ∈ DECNET_TRUSTED_PROXIES (comma-separated IPs or
CIDRs) → legitimate reverse-proxy forwarding, skip.
3. Walk proxy-family headers in priority order: Forwarded (RFC 7239)
→ X-Forwarded-For → X-Real-IP → True-Client-IP → CF-Connecting-IP.
4. Extract the left-most parseable IP from the winning header.
5. If that IP differs from the TCP source → emit a bounty with
bounty_type="ip_leak" carrying {source_ip, real_ip_claim,
source_header, headers_seen, path, method}.
Storage is the existing Bounty table — no schema change; de-dup is
handled by Bounty's (attacker_ip, bounty_type, payload_hash) key, so
repeat requests with the same leaked IP don't spam.
AttackerDetail renders a warn-accent "LEAKED IPs:" row under ORIGIN
listing distinct real_ip_claim values; hover tooltip shows the source
header + path of the most recent leak. Only shown when at least one
ip_leak bounty exists.
RFC 7239 Forwarded parser handles the full vocabulary — bare IPv4,
IPv4:port, quoted, IPv6 in brackets, IPv6 with port — returning only
IPs that actually parse.
Closes DEVELOPMENT.md "Network Topology Leakage → X-Forwarded-For
mismatches". Phase 3 of the three-phase Attacker Intelligence series
(phases 1: scanned-vs-interacted, 2: PTR records already shipped).
DECNET_TRUSTED_PROXIES env shape matches THREAT_MODEL DA-08's
"revisit when verified-proxy config lands" note — same token set
future rate-limit work will consume.
Resolve each attacker IP's rDNS name once at first sighting, store on
Attacker.ptr_record, render on AttackerDetail under ORIGIN. Many
attackers run infrastructure with forgotten rDNS that instantly
identifies them once surfaced: scan-node-42.shodan.io,
shady-vps.leasecloud.net, etc.
Resolver lives in decnet/geoip/ptr.py — colocated with enrich_ip
because the shape matches (take an IP, return supplementary
metadata, never raise). Uses the OS resolver via socket.gethostbyaddr
offloaded to the default executor, wrapped with asyncio.wait_for
timeout=2s so a slow authoritative NS can't stall the profiler tick.
Profiler side: _WorkerState grows a ptr_attempted: set[str] bounding
resolution to once per worker lifetime. Cold-start batches resolve
concurrently (Semaphore(_PTR_CONCURRENCY=10)) so a backlog doesn't
serialize 2s ceilings. _build_record gains a keyword-only ptr_record
parameter that, when _UNSET, omits the key from the record dict —
upsert_attacker's attribute-merge loop then preserves whatever's
stored on the row. Explicit None is a "fresh failed attempt" signal
and gets written through.
Env kill-switch DECNET_PTR_ENABLED=false for locked-down deploys
where egress DNS is forbidden. Private / loopback / link-local /
multicast / reserved addresses short-circuit before any DNS call.
IPv6 reverse DNS works transparently through the stdlib resolver.
Schema change — run once on upgrade:
ALTER TABLE attackers
ADD COLUMN ptr_record VARCHAR(256) NULL DEFAULT NULL;
Or drop-and-recreate on dev boxes (db-reset's SQLModel.metadata-driven
table discovery now picks it up automatically since ba155b7).
tests/conftest.py disables DECNET_PTR_ENABLED globally for the same
reason it disables DECNET_GEOIP_ENABLED — unit tests must never hit
the network. tests/geoip/test_ptr.py re-enables explicitly via an
autouse fixture.
Adds a new card on AttackerDetail: SCANNED · N services | INTERACTED
WITH · M services. Distinguishes port-scanners (N high, M=0) from
actual engagement (M>0) at a glance — the analyst's first question
when triaging a new attacker row.
Classifier lives in decnet/correlation/event_kinds.py, a single
source of truth for the event-type vocabulary:
- INTERACTION_EVENT_TYPES — command-family (command/exec/query/...),
SMTP engagement (mail_from/rcpt_to/message_accepted), file/payload
activity (file_captured/upload/download_attempt/retr), pub/sub
(publish/subscribe), recorded TTY sessions.
- NOISE_EVENT_TYPES — DECNET-internal (startup/shutdown/parse_error/
unknown_*).
- Everything else defaults to scan. Conservative by design: new
template verbs show up as "scanned" until explicitly promoted.
Bucket logic: a service is "interacted" if ≥1 of its events
classifies as interaction; otherwise "scanned" if ≥1 scan event;
noise-only services drop. Disjoint by construction.
Deliberate no-schema path: compute on-the-fly in the detail endpoint
via SELECT DISTINCT service, event_type FROM logs. Small result set
(tens of pairs per attacker), cost is trivial vs. the existing
behavior/commands queries. Trade-off: one more DB round-trip per
detail view in exchange for zero ALTER TABLE migration pain and
immediate classifier-change feedback loop.
Profiler's _COMMAND_EVENT_TYPES stays as-is (strict subset of
interactions that carry executable text), with a comment pointing at
the new canonical module.
Closes DEVELOPMENT.md "Attacker Intelligence §Service-Level Behavioral
Profiling — Services actively interacted with".
test_lifespan_db_retry patched decnet.web.api.asyncio.sleep to skip the
DB-retry backoff. Problem: asyncio is a shared module — the patch leaks
to every caller that looked up asyncio.sleep via `import asyncio`,
including run_health_heartbeat's own sleep loop. That heartbeat task
spawns inside the same lifespan; with its sleep mocked, the while-loop
spins tight, starves cancellation, and leaves an orphan task that
pytest-timeout eventually signals — surfacing as the 'Task exception
was never retrieved' warnings the user saw when running the suite.
Fix: give decnet.web.api a local binding `_retry_sleep = asyncio.sleep`
for the DB-retry wait, and have the test patch that instead. Narrowly
scoped, no impact on asyncio.sleep callers elsewhere.
Test timing before: 12s with --timeout=10 (interrupted by signal).
Test timing after: 0.58s. Full tests/web slice: 27s → 7.1s with the
spurious warnings gone.
Before: if the bus was unreachable at worker start, we logged
"running in idle mode" once and parked on shutdown forever. systemd
doesn't guarantee bus is fully up before the webhook worker starts,
so a race on boot left the worker permanently dead until restart.
Now: wrap the whole bus-use in an outer reconnect loop.
while not shutdown:
try: connect()
except: sleep(RECONNECT_SECS) ; continue
try: run_with_bus(...) # heartbeat + dispatch
except: log+close ; reconnect on next iter
Clean consequence: if the bus dies mid-operation the dispatch loop's
subscriptions raise inside the consumer tasks, `_run_with_bus` exits,
the outer loop closes the stale connection and reconnects. No partial
state leaks across epochs — fresh bus, fresh subs, fresh heartbeat.
Interval is 60s by default, overridable via
DECNET_WEBHOOK_BUS_RECONNECT_SECS. Shutdown wakes the wait so
systemctl stop doesn't hang for a minute.
Test added: flaky get_bus that fails once, then returns a live
FakeBus — asserts retry + successful delivery.
get_app_bus() in decnet/bus/app.py already has a 2s backoff retry so
the FastAPI hot path self-heals; this commit brings the standalone
webhook worker in line with the same posture.
Add "webhook" to KNOWN_WORKERS + the start-all preferred order so the
Config → Workers panel picks up the row automatically: heartbeat
subscription, start/stop controls via the existing systemd helper
(decnet-webhook.service.j2 already lands via decnet init's unit
glob), and the status-dot lifecycle all come for free.
Placed between mutator and the swarm-only agent/forwarder/updater
trio — matches the intended startup sequence (bus → api → data-plane
workers → egress → swarm management).
No frontend change needed; Config.tsx reads the worker list
dynamically from GET /api/v1/workers.
The hardcoded _DB_RESET_TABLES tuple had drifted — session_profile,
smtp_targets, and webhook_subscriptions were all missing, so
`decnet db-reset --i-know-what-im-doing drop-tables` silently left
them behind. Running it on a post-webhook install then letting
SQLModel.metadata.create_all() re-create tables produced a partial
schema: old rows survived, new columns didn't land, and endpoints
500'd on the missing columns (e.g. auto_disabled_at after the
circuit breaker merge).
Replace the hardcoded list with `SQLModel.metadata.sorted_tables`,
reversed for DROP safety (children first). Any future model addition
is auto-enrolled — no manual step, no more drift.
No behavior change on reset semantics; the SET FOREIGN_KEY_CHECKS=0
fence still covers any edge case the sort order misses.
After DECNET_WEBHOOK_CIRCUIT_THRESHOLD (default 5) consecutive failed
deliveries, the worker calls trip_webhook_circuit(uuid, ts) which
flips enabled=False and stamps auto_disabled_at. The worker sets its
reload flag so the next dispatch epoch stops consuming events for the
tripped sub entirely — one dead receiver can't poison the shared
egress pool anymore.
Operator clears the trip via PATCH — setting enabled=True when the
sub was previously disabled clears auto_disabled_at, zeros
consecutive_failures, and clears last_error. Admin-pause → re-enable
hits the same path harmlessly.
Three observable states now distinguishable in the UI:
- Active enabled=True, auto_disabled_at=NULL
- Admin-paused enabled=False, auto_disabled_at=NULL
- Tripped enabled=False, auto_disabled_at=<ts>
UI surfaces a TRIPPED · <ts> chip on the row (red, alert-styled) and
a "N TRIPPED" count in the page header. Hover tooltip tells the
operator how to reset ("Re-enable via Edit").
record_webhook_failure now returns the new consecutive_failures count
so the worker can compare against the threshold without a second
roundtrip. trip_webhook_circuit is idempotent — re-tripping just
re-stamps auto_disabled_at.
Closes THREAT_MODEL WH-02 and DEBT-037 §1.
The per-row test-delivery action already existed as an icon-only ⚡
zap in the ACTIONS column — backed by POST /webhooks/{uuid}/test,
which fires a synthetic test.ping event through the normal HMAC-
signed delivery path with retries disabled. Too easy to miss.
Replace the icon-only button with a labeled [⚡ FIRE] violet-accented
button so it reads as an emphasized dev-tool action right next to
edit/delete. Tooltip now spells out the backend endpoint and "fire
a synthetic test event" intent.
No backend change. Widens the actions column to 180px to accommodate
the label.
Python stdlib ThreadingHTTPServer that accepts any POST path, optionally
verifies HMAC against --secret / $DECNET_MOCK_SECRET, and pretty-prints
each delivery with topic / event-id / signature status. Pass --fail 503
to exercise the worker's retry/backoff path.
Point a webhook at http://localhost:8765/ and you'll see every delivery
land with color-coded HMAC OK / MISMATCH / UNVERIFIED badges. No deps.
The webhooks page used a bespoke .webhooks-header wrapper that didn't
line up with the rest of the dashboard (Fleet / Logs / Swarm all use
the .<page>-root + .page-header + .page-title-group + .actions
pattern). Swapped to that convention:
- .webhooks-root wrapper, matching .logs-root / .fleet-root spacing.
- H1 "WEBHOOKS" in .page-title-group; subtitle shows
`N CONFIGURED · M ENABLED [· K FAILING] [· L INSECURE]` in
.page-sub, same voice as the LOGS stream summary.
- Actions (CREATE WEBHOOK, DELETE SELECTED) sit in .actions.
- Table lives in a proper .logs-section shell with a .section-header
carrying the Webhook icon + "SUBSCRIPTIONS" title.
- All scoped button overrides (violet/alert/warn/ghost) copied from
the LiveLogs scope so theme switches behave identically.
Also improve error messaging: extractErrorDetail now maps 401 to
"Session expired" and 403 to "Insufficient permissions (admin only)"
instead of falling through to the generic "Failed to load webhooks".
Helps users who hit the page as viewer or with a stale token see why
it failed.
New /webhooks admin page with table-based subscription management:
- CREATE WEBHOOK (inline form row — no modal) with simple-event
checkboxes (AttackerDetail / DeckyStatus / SystemStatus) that
expand to bus-topic patterns server-side, and an advanced-mode
textarea for raw NATS-style patterns.
- Bulk-select + DELETE SELECTED with two-click arm pattern.
- Per-row test-ping (zap), pencil edit, and delete actions.
- Last-fired timestamp column.
- Yellow banner surfacing insecure_url warnings (WH-03): http:// is
allowed but flagged so operators see it on every page load.
- Post-create secret modal — the secret is shown exactly once with
a COPY button and a clear "won't see this again" notice.
Sidebar nav regrouped: /live-logs and /webhooks now live under a new
ALERTS NavGroup (Bell icon). The alertCount badge rides the Live
Logs sub-item. Command palette gains a "Webhooks" GO TO entry with
the `G W` chord.
Side-fix: useFocusSearch.ts was failing the build under
verbatimModuleSyntax (pre-existing, unrelated). Split the React
import to satisfy tsc; no behavioural change.
The webhook MVP shipped with deliberate deferrals; this entry names
them so future PRs know exactly what's left to close: circuit
breaker, dead-letter table, delivery audit log, batch/coalescing,
per-subscription rate limiting, payload templates per destination,
and secret encryption at rest.
Non-negotiable even at MVP scope (HMAC signing, bus-off degraded
mode, jittered retry backoff) is called out explicitly to prevent
future contributors from weakening it under the banner of
"simplification."
WebhookResponse now carries a `warnings: list[str]` field. When the
subscription's URL starts with http://, an `insecure_url` advisory is
surfaced on every GET/CREATE without blocking the request. HMAC still
detects tampering regardless of transport — only read-confidentiality
is lost over plaintext — and test/dev environments without TLS stay
usable.
Matches the operator-trust posture already established by DA-06
(admin-on-admin protection is out of scope). The alternative — hard
rejection at admin time — was considered and declined; warning-plus-
visibility is the right shape.
THREAT_MODEL WH-03 accepted risk registered; revisit triggers are
multi-admin delegation, a regulated customer, or an operator ticket
asking for a DECNET_WEBHOOK_REQUIRE_HTTPS enforcement knob.
- DEVELOPMENT.md: tick the "Real-time alerting" roadmap item with a
note that Slack/Telegram-specific senders remain per-destination
follow-ups (they accept generic webhook payloads already).
- THREAT_MODEL.md: new Component 2 — DECNET↔External webhook
destination. DFD, full STRIDE table, WH-01 (secret at rest) and
WH-02 (half-dead-receiver retry waste) registered as accepted
risks pointing at DEBT-037 for post-MVP hardening. Checklist lists
two open items: OpenAPI schema omits `secret`, and http:// URL
rejection at admin time.
Introduces the `decnet webhook` long-running worker that consumes the
internal bus and POSTs matching events to configured subscriptions.
Design: one task per (subscription, pattern) pair. Each task opens
its own bus subscription, iterates events, and dispatches via the
shared deliver() client. No intermediate queue, no in-memory filter
matching — the bus's own pattern matcher is the filter. Reloads on
`system.webhook.subscriptions_changed` signals from the CRUD router,
with a 60s fallback timer in case a signal is lost.
Shutdown propagates via CancelledError on the outer task; all inner
subscription tasks are cancelled and awaited in a finally block.
Bus unavailable → worker stays up in idle mode per the DEBT-031
pattern, logging one warning.
Registered as a master-only CLI command (agents don't configure
webhooks — the subscription store lives on master). systemd unit
mirrors the profiler template; added to decnet.target Wants= list so
`systemctl start decnet.target` brings it up alongside everything
else. `decnet init` auto-picks up the new .service.j2 via its
existing `glob("decnet-*.service.j2")` sweep.
Introduces the webhook egress foundation — a new WebhookSubscription
table, admin-gated CRUD under /api/v1/webhooks, and the shared
delivery client that both the test-ping route and the upcoming worker
will use. No worker yet; this commit is API + model + client only.
Simple-mode enum (AttackerDetail / DeckyStatus / SystemStatus) expands
to bus-topic patterns at the router layer; storage is always the raw
pattern list. Advanced mode lets admins supply raw NATS-style patterns
directly. Filter-at-subscribe: the worker (next commit) will subscribe
to the union of patterns across enabled subscriptions.
Delivery client handles HMAC-SHA256 signing (X-DECNET-Signature),
retry on 429/5xx/network errors with jittered backoff, no-retry on
4xx. Secrets never leave the server on GET/LIST — only the create
response carries the secret for copy-out.
CRUD routes publish WEBHOOK_SUBSCRIPTIONS_CHANGED on the bus after
every mutation so the (future) worker can hot-reload.
Opens DEBT-037 for the deferred items (circuit breaker, dead-letter,
batch delivery, payload templates, secret-at-rest).
New decnet/web/sse_limits.py provides sse_connection_slot, an async
context manager that counts live SSE connections per user UUID and
raises 429 when a per-user cap is exceeded (default 5, override via
DECNET_SSE_MAX_PER_USER). Wired into both SSE generators as their
first async with, so the cap check fires before any stream data is
yielded.
The cap must sit inside the generator — StreamingResponse returns
before the generator body runs, so a handler-level wrapper would
release the slot immediately. Put prefetch + slot + loop all under
the one async with.
Also documents F6/I (role leakage) as mitigated-by-construction via
handler docstrings: every event type on both streams wraps data
already reachable via viewer-gated REST, so no per-event filter is
needed until a new event family is introduced. The invariant is
written into the handler docstrings so a future PR can't silently
add admin-only events.
Resolves THREAT_MODEL F6/I and F6/D.
Every mutation route that returned an untyped dict now declares
response_model at the decorator. MessageResponse covers the eight
{"message": ...} envelopes (change-password, mutate-decky, mutate-
interval, update-deployment-limit, update-global-mutation-interval,
delete-user, update-user-role, reset-user-password). Purpose-built
models cover the richer shapes (DeployResponse for /deckies/deploy,
PurgeResponse for /config/reinit, ReapReportResponse for /reap-orphans,
UserResponse for /config/users). 204-No-Content and Response/
ORJSONResponse routes stay as-is.
The wire shape for clients is unchanged — the envelopes already only
shipped a message field. What changes is that a handler which
accidentally returns a richer dict (e.g. a full user row including
password_hash) would be silently stripped to the declared fields at
serialization time.
Also flips F4/D "expensive LIKE" to accepted (new DA-09) — the /logs
and /attackers search routes LIKE-scan unbounded columns, but both are
admin-gated, limit-capped, and operator rate-limit scope per DA-04.
FTS5 stays a performance TODO, not a security blocker.
The other five query endpoints (/logs, /attackers, /attacker-commands,
/bounties, /topologies/{id}) already declared le=2147483647 on offset;
these two were inconsistently uncapped. Bring them in line to close
the F4/D deep-pagination row.
Also resolves F4/T (ORM sort injection — already mitigated by the
regex pattern on /attackers sort_by, no other route accepts a column
name) and F4/D (limit cap — already universal) with code pointers.
New test walks app.routes, classifies each APIRoute as admin/viewer/open
by identity-matching require_admin / require_viewer closures inside the
route's dependency tree, then asserts:
- admin routes return 403 to a viewer JWT
- viewer routes return neither 401 nor 403 to a viewer JWT
SSE routes skipped (separate scope under F6). Role hints deliberately
NOT encoded in the OpenAPI spec — classification stays server-side so
/openapi.json can't be used to enumerate admin routes.
Resolves THREAT_MODEL F2/I + F5/E; paired with the existing
test_schemathesis.py::test_auth_enforcement (401-half coverage).
Harden the attacker-controlled artifact download path (F7) with explicit
response headers instead of relying on Starlette's defaults (which only
emit attachment for non-ASCII filenames and never set nosniff). Also
resolves the THREAT_MODEL F7 path-traversal row (containment check was
already in _resolve_artifact_path) and the fleet-deploy detail=str(e)
audit (all four sites are admin-gated deliberate validator UX or
structured worker-response fields).
The ~30-signature hand-rolled p0f-lite table in decnet/sniffer/p0f.py
misses most real-world attackers (yesterday's SLOW SCAN being a
textbook case — 9 hours of events, 19 hits, os_guess = NULL). The
375-sig vendored p0f v2 DB was already there; this commit actually
calls it.
New resolution chain in sniffer_rollup:
1. Enabled OS-fingerprint providers (p0f-v2 default, via
DECNET_OSFP_PROVIDERS) tried in declared order. Provider with
highest-confidence match across all enabled sources wins.
2. Modal os_guess label from the sniffer's hand-rolled p0f.py.
Kept as fallback because v2's DB predates post-2006 kernels.
3. TTL bucket (linux / windows / embedded). Coarse but never wrong.
Wiring details:
- _match_via_osfp_providers: never raises — factory / provider
failures collapse to None and the chain falls through to the
old modal-label / TTL path. A corrupt .fp file or misconfigured
DECNET_OSFP_PROVIDERS must never wedge a profile rebuild.
- tcp_fp_context tracks whether the LATEST tcp_fp snapshot came
from a passive SYN ('syn' → p0f.fp) or an active prober probe
('synack' → p0fa.fp). Routes to the right sig list.
- initial-TTL normalisation via decnet.sniffer.p0f.initial_ttl.
Observation's TTL may be N hops below the OS's initial; v2
signatures match on the canonical bucket.
Soft-field semantics on Signature.score(): df and total_len are now
skip-checked when the observation is missing them. Sniffer doesn't
currently emit either SD field; a literal-constraint sig
shouldn't hard-reject a match solely because of upstream
incompleteness. Hard fields (window, ttl, options_sig, quirks)
still hard-reject on absent/mismatched input — those are the real
discriminators. Promote df / total_len back to hard the moment the
sniffer starts emitting them.
+2 integration tests on TestSnifferRollup, +2 soft-field tests on
test_signature. Full regression: 166 tests across tests/prober/osfp
+ tests/profiler all green.
- decnet/prober/osfp/p0f/provider.py: P0fV2Provider loads the four
vendored .fp files into per-context signature lists (syn / synack /
rst / stray) and matches via highest-specificity score across the
relevant list. Also auto-picks up p0f-decnet.fp if present (GPL-3.0
additions land there later, empty for now).
- decnet/prober/osfp/factory.py: get_provider / get_all_providers /
reset_cache, mirrors decnet/geoip/factory exactly. Env-dispatched
via DECNET_OSFP_PROVIDERS (default "p0f-v2"). Reserved names
"nmap-osdb" (pending Fyodor's grant) and "decnet-observed" (our
future curated DB) raise NotImplementedError — visible on the
factory surface so a typo doesn't silently fall through.
- decnet/prober/osfp/__init__.py now re-exports the public API so
callers use `from decnet.prober.osfp import get_provider` without
reaching into submodules (upholds the provider-subpackage rule).
15 new provider+factory tests covering:
- All four DB contexts load (262/61/46/6 sigs per inventory).
- Known-good Linux 2.6 SYN + Linux 2.2 SYN-ACK match end-to-end.
- Unknown observations / contexts return None, not raise.
- Factory memoises, env override honoured, unsupported names raise.
- Reserved names raise NotImplementedError (not silent None).
`sniffer_rollup` wiring lands in the next commit.
First code layer of the OS-fingerprinting work on top of yesterday's
vendored p0f v2 database. Three new modules, all pure (no I/O outside
of the parser's file read):
- decnet/prober/osfp/base.py — Provider protocol + OsMatch dataclass
matching the established Provider convention in decnet/geoip and
decnet/bus. Docstring spells out the never-raise invariant: malformed
input returns None, so a single bad event can't wedge a whole
attacker-profile rebuild.
- decnet/prober/osfp/p0f/signature.py — Signature dataclass + three
predicate helpers (WindowSpec / IntSpec / OptionToken) encoding the
p0f v2 DSL's wildcard / modulo / MSS-multiple / MTU-multiple
semantics. Scoring is our extension on top of upstream p0f's
first-match-wins policy: each signature carries a precomputed
specificity in [0, 1] so the factory can pick the most-specific
match when multiple signatures fire against one observation.
- decnet/prober/osfp/p0f/format.py — .fp line parser. Every shipped
field variant from the DSL spec at the top of p0f.fp is covered
(Snn / Tnn / %nnn / * for window; T0 vs T; -/@/* os-genre prefixes;
quirks as concatenated single-letter flags; '.' sentinels for
no-options / no-quirks). Malformed lines log a warning and skip
instead of aborting the whole file — 1 bad row must not cost the
other 374.
20 parser tests + 14 scoring tests. Full vendored-DB smoke tests
confirm all 375 signatures parse round-trip (262 SYN + 61 SYN-ACK +
46 RST + 6 stray) and every computed specificity lands in [0, 1].
Ships the p0f v2.0.8 signature database for passive + active OS
fingerprinting. 375 total signatures across four probe contexts:
- p0f.fp (262 sigs) — passive SYN fingerprints
- p0fa.fp ( 61 sigs) — SYN-ACK response, for active probes
- p0fr.fp ( 46 sigs) — RST response quirks
- p0fo.fp ( 6 sigs) — "stray" packet fingerprints
Replaces reliance on the 10-signature hand-rolled p0f-lite table in
decnet/sniffer/p0f.py for any match job the upstream DB covers.
Keeping the hand-rolled table as a fallback for modern kernels the
v2 DB pre-dates — v2 froze in 2006 so post-Win10 / post-Linux-3.x
kernels won't match against upstream directly. DECNET-authored
additions will go in a sibling p0f-decnet.fp under GPLv3 (not yet
committed; added as the ingester observes real honeypot traffic).
Provenance (full chain in data/README.md):
- Source: Debian snapshot of p0f_2.0.8.orig.tar.gz
- SHA1 matches Debian-recorded 7b4d5b2f24af4b5a299979134bc7f6d7b1eaf875
- Files byte-identical to upstream tarball (verified by hash)
License chain:
- Upstream: LGPL-2.1 (doc/COPYING preserved verbatim as
data/LICENSE.p0f-upstream, Michal Zalewski's copyright intact).
- DECNET uses the LGPL-2.1 §3 explicit permission to convert to any
version of the GPL. These files, as consumed in DECNET, are
effectively GPL-3.0. Chain documented in data/README.md so an
auditor sees the full reasoning.
- LGPL-2.1 → GPL-3.0 §3 conversion is a settled compat path; same
mechanism the kernel uses for LGPL userland glue and many other
projects apply daily.
Rejected path — nmap-os-db under NPSL — because NPSL adds
restrictions GPLv3 §7 prohibits us from accepting. An email is out
to Fyodor requesting an open-source-author exception grant, but we
don't block on it: p0f v2 is a genuine accuracy improvement in
its own right, and adding nmap-osdb later (if granted) plugs into
the same provider interface with zero refactor.
Directory layout mirrors the established provider-subpackage pattern
(see decnet/geoip/, decnet/bus/) per the feedback_provider_
subpackages memory: base + factory + impl/ subpackages, no flat
files. Parser + matcher + factory wiring land in the next commit
sequence.
DECNET had no LICENSE file and no license metadata in pyproject.toml
despite intent being GPLv3. Legally that meant the code was "all
rights reserved" by default, so anyone distributing it (including via
GitHub clones, mirrors, or the forthcoming swarm enroll bundles) was
technically in violation even though the operator's own intent was
copyleft.
- Add canonical GPL-3.0 text from gnu.org/licenses/gpl-3.0.txt as
LICENSE (verbatim, 674 lines).
- Add license = "GPL-3.0-or-later" and license-files = ["LICENSE"]
to pyproject.toml [project] (SPDX identifier per PEP 639).
- Add the matching OSI classifier plus a few other standard ones
(Python 3.11, Linux, Security, Network Monitoring, Beta) that
pyproject was silently missing.
Prereq for the forthcoming p0f-db vendoring: establishing DECNET's
own license explicitly closes the first question an auditor would
ask about any third-party data we embed.
Follow-ups on 9232031 per review:
- Module-level constants KD_PAUSE_BURST_MAX_S (0.2s),
KD_PAUSE_THINK_MAX_S (1.5s), KD_START_OF_ACTION_IDLE_S (2.0s).
Docstrings reference them by name; future calibration against real
session data only has to touch one place. Threshold for "started
a new action" raised from 1s → 2s — 1s catches too much
mid-command hesitation to be empirically bimodal.
- New column kd_max_pause_gap (seconds). The distracted bucket count
alone can't distinguish one 3s pause from three 60s pauses;
max-gap carries that signal in one cheap scalar (vs widening the
histogram to a fourth bucket).
- Scope-framing docstring above the whole kd_* section: intended
use is session clustering / tooling attribution, explicitly NOT
biometric identity, admission decisions, or ML-driven user ID.
Keeps a future well-intentioned contributor from walking the
project into legal/ethics territory by accident.
- TODO comment on kd_top_bigrams: v1's JSON-in-TEXT is fine for
"show the top digraphs on the attacker page". If bigram-similarity
queries become hot, promote to a session_bigram_stats(sid, bigram,
count, mean_iat_s) table or Postgres JSONB + GIN. Neither changes
the write-side ingester materially.
No new migration helper — pre-v1 schema additions go through
create_all on fresh DBs; the existing _migrate_session_profile_table
stays but does not get extended. Alembic lands at v1 and sweeps all
the ad-hoc migrations at once.
Adds the three signal columns motivated by the manual keystroke
analysis in DEBT-036 directly to the SessionProfile table. Pre-v1 so
we modify the schema in place — Alembic arrives at v1.
Columns:
- kd_top_bigrams (TEXT) — JSON of top-N most-common digraphs with
mean IAT per bigram. Complements kd_digraph_simhash ("same typist?")
with "same typist in same mental state?" (tired / rested / distracted
shifts bigram-specific IATs measurably).
- kd_start_of_action_latency (REAL/DOUBLE) — median IAT of the first
keystroke after an idle gap > 1s. Separates "initiating a command"
from "executing a remembered one"; real humans have measurable
start-of-action latency, bots don't.
- kd_pause_hist_burst / _think / _distracted (INT) — three-bucket
histogram (counts, <0.2s / 0.2-1.5s / >1.5s). More discriminating
than the existing flat burst_ratio / think_ratio pair: C2 operators
concentrate in burst with a thin tail; opportunistic humans have a
fat think bucket and a long distracted tail.
Both backends get an idempotent ADD COLUMN migration
(_migrate_session_profile_table) wired into initialize() alongside
the existing _migrate_attackers_table path — guards on PRAGMA
table_info (SQLite) / information_schema.COLUMNS (MySQL) so reruns
are safe.
PII discipline comment on kd_digraph_simhash and kd_top_bigrams:
both operate on bigram CHARACTERS, never on raw input stream content.
Attacker passwords typed over SSH must not land here.
Test updated for the MySQL initialize() migration-order contract.
The SessionProfile SQLModel table has shipped with every column
nullable since session-recording v1 landed — because the ingester
that populates them from the [t,"i",d] events in the transcript
shards does not exist yet (known as gap #2 in SIGNAL_CAPTURE_AUDIT).
A manual keystroke-dynamics pass over one real session (wget scanme.
nmap.orgh) trivially recovered CoV ≈ 0.74 (human band), a 467 ms
semantic pause before the URL argument, tight intra-word bigrams
(ge 79 ms, t<space> 83 ms), and slow start-of-action latency (w→g
225 ms) — all signals the existing schema columns were designed to
hold. So the missing piece is purely the ingester.
Entry captures:
- the manual case as the motivating + sanity-check target
(ingester should produce CoV ≈ 0.74 ± 0.05 on the same shard),
- three schema extensions the manual analysis suggests beyond what
the table carries today: kd_start_of_action_latency_ms,
kd_pause_hist_{burst,think,distracted}, kd_top_bigrams,
- a non-PII discipline line: raw keystroke content (including
captured passwords) MUST NOT land in SessionProfile columns —
only timing and frequency aggregates.
Poll-driven ingestion can ship first; the bus-trigger path
piggybacks on DEBT-031's deferred session-boundary topics.
The drawer used onClick={onClose} on the backdrop + onClick={e =>
e.stopPropagation()} on the panel to stop inside-clicks from closing
the drawer. That pattern is fine for most React trees, but React's
stopPropagation() also aborts the NATIVE DOM event — and asciinema-
player wires its click-to-play handler via document-level event
delegation. So every click inside the drawer (including the big
play button) died at the panel boundary and never reached the
player's dispatcher. Confirmed end-to-end by calling window.__ap.
play() directly from DevTools: playback started, cast rendered in
full, ended event fired.
Swap to the idiomatic target===currentTarget guard on the backdrop
so only genuine backdrop clicks close the drawer; everything inside
(including native-delegated handlers) gets its events untouched.
All the debug instrumentation from b5c6b8a, 4424138, 6d031ae, and
f032ece (cast logging, lifecycle listeners, window.__ap) is
reverted here — symptom root-cause is known, it was event delegation
not the parser or the cast.
The parse path works (metadata event fires with duration: 24.58s,
idle event fires); next unknown is whether clicking play even
reaches core.play(). Stash the player on window so the operator can
call __ap.play() from DevTools to diff UI-click vs direct-call
behaviour and see whether 'play' / 'playing' events fire.
To be reverted once we pin the failure.
The original short subscribe list missed 'metadata' — which is the
one that carries the parsed duration + theme + marker info AFTER
_initializeDriver (the step that actually parses the cast). Without
it we only saw 'ready' (= UI mounted, parse not yet run) and jumped
to conclusions about the parser.
Add the full lifecycle set so the next repro pins which step the
player is actually getting stuck at.
Without preload:true the player only parses the recording when the
user first clicks play. Any parse error during that lazy step
bypasses our lifecycle instrumentation (we only see "ready", which
just means UI mounted), and from the user's POV the play button
stays black because they never see the actual failure.
Forcing preload makes the driver's init() run synchronously-ish with
the "ready" dispatch, so getDuration() resolves to a real number
(or we see an "errored" event with a payload that tells us why).
The sync try/catch around AsciinemaPlayer.create() misses async
failures in the player's internal init() promise — those land as
unhandled rejections and are invisible from the component's POV.
Subscribe to every lifecycle event (ready / play / pause / ended /
error / errored / loading) and log the resolved duration. If the
parser produces zero events despite a well-formed cast, duration
resolves to 0 / NaN / rejected — one of those signals will point at
whichever frame the render path is silently failing at.
Diagnostic for the persistent "player mounts with chrome but plays
black" symptom after the blob-URL fix. The player now gets
{data: cast} correctly and parses at least enough to render the
control bar, but duration shows --:-- and the terminal stays blank.
Log the first 400 chars of the built cast + event/cols/rows so the
operator can confirm in DevTools whether the malformed input is the
cast itself or something downstream in the asciinema parser.
SessionDrawer built a cast blob, pushed it through URL.createObjectURL,
and passed the blob URL to AsciinemaPlayer.create(). That's racy with
useEffect's cleanup: each new page of events re-fires the effect, the
cleanup revokes the URL, and the player's already-in-flight async
loadRecording() lands on a dead URL with no visible error — result was
a centered play button with an empty black pane, playback never starts.
asciinema-player v3's recording driver accepts {data: <string>} as a
first-class source (see core-DnNOMtZn.js:905-930 doFetch — string/
ArrayBuffer data is wrapped in `new Response(value)` and handed to the
parser). Skip the blob detour entirely, pass the cast text inline.
Also filter events to valid asciicast channels (o/i/r) before feeding
so a future stray SD field can't derail the parser, and log mount
errors to console for next-time debugging.
Tracks the durable follow-up to 323077b. The transcripts soft-fail
shipped in that commit keeps the API from 500-ing on
/var/lib/decnet/artifacts/** permission mismatches, but the real
issue is that decoy containers write artifacts under a uid the API
can't read — today's workaround is a manual `sudo chown -R` after
every new deploy.
Three design options documented (container-runs-as-host-uid, setgid
+ shared group, inotify sidecar) with a recommendation, plus an
acceptance criterion: fresh init + deploy + record session → the
API can read the transcripts with no manual chown.
sessrec.c emits the session_recorded SD blob with sid/service/src_ip/
duration_s/bytes/truncated — it never emitted shard_path. The web
handler still asked for fields.shard_path, got "", tripped the
sessions-YYYY-MM-DD.jsonl basename regex and returned
400 "invalid shard name" for every legitimate transcript request.
Handler now:
- Fast-paths when fields.shard_path IS present and validates
(for any future emitter or ingester that backfills it).
- Otherwise enumerates sessions-YYYY-MM-DD.jsonl shards under
ARTIFACTS_ROOT/{decky}/{service}/transcripts/ (newest first) and
returns the first one whose per-sid index contains our sid.
- Security invariant preserved: only files whose basename matches the
_SHARD_BASENAME_RE are ever opened, and they always resolve inside
ARTIFACTS_ROOT. A forged fields.shard_path is silently ignored.
- Soft-fails OSError/PermissionError on the transcripts dir (decky
containers often write it with a uid the API can't read) — returns
404 instead of a 500 traceback.
test_forged_shard_path_blocked updated to match the new semantics:
forgery is ignored, the real shard is served via fallback. The
invariant (no /etc/passwd access) is still asserted by the fact
that status is 200 with data from the test shard.
decnet-bus.service.j2 ran with User={{ user }} / Group={{ group }}
but the actual bus CLI invocation hardcoded --group decnet. The bus
chowns /run/decnet/bus.sock to that group at 0660 — so when an
operator ran `decnet init --group anti`, the socket ended up
owned by decnet:decnet while every worker (agent, api, collector,
forwarder, prober, updater) ran as anti and got EACCES on connect().
Each worker's bus-wiring catches the error, logs a warning, sets
bus=None, and carries on — which is correct for the data-plane but
silently kills Workers-panel heartbeats (run_health_heartbeat(None,
...) no-ops). So half the worker grid showed UNKNOWN even though
systemctl confirmed the processes were alive.
Swap the hardcoded --group decnet for --group {{ group }} so the
socket is owned by the same group the workers run under.
polkit rule 50-decnet-workers.rules hardcoded isInGroup("decnet"),
so when 'decnet init --group anti' installed systemd units as
User=anti / Group=anti, the API (running as anti) could no longer
systemctl start/stop decnet-*.service — polkit fell back to
'interactive authentication required', which in a daemon context is
a hard fail:
START FAILED · COLLECTOR — Failed to start decnet-collector.service:
Access denied as the requested operation requires interactive
authentication.
Rename the rule to .j2, parameterise the group on {{ group }}, and
route _install_polkit through _render_template /
_write_rendered_if_changed. Now the polkit rule matches whatever
group was passed to 'decnet init'.
Test fixture updated to seed the .j2 variant.
Four templates use backslash line-continuation on ExecStart
(decnet-bus, decnet-forwarder, decnet-listener, decnet-updater). My
earlier sed inserted StandardOutput= and StandardError= right after
the first ExecStart= line, which split the command and systemd fed
those two lines back to the binary as extra positional arguments —
the bus in particular crashed with:
Got unexpected extra argument
(StandardOutput=append:/var/log/decnet/decnet.bus.log)
Walk the ExecStart block (follow \-continuation lines) and insert
the two Standard* directives AFTER the last continuation line. The
nine single-line ExecStart templates are unaffected in shape but
re-written through the same path to keep the whole set uniform.
_configure_logging opened InodeAwareRotatingFileHandler against
DECNET_SYSTEM_LOGS (default: relative decnet.system.log) without
guarding OSError. Under systemd with ProtectSystem=full +
ProtectHome=read-only and no writable path baked into the unit, the
first import of decnet.config raised OSError and the daemon died
before it could even print a useful error — the root-cause log line
showed up in journalctl as a stack trace rather than a warning.
Wrap the handler attachment in try/except OSError and log a single
WARNING via the already-installed stream handler. stderr is always
attached, so losing the file handler means operators tail
journalctl / docker logs instead — the daemon keeps running.
The agent-side enroll-bundle templates (decnet/web/templates/*) always
set DECNET_SYSTEM_LOGS + StandardOutput/StandardError to a per-unit
file under /var/log/decnet. The master-side init templates (deploy/*)
never did, so every 'decnet init'-installed service:
- inherited the default DECNET_SYSTEM_LOGS=decnet.system.log — a
relative path, landing in the unit's WorkingDirectory. All 13 units
shared the same cwd and fought for the same file, or more often
just failed to write it under ProtectSystem=full,
- emitted stdout/stderr to the journal by default, which is fine for
uvicorn's INFO banter but makes per-service grepping a pain when
you're chasing a single worker's trace.
Mirror the agent-side wiring on all 13 master templates:
- Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.<name>.log
- StandardOutput=append:/var/log/decnet/decnet.<name>.log
- StandardError=append:/var/log/decnet/decnet.<name>.log
/var/log/decnet is already in ReadWritePaths so ProtectSystem=full
stays compatible. Operators now get a dedicated
/var/log/decnet/decnet.<unit>.log per service, both from the app's
structured logger and from any stray stderr — journalctl still
works too, but no longer the only option.
Key:value chips in the live-feed event cell used the default .chip
style, which is white-space: nowrap + inline-flex. A long cmd: value
(attacker-controlled shell strings, URLs, base64 payloads) stretched
the chip horizontally past the column, pushing the whole table into
horizontal scroll and clipping subsequent columns off-screen.
Add a chip-kv variant that allows the value to wrap inside a
max-width: 100% chip (word-break: break-word, overflow-wrap: anywhere
for dense strings with no natural break). The key-label stays on the
first line via flex-shrink: 0. Short values (uid: 0, user: root)
stay tight; long ones wrap onto multiple lines inside the chip.
Also set minWidth: 0 on the EVENT td + nested flex containers so
flex children honour the column width instead of growing to fit
content. Added title={k: v} on each chip for full-value hover in
case the wrap is still clipped.
The API lifespan unconditionally spawned log_collector_worker,
appending every container line to DECNET_INGEST_LOG_FILE. On hosts
that also run decnet-collector.service (installed by 'decnet init')
that's two tailers writing the same events to the same file — the
ingester then inserts each event twice and the dashboard shows every
command duplicated.
Add DECNET_EMBED_COLLECTOR (default false), matching the existing
DECNET_EMBED_PROFILER and DECNET_EMBED_SNIFFER pattern directly
above this block. Single-process dev setups without systemd can flip
it on to restore the all-in-one behaviour; multi-process production
gets the single-writer invariant by default.
Every plain `decnet deinit` ran userdel + groupdel unconditionally. In
dev the operator may pass `--user $USER --group $USER` to avoid file
ownership churn against a source checkout — at which point deinit
would cheerfully delete their own login account.
Move user/group removal behind --purge, matching the existing
behaviour for /var/lib/decnet + /var/log/decnet. Help text updated:
--purge now clearly advertises that it also wipes the service
user/group, with an explicit warning to only run it when `decnet init`
created the account in the first place.
Test updated: plain --deinit must NOT invoke userdel/groupdel;
--deinit --purge must.
Every decnet-*.service.j2 hardcoded User=decnet / Group=decnet. The
init CLI accepted --user / --group and used them for useradd,
chown, /etc/decnet ownership and ReadWritePaths — but the Jinja
context omitted them entirely, so
sudo decnet init --install-dir $PWD --user anti --group anti
rendered
User=decnet
Group=decnet
into every unit, which at best ran the workers as a user that didn't
match the files (fails to read the venv / config), and at worst spun
a parallel system user the operator never asked for.
Swap the hardcoded lines to {{ user }} / {{ group }} across all 13
templates and add both to the Jinja context in _install_units.
The systemd unit templates hardcoded {{ install_dir }}/venv/bin/decnet.
On production hosts enroll_bootstrap.sh creates exactly that path so it
worked. On dev boxes where the operator runs `sudo decnet init` against
a source checkout with a differently-named venv (.venv, .311, .312),
every decnet-*.service looped forever in auto-restart with:
Failed at step EXEC spawning .../venv/bin/decnet: No such file or
directory
Templates now use {{ venv_dir }} as an independent Jinja2 var. `decnet
init` adds --venv-dir (explicit override), otherwise autodetects:
1. $VIRTUAL_ENV (only when inside --install-dir, so a user-home venv
never gets baked into a root-owned unit),
2. {install_dir}/venv (production default; what enroll_bootstrap
creates),
3. {install_dir}/{.venv,.311,.312,.313} (common dev conventions).
Init aborts before any file writes if nothing resolves — an
operator-friendly error beats journalctl spam on every unit restart.
python3-venv doesn't set a persistent system variable — $VIRTUAL_ENV
lives in the activated shell only — so this has to be decided + baked
in at init time; there's no way for systemd to "inherit the current
venv" at unit start.
Test mode (--prefix) skips venv validation so the existing test suite
doesn't need to stub up a venv tree per case.
'decnet status' used to psutil-scan for cmdlines matching hand-coded
service launch args. That worked on dev boxes running workers via
'python -m decnet.cli ...' but missed the systemd reality on real
hosts: units may be installed but not started, failed, or in
auto-restart — all invisible to a cmdline grep.
New behaviour: status calls `systemctl list-units --type=service --all
--output=json 'decnet-*.service'` and renders the unit/load/active/
sub/description matrix. One view works for masters, agents, and
mixed hosts — iterates over whatever 'decnet-*' units were installed
by 'decnet init' / the enroll-bundle. Agent/master mode filtering is
no longer needed in the CLI; the host literally does not have
master-only units installed if it enrolled as an agent.
The psutil path survives as a fallback for boxes without systemd
(dev laptops, CI containers, minimal init systems) so the command
stays useful there. Clearly labelled 'psutil fallback' in the table
title so operators know which view they're looking at.
Locust spawns N virtual users (default 1000), all from 127.0.0.1 as
admin. /auth/login is rate-limited 10/5min per-IP AND per-username, so
the 11th on_start() got 429 and a RuntimeError. A @task(2) login in
the task weights turned the whole run into a 429 factory even after
ramp-up. And _login_with_retry treated 429 as non-retryable, so there
was no graceful degradation path.
Three changes, one root cause:
- decnet/web/limiter.py: read DECNET_LIMITER_ENABLED (default true).
When false, slowapi's Limiter(enabled=False) makes @limiter.limit a
no-op. Default ships unchanged; nobody should ever release with this
off.
- tests/stress/conftest.py: set DECNET_LIMITER_ENABLED=false in the
uvicorn subprocess env. Stress tests measure throughput, not rate
limiting.
- tests/stress/locustfile.py: drop the @task(2) login — it added zero
coverage (every user already logs in at on_start) and only generated
contention. Teach _login_with_retry to honour 429 + Retry-After so a
Locust pointed at a limiter-enabled server degrades gracefully
instead of crashing on_start.
Three unrelated test-correctness fixes exposed by running tests/live:
- test_mqtt_live: honeypot defaults to auth-required (post-2018
realistic broker). Anonymous CONNECT is rejected with CONNACK rc=5,
which the "accept" / "subscribe" tests misread as a failure. Pass
MQTT_ACCEPT_ALL=1 via a new env= override on the live_service factory
so only those two tests opt into accept-all.
- test_postgres_live::test_auth_hash_logged: connected with
dbname='prod', which isn't in the honeypot's per-instance DB list, so
Postgres (correctly) rejected at startup before asking for a
password — blowing past the auth event the test asserts on. Target
'postgres' (always in _BASE_DBS) to reach the auth stage.
- test_mysql_backend_live: the module-scoped mysql_test_db_url fixture
is bound to the module loop, but function-scoped tests default to
their own per-function loops. Any reuse of the asyncmy pool then
tripped "Future attached to a different loop". Pin the whole module
with pytest.mark.asyncio(loop_scope='module').
MySQL can't index a BLOB/TEXT column without a prefix length, so
create_all() on a fresh MySQL schema blew up with "BLOB/TEXT column
'kd_digraph_simhash' used in key specification without a key length".
SimHashes are a fixed 8 bytes — the variable-length type was a
SQLAlchemy-side auto-mapping from 'Optional[bytes]', not an actual
schema requirement. Switch to BINARY(8), which is portable: MySQL gets
a fixed-width indexable BINARY, SQLite treats it as BLOB and doesn't
care about key length.
- .311/ and .3[0-9][0-9]/ + .venv*/ — cpython-version-suffixed venvs
(common convention) now covered alongside the existing .venv/.
- wiki-checkout/ — local nested clone of the wiki; never a submodule.
- hang.log / schem / *.pytest.log — scratch dumps from saved pytest
output redirections.
- deps.txt — pydeps-style dependency graph from local analysis runs.
No tracked files affected; just stops new working-tree noise from
showing up in git status.
- SIGNAL_CAPTURE_AUDIT.md: end-to-end walkthrough of what attacker
signals DECNET captures at each pipeline stage, where the gaps are
(session profile ingestion, keystroke dynamics), and what ships for
v1 vs what lands post-v1.
- api-audit.md: FastAPI /api/v1 route audit — surface area, auth
requirements, status-code coverage, and where schema drift would bite
the schemathesis suite.
Both are operator/engineering reference docs, not user-facing.
Adds instance_seed.py to every service template (conpot, docker_api,
imap, k8s, llmnr, pop3, rdp, sip, smb, snmp, ssh, telnet, tftp, vnc).
Derives a stable per-instance seed from NODE_NAME (+ optional
INSTANCE_ID) and exposes deterministic helpers for the boring details
scanners would otherwise use to fingerprint the whole fleet as one
machine: cluster UUIDs, auth salts, uptime fixtures, minor version
strings. Connection-time jitter is intentionally NOT seeded — two hits
to the same decky must not replay the same latency curve.
Identical source across every template; lives next to each service so
the Docker build context picks it up without a shared package-data hop.
Exclude lists fail open — anything new at the master's repo root (venvs,
logs, dev notes, .env.local, local DB dumps) silently leaks into every
agent bundle. On this box a stray .311 venv (335 MB) + logs/ (220 MB)
bloated the tarball to ~150 MB and blew test_enroll_bundle timeouts.
Replace _EXCLUDES + _is_excluded with _INCLUDED_ROOT_FILES +
_INCLUDED_DIRS + _EXCLUDED_DECNET_SUBTREES and iterate via os.walk with
in-place dirnames[:] pruning so master-only subtrees (decnet/web,
decnet/mutator, decnet/profiler) and __pycache__ aren't descended into
at all.
Bundle contents are now strictly: pyproject.toml + the decnet/ package
minus the three master-only subtrees. Synthetic entries (INI, certs,
systemd units) unchanged — they were always added inline, not from the
tree walk.
test_enroll_bundle.py: 20/20 pass in 24s (was timing out at 15s/test).
Groups every flat test_*.py under the module it exercises, matching the
existing tests/{profiler,sniffer,prober,collector,correlation,cli,web,
topology,swarm,bus,updater,api,docker,geoip,...} layout. New folders:
services/, fleet/, config/, logging/, db/ (+ db/mysql/), telemetry/,
mutator/, core/.
Path-dependent __file__ references bumped an extra .parent in three
files that moved one level deeper:
- tests/sniffer/test_sniffer_ja3.py (template path)
- tests/services/test_ssh_capture_emit.py (template path)
- tests/cli/test_mode_gating.py (REPO root)
- tests/web/test_env_lazy_jwt.py (repo var)
Also drops two SQLite runtime artifacts (test_decnet.db-{shm,wal}) that
were leaking into the repo from a previous test run.
Fixes two test_service_isolation cases that patched asyncio.sleep (no
longer on the profiler main-loop hot path — same pre-existing bug I
fixed earlier in test_attacker_worker.py) by patching asyncio.wait_for
and passing interval=0.
- Attackers list: small country-code chip next to the IP on each card,
title-tooltip shows the source (e.g. "rir")
- AttackerDetail: country-code tag next to the IP in the header plus an
ORIGIN field in the TIMELINE section for always-visible origin
- TypeScript interfaces updated with country_code/country_source
Since the event-driven shutdown refactor (0fbb07c), the profiler main
loop is asyncio.wait_for(shutdown.wait(), timeout=interval) — no sleep
on the hot path. The four worker tests that patched asyncio.sleep to
raise CancelledError on the Nth call were silently no-op'ing and
hanging on the real 30 s wait_for timeout.
Replace the sleep patches with a shared _cancel_after helper that
patches wait_for itself. Pass interval=0 so the loop ticks without
delay between iterations.
Populates Attacker.country_code + country_source (MVP) using the five
RIR delegated-stats files (ARIN/RIPE/APNIC/LACNIC/AFRINIC). Offline,
license-free, no outbound traffic that could burn honeypot stealth.
- decnet.geoip package with factory/base/lookup + rir/ subpackage
(fetch/parse/provider) mirroring the db + bus factory convention
- Profiler._build_record calls enrich_ip on every upsert
- Idempotent ALTER TABLE migrations for both SQLite and MySQL
- decnet geoip refresh/lookup CLI (master-only)
- /var/lib/decnet/geoip seeded by decnet init
- DECNET_GEOIP_ENABLED=false kill-switch; set in tests/conftest.py so
unit tests never trigger the first-access fetch
The config file `decnet init` dropped at /etc/decnet/config.ini was a
stub with a single [decnet] header saying 'reserved for future structured
settings.' Admins who wanted to tune DECNET_API_HOST, DECNET_DB_URL,
DECNET_BATCH_SIZE, etc. had to hunt env.py for the exact variable name
and drop it in .env.local.
Changes:
- decnet/config_ini.py — adds a _DOMAIN_MAP translation table covering
[api], [web], [database], [bus], [swarm], [logging], [ingester],
[tracing]. Loads regardless of mode; unknown keys inside a known
section log a WARNING (operator typos shouldn't be silent).
Explicit key map (not auto kebab-to-snake) so [web] admin-user lands
in DECNET_ADMIN_USER without silently renaming the env-var contract
consumers import from decnet.env.
- decnet/cli/init.py — renames the placeholder target config.ini →
decnet.ini (unifies with the name already used by load_ini_config and
the enroll bundle's _render_decnet_ini). Placeholder body now shows
every domain section as a commented example so admins learn the
shape by reading. Deinit removes both decnet.ini and the legacy
config.ini so upgrading hosts leave no orphan file.
Precedence is unchanged: real env > INI > built-in default in env.py.
os.environ.setdefault means systemd EnvironmentFile= and one-off
DECNET_FOO=bar decnet ... invocations always win.
Secrets explicitly NOT moved to the INI:
- DECNET_JWT_SECRET
- DECNET_ADMIN_PASSWORD
- DECNET_DB_PASSWORD
They stay in .env.local / EnvironmentFile= — never in a group-readable
INI, never in a diff, never on the dashboard.
Dev/profiling flags (DECNET_DEVELOPER, DECNET_EMBED_*, DECNET_PROFILE_*)
also stay env-only per maintainer direction — dev knobs shouldn't
be one 'I'll flip this for tonight' away.
Tests: +5 in test_config_ini.py (domain sections load regardless of mode,
env beats INI for domain keys, unknown key warns, absent section is
no-op, role section beats domain section via setdefault precedence). +1
in test_init.py (placeholder writes decnet.ini with every section
header present as commented guidance).
31 tests pass across the two files (was 26).
Distros reserve /opt for different things (some package managers own it
outright), and a DECNET install that wants to live at /srv/decnet or
/usr/local/decnet had to hand-edit 13 service files post-install.
Converts every deploy/decnet-*.service to a .j2 template keyed on
{{ install_dir }}, rendered by `decnet init` at install time. All other
paths (log_dir, state_dir, runtime_dir, user, group) stay standard —
only install_dir varies.
Changes:
- deploy/decnet-*.service → deploy/decnet-*.service.j2 (13 files).
- decnet init gains --install-dir (default /opt/decnet, preserves
existing behaviour byte-for-byte). Validates absolute-path at the
CLI boundary. Threads through useradd --home-dir and the dir-creation
list so the filesystem layout matches the rendered templates.
- _install_units renders via Jinja2 with StrictUndefined (typo → loud
error, not a silent broken unit). SHA over rendered output so
operators with a custom install_dir get idempotent re-runs.
- decnet.target, tmpfiles.d, polkit rule stay static — they don't
reference install paths.
- 4 new tests: custom install_dir renders into units, default remains
/opt/decnet, relative paths rejected, second run with same custom
dir is idempotent.
Worker bus instances (collector, ingester) close their private buses
in finally blocks on shutdown, but stream threads holding closure
references kept calling publish after close — one `RuntimeError:
publish on closed bus` per stream line, caught by publish_safely
and logged per call, flooding server logs.
Changes:
- `UnixSocketBus.publish()` now drops post-close calls. First drop
WARNs loudly (bus is critical infra — silent drops would hide real
problems); subsequent drops on the same instance log at DEBUG to
prevent the flood. Sticky `_closed_publish_warned` flag, reset
naturally per new bus instance.
- `make_thread_safe_publisher` short-circuits on a closed bus before
marshalling a coroutine onto the loop. Avoids the wasted scheduling
work in the hot shutdown path.
Degradation is safe: callers go through `publish_safely`, which
already treats exceptions as 'dropped notification, DB is source of
truth.' We just stop manufacturing the exception in the first place
for a known-benign condition.
A startup race between `decnet bus` being ready and the API's lifespan
hitting `get_app_bus()` at api.py:135 would set `_tried = True`
permanently, poisoning the singleton for the rest of the process: the
dashboard shows BUS OFFLINE, topology SSE falls into the bus-is-None
snapshot-only branch, mutator publish calls no-op. Only an API
restart recovered.
Replaces the one-shot veto with a time-gated retry keyed on a
`_last_failure_ts` monotonic timestamp plus a 2 s backoff. Publishers
on the hot path still pay at most one connect attempt every 2 s when
the bus is down, but the singleton auto-recovers within 5 s (one
dashboard poll) once the bus comes up.
The asyncio lock still serialises concurrent callers so the bus server
doesn't get stampeded with parallel connect attempts on startup.
Registers a generic @app.exception_handler(Exception) that catches anything
uncaught in route handlers / dependencies. Prod response is opaque:
{detail: 'Internal Server Error', error_id: <uuid4 hex>}. Dev mode
(DECNET_DEVELOPER=True) adds exception_type and traceback fields so
failures are debuggable without tailing server logs.
The error_id is logged alongside the full traceback server-side, letting
operators correlate a user's 500 report with the exact exception via
`grep <error_id> /var/log/decnet.log`.
FastAPI's own HTTPException routing and the existing
RequestValidationError / ValidationError / RateLimitExceeded handlers
still take precedence — this handler only fires on genuinely-uncaught
exceptions.
Flips threat model F1/I 'traceback / stack trace leakage' from ? to M
and logs a follow-up checklist entry for 4 detail=str(e) sites in the
fleet deploy router (admin-gated, different threat class, separate
audit).
Adds slowapi two-bucket rate limit on /auth/login — 10 attempts per
5 minutes per-IP AND per-username, tripping either → 429. Per-IP
catches botnets hitting one account; per-username catches distributed
credential stuffing against one account. In-memory storage: dashboard
API is single-process, Redis is disproportionate for v1.
X-Forwarded-For is deliberately NOT trusted (spoofable); reverse-proxy
deployments get one shared bucket per proxy IP. Logged in the threat
model as accepted risk DA-08, to be revisited when a verified-proxy
config lands.
Also scaffolds development/THREAT_MODEL.md with STRIDE-per-element
methodology, system-context DFD, and Dashboard↔API as the first fully
worked component (7 sub-flows, ~50 threat entries). F1 Authn ships
with 3 threats mitigated: rate limit (new), uniform 401 (verified
already in place), bcrypt length clamp (verified already in place via
Pydantic max_length=72).
Adds GET /attackers/{uuid}/smtp-targets (viewer) and GET /attackers/{uuid}/mail
(admin) endpoints, plus two new sections on the attacker detail page:
VICTIM DOMAINS rollup (aggregate-only, federation-gossip-safe) and STORED MAIL
with a drawer that decodes headers, lists attachments, and downloads the raw
.eml via the existing artifact endpoint (?service=smtp).
New SmtpTarget table records each (attacker, domain) pair observed via
the SMTP honeypots. Only the domain is stored — local-parts are dropped
at ingestion, so this table holds no user-identifying data beyond the
target organisation's identity.
The profiler worker extracts domains from rcpt_to / rcpt_denied /
message_accepted events, normalizes them (lowercase, strip local-part,
drop blocked TLDs), and upserts one row per pair with a running count +
first_seen / last_seen.
Three repo methods shipped:
* increment_smtp_target(attacker, domain) — upsert + bump
* list_smtp_targets(attacker) — per-attacker view
* smtp_target_seen(domain) — cross-attacker aggregate, shaped as the
federation-gossip RPC that V2 will expose.
The gossip-query shape is load-bearing: each operator can answer
"have any of your attackers targeted corp1.com?" without leaking
which attackers or when — the aggregate returns a bool + total count
+ first/last seen, nothing else.
SMTP template now writes each accepted DATA body as a .eml file into a
bind-mounted per-decky quarantine dir and emits a `message_stored` log
with sha256, size, decoded headers, and an attachment manifest
(filename + sha256 + size + content-type). Attachment hashing uses the
*decoded* payload so operators can match against VT / MalwareBazaar
directly. Body accumulator is capped at SMTP_MAX_BODY_BYTES (default
10 MB, matching the EHLO SIZE advert) so a streaming client can't OOM
the container.
The existing /api/v1/artifacts/{decky}/{stored_as} endpoint now takes
an optional ?service= query param (defaults to ssh for back-compat)
and can serve .eml files out of the smtp subdir. Forensic metadata
rides the normal log pipeline, same as SSH file_captured.
decnet/web/db/models.py was approaching 1000 lines across User/Log/
Attacker/Swarm/Topology/Workers/Updater/Health domains. Split into a
package with one module per domain; __init__.py re-exports every symbol
so all 52 call sites keep importing from decnet.web.db.models
unchanged.
New purpose-built table with schema_version column committed from day one
so V2 federation gossip can cluster sessions across operators without
retrofitting. Ships with the empty write path (upsert_session_profile);
ingestion of keystroke features (IKI moments, control-char rates, digraph
SimHash) is tracked as V2 work.
Closes gap #2 from SIGNAL_CAPTURE_AUDIT.md.
Parse RFC 4253 §4.2 identification strings from the first attacker→decky
data segment on TCP/22; emit ssh_client_banner syslog events and bus
fan-out. Profiler's sniffer_rollup dedupes observed banners into a new
AttackerBehavior.ssh_client_banners JSON column.
Closes gap #3 from SIGNAL_CAPTURE_AUDIT.md.
Prober already emits kex_algorithms in hassh_fingerprint syslog events, but
the raw ordered list was only queryable via the generic bounty store. Add a
dedicated AttackerBehavior.kex_order_raw column (TEXT, JSON list) so
post-v1 KEX-order fingerprinting has a typed, indexable home.
Pipeline:
- sniffer_rollup() now consumes hassh_fingerprint events and collects
distinct kex_algorithms strings across ports.
- build_behavior_record() JSON-encodes the list (NULL when empty).
- sqlmodel_repo._deserialize_behavior() parses it back into a list.
Closes pre-v1 gap #1 from SIGNAL_CAPTURE_AUDIT.md.
Break the 603-line behavioral.py into timing/classify/tools/phases/fingerprint
sibling modules plus a slim orchestrator. Public API unchanged: behavioral.py
re-exports every previously-exposed symbol, so worker.py and existing tests
keep working with zero import changes.
No behavior change; all 64 profiler tests pass.
- TopologyList header now uses .page-header + .page-title-group +
.page-sub like Dashboard/Attackers/DeckyFleet; title typography and
separator match the rest of the app.
- Pluralisation fix: '0 topologyies' → '0 TOPOLOGIES', singular '1
TOPOLOGY'.
- When the list is empty the EmptyState renders in its own flex
container that fills the viewport so the card is centered both
axes, with bumped icon/title/hint sizing for the hero treatment.
delete_topology_cascade manually deletes status_events, edges, deckies
and lans but overlooked topology_mutations, so deleting any topology
that ever had a mutation enqueued (i.e. edits while active|degraded)
failed with an FK IntegrityError. Add the missing DELETE and extend
the cascade test to seed a mutation row.
MazeNET header now reports '{running}/{total} DECKIES RUNNING' so
operators can see per-topology runtime status at a glance.
Dashboard ACTIVE DECKIES counters used to reflect only the fleet state
file; TopologyDecky rows (MazeNET deployments) are now added in —
deployed_deckies = fleet + all topology rows, active_deckies = fleet
(no runtime field) + topology rows whose state is 'running'.
Hovering the empty-state row in LiveLogs/Dashboard tables briefly lit
the full-width td with the data-row glow. Tag the placeholder tr with
.empty-row and scope the .logs-table hover rule to :not(.empty-row).
Base .empty-state now flex-centers its icon/title/hint/CTA with a
140px min-height so icon-bearing empty states in the Dashboard side
panels (DECKIES UNDER SIEGE, TOP ATTACKERS) stop looking cramped.
Component-scoped rules (attackers-root, bounty-root, logs-root)
remain more specific and are unaffected.
- New ShortcutsHelp modal enumerates global, nav G-chord and palette
bindings; openable via ? (Shift+/) or the command palette.
- / dispatches a global decnet:focus-search event; Attackers, Bounty
and LiveLogs listen and focus their in-page search inputs (pages
without a local search are skipped per plan).
- Respects the existing editable-element guard and Alt+K palette
toggle; no rebinds to prior shortcuts.
Replace ad-hoc empty-state markup across Dashboard, TopologyList,
LiveLogs, Attackers, Bounty, AttackerDetail, SwarmHosts, RemoteUpdates
and CommandPalette with the new <EmptyState> component. Themed icons
+ hints improve discoverability; TopologyList and SwarmHosts gain
CTAs to their respective creation flows.
Each page gets its own scoped stylesheet and is rewritten around the
shared design language: filter bars, paginated lists, empty-state
blocks, BountyInspector drawer. Behavioural surface is unchanged —
same API calls, same routes, same RBAC gating.
Rewrites Dashboard.tsx around three stacked panels — live interactions,
deckies-under-siege, and top-attackers — each with its own header,
empty state, and status accents. Dashboard.css fills in the supporting
grid + type system.
- CommandPalette (Alt+K): fuzzy action launcher with keyboard nav.
- Toasts: ephemeral notification stack + provider.
- useGlobalHotkeys: Alt+K palette toggle, G-chord navigation
(G D/F/M/L/B/A/S/U/E/C), respects editable-element focus.
- Layout/App: wire ToastProvider at root, mount the palette inside the
authed shell, introduce the global search box in the top bar.
- MazeNETRoute now renders TopologyList inline when no ?topology is
present, instead of bouncing through a redirect.
- index.css: a few global token tweaks consumed by the new chrome.
Fixes a latent breakage: Config.tsx and MazeNET already imported
./Toasts/useToast but the directory was never committed.
The DELETE path on a topology whose containers are still up is a
footgun — even if the backend rejects the delete, surfacing the
button invites mistakes. Gate it so DELETE only shows for pending,
failed, and torn-down topologies. Active/degraded/deploying topologies
must be torn down first, which then reveals DELETE again.
POST /topologies/{id}/lans previously called _auto_attach_gateway()
whenever a non-DMZ LAN was created, which wired the DMZ gateway decky
to every new subnet. That's why a deployed gateway ended up with
eth0..ethN on every LAN regardless of what the user drew in MazeNET.
Drop the auto-attach helper entirely. The DMZ_ORPHAN deploy-time
validator (decnet/topology/validate.py:65-110) stays strict — users
must explicitly wire the gateway to each subnet they want bridged,
which is the whole point of having a topology editor.
useMazeApi.ts: drop stale auto-bridge reference from comment.
ArtifactDrawer, SessionDrawer, CreateTopologyWizard all now:
- close on ESC
- trap Tab/Shift+Tab focus within the panel
- lock body scroll while open
- restore prior focus on unmount
Uses the new useEscapeKey + useFocusTrap hooks. No visual changes;
the bespoke CSS shells (ctw-*, inline drawer styling) are preserved.
- Modal: shared backdrop/panel with ESC-close, backdrop-click-close,
focus trap, body scroll lock; supports center + drawer-right variants,
matrix/violet accents, default/wide widths.
- EmptyState: icon + title + hint + optional CTA; compact variant
for tight rails.
- useEscapeKey, useFocusTrap: reusable hooks powering Modal; will also
be adopted by CommandPalette and ContextMenu in follow-up commits.
No retrofits yet — primitives only. tsc clean.
Pan drag previously required mousedown on the bare canvas (target ===
currentTarget). When zoomed in, net-boxes cover most of the viewport
so there was no bare grid to grab. Drop the guard — node/header/port/
resize handlers all call stopPropagation() already, so only net-box
body mousedowns bubble up to start the pan, which is exactly what
we want.
Wheel-to-zoom anchored at the cursor, ZOOM IN/OUT toolbar buttons, and
a live zoom% in the status bar. Pan layer gets transform-origin 0 0 and
a scale(zoom) factor; grid pattern tile scales with zoom; edge SVG is
overflow:visible so long edges don't clip at high zoom. World-space
hit-testing, resize deltas, and palette drops all divide by zoom.
Reset View zeroes pan AND zoom.
Clicking a service tag selects it (stops node drag), extends Selection
discriminant with {type:'service',id,nodeId}, and renders an inspector
panel showing proto/port/subnet/risk chip + REMOVE SERVICE button
(gated off for observed nodes and degraded topologies). Service-tag
styling now pulls `risk` from DEFAULT_SERVICES metadata instead of
node.status alone.
Reverse of init, step-by-step: systemctl disable --now decnet.target,
remove every decnet-*.service + decnet.target unit file, drop the
polkit rule, drop the tmpfiles.d entry, daemon-reload, remove
/etc/decnet + /etc/decnet/config.ini, /run/decnet, /opt/decnet, and
userdel/groupdel the decnet identity.
Preserves /var/lib/decnet and /var/log/decnet by default — those
hold operator data. Pass `--deinit --purge` to rm -rf them too.
Idempotent on a clean host (every step prints [SKIP]). Honours
--dry-run.
5 new tests cover the full-undo path, --purge, idempotent clean-host
deinit, dry-run side-effect-free behaviour, and the --purge without
--deinit guard.
Creates the decnet system user/group, installs every unit file from
deploy/ into /etc/systemd/system, drops the polkit rule, seeds
/opt/decnet + /var/{lib,log}/decnet + /etc/decnet + /run/decnet,
writes a placeholder /etc/decnet/config.ini, applies the new
tmpfiles.d entry so /run/decnet survives reboots, daemon-reloads,
and `systemctl enable --now decnet.target`.
Idempotent (re-runs print [SKIP] on already-configured items),
--dry-run previews the plan without touching anything, --no-start
defers the target start, --force overwrites even matching unit
files. Master-only (added to MASTER_ONLY_COMMANDS).
9 orchestration tests cover the non-root gate, dry-run, useradd/
groupadd argv, SKIP on present user/group, unit-file idempotency,
--force overwrite, --no-start suppression, happy path, and the
"deploy/ not found" error message.
Units + polkit rule + systemd_control helper + start endpoints +
installed flag + UI wiring all landed. SWARM-host start/stop and
crash-quarantine policy stay as named deferrals.
Per-row START button enabled iff `installed && status !== 'ok'`;
tooltip explains why it's disabled ("Unit not installed" /
"Already running"). Transient `starting` state shows `...` on the
button and auto-clears after 15s so the UI never gets stuck if the
heartbeat is slow.
START ALL WORKERS button in the header calls /workers/start-all and
renders the three counts in the toast:
`STARTED · N · ALREADY RUNNING · M · FAILED · K (first failure: …)`.
Tone flips to alert when K > 0.
POST /api/v1/workers/{name}/start — 202 on acceptance, 404 unknown
worker, 503 if the unit file is not installed, 502 if systemctl
returns non-zero (stderr snippet in detail, full stack logged).
Admin only.
POST /api/v1/workers/start-all — best-effort: walks the worker list
in dependency order (bus → api → data-plane), skips already-active
and uninstalled units, aggregates outcomes into
{started, already_running, failed[]}. Returns 200 even on partial
failure; the caller reads the three lists.
Both endpoints delegate to the systemd_control helper, so the attack
surface for "what gets executed" is locked to `decnet-<validated-name>
.service` at two layers (router KNOWN_WORKERS + helper regex).
Ships the backend half of Config → Workers:
* Worker registry aggregates `system.*.health` + `system.bus.health`
heartbeats into a last-seen dict; OK / STALE / UNKNOWN tiers drop
out of a 90s window (3× the 30s heartbeat interval).
* `GET /api/v1/workers` returns the snapshot plus `bus_connected`
(so the UI can explain "all UNKNOWN" when the bus socket is down)
and a per-row `installed` flag populated from
`systemctl list-unit-files decnet-*.service` (cached 30s).
* `POST /api/v1/workers/{name}/stop` publishes a stop intent on
`system.<name>.control`; workers listen via the shared control
listener in `bus/publish.py`.
* Heartbeat + control listener wired into collector / profiler /
sniffer / prober / mutator worker loops. API self-heartbeats too
so the panel always has one ground-truth row.
* Topic helper `system_control(name)` + tests covering builder
validation, control listener shutdown path, and the API surface
(auth gating, bus-connected field, unknown-name 404).
Adds `StartFailure` / `StartAllResponse` models in anticipation of
the upcoming start endpoints (DEBT-034).
Thin async wrapper over `systemctl` — never shell=True, always
create_subprocess_exec. Unit names are built from
`decnet-<validated-name>.service`; the regex check is defence in depth
on top of the router-level KNOWN_WORKERS validation.
Exposes start / stop / is_active / list_installed; last is cached for
30s to keep the Workers panel cheap under REFRESH spam. On non-systemd
hosts list_installed returns an empty set, so the UI renders with
every row marked not-installed instead of 500-ing.
Scoped rule — matches only `decnet-<name>.service` and `decnet.target`.
Any unit outside that regex falls through to the default polkit policy.
Required so the API (running as the `decnet` user) can invoke
`systemctl start decnet-<name>.service` non-interactively.
Adds the five missing worker units plus a grouping target so
`systemctl start decnet.target` brings the whole fleet up in order.
Sniffer gets CAP_NET_RAW for scapy; collector and mutator join the
docker supplementary group for docker.sock access. Repoints
Documentation= across all existing units to the canonical
git.resacachile.cl wiki.
Add tests/service_testing/test_instance_seed.py — pins NODE_NAME to assert
determinism of seeded functions and sweeps NODE_NAMEs to assert cross-fleet
divergence. Conftest gains load_real_instance_seed() so template tests see
the real seeding behavior instead of a stub. Existing template tests updated
to pin NODE_NAME and match seeded outputs.
Every service template now pulls version strings, cluster/node UUIDs, auth
salts, greeting banners, and uptime from the seeded per-instance RNG instead
of hard-coded defaults. Scanners sweeping the fleet now see legitimately
diverging fingerprints per decky while each decky's own responses stay
internally consistent across restarts.
Covers elasticsearch, ftp, http, https, ldap, mongodb, mqtt, mssql, mysql,
postgres, redis, and smtp templates.
Each decky now gets a deterministic-per-instance seeded RNG derived from
NODE_NAME, so cluster UUIDs, version strings, uptime, and credentials diverge
across the fleet while staying stable within one container. The canonical
helper lives at decnet/templates/instance_seed.py; the deployer copies it into
every active template build context alongside syslog_bridge.py. Dockerfiles
COPY it to /opt/ so server.py can import it.
Connection-time jitter intentionally stays unseeded — two hits to the same
decky must not replay the same latency curve.
The ssh and telnet services hard-coded /var/lib/decnet/artifacts as the host
quarantine mount. Read it from DECNET_ARTIFACTS_ROOT with the same default so
dev/rootless deploys can point it elsewhere.
Paging, truncation surfacing, admin gate, path traversal, sid-regex and
decky-mismatch rejection for /transcripts; mirror coverage for
/attackers/{uuid}/transcripts. Flips the Session Recording box in the
roadmap (sessrec pty relay now shipping end-to-end).
Adds asciinema-player dependency, SessionDrawer.tsx that pages the
transcripts API (500 events per request) and rebuilds a v2 .cast blob
for playback, and a Session Transcripts section in AttackerDetail that
deep-links into the drawer. Truncation banner surfaces the 10 MB
per-session cap when it's been hit.
Adds get_attacker_transcripts (mirror of artifacts for session_recorded
logs) and get_session_log for sid→shard resolution. New
/api/v1/transcripts/{decky}/{sid}?offset=&limit= pages asciinema events
out of the shared JSONL day-shard via an mtime-keyed byte-offset index
— never scans the whole shard per request. New
/api/v1/attackers/{uuid}/transcripts lists sessions for drilldown. Both
endpoints admin-gated.
Build login-session into both images as the swapped root shell, add a
quarantine bind mount for telnet (symmetric to SSH), seed transcripts/
dir and service discriminant at entrypoint. Deployer syncs sessrec.c +
Makefile into each build context alongside the existing syslog_bridge
helper. sessrec falls back to /etc/sessrec.service when env is stripped
(busybox /bin/login).
New decnet/templates/_shared/sessrec/ — a small C program installed as the
login shell in SSH / Telnet deckies. Forkpty-relays /bin/bash, records each
chunk as an asciinema v2 event into a shared JSONL day-shard keyed by sid,
and emits one RFC 5424 session_recorded line on exit (direct to PID 1's
stdout, same pattern syslog_bridge.py uses).
Storage: one shard per (decky, UTC day) at
/var/lib/systemd/coredump/transcripts/sessions-YYYY-MM-DD.jsonl. Concurrent
appends are lock-free: each write is chunked below PIPE_BUF so O_APPEND
interleaves atomically. Per-session cap 10 MB with a trunc sentinel; disk-
free precheck (<200 MB) falls through to plain bash with a session_skipped
log event. Attacker src_ip resolves from \$SSH_CONNECTION, getpeername(0),
or utmp in that order. SIGWINCH appends a 'r' resize event so ncurses
replays stay aligned.
Stealth for v1: /etc/passwd shell-swap to /usr/libexec/login-session
(plausible login-machinery path) + prctl comm disguise. Full LD_PRELOAD
argv-zap is deferred — sshd strips LD_PRELOAD from the session env, so
wiring the existing argv_zap.so into this path needs a separate wrapper.
DEBT-033 opened for size-based day-shard rotation; v1's disk-free precheck
covers the worst case but can be blinded by a one-shot disk fill.
Exposes POST /topologies/reap-orphans via an arm-to-confirm button in
the topology list header. Shows a transient status line with removal
counts or the error. Admin-only on the backend; non-admins see the 403.
Topology rows deleted without a proper teardown leave Docker containers
and bridge networks behind, holding IPAM pools that cause 403 "Pool
overlaps" on the next deploy at the same subnet.
- engine/reaper.py walks the local Docker daemon, extracts the 8-char
topology prefix from every decnet_t_* resource, and force-removes
containers + networks whose prefix is not in the repo.
- POST /api/v1/topologies/reap-orphans (admin-only) returns a report
of live/orphan prefixes and what was removed.
- Resources belonging to live topologies are never touched; per-resource
errors are captured without aborting the sweep.
When create_bridge_network or compose-up raised mid-deploy, the
deployer marked the topology FAILED and re-raised — but left every
network it had already created alive. The next deploy attempt tripped
over the orphans with 'Pool overlaps with other one on this address
space' (IPAM conflict).
Track networks created in the current attempt; on exception, tear down
the started compose stack (if any), remove the networks in reverse
order, and delete the compose file before marking FAILED. Rollback
errors are logged but never mask the original failure.
Covered by a new regression test that drives a docker client which
succeeds once then raises, and asserts every created network is also
removed.
useTopologyEditor imported 'UseMazeApi' but the actual exported type
is 'MazeApi'. tsc --noEmit missed it because the file isn't in the
default tsconfig include path; tsc -b (project references, used by
'npm run build') catches it.
apply_attach_decky requires an existing decky, so the MazeNET editor
had no way to grow a live topology: creating a new decky on active
topologies 409'd on the direct-CRUD createDecky call.
- Backend: new apply_add_decky that creates the decky row + its
home-LAN edge atomically, auto-allocating an IP if none pinned.
Post-apply validation still runs. Added to DISPATCH + _MUTATION_OPS
Literal + CLI help text.
- Tests: 3 new ops tests (happy path, duplicate-name rejection,
missing-LAN rejection) plus dispatch coverage update.
- Frontend: useTopologyEditor gains addDeckyToLan() composite. Pending
routes through createDecky + attachEdge as before; active routes
through a single add_decky enqueue. MazeNET.tsx drag-archetype,
duplicate, DMZ-gateway, and ctx-menu add-decky paths all use the
composite so active topologies stop 409'ing on new-decky drops.
useTopologyEditor now branches on topoStatus: pending keeps direct CRUD,
active/degraded routes through enqueueMutation with expected_version.
Every primitive returns a tagged PrimitiveResult; callers skip local
state updates on enqueued and wait for the SSE mutation.applied refetch
to reflect DB truth.
- remove_lan/remove_decky/detach_decky: direct name-keyed enqueues.
- update_decky/update_lan: services/x/y lifted to top-level payload keys,
remainder placed under patch (matches apply_update_* contract).
- attach_decky: enqueued with decky+lan names; requires the decky to
already exist (Phase B step 3 adds the create+attach composite).
- createDecky stays direct-CRUD this pass — no add_decky op yet, so
new-decky drag will 409 on active until a follow-up commit.
- MazeNET surfaces mutation.failed payload.reason/error into actionErr
so the status bar tells the user WHY a queue op was rejected.
apply_update_decky only merged payload.patch into decky_config. Since
services is a separate DB column, there was no way to replace a decky's
services list via a mutation. Add a top-level services key to the op
payload that maps straight onto the services column.
Unblocks the MazeNET editor routing service-add/service-drop actions
through the mutation queue on active topologies.
Phase B step 1 of DEBT-030: introduce a status-aware editor hook that
wraps useMazeApi. Every primitive currently pass-throughs to direct
CRUD and returns {kind: 'applied', data} — behavior is unchanged.
Follow-up commits route active/degraded topologies through
enqueueMutation when status != pending.
Also tighten the SSE LIVE indicator: flip setStreamLive(true) only on
snapshot, mutation.*, or status events, not on any incidental frame.
The mutation-event stream landed this session closes the "deckies are
atomic nodes" gap for service-list changes, but substrate identity is
really ``(service, implementation_fingerprint)``. A base-image
rebuild that rotates OpenSSH 8.4 → 9.2 without changing the service
list is invisible to the correlation graph today because the prober's
dedup set is in-memory and per-run — no cross-run diff, no
"fingerprint changed" event.
DEBT-032 documents the required piece: a per-(decky, service,
probe_type) persistence layer + diff-on-change emission, with the
correlator's existing mutation-marker interleaving pattern as the
model for fingerprint markers. A mutation-vs-fingerprint divergence
detector then falls out of the data model for free — fingerprint drift
without a preceding mutation ⇒ substrate_divergence finding.
Parser now tags ``mutator`` / ``decky_mutated`` lines with ``kind="mutation"``
so the engine can route them into a sibling ``_mutations`` index keyed
by decky name instead of the per-IP attacker index. ``traversals()``
joins the two streams: every attacker gets a ``mutations_during`` list
of markers from touched deckies bounded by their first/last-seen
window. ``AttackerTraversal.to_dict()`` grows a ``mutations_during``
field and a ``timeline`` that chronologically interleaves hops and
markers, so an ``SSH at T5 → mutation at T6 → HTTP at T7`` substrate
transition is visible to UI consumers instead of reading as a silent
discontinuity.
The existing hops-only JSON shape is preserved; old clients that
ignore unknown keys keep working.
Close the lifecycle loop for the correlation graph: every decky now
enters the substrate with an explicit `trigger=creation` event
(old_services=[] ⇒ new_services=<initial>) and leaves it with
`trigger=retirement` (old=<current> ⇒ new=[]). With scheduled/operator
mutations already flowing through emit_decky_mutated, the entire decky
lifecycle is now a well-formed sequence of mutation events — the
correlator can fold substrate_state(t) at any T by replaying them.
Lazy-imports mutator.events to dodge the engine↔mutator circular
dependency. Bus is None at CLI sites; the syslog write is what the
correlator consumes. Emission is soft-failing so a broken log path
never aborts a deploy.
Mutator now emits one decky_mutated event (RFC 5424 + bus) per
successful mutation instead of the inline decky.<id>.state bus
publish. The previous state topic published new_services only;
mutation events carry old/new/trigger, which is what the correlation
engine needs to interleave substrate-change markers into attacker
traversals.
- mutate_decky gains trigger: MutationTrigger = "operator" and
captures old_services before the shuffle; replaces the inline
_publish_safely(decky.<id>.state) with emit_decky_mutated(...).
- mutate_all derives trigger internally: operator when force or
only-filter is set (CLI --all, API mutate-now, UI bus request);
scheduled on interval ticks. Passed through to each mutate_decky
call.
- Tests updated: the old decky.<id>.state assertion is replaced
with decky.<id>.mutation topic + mutation payload shape; 3 new
tests cover trigger derivation for scheduled / force / only paths.
26 tests in test_mutator.py green; 116 across mutator + topology
+ bus.
First step toward making mutation events first-class nodes in the
correlation graph. Today the graph silently reflects post-mutation
state with no marker of the transition; this helper lands the
emitter the mutator and deploy paths will call.
- decnet/mutator/events.py: emit_decky_mutated(bus, *, decky,
old_services, new_services, trigger, actor=None, log_path=None)
writes an RFC 5424 line (service=mutator, hostname=<decky>,
MSGID=decky_mutated, SD params for old/new services + trigger +
optional actor) to DECNET_INGEST_LOG_FILE, then fire-and-forget
publishes on decky.<id>.mutation. Either side failing is soft —
the other path still completes.
- MutationTrigger Literal covers creation, retirement, scheduled,
operator, behavioral, healer, federation. Reserved values for v2/v3
(behavioral + federation) stay nullable so the schema is stable.
- decnet/bus/topics.py: DECKY_MUTATION constant + decky_mutation(id)
builder. Distinct from DECKY_STATE ("current shape") because a
mutation is a transition event, not a steady-state snapshot.
- Empty-set symmetry: creation emits old_services=[], retirement
emits new_services=[]. Every decky lifecycle becomes a well-formed
fold sequence on the correlator side.
- 4 new tests: FakeBus + correlator parser round-trip; creation and
retirement empty-set cases; bus=None still writes syslog;
unwritable log path doesn't block bus publish. 95 tests green
across test_mutator + tests/bus.
The flat-fleet mutator was DB-poll-only and noisy — it logged
"no active deployment found" every 10s on idle hosts and ran
mutate_all at a fixed tick regardless of when the next decky
was due.
- mutate_all returns seconds-until-next-due; watch loop sleeps
min(next_due, poll_interval_secs) with a 1s floor.
- "No deployment" is now idle, not an error: edge-triggered log
on present<->absent transition instead of every tick.
- mutate_decky publishes decky.<name>.state on successful compose
so UIs react in real time.
- New decky.*.mutate_request subscription lets API/CLI/UI force
an immediate mutation of a specific decky without waiting for
its interval; target name feeds mutate_all(only={...}).
- system.mutator.health heartbeat via run_health_heartbeat helper,
bringing the mutator in line with DEBT-031 workers.
Tests: next_due return, only= filter, decky.<name>.state publish
on success, no publish on compose failure. Full mutator+topology-
mutator+bus suite (109) green.
All nine service workers now participate in the host-local bus: sniffer,
prober, correlator (via profiler), profiler, collector, ingester, agent,
forwarder, updater. Pre-bus behavior is preserved end-to-end for
DECNET_BUS_ENABLED=false and get_bus() failures.
Three items intentionally deferred: realism-probe decky.{id}.state
(needs a realism probe path that doesn't exist yet), correlator session
boundaries (needs session state), and bus-wake subscriptions (publishes
landed; wake side wired to no subscriber today).
All three workers now share a run_health_heartbeat helper in
decnet.bus.publish. Each publishes system.<worker>.health on a 30s tick
with {worker, ts} plus optional per-worker extras. Subscribers can
watch system.*.health to see every DECNET worker on a host at once.
- agent: heartbeat runs inside the FastAPI lifespan alongside the
existing master-facing heartbeat; bus-disabled path is a no-op.
- forwarder: heartbeat task spawned at run_forwarder entry, cancelled
in the finally block so a crashed master loop never leaks the task.
- updater: new FastAPI lifespan hosts the heartbeat.
Heartbeat helper swallows extra() failures and is cancellation-safe so
lifespan teardown never hangs on it.
Ingester connects the bus at startup, emits a batch-committed summary
(component/flushed/position) after each successful _flush_batch. Zero-
row flushes are suppressed so the topic stays meaningful.
Complements the collector's per-line system.log publishes: collector
signals ingress, ingester signals DB-persisted progress. Federation
forwarder (worker 8) will subscribe to the batch-committed leaf to
trigger its upstream push.
Bus stays optional: publish_safely swallows failures, get_bus() can
return None, DECNET_BUS_ENABLED=false leaves the ingestion loop fully
functional.
log_collector_worker connects the bus at startup, builds a thread-safe
system.log publisher, and hands it to each container-stream thread
through _stream_container's new publish_fn parameter. Publishing fires
right after the JSON record is written — same rate-limiter path, no
extra parsing, compact payload (decky/service/event_type/attacker_ip/
timestamp) so subscribers can redraw without re-reading the DB.
Bus stays optional: if get_bus() fails or DECNET_BUS_ENABLED=false the
factory returns a no-op publisher and the stream thread calls it
unconditionally. Hook failures are logged and never abort the thread.
The profiler worker threads its bus publisher through _WorkerState so
_update_profiles can emit a compact attacker.scored event for every
upsert. Payload carries the headline counts (event/service/decky/
bounty/credential) plus is_traversal, so the MazeNET attacker pool can
redraw without a round-trip.
Bus stays optional: publish_attacker=None when DECNET_BUS_ENABLED=false
or get_bus() fails, and hook exceptions are logged without breaking the
upsert path.
CorrelationEngine gains an optional publish_fn hook fired once per unique
attacker IP. The profiler worker — sole caller of the engine today —
carries the bus physically, builds a thread-safe publisher, and wraps it
with the attacker.observed topic before handing it in.
Bus stays optional: if get_bus() fails or DECNET_BUS_ENABLED=false, the
engine runs publish_fn=None and the worker degrades to DB-only. Hook
failures log a warning and never break ingestion.
Each successful JARM / HASSH / TCPfp probe fans out an
attacker.fingerprinted event; the probe family goes in event.type so a
single subscription covers all three. Payload carries the attacker IP,
port, and probe-specific hash — enough for the MazeNET live map to
render fingerprint info on observed attackers.
Lifts the thread-safe publisher helper out of the sniffer worker into
decnet/bus/publish.py so the prober (and every future worker with a
to_thread hot path) can reuse it without copy-pasting the
run_coroutine_threadsafe dance. Sniffer rewires onto the shared helper
in passing.
Adds ATTACKER_FINGERPRINTED as a new leaf — distinct from
ATTACKER_OBSERVED (correlator's first-sight signal) because an active
probe result is additional evidence about an already-observed attacker.
Note: the plan's decky.{id}.state realism-probe publish path is
deferred — the current prober fingerprints attackers, not decky
realism. Will revisit when realism probes exist.
SnifferEngine gains an optional publish_fn hook, invoked after the
dedup + syslog write for traffic-summary events only (tls_session,
tcp_flow_timing, tcp_syn_fingerprint) — intermediate parser artifacts
like tls_client_hello stay off the bus.
The sniffer worker wires get_bus() + a thread-safe shim that marshals
sync calls from the scapy sniff thread back onto the asyncio loop via
run_coroutine_threadsafe. Bus failure at startup degrades cleanly to
publish-off mode; publish failures at runtime never escape the sniff
thread.
Shared publish_safely helper at decnet/bus/publish.py so the nine
workers about to be wired into the bus don't each copy-paste the
"never raise back at the caller" contract. Mutator drops its private
copy and imports the canonical one.
topics.py gains the attacker.* hierarchy (observed, scored,
session.started, session.ended) and a system_health(worker) builder
for per-worker health heartbeats — both prerequisites for the worker
rollout under DEBT-031.
Per-worker integration of the service bus shipped in DEBT-029. Publishes
are fire-and-forget; subscribes wake polling loops. Bus stays optional —
if get_bus() fails or DECNET_BUS_ENABLED=false, workers log once and
continue in poll-only mode (mirrors decnet/mutator/engine.py:run_watch_loop).
- scripts/bus/smoke-mutator.sh: boots decnet bus, subscribes to
topology.>, publishes one event per mutation-lifecycle state plus
a topology.status transition, asserts all four land on the
subscriber. Cheap E2E for the topic hierarchy the mutator + SSE
route rely on.
- development/DEBT.md: mark DEBT-030 ✅ resolved (Phase A) with a
summary of what shipped; flag the optimistic staged-buffer editor
as Phase B follow-up, not debt.
- tests/topology/test_mutator.py: reconcile_topologies publishes
applying+applied on success, applying+failed+status on failure; and
stays safe when bus=None. _wake_on_enqueue sets its asyncio.Event
on every matching enqueue event.
- tests/api/topology/test_mutations.py: POST /mutations publishes
mutation.enqueued after a successful DB write, via a FakeBus
injected in place of the app-wide bus singleton.
- tests/api/topology/test_events_stream.py: SSE route returns 401
unauthenticated, 404 for unknown topologies, and (driving the
async generator directly) emits a snapshot on connect plus
forwards a published mutation.applied as an `event: mutation.applied`
SSE frame.
Wire the MazeNET editor to the new /topologies/{id}/events SSE route
so live (active|degraded) topologies reflect mutator state transitions
without reload:
- useTopologyStream hook opens an EventSource against
/topologies/{id}/events?token=<jwt>, with 3s reconnect matching the
dashboard's /stream consumer. Callback refs avoid tearing down the
connection on consumer rerenders.
- useMazeApi gains enqueueMutation(topologyId, op, payload,
expectedVersion?) — thin wrapper over POST /mutations.
- MazeNET.tsx opens the stream only when topoStatus is active|degraded
(pending editors have nothing to stream) and refetches on
mutation.applied|failed|status events. Header shows a LIVE /
CONNECTING… indicator.
Phase A slice — Apply (N changes) with an optimistic staged buffer
lands in a follow-up; the hooks + API method it'll need are already
here.
Wire the mutator and web API into the service bus so live-topology
edits flow sub-second from enqueue to UI:
- Mutator publishes every state transition on the bus (mutation.applying
/applied/failed + topology.status). Fire-and-forget; DB stays source
of truth.
- Mutator watch loop subscribes to topology.*.mutation.enqueued and
wakes early via asyncio.Event — the 10s poll becomes a fallback
heartbeat, not the primary dispatch trigger.
- POST /topologies/{id}/mutations publishes mutation.enqueued after
the DB write succeeds.
- New GET /topologies/{id}/events SSE route: snapshot on connect
(status + in-flight mutations), live forwards topology.{id}.>
bus events, 15s keepalive. ?token= auth mirrors /stream.
- New decnet/bus/app.py — process-wide lazy bus singleton for the
API, closed cleanly on lifespan shutdown.
start.sh boots a local bus on /tmp (no root, no decnet group).
sub.py / pub.py are thin CLIs over UnixSocketBus for manual poking.
smoke.sh is a self-contained end-to-end check — spawns a worker,
subscribes, publishes, asserts delivery, cleans up.
Land the `decnet bus` worker and `get_bus()` factory. Transport is a
host-local UNIX-domain socket (0660, group=decnet); authz is the file
mode. Wire framing is a tiny verb-line + 4-byte-BE length + orjson body.
NATS-style wildcard topics (`*`, `>`). At-most-once, fire-and-forget —
DB stays the source of truth. `FakeBus` / `NullBus` for tests and the
disabled path. Cross-host federation is deferred to a future
`--bridge-tcp` mode; DEBT-030 is master-only and unblocked.
Port the design-handoff layout into a scoped DeckyFleet.css (no more
piggybacking on Dashboard.css). Add an archetype-first creation wizard
that consumes /api/v1/topologies/archetypes, falling back to the
MazeNET ARCHETYPES constant when the endpoint is unavailable.
Canvas grew a deployed prop so nodes can visually distinguish "live in
docker" from "planned". ContextMenu learned nested submenus with
ChevronRight affordance; NetBox renders a ShieldAlert for DMZ LANs;
Palette got additional lucide icons. Dead PendingChange union pulled
out of types.ts — Phase-3 mutation ops are driven by the API layer now,
not a frontend type.
New /topologies page lists topologies; a bare /mazenet now redirects
there since the editor has no meaning without ?topology=<id>. Wizard
picks up a note style + tweaked copy.
test_compose asserts the new decnet.topology.* labels land on both base
deckies (role=base, no service marker) and service fragments
(service=true). The stub docker client in test_deploy grew a filters
kwarg so it keeps matching the real .networks.list(filters=...) call
signature now used by the deployer.
/api/v1/topologies/archetypes returns the archetype registry (slug,
display name, description, preferred services/distros, nmap_os
fingerprint) so the frontend wizard can render a live catalog instead
of hardcoding a copy.
The web bundle proxy handled GET/POST/PUT/DELETE but not PATCH or
preflight OPTIONS, which broke browser calls to PATCH endpoints behind
the static-bundle server. CORS middleware had the same gap.
db reset drops-and-recreates a fixed table set in FK order. Topology
tables weren't in the list, so reset left orphan topology rows behind
and a fresh MazeNET deploy could collide with stale child records.
topology delete cascades children (LANs, deckies, edges, mutations) but
refuses while containers are still running — teardown is prerequisite.
show stopped assuming every decky carried a full decky_config blob;
MazeNET-generated deckies only get hydrated on deploy, so fall back to
top-level name/services when the config isn't there.
Legacy fleet deckies live in decnet-state.json; MazeNET topology
containers don't. Tag them at compose-time with
decnet.topology.service=true and let the collector match on that label.
Spin up the agent's log collector on the first successful /topology/apply
(not in the lifespan — that would break the no-docker-on-boot invariant)
and tear it down with the app. Land log lines in DECNET_AGENT_LOG_FILE,
separate from master-side DECNET_INGEST_LOG_FILE, so a dev box running
both roles can't forward its own ingest back to itself.
When master pushes a topology that differs from whatever is pinned
locally, teardown the predecessor and accept the new one. Refusing with
409 left the agent stranded after partial deploys. record_error now
persists the hydrated blob so a later teardown can still walk the LAN
list — otherwise a half-failed apply strands containers + bridges with
no breadcrumb back to them.
Replaces the single-line name input with a modal that mirrors the
design-handoff DeployWizard shape (backdrop + violet-bordered panel,
wizard-step tabs, card-picker body):
- Step 1 — TARGET: a RUN LOCALLY card plus one card per enrolled
swarm host. Non-routable hosts render disabled with their status as
the tooltip. Selecting an agent pins the topology via
target_host_uuid; local stays unihost.
- Step 2 — TYPE: BLANK (POST /topologies/blank) or SEED-BASED
(POST /topologies/ with depth, branching, deckies-per-LAN, optional
seed). Name is required on both.
Existing navigate-to-editor-on-create behavior is preserved.
Two small observability follow-ups to the phase-1 agent/topology wiring:
TopologySummary now carries needs_resync so operators can see the
heartbeat's resync flag via the topology list/detail API without
dropping into the DB.
TopologyStore.record_error becomes an upsert — when a docker/compose
failure fires during the first materialise (put() never reached), we
still land a marker row so GET /topology/state surfaces the error and
the next heartbeat carries an empty applied_version_hash. That empty
hash is what master's heartbeat check relies on to flag the topology
for resync instead of assuming the apply succeeded.
Four regression tests guarding Step 8 of the agent/topology wiring:
- Lifespan startup must not call docker.from_env even with a populated
topology.db — replace docker with a boom-stub and assert zero calls.
- GET /topology/state returns the cached row verbatim without
re-materialising bridges/containers; live observation is read-only.
- Static guard: TopologyStore must not grow a restore/replay/reapply
method without someone re-reading the module docstring.
- Raw sqlite read + a second TopologyStore instance confirm the store
is passive — nothing scrubs stale rows on open, which is the
behaviour master's resync flow depends on.
Agent heartbeats now carry an applied-topology snapshot. The master
heartbeat handler compares the reported version_hash against what
canonical_hash yields for the hydrated topology pinned to that host
and flags Topology.needs_resync on divergence (or when the agent
reports no topology at all while master expects one).
The mutator watch loop gains reconcile_agent_resyncs, which re-pushes
the current hydrated blob via AgentClient.apply_topology without
touching status, then clears the flag on success. Push failures leave
the flag set so the next tick retries.
deploy_topology and teardown_topology now branch on
target_host_uuid. When set:
- Hydrate the topology locally (validator runs exactly as before).
- Compute canonical_hash; push {hydrated, version_hash} to the
pinned agent through AgentClient.apply_topology.
- Status machine still moves PENDING -> DEPLOYING -> ACTIVE on 2xx,
PENDING -> DEPLOYING -> FAILED on error; master remains the sole
owner of the row.
Teardown flips to TEARING_DOWN, fires /topology/teardown, then
TORN_DOWN — we log a warning on agent error but still settle to
TORN_DOWN so operators can delete the row (agent garbage is cleaned
on the next re-enroll).
Unihost deploys are unchanged — the field defaults to NULL so every
existing flow takes the local path.
Step 6 of the agent <-> topology integration.
Three new RPCs mirroring the existing deploy/teardown/status pattern:
- apply_topology(hydrated, version_hash) — long-timeout (600s) for
image pulls + compose up.
- teardown_topology(topology_id) — 300s timeout; enough for a
stubborn compose-down without hanging a heartbeat.
- get_topology_state() — short control-plane read for reconcile.
The per-call timeout swap uses the same trick as .deploy().
Step 5 of the agent <-> topology integration.
New mTLS-protected routes on the agent:
- POST /topology/apply — master pushes {hydrated, version_hash}.
Validates the hash matches locally (serialisation drift guard),
runs the topology through the same validator/composer pipeline
used master-side, then creates bridges + compose up + records the
apply in topology.db.
- POST /topology/teardown — dismantles compose, removes bridges,
clears topology.db. Idempotent.
- GET /topology/state — returns applied row + live docker
observation for the heartbeat.
Implementation lives in decnet/agent/topology_ops.py; it reuses the
private compose helpers from decnet.engine.deployer so we don't
duplicate compose/project-name plumbing. The apply path is sync
under the hood (docker SDK + subprocess); we hop to a thread so the
event loop keeps servicing other agent traffic.
v1 is one-topology-per-agent; cross-topology apply returns 409.
Step 4 of the agent <-> topology integration.
Single-row sqlite tracking which topology the agent last applied and
its version hash. Sync/stdlib, same pattern as the log-forwarder
offset store. v1 is one-topology-per-agent; attempting to apply a
different topology over a populated row raises AlreadyApplied so the
endpoint can return 409. observed() snapshots live docker state
(decnet-topology-* bridges + decnet-* containers) for the heartbeat.
The store is a cache, not authority — no auto-restore on boot.
Master remains the only source of truth.
Step 3 of the agent <-> topology integration.
Tiny pure helper both master and agent will use to answer "is the
applied state the one we expect?". SHA-256 of canonical JSON with
volatile keys (timestamps, status, version, canvas x/y/w/h) stripped
so the hash only captures deployment-relevant state.
Step 2 of the agent <-> topology integration.
Adds the `target_host_uuid` FK on `Topology` plus wiring through the
two create endpoints (`POST /topologies`, `POST /topologies/blank`).
Validates the mode/host pair: `mode='agent'` now requires a known,
routable host; `mode='unihost'` must leave the field unset.
Surfaced on `TopologySummary` so list/detail responses expose it.
Purely additive at the schema level — existing unihost flows unchanged
(field defaults to `NULL`).
Step 1 of the agent <-> topology integration.
Dragging a LAN or decky, or resizing a NetBox, updates React state
but previously vanished on reload because the grid-layout adapter
rewrote everything from the graph. Add a per-topology localStorage
snapshot (key: mazenet.layout.<topologyId>) that captures net
x/y/w/h and decky x/y; useLayoutPersistor writes it debounced, and
getTopology merges it over adaptTopology's grid so entities without
a stored entry still fall back to a clean auto-layout. Deleting a
topology calls clearLayout to drop its snapshot.
Dropping more than one LAN near the same spot stacked the NetBox
rectangles on top of each other, and multiple deckies in a LAN
landed on identical per-LAN coordinates. Since canvas position
persistence is deferred (localStorage pass), the stored x/y are
not load-bearing — compute layout from the topology graph instead.
adaptTopology now lays LANs out in a 3-col grid with the DMZ first
and stacks deckies 2-wide inside their home LAN. New LAN palette
drops append to the same grid, ignoring the raw drop point.
Active/degraded/failed/deploying topologies cannot be deleted
without first transitioning to torn_down, but the UI had no way
to trigger that. Add POST /topologies/{id}/teardown mirroring the
deploy endpoint (background task, 202 Accepted), and a
click-to-arm TEARDOWN button on the topology list card that shows
whenever the row is in a teardown-eligible state.
MazeNET publishes gateway ports on the host via Docker. With the
default userland-proxy enabled, attacker connections appear to
originate from the bridge gateway instead of the real remote IP.
Log a soft warning at deploy time when the topology publishes any
ports and docker info reports UserlandProxy=true, pointing the
operator at the daemon.json toggle. Best-effort: daemon talk
failures silently no-op.
Rebuild the inspector panel to match the handoff mock: crosshair-titled
header with dim type label and close X, status-dot + archetype-chip
head rows, connection list with directional arrows, member list with
click-to-select, and a pending-diff block at the foot. Carry the
gateway/observed disable titles over from the ctx menu so the 'remove'
action stays honest.
Also prefix the subtitle with 'NETWORK OF NETWORKS' so the purpose of
this editor reads at a glance.
A prior half-torn-down topology can leave a bridge network alive under
a different name that still owns our intended subnet. Docker then
rejects our create with 'Pool overlaps with other one on this address
space', and the topology deploy fails.
Extend create_bridge_network to sweep any unused bridge whose IPAM
subnet matches the one we're about to claim (skipping networks with
running containers — those are live use).
UI-created deckies (api_decky_crud, api_create_blank_topology) write
decky_config as sent by the client — typically just archetype flags,
without the name/ips_by_lan fields compose.py requires. The generator
path populates them at persist() time, so compose worked for generated
topologies but KeyError'd on UI-created ones.
Normalise in hydrate() so every write path feeds the same shape
downstream: mirror decky.name into decky_config.name, and allocate
per-LAN IPs deterministically (reserving the primary decky.ip where it
falls in-subnet, then filling remaining edges with next_free).
Gateway detection in the editor previously matched
archetype === 'host-gateway' (a fictional archetype that never
existed in decnet/archetypes.py). Switch to
decky_config.forwards_l3 — the real runtime marker the composer
already reads — so deletion guards, drag-pinning, context menu
locking, and NodeCard DMZ-gateway styling all line up with what
actually ships at deploy time.
On DMZ palette drop, create the gateway with archetype=deaddeck,
services=['ssh'], forwards_l3=true, and mark the edge
is_bridge=true, forwards_l3=true. attachEdge now accepts those
flags so callers can seed a real bridge attachment.
Add check_no_host_port_collision: enumerate the ports the topology's
gateways will publish (forwards_l3=True × svc.ports), probe live
listeners via psutil, emit a 'warning'-severity PORT_COLLISION
issue per overlap. Live-only — invoked from deploy_topology just
after dry-run branching, so unit tests that exercise validate()
stay hermetic.
Warning rather than error because docker-compose up will hard-fail
on a real collision anyway; this just gives operators a cleaner log
line ahead of the compose failure.
When a non-DMZ LAN is created via POST /lans, look up the topology's
gateway (decky with forwards_l3=True attached to the DMZ) and insert
an edge binding it to the new LAN. The gateway becomes multi-homed
to every internal LAN automatically, so DMZ_ORPHAN cannot arise
from ordinary editor use.
Also fixes delete_lan: the home-decky guard used scalar_one_or_none,
which blew up when the gateway already had >1 'other' LAN edge.
Switch to scalars().first() — we only need to know *some* other
edge exists, not a unique one.
Gateway deckies (forwards_l3=True) are the DMZ's ingress. Their
service containers share the base namespace via network_mode:service,
so any listener inside the gateway is reachable through the base
container's published ports. Emit 'ports: [<p>:<p>, ...]' on the
gateway base from svc.ports across the decky's service list.
This is the principled replacement for the broken network_mode: host
stub — with docker-proxy publishing, the DMZ works on any single-NIC
VPS (no MACVLAN, no promiscuous mode required).
POST /topologies/blank seeded the gateway decky with
archetype=host-gateway + network_mode=host, but neither was wired:
no compose writer reads network_mode and host-gateway is not a real
archetype. Replace with archetype=deaddeck + forwards_l3=true so the
gateway is a normal multi-homed bridge decky, consistent with how
compose.py interprets forwards_l3 (sysctl + NET_ADMIN).
Edge marked is_bridge=true, forwards_l3=true so downstream readers
(generator, compose, validator) see a real bridge attachment.
JA3/JA3S fingerprints are defined by their specs as MD5 digests of
the ClientHello/ServerHello feature tuples — they are identifiers,
not security primitives. Pass usedforsecurity=False at the two call
sites so bandit stops flagging them as B324 High when scanning
outside the templates/ exclude.
DECNET's app-level RequestValidationError handler remaps structural
422→400, including query/path constraint violations (limit bounds,
the next-subnet base pattern, etc.). Schemathesis fuzzing will drive
those code paths and fail response_schema_conformance unless 400 is
declared in responses={}. Adds the entry to every phase-3 read route.
GET /api/v1/topologies — paginated list with status filter. Extends
repo.list_topologies() to accept limit/offset and adds count_topologies()
for the total envelope field.
GET /api/v1/topologies/{id} — hydrated TopologyDetail; 404 if missing.
GET /api/v1/topologies/{id}/status-events — audit trail, limit-capped.
Catalog helpers for the phase-4 canvas UI:
* GET /topologies/services — full service catalog.
* GET /topologies/next-subnet?base=172.20 — wraps SubnetAllocator against
reserved_subnets across non-torn-down topologies.
* GET /topologies/{id}/lans/{lan_id}/next-ip — IPAllocator pre-seeded
with existing decky IPs in that LAN.
All read routes are viewer-or-admin. Sub-routers are included in an
order that keeps literal catalog paths (/services, /next-subnet) from
being shadowed by the /{topology_id} trie branch.
Add Pydantic DTOs in decnet/web/db/models.py covering every phase-3
endpoint shape: TopologyGenerateRequest, TopologySummary/Detail, child
create/update requests, MutationEnqueueRequest (Literal op guard),
MutationRow with JSON-payload decoder, validation/version/not-editable
error envelopes, and the three catalog responses.
Create decnet/web/router/topology/ as an import-safe package exporting
topology_router (prefix /topologies) — sub-routers land step-by-step in
subsequent commits. Mount under the main api router alongside swarm_mgmt.
tests/api/topology/test_models.py pins repo-dict ↔ DTO parity so future
repo-row drift breaks the contract test before the endpoints.
Adds the live-mutation pipeline for active/degraded topologies:
* TopologyMutation table with composite index (state, topology_id)
so the watch-loop guard query stays O(log n).
* claim_next_mutation is a single atomic UPDATE ... WHERE
state='pending' so racing reconcilers deterministically pick one
winner; losers see rowcount=0 and skip.
* reconcile_topologies drains pending rows per live topology, applies
via decnet.mutator.ops.dispatch, and on failure marks the mutation
failed + transitions topology to degraded.
* run_watch_loop gains a gated branch: flat-fleet mutate_all runs
every tick unchanged; the reconciler only enters when the cheap
has_pending_topology_mutation guard returns True.
* apply_* ops re-check hard invariants (names, IP collisions, subnet
overlap, known services, service_config shape) after every mutation
so the repo never lands in an invalid state.
* CLI: 'decnet topology mutate' / 'mutations' subcommands.
MazeNET phase 2 step 6. Equips the repo layer with the CRUD the web
editor needs before deploy.
- TopologyNotEditable exception: raised when a pending-only method hits
a non-pending topology. The intent is "free-form edits stop at deploy;
the mutator (step 7) takes over for live topologies."
- _assert_pending helper checks status inside the session.
- update_lan / update_topology_decky accept enforce_pending=True for
pre-deploy callers (existing internal callers default to False so
behavior is unchanged).
- delete_lan: cascades edges; refuses if any decky has only one edge
(= this LAN is its home) to prevent orphans.
- delete_topology_decky: cascades edges.
- delete_topology_edge: bare-bones removal.
All four mutators accept expected_version for optimistic concurrency.
Existing tests continue to pass (no behavior change for persist/deploy).
MazeNET phase 2 step 5. Pure storage — the generator emits None for
x/y and the web canvas fills them in later. No logic changes; no
compose, deploy, or validator impact.
MazeNET phase 2 step 4. Readies the repo layer for concurrent editors
(web canvas + CLI + mutator) without lost-write races.
- Topology.version: monotonically bumped on supervised child-row writes.
- VersionConflict exception carries {current, expected} for the UI.
- _check_and_bump_version helper reads Topology in the same session,
compares against expected_version, raises on mismatch, bumps on match.
Commit happens in the caller's existing transaction so check+bump+write
are atomic per mutation.
- add_lan / update_lan / add_topology_decky / update_topology_decky /
add_topology_edge accept expected_version=None by default, preserving
every existing caller's behavior.
When expected_version is None, no check runs and version stays put —
internal callers (persist) that don't care about concurrency keep
working unchanged.
MazeNET phase 2 step 3. Blocks deploys of hand-authored topologies that
would fail mid-bring-up (orphan deckies, duplicate IPs, overlapping
subnets, unknown services) with a structured error list instead of a
docker error at startup.
Rules (one function each, composable by the editor for inline hints):
- exactly one DMZ
- every LAN has a bridge chain to the DMZ (BFS via multi-homed deckies)
- no orphan deckies
- unique LAN and decky names per topology
- no IP collisions + IPs inside their LAN's subnet
- no LAN subnet overlaps
- every service in decnet.fleet.all_service_names()
- service_config keys match the decky's declared services
deploy_topology runs the validator after hydrate, before any status
transition or Docker call; errors raise ValidationError and status
stays at pending.
MazeNET phase 2 step 2. Mirrors the flat-fleet service_config pattern
(DeckyConfig.service_config → composer → svc.compose_fragment) into the
topology compose pipeline, so a hand-authored decky can carry overrides
like {"ssh": {"password": "megapassword"}} and the ssh fragment reads
them just like the flat path does.
- _PlannedDecky gains service_config: dict[str, dict].
- persist() stores it under decky_config["service_config"].
- topology/compose.py passes cfg.get("service_config", {}).get(svc, {})
to svc.compose_fragment(service_cfg=...).
Schema unchanged — service_config lives inside the existing
decky_config JSON blob. Zero changes in decnet/services/*.
MazeNET phase 2 step 1. Pulls inline IP/subnet allocation out of the
generator into decnet/topology/allocator.py so the editor + reconciler
can reuse the same primitives without duplicating logic.
- IPAllocator: stateful host-IP handout with reserve/release/is_free.
- SubnetAllocator: /24 handout under a base prefix, skips reservations.
- reserved_subnets(repo): collects claimed subnets across every
non-torn_down topology so concurrent drafts cannot collide.
- generate() accepts reserved_subnets= to skip existing claims.
Generator output is byte-identical under seed (behavior preserved).
Covers dry-run compose emission (no status change), FAILED transition
with reason logged on daemon errors, teardown from FAILED, and a
live-marked end-to-end test that creates/removes bridge networks
against a real docker daemon (skipped on CI).
decnet topology {generate,list,show,deploy,teardown} wraps the new
persistence and deployer APIs. Structured text output, no ASCII art —
visual DAG rendering belongs in the web dashboard. Group is master-only
via MASTER_ONLY_GROUPS and a _require_master_mode guard on each body.
Adds per-topology compose generation (one Docker bridge network per
LAN, multi-homed bridge deckies, ip_forward sysctl for L3 forwarders)
plus async deploy_topology/teardown_topology in the engine. Leaf-first
teardown via BFS-named LAN reverse sort; partial-state safe on failure.
Adds decnet/topology/ with:
- config.TopologyConfig: pydantic model driving generation (depth,
branching_factor, deckies_per_lan_min/max, bridge_forward_probability,
cross_edge_probability, subnet_base_prefix, service selection, seed).
Emits GeneratedTopology dataclass (lans, deckies, edges).
- status.TopologyStatus + assert_transition: seven-state machine with
an explicit legal-transition table. torn_down is terminal; degraded
is schema-reserved for future Healer use.
- generator.generate: deterministic DAG generation under config.seed.
Builds a tree of LANs (DMZ at root), plants deckies in each LAN,
promotes one decky per non-DMZ LAN to a parent bridge, and rolls
cross-edges per cross_edge_probability for DAG shape.
- persistence: persist() writes a plan to the repo as pending;
transition_status() enforces state-machine legality; hydrate() loads
topology + children into a single dict.
Covered by tests/topology/{test_status,test_generator,test_persistence}.
Adds topology CRUD to BaseRepository (NotImplementedError defaults) and
implements them in SQLModelRepository: create/get/list/delete topologies,
add/update/list LANs and TopologyDeckies, add/list edges, plus an atomic
update_topology_status that appends a TopologyStatusEvent in the same
transaction. Cascade delete sweeps children before the topology row.
Covered by tests/topology/test_repo.py (roundtrip, per-topology name
uniqueness, status event log, cascade delete, status filter) and an
extension to tests/test_base_repo.py for the NotImplementedError surface.
Introduces five new SQLModel tables for MazeNET (nested deception
topologies): Topology, LAN, TopologyDecky, TopologyEdge, and
TopologyStatusEvent. DeckyShard is intentionally not touched —
TopologyDecky is a purpose-built sibling for MazeNET's lifecycle
(topology-scoped UUIDs, per-topology name uniqueness).
Part of MazeNET v1 (nested self-container network-of-networks).
Schemathesis was failing CI on routes that returned status codes not
declared in their OpenAPI responses= dicts. Adds the missing codes
across swarm_updates, swarm_mgmt, swarm, fleet and attackers routers.
Also adds 400 to every POST/PUT/PATCH that accepts a JSON body —
Starlette returns 400 on malformed/non-UTF8 bodies before FastAPI's
422 validation runs, which schemathesis fuzzing trips every time.
No handler logic changed.
- tests/**: update templates/ → decnet/templates/ paths after module move
- tests/mysql_spinup.sh: use root:root and asyncmy driver
- tests/test_auto_spawn.py: patch decnet.cli.utils._pid_dir (package split)
- tests/test_cli.py: set DECNET_MODE=master in api-command tests
- tests/stress/conftest.py: run locust out-of-process via its CLI + CSV
stats shim to avoid urllib3 RecursionError from late gevent monkey-patch;
raise uvicorn startup timeout to 60s, accept 401 from auth-gated health,
strip inherited DECNET_* env, surface stderr on 0-request runs
- tests/stress/test_stress.py: loosen baseline thresholds to match hw
The 1,878-line cli.py held every Typer command plus process/HTTP helpers
and mode-gating logic. Split into one module per command using a
register(app) pattern so submodules never import app at module scope,
eliminating circular-import risk.
- utils.py: process helpers, _http_request, _kill_all_services, console, log
- gating.py: MASTER_ONLY_* sets, _require_master_mode, _gate_commands_by_mode
- deploy.py: deploy + _deploy_swarm (tightly coupled)
- lifecycle.py: status, teardown, redeploy
- workers.py: probe, collect, mutate, correlate
- inventory.py, swarm.py, db.py, and one file per remaining command
__init__.py calls register(app) on each module then runs the mode gate
last, and re-exports the private symbols tests patch against
(_db_reset_mysql_async, _kill_all_services, _require_master_mode, etc.).
Test patches retargeted to the submodule where each name now resolves.
Enroll-bundle tarball test updated to assert decnet/cli/__init__.py.
No behavioral change.
Uvicorn's h11/httptools HTTP protocols don't populate scope['extensions']['tls'], so /swarm/heartbeat's per-request cert pinning was 403ing every call despite CERT_REQUIRED validating the cert at handshake. Patch RequestResponseCycle.__init__ on both protocol modules to read the peer cert off the asyncio transport and write DER bytes into scope['extensions']['tls']['client_cert_chain']. Importing the module from swarm_api.py auto-installs the patch in the swarmctl uvicorn worker before any request is served.
DeckyFleet now branches on /system/deployment-mode: in swarm mode it
pulls /swarm/deckies and normalises DeckyShardView into the shared
Decky shape so the same card grid renders either way. Swarm cards gain
a host badge (host_name @ address), a state pill (running/degraded/
tearing_down/failed/teardown_failed with matching colors), an inline
last_error snippet, and a two-click arm/commit Teardown button lifted
from the old SwarmDeckies component. Mutate + interval controls are
hidden in swarm mode since the worker /mutate endpoint still 501s —
swarm-side rotation is a separate ticket.
Drops the standalone /swarm/deckies route + nav entry; SwarmDeckies.tsx
is deleted. The SWARM nav group keeps SwarmHosts, Remote Updates, and
Agent Enrollment.
New decnet.agent.heartbeat asyncio loop wired into the agent FastAPI
lifespan. Every 30 s the worker POSTs executor.status() to the master's
/swarm/heartbeat with its DECNET_HOST_UUID for self-identity; the
existing agent mTLS bundle provides the client cert the master pins
against SwarmHost.client_cert_fingerprint.
start() is a silent no-op when identity env (HOST_UUID, MASTER_HOST) is
unset or the worker bundle is missing, so dev runs and un-enrolled hosts
don't crash the agent app. On non-204 responses the loop logs loudly but
keeps ticking — an operator may re-enrol mid-session, and fail-closed
pinning shouldn't be self-silencing.
swarmctl CLI gains --tls/--cert/--key/--client-ca flags. With --tls the
controller runs uvicorn under HTTPS + mTLS (CERT_REQUIRED) so worker
heartbeats can reach it cross-host. Default is still 127.0.0.1 plaintext
for backwards compat with the master-CLI enrollment flow.
Auto-issue path (no --cert/--key given): a server cert signed by the
existing DECNET CA is issued once and parked under ~/.decnet/swarmctl/.
Workers already ship that CA's ca.crt from the enroll bundle, so they
verify the endpoint with no extra trust config. BYOC via --cert/--key
when the operator wants a publicly-trusted or externally-managed cert.
The auto-cert path is idempotent across restarts to keep a stable
fingerprint for any long-lived mTLS sessions.
The rendered /etc/decnet/decnet.ini now carries host-uuid and
swarmctl-port in [agent], which config_ini seeds into DECNET_HOST_UUID
and DECNET_SWARMCTL_PORT. Gives the worker a stable self-identity for
the heartbeat loop — the INI never has to be rewritten because cert
pinning is the real gate (a rotated UUID with a matching CA-signed
cert would still be blocked by SHA-256 fingerprint mismatch against
the stored SwarmHost row).
Also adds DECNET_MASTER_HOST so the agent can find the swarmctl URL
via the INI's existing master-host key.
New POST /swarm/heartbeat on the swarm controller. Workers post every
~30s with the output of executor.status(); the master bumps
SwarmHost.last_heartbeat and re-upserts each DeckyShard with a fresh
DeckyConfig snapshot and runtime-derived state (running/degraded).
Security: CA-signed mTLS alone is not sufficient — a decommissioned
worker's still-valid cert could resurrect ghost shards. The endpoint
extracts the presented peer cert (primary: scope["extensions"]["tls"],
fallback: transport.get_extra_info("ssl_object")) and SHA-256-pins it
to the SwarmHost.client_cert_fingerprint stored for the claimed
host_uuid. Extraction is factored into _extract_peer_fingerprint so
tests can exercise both uvicorn scope shapes and the both-unavailable
fail-closed path without mocking uvicorn's TLS pipeline.
Adds get_swarm_host_by_fingerprint to the repo interface (SQLModel
impl reuses the indexed client_cert_fingerprint column).
Dispatch now writes the full serialised DeckyConfig into
DeckyShard.decky_config (plus decky_ip as a cheap extract), so the
master can render the same rich per-decky card the local-fleet view
uses — hostname, distro, archetype, service_config, mutate_interval,
last_mutated — without round-tripping to the worker on every page
render. DeckyShardView gains the corresponding fields; the repository
flattens the snapshot at read time. Pre-migration rows keep working
(fields fall through as None/defaults).
Columns are additive + nullable so SQLModel.metadata.create_all handles
the change on both SQLite and MySQL. Backfill happens organically on
the next dispatch or (in a follow-up) agent heartbeat.
The reaper was being SIGTERM'd mid-rm because `start_new_session=True`
only forks a new POSIX session — it does not escape decnet-agent.service's
cgroup. When the reaper ran `systemctl stop decnet-agent`, systemd
tore down the whole cgroup (reaper included) before `rm -rf /opt/decnet*`
finished, leaving the install on disk.
Spawn the reaper via `systemd-run --collect --unit decnet-reaper-<pid>`
so it runs in a fresh transient scope, outside the agent unit. Falls
back to bare Popen for non-systemd hosts.
Decommissioning a worker from the dashboard (or swarm controller) now
asks the agent to wipe its own install before the master forgets it.
The agent stops decky containers + every decnet-* systemd unit, then
deletes /opt/decnet*, /etc/systemd/system/decnet-*, /var/lib/decnet/*,
and /usr/local/bin/decnet*. Logs under /var/log are preserved.
The reaper runs as a detached /tmp script (start_new_session=True) so
it survives the agent process being killed. Self-destruct dispatch is
best-effort — a dead worker doesn't block master-side cleanup.
Teardowns were synchronous all the way through: POST blocked on the
worker's docker-compose-down cycle (seconds to minutes), the frontend
locked tearingDown to a single string so only one button could be armed
at a time, and operators couldn't queue a second teardown until the
first returned. On a flaky worker that meant staring at a spinner for
the whole RTT.
Backend: POST /swarm/hosts/{uuid}/teardown returns 202 the instant the
request is validated. Affected shards flip to state='tearing_down'
synchronously before the response so the UI reflects progress
immediately, then the actual AgentClient call + DB cleanup run in an
asyncio.create_task (tracked in a module-level set to survive GC and
to be drainable by tests). On failure the shard flips to
'teardown_failed' with the error recorded — nothing is re-raised,
since there's no caller to catch it.
Frontend: swap tearingDown / decommissioning from 'string | null' to
'Set<string>'. Each button tracks its own in-flight state; the poll
loop picks up the final shard state from the backend. Multiple
teardowns can now be queued without blocking each other.
Submitting an INI with a single [decky1] was silently redeploying the
deckies from the *previous* deploy too. POST /deckies/deploy merged the
new INI into the stored DecnetConfig by name, so a 1-decky INI on top of
a prior 3-decky run still pushed 3 deckies to the worker. Those stale
decky2/decky3 kept their old IPs, collided on the parent NIC, and the
agent failed with 'Address already in use' — the deploy the operator
never asked for.
The INI is the source of truth for which deckies exist this deploy.
Full replace: config.deckies = list(new_decky_configs). Operators who
want to add more deckies should list them all in the INI.
Update the deploy-limit test to reflect the new replace semantics, and
add a regression test asserting prior state is dropped.
Teardown and Decommission buttons were silently dead in the browser.
Root cause: every handler started with 'if (!window.confirm(...)) return;'
and browsers permanently disable confirm() for a tab once the user ticks
'Prevent this page from creating additional dialogs'. That returns false
with no UI, the handler early-exits, and no request is ever fired — no
network traffic, no console error, no backend activity.
Swap to an inline two-click pattern: first click arms the button (label
flips to 'Click again to confirm', resets after 4s); second click within
the window commits. Same safety against misclicks, zero dependency on
browser-native dialog primitives.
docker compose up is partial-success-friendly — a build failure on one
service doesn't roll back the others. But the master was catching the
agent's 500 and tagging every decky in the shard as 'failed' with the
same error message. From the UI that looked like all three deckies died
even though two were live on the worker.
On dispatch exception, probe the agent's /status to learn which deckies
actually have running containers, and upsert per-decky state accordingly.
Only fall back to marking the whole shard failed if the status probe
itself is unreachable.
Enhance agent.executor.status() to include a 'runtime' map keyed by
decky name with per-service container state, so the master has something
concrete to consult.
Two compounding root causes produced the recurring 'Address already in use'
error on redeploy:
1. _ensure_network only compared driver+name; if a prior deploy's IPAM
pool drifted (different subnet/gateway/range), Docker kept handing out
addresses from the old pool and raced the real LAN. Now also compares
Subnet/Gateway/IPRange and rebuilds on drift.
2. A prior half-failed 'up' could leave containers still holding the IPs
and ports the new run wants. Run 'compose down --remove-orphans' as a
best-effort pre-up cleanup so IPAM starts from a clean state.
Also surface docker compose stderr to the structured log on failure so
the agent's journal captures Docker's actual message (which IP, which
port) instead of just the exit code.
Operators want to know what address to poke when triaging a swarm decky;
the compose-hash column was debug scaffolding that never paid off.
DeckyShard has no IP column (the deploy-time IP lives on DecnetConfig),
so the list endpoint resolves it at read time by joining shards against
the stored deployment state by decky_name. Missing lookups render as "—"
rather than erroring — the list stays useful even after a master restart
that hasn't persisted a config yet.
The nested list-comp `[f"{id}-{svc}" for svc in [d.services for d ...]]`
iterated over a list of lists, so `svc` was the whole services list and
f-string stringified it -> `decky3-['sip']`. docker compose saw "no such
service" and the per-decky teardown failed 500.
Flatten: find the matching decky once, then iterate its services. Noop
early on unknown decky_id and on empty service lists. Regression test
asserts the emitted compose args have no '[' or quote characters.
Agents already exposed POST /teardown; the master was missing the plumbing
to reach it. Add:
- POST /api/v1/swarm/hosts/{uuid}/teardown — admin-gated. Body
{decky_id: str|null}: null tears the whole host, a value tears one decky.
On worker failure the master returns 502 and leaves DB shards intact so
master and agent stay aligned.
- BaseRepository.delete_decky_shard(name) + sqlmodel impl for per-decky
cleanup after a single-decky teardown.
- SwarmHosts page: "Teardown all" button (keeps host enrolled).
- SwarmDeckies page: per-row "Teardown" button.
Also exclude setuptools' build/ staging dir from the enrollment tarball —
`pip install -e` on the master generates build/lib/decnet_web/node_modules
and the bundle walker was leaking it to agents. Align pyproject's bandit
exclude with the git-hook invocation so both skip decnet/templates/.
The docker build contexts and syslog_bridge.py lived at repo root, which
meant setuptools (include = ["decnet*"]) never shipped them. Agents
installed via `pip install $RELEASE_DIR` got site-packages/decnet/** but no
templates/, so every deploy blew up in deployer._sync_logging_helper with
FileNotFoundError on templates/syslog_bridge.py.
Move templates/ -> decnet/templates/ and declare it as setuptools
package-data. Path resolutions in services/*.py and engine/deployer.py drop
one .parent since templates now lives beside the code. Test fixtures,
bandit exclude path, and coverage omit glob updated to match.
Agents now ship with collector/prober/sniffer as systemd services; mutator,
profiler, web, and API stay master-only (profiler rebuilds attacker profiles
against the master DB — no per-host DB exists). Expand _EXCLUDES to drop the
full decnet/web, decnet/mutator, decnet/profiler, and decnet_web trees from
the enrollment bundle.
Updater now calls _heal_path_symlink + _sync_systemd_units after rotation so
fleets pick up new unit files and /usr/local/bin/decnet tracks the shared venv
without a manual reinstall. daemon-reload runs once per update when any unit
changed.
Fix _service_registry matchers to accept systemd-style /usr/local/bin/decnet
cmdlines (psutil returns a list — join to string before substring-checking)
so agent-mode `decnet status` reports collector/prober/sniffer correctly.
The bootstrap installer copies etc/systemd/system/*.service into
/etc/systemd/system at enrollment time, but the updater was skipping
that step — a code push could not ship a new unit (e.g. the four
per-host microservices added this session) or change ExecStart on an
existing one. systemctl alone doesn't re-read unit files; daemon-reload
is required.
run_update / run_update_self now call _sync_systemd_units after
rotation: diff each .service file against the live copy, atomically
replace changed ones, then issue a single `systemctl daemon-reload`.
No-op on legacy tarballs that don't ship etc/systemd/system/.
Previously `decnet status` on an agent showed every microservice as DOWN
because deploy's auto-spawn was unihost-scoped and the agent CLI gate
hid the per-host commands. Now:
- collect, probe, profiler, sniffer drop out of MASTER_ONLY_COMMANDS
(they run per-host; master-side work stays master-gated).
- mutate stays master-only (it orchestrates swarm-wide respawns).
- decnet/mutator/ excluded from agent tarballs — never invoked there.
- decnet/web exclusion tightened: ship db/ + auth.py + dependencies.py
(profiler needs the repo singleton), drop api.py, swarm_api.py,
ingester.py, router/, templates/.
- Four new systemd unit templates (decnet-collector/prober/profiler/
sniffer) shipped in every enrollment tarball.
- enroll_bootstrap.sh enables + starts all four alongside agent and
forwarder at install time.
- updater restarts the aux units on code push so they pick up the new
release (best-effort — legacy enrollments without the units won't
fail the update).
- status table hides Mutator + API rows in agent mode.
Agents never run the FastAPI master app (decnet/web/) or serve the React
frontend (decnet_web/) — they run decnet.agent, decnet.updater, and
decnet.forwarder, none of which import decnet.web. Shipping the master
tree bloats every enrollment payload and needlessly widens the worker's
attack surface.
Excluded paths are unreachable on the worker (all cli.py imports of
decnet.web are inside master-only command bodies that the agent-mode
gate strips). Tests assert neither tree leaks into the tarball.
The bootstrap was installing into /opt/decnet/.venv with an editable
`pip install -e .`, and /usr/local/bin/decnet pointed there. The updater
writes releases to /opt/decnet/releases/active/ with a shared venv at
/opt/decnet/venv — a parallel tree nothing on the box actually runs.
Result: updates appeared to succeed (release dir rotated, SHA changed)
but systemd kept executing the untouched bootstrap code.
Changes:
- Bootstrap now installs directly into /opt/decnet/releases/active
with the shared venv at /opt/decnet/venv and /opt/decnet/current
symlinked. Same layout the updater rotates in and out of.
- /usr/local/bin/decnet -> /opt/decnet/venv/bin/decnet.
- run_update / run_update_self heal /usr/local/bin/decnet on every
push so already-enrolled hosts recover on the next update instead
of needing a re-enroll.
- run_update / run_update_self now log each phase (receive, extract,
pip install, rotate, restart, probe) so the updater log actually
shows what happened.
Agents run deckies locally and need to inspect their own state. Removed
`status` from MASTER_ONLY_COMMANDS so it survives the agent-mode gate.
Useful for validating remote updater pushes from the master.
Three holes in the systemd integration:
1. _spawn_agent_via_systemd only restarted decnet-agent.service, leaving
decnet-forwarder.service running the pre-update code (same /opt/decnet
tree, stale import cache).
2. run_update_self used os.execv regardless of environment — the re-execed
process kept the updater's existing cgroup/capability inheritance but
systemd would notice MainPID change and mark the unit degraded.
3. No path to surface a failed forwarder restart (legacy enrollments have
no forwarder unit).
Now: agent restart first, forwarder restart as best-effort (logged but
non-fatal so legacy workers still update), MainPID still read from the
agent unit. For update-self under systemd, spawn a detached sleep+
systemctl restart so the HTTP response flushes before the unit cycles.
Bootstrap used to end with `decnet updater --daemon` which forks and
detaches — invisible to systemctl, no auto-restart, dies on reboot.
Ships a decnet-updater.service template matching the pattern of the
other units (Restart=on-failure, log to /var/log/decnet/decnet.updater.log,
certs from /etc/decnet/updater, install tree at /opt/decnet), bundles
it alongside agent/forwarder/engine units, and the installer now
`systemctl enable --now`s it when --with-updater is set.
The create helpers short-circuited on name alone, so a prior macvlan
deploy left Docker's decnet_lan network in place. A subsequent ipvlan
deploy would no-op the network create, then container attach would try
to add a macvlan port on enp0s3 that already had an ipvlan slave —
EBUSY, agent 500, docker ps empty.
Now: when the existing network's driver disagrees with the requested
one, disconnect any live containers and DROP it before recreating.
Parent-NIC can host one driver at a time.
Also: setup_host_{macvlan,ipvlan} opportunistically delete the opposite
host-side helper so we don't leave cruft across driver swaps.
_DB_RESET_TABLES was missing the swarm tables, so drop-tables mode left
them intact. create_all doesn't alter columns on existing tables, so any
schema change to SwarmHost (like use_ipvlan) never took effect after a
reset. Child FK first (decky_shards -> swarm_hosts).
Wi-Fi APs bind one MAC per associated station, so VirtualBox/VMware
guests bridged over Wi-Fi rotate the VM's DHCP lease when Docker's
macvlan starts emitting container-MAC frames through the vNIC. Adds a
`use_ipvlan` toggle on the Agent Enrollment tab (mirrors the updater
daemon checkbox): flips the flag on SwarmHost, bakes `ipvlan=true` into
the agent's decnet.ini, and `_worker_config` forces ipvlan=True on the
per-host shard at dispatch. Safe no-op on wired/bare-metal agents.
Deckies merged in from a prior deployment's saved state kept their
original host_uuid — which dispatch_decnet_config then 404'd on if that
host had since been decommissioned or re-enrolled at a different uuid.
Before round-robin assignment, drop any host_uuid that isn't in the live
swarm_hosts set so orphaned entries get reassigned instead of exploding
with 'unknown host_uuid'.
tar_working_tree (walks repo + gzips several MB) and detect_git_sha
(shells out) were called directly on the event loop, so /swarm-updates/push
and /swarm-updates/push-self froze every other request until the tarball
was ready. Wrap both in asyncio.to_thread.
systemd daemons run with WorkingDirectory=/ by default; docker compose
derives the project name from basename(cwd), which is empty at '/', and
aborts with 'project name must not be empty'. Pass -p decnet explicitly
so the project name is independent of cwd, and set WorkingDirectory=/opt/decnet
on the three DECNET units so compose artifacts (decnet-compose.yml,
build contexts) also land in the install dir.
POST /deckies/deploy now branches on DECNET_MODE + enrolled host presence:
when the caller is a master with at least one reachable swarm host, round-
robin host_uuids are assigned over new deckies and the config is dispatched
via AgentClient. Falls back to local docker-compose otherwise.
Extracts the dispatch loop from api_deploy_swarm into dispatch_decnet_config
so both endpoints share the same shard/dispatch/persist path. Adds
GET /system/deployment-mode for the UI to show 'will shard across N hosts'
vs 'will deploy locally' before the operator clicks deploy.
Stateless /api/v1/deckies/deploy previously instantiated DecnetConfig with
deckies=[] so it could merge entries later — but DecnetConfig.deckies is
min_length=1, so Pydantic raised and the global handler mapped it to 422
'Internal data consistency error'. Construct the config after
build_deckies_from_ini returns at least one DeckyConfig.
Rename log-file-path -> log-directory (maps to DECNET_LOG_DIRECTORY). Bundle
now ships three systemd units rendered with agent_name/master_host and installs
them into /etc/systemd/system/. Bootstrap replaces direct 'decnet X --daemon'
calls with systemctl enable --now. Each unit pins DECNET_SYSTEM_LOGS so agent,
forwarder, and deckies logs land at decnet.{agent,forwarder}.log and decnet.log
under /var/log/decnet.
Mirrors the agent→forwarder pattern: `decnet swarmctl` now fires the
syslog-TLS listener as a detached Popen sibling so a single master
invocation brings the full receive pipeline online. --no-listener opts
out for operators who want to run the listener on a different host (or
under their own systemd unit).
Listener bind host / port come from DECNET_LISTENER_HOST and
DECNET_SWARM_SYSLOG_PORT — both seedable from /etc/decnet/decnet.ini.
PID at $(pid_dir)/listener.pid so operators can kill/restart manually.
decnet.ini.example ships alongside env.config.example as the
documented surface for the new role-scoped config. Mode, forwarder
targets, listener bind, and master ports all live there — no more
memorizing flag trees.
Extends tests/test_auto_spawn.py with two swarmctl cases: listener is
spawned with the expected argv + PID file, and --no-listener suppresses.
New _spawn_detached(argv, pid_file) helper uses Popen with
start_new_session=True + close_fds=True + DEVNULL stdio to launch a
DECNET subcommand as a fully independent process. The parent does NOT
wait(); if it dies the child survives under init. This is deliberately
not a supervisor — if the child dies the operator restarts it manually.
_pid_dir() picks /opt/decnet when writable else ~/.decnet, so both
root-run production and non-root dev work without ceremony.
`decnet agent` now auto-spawns `decnet forwarder --daemon ...` as
that detached sibling, pulling master host + syslog port from
DECNET_SWARM_MASTER_HOST / DECNET_SWARM_SYSLOG_PORT. --no-forwarder
opts out. If DECNET_SWARM_MASTER_HOST is unset the auto-spawn is
silently skipped (single-host dev or operator wants to start the
forwarder separately).
tests/test_auto_spawn.py monkeypatches subprocess.Popen and verifies:
the detach kwargs are passed, the PID file exists and contains a
valid positive integer (PID-file corruption is a real operational
headache — catching bad writes at the test level is free), the
--no-forwarder flag suppresses the spawn, and the unset-master-host
path silently skips.
- MASTER_ONLY_COMMANDS / MASTER_ONLY_GROUPS frozensets enumerate every
command a worker host must not see. Comment block at the declaration
puts the maintenance obligation in front of anyone touching command
registration.
- _gate_commands_by_mode() filters both app.registered_commands (for
@app.command() registrations) and app.registered_groups (for
add_typer sub-apps) so the 'swarm' group disappears along with
'api', 'swarmctl', 'deploy', etc. on agent hosts.
- _require_master_mode() is the belt-and-braces in-function guard,
added to the four highest-risk commands (api, swarmctl, deploy,
teardown). Protects against direct function imports that would
bypass Typer.
- DECNET_DISALLOW_MASTER=false is the escape hatch for hybrid dev
hosts that legitimately play both roles.
tests/test_mode_gating.py exercises help-text listings via subprocess
and the defence-in-depth guard via direct import.
- decnet/__init__.py now calls load_ini_config() on first import of any
decnet.* module, seeding os.environ via setdefault() so env.py's
module-level reads pick up INI values before the shell had to export
them. Real env vars still win.
- env.py exposes DECNET_MODE (default 'master') and
DECNET_DISALLOW_MASTER (default true), consumed by the upcoming
master-command gating in cli.py.
Back-compat: missing /etc/decnet/decnet.ini is a no-op. Existing
.env.local + flag-based launches behave identically.
- decnet/agent/app.py /health: drop leftover 'push-test-2' canary
planted during live VM push verification and never cleaned up;
test_health_endpoint asserts the exact dict shape.
- tests/test_factory.py: switch the lazy-engine check from
mysql+aiomysql (not in pyproject) to mysql+asyncmy (the driver the
project actually ships). The test does not hit the wire so the
dialect swap is safe.
Both were red on `pytest tests/` before any config/auto-spawn work
began; fixing them here so the upcoming commits land on a green
full-suite baseline.
New decnet/config_ini.py parses a role-scoped INI file via stdlib
configparser and seeds os.environ via setdefault — real env vars still
win, keeping full back-compat with .env.local flows.
[decnet] holds role-agnostic keys (mode, disallow-master, log-file-path);
the role section matching `mode` is loaded, the other is ignored
silently so a worker never reads master-only keys (and vice versa).
Loader is standalone in this commit — not wired into cli.py yet.
The module-level _require_env('DECNET_JWT_SECRET') call blocked
`decnet agent` and `decnet updater` from starting on workers that
legitimately have no business knowing the master's JWT signing key.
Move the resolution into a module `__getattr__`: only consumers that
actually read `decnet.env.DECNET_JWT_SECRET` trigger the validation,
which in practice means only decnet.web.auth (master-side).
Adds tests/test_env_lazy_jwt.py covering both the in-process lazy path
and an out-of-process `decnet agent --help` subprocess check with a
fully sanitized environment.
React component for /swarm-updates: per-host table polled every 10s,
row actions for Push Update / Update Updater / Rollback, a fleet-wide
'Push to All' modal with the include_self toggle, and toast feedback
per result.
Admin-only (both server-gated and UI-gated). Unreachable hosts surface
as an explicit state; actions are disabled on them. Rollback is
disabled when the worker has no previous release slot (previous_sha
null from /hosts).
Adds /api/v1/swarm-updates/{hosts,push,push-self,rollback} behind
require_admin. Reuses the existing UpdaterClient + tar_working_tree + the
per-host asyncio.gather pattern from api_deploy_swarm.py; tarball is
built exactly once per /push request and fanned out to every selected
worker. /hosts filters out decommissioned hosts and agent-only
enrollments (no updater bundle = not a target).
Connection drops during /update-self are treated as success — the
updater re-execs itself mid-response, so httpx always raises.
Pydantic models live in decnet/web/db/models.py (single source of
truth). 24 tests cover happy paths, rollback, transport failures,
include_self ordering (skip on rolled-back agents), validation, and
RBAC gating.
Add deploy/ unit files for every DECNET daemon (agent, updater, api, web,
swarmctl, listener, forwarder). All run as User=decnet with NoNewPrivileges,
ProtectSystem, PrivateTmp, LockPersonality; AmbientCapabilities=CAP_NET_ADMIN
CAP_NET_RAW only on the agent (MACVLAN/scapy). Existing api/web units migrated
to /opt/decnet layout and the same hardening stanza.
Make the updater's _spawn_agent systemd-aware: under systemd (detected via
INVOCATION_ID + systemctl on PATH), `systemctl restart decnet-agent.service`
replaces the Popen path so the new agent inherits the unit's ambient caps
instead of the updater's empty set. _stop_agent becomes a no-op in that mode
to avoid racing systemctl's own stop phase.
Tests cover the dispatcher branch selection, MainPID parsing, and the
systemd no-op stop.
- _run_pip: on first venv use, install decnet with its full dep tree so the
bootstrapped environment actually has typer/fastapi/uvicorn. Subsequent
updates keep --no-deps for a near-no-op refresh.
- run_update_self: do not reuse sys.argv to re-exec the updater. Inside the
live process, sys.argv is the uvicorn subprocess invocation (--ssl-keyfile
etc.), which 'decnet updater' CLI rejects. Reconstruct the operator-visible
command from env vars set by updater.server.run.
If the agent was started outside the updater (manually, during dev,
or from a prior systemd unit), there is no agent.pid for _stop_agent
to target, so a successful code install leaves the old in-memory
agent process still serving requests. Scan /proc for any decnet agent
command and SIGTERM all matches so restart is reliable regardless of
how the agent was originally launched.
Adds a separate `decnet updater` daemon on each worker that owns the
agent's release directory and installs tarball pushes from the master
over mTLS. A normal `/update` never touches the updater itself, so the
updater is always a known-good rescuer if a bad agent push breaks
/health — the rotation is reversed and the agent restarted against the
previous release. `POST /update-self` handles updater upgrades
explicitly (no auto-rollback).
- decnet/updater/: executor, FastAPI app, uvicorn launcher
- decnet/swarm/updater_client.py, tar_tree.py: master-side push
- cli: `decnet updater`, `decnet swarm update [--host|--all]
[--include-self] [--dry-run]`, `--updater` on `swarm enroll`
- enrollment API issues a second cert (CN=updater@<host>) signed by the
same CA; SwarmHost records updater_cert_fingerprint
- tests: executor, app, CLI, tar tree, enroll-with-updater (37 new)
- wiki: Remote-Updates page + sidebar + SWARM-Mode cross-link
`swarm list` only shows enrolled workers — there was no way to see which
deckies are running and where. Adds GET /swarm/deckies on the controller
(joins DeckyShard with SwarmHost for name/address/status) plus the CLI
wrapper with --host / --state filters and --json.
deploy --mode swarm was failing on every heterogeneous fleet: the master
populates config.interface from its own box (detect_interface() → its
default NIC), then ships that verbatim. The worker's deployer then calls
get_host_ip(config.interface), hits 'ip addr show wlp6s0' on a VM whose
NIC is enp0s3, and 500s.
Fix: agent.executor._relocalize() runs on every swarm-mode deploy.
Re-detects the worker's interface/subnet/gateway/host_ip locally and
swaps them into the config before calling deployer.deploy(). When the
worker's subnet doesn't match the master's, decky IPs are re-allocated
from the worker's subnet via allocate_ips() so they're reachable.
Unihost-mode configs are left untouched — they're already built against
the local box and second-guessing them would be wrong.
Validated against anti@192.168.1.13: master dispatched interface=wlp6s0,
agent logged 'relocalized interface=enp0s3', deployer ran successfully,
dry-run returned ok=deployed.
4 new tests cover both branches (matching-subnet preserves decky IPs;
mismatch re-allocates), the end-to-end executor.deploy() path, and the
unihost short-circuit.
The swarmctl API already exposes POST /swarm/check — an active mTLS
probe that refreshes SwarmHost.status + last_heartbeat for every
enrolled worker. The CLI was missing a wrapper, so operators had to
curl the endpoint directly (which is how the VM validation run did it,
and how the wiki Deployment-Modes / SWARM-Mode pages ended up doc'ing
a command that didn't exist yet).
Matches the existing list/enroll/decommission pattern: typer subcommand
under swarm_app, --url override, Rich table output plus --json for
scripting. Three tests: populated table, empty-swarm path, and --json
emission.
New `decnet listener` command runs the master-side RFC 5425 syslog-TLS
receiver as a standalone process (mirrors `decnet api` / `decnet swarmctl`
pattern, SIGTERM/SIGINT handlers, --daemon support).
`decnet agent` now accepts --agent-dir so operators running the worker
agent under sudo/root can point at a bundle outside /root/.decnet/agent
(the HOME under sudo propagation).
Both flags were needed to stand up the full SWARM pipeline end-to-end on
a throwaway VM: mTLS control plane reachable, syslog-over-TLS wire
confirmed via tcpdump, master-crash/resume proved with zero loss and
zero duplication across 10 forwarded lines.
pyproject: bump asyncmy floor to 0.2.11 (resolver already pulled this in).
Covers failure modes the happy-path tests miss:
- log rotation (copytruncate): st_size shrinks under the forwarder, it
resets offset=0 and reships the new contents instead of getting wedged
past EOF;
- listener restart: forwarder retries, resumes from the persisted offset,
and the previously-acked lines are NOT duplicated on the master;
- listener tolerates a well-authenticated client that sends a partial
octet-count frame and drops — the server must stay up and accept
follow-on connections;
- peer_cn / fingerprint_from_ssl degrade to 'unknown' / None when no
peer cert is available (defensive path that otherwise rarely fires).
New sub-app talks HTTP to the local swarm controller (127.0.0.1:8770 by
default; override with --url or $DECNET_SWARMCTL_URL).
- enroll: POSTs /swarm/enroll, prints fingerprint, optionally writes
ca.crt/worker.crt/worker.key to --out-dir for scp to the worker.
- list: renders enrolled workers as a rich table (with --status filter).
- decommission: looks up uuid by --name, confirms, DELETEs.
deploy --mode swarm now:
1. fetches enrolled+active workers from the controller,
2. round-robin-assigns host_uuid to each decky,
3. POSTs the sharded DecnetConfig to /swarm/deploy,
4. renders per-worker pass/fail in a results table.
Exits non-zero if no workers exist or any worker's dispatch failed.
The forwarder module existed but had no runner — closes that gap so the
worker-side process can actually be launched and runs isolated from the
agent (asyncio.run + SIGTERM/SIGINT → stop_event).
Guards: refuses to start without a worker cert bundle or a resolvable
master host ($DECNET_SWARM_MASTER_HOST or --master-host).
Worker-side log_forwarder tails the local RFC 5424 log file and ships
each line as an octet-counted frame to the master over mTLS. Offset is
persisted in a tiny local SQLite so master outages never cause loss or
duplication — reconnect resumes from the exact byte where the previous
session left off. Impostor workers (cert not signed by DECNET CA) are
rejected at TLS handshake.
Master-side log_listener terminates mTLS on 0.0.0.0:6514, validates the
client cert, extracts the peer CN as authoritative worker provenance,
and appends each frame to the master's ingest log files. Attacker-
controlled syslog HOSTNAME field is ignored — the CA-controlled CN is
the only source of provenance.
7 tests added covering framing codec, offset persistence across
reopens, end-to-end mTLS delivery, crash-resilience (offset survives
restart, no duplicate shipping), and impostor-CA rejection.
DECNET_SWARM_SYSLOG_PORT / DECNET_SWARM_MASTER_HOST env bindings
added.
_schemas.py was a local exception to the codebase convention. The rest
of the app keeps all API request/response DTOs in decnet/web/db/models.py
alongside UserResponse, DeployIniRequest, etc. — the swarm endpoints now
follow the same convention (SwarmEnrollRequest, SwarmHostView, etc).
Deletes decnet/web/router/swarm/_schemas.py.
Splits the three grouped router files into eight api_<verb>_<resource>.py
modules under decnet/web/router/swarm/ to match the convention used by
router/fleet/ and router/config/. Shared request/response models live in
_schemas.py. Keeps each endpoint easy to locate and modify without
stepping on siblings.
Adds decnet/web/swarm_api.py as an independent FastAPI app with routers
for host enrollment, deployment dispatch (sharding DecnetConfig across
enrolled workers via AgentClient), and active health probing. Runs as
its own uvicorn subprocess via 'decnet swarmctl', mirroring the isolation
pattern used by 'decnet api'. Also wires up 'decnet agent' CLI entry for
the worker side.
29 tests added under tests/swarm/test_swarm_api.py cover enrollment
(including bundle generation + duplicate rejection), host CRUD, sharding
correctness, non-swarm-mode rejection, teardown, and health probes with
a stubbed AgentClient.
- decnet.models.DeckyConfig grows an optional 'host_uuid' (the SwarmHost
that runs this decky). Defaults to None so legacy unihost state files
deserialize unchanged.
- decnet.agent.executor: replace non-existent config.name references
with config.mode / config.interface in logs and status payload.
- tests/swarm/test_state_schema.py covers legacy-dict roundtrip, field
default, and swarm-mode assignments.
decnet.swarm.client exposes:
- MasterIdentity / ensure_master_identity(): the master's own CA-signed
client bundle, issued once into ~/.decnet/ca/master/.
- AgentClient: async-context httpx wrapper that talks to a worker agent
over mTLS. health/status/deploy/teardown methods mirror the agent API.
SSL context is built from a bare ssl.SSLContext(PROTOCOL_TLS_CLIENT)
instead of httpx.create_ssl_context — the latter layers on default-CA
and purpose logic that broke private-CA mTLS. Server cert is pinned by
CA + chain, not DNS (workers enroll with arbitrary SANs).
tests/swarm/test_client_agent_roundtrip.py spins uvicorn in-process
with real certs on disk and verifies:
- A CA-signed master client passes health + status calls.
- An impostor whose cert comes from a different CA cannot connect.
Worker agent (decnet.agent):
- mTLS FastAPI service exposing /deploy, /teardown, /status, /health,
/mutate. uvicorn enforces CERT_REQUIRED with the DECNET CA pinned.
- executor.py offloads the blocking deployer onto asyncio.to_thread so
the event loop stays responsive.
- server.py refuses to start without an enrolled bundle in
~/.decnet/agent/ — unauthenticated agents are not a supported mode.
- docs/openapi disabled on the agent — narrow attack surface.
tests/test_base_repo.py: DummyRepo was missing get_attacker_artifacts
(pre-existing abstractmethod) and so could not be instantiated. Added
the stub + coverage for the new swarm CRUD surface on BaseRepository.
decnet.swarm.pki provides:
- generate_ca() / ensure_ca() — self-signed root, PKCS8 PEM, 4096-bit.
- issue_worker_cert() — per-worker keypair + cert signed by the CA with
serverAuth + clientAuth EKU so the same identity backs the agent's
HTTPS endpoint AND the syslog-over-TLS upstream.
- write_worker_bundle() / load_worker_bundle() — persist with 0600 on
private keys.
- fingerprint() — SHA-256 DER hex for master-side pinning.
tests/swarm/test_pki.py covers:
- CA idempotency on disk.
- Signed chain validates against CA subject.
- SAN population (DNS + IP).
- Bundle roundtrip with 0600 key perms.
- End-to-end mTLS handshake between two CA-issued peers.
- Cross-CA client rejection (handshake fails).
Introduces the master-side persistence layer for swarm mode:
- SwarmHost: enrolled worker metadata, cert fingerprint, heartbeat.
- DeckyShard: per-decky host assignment, state, last error.
Repo methods are added as default-raising on BaseRepository so unihost
deployments are untouched; SQLModelRepository implements them (shared
between the sqlite and mysql subclasses per the existing pattern).
decnet.collector.log / decnet.system.log and the *.db-shm / *.db-wal
sidecars produced by the sqlite WAL journal were slipping through the
existing rules. Extend the patterns so runtime state doesn't show up
in git status.
Reference template for .env / .env.local showing every variable that
decnet/env.py consumes, with short rationale per section (system
logging, embedded workers, profiling, API server, …). Copy to .env
and fill in secrets; .env itself stays gitignored.
Exercises the JSON → syslog formatter end to end: flat fields ride as
SD params, bulky nested metadata collapses into the meta_json_b64 blob,
and the event_type / hostname / service mapping lands in the right
RFC 5424 header slots.
Frontend now handles syslog lines from producers that don't use
structured-data (notably the SSH PROMPT_COMMAND hook, which emits
'CMD uid=0 user=root src=IP pwd=… cmd=…' as a plain logger message).
A new parseEventBody utility splits the body into head + key/value
pairs and preserves the final value verbatim so commands stay intact.
Dashboard and LiveLogs use this parser to render consistent pills
whether the structure came from SD params or from the MSG body.
The host-side sniffer interface depends on the deploy's driver choice
(--ipvlan flag). Instead of hardcoding HOST_MACVLAN_IFACE, probe both
names and pick whichever exists; warn and disable cleanly if neither
is present. Explicit DECNET_SNIFFER_IFACE still wins.
- Relaxed RFC 5424 regex to accept either NILVALUE or a numeric PROCID;
sshd / sudo go through rsyslog with their real PID, while
syslog_bridge emitters keep using '-'.
- Added a fallback pass that scans the MSG body for IP-shaped
key=value tokens. This rescues attacker attribution for plain logger
callers like the SSH PROMPT_COMMAND shim, which emits
'CMD … src=IP …' without SD-element params.
Each honeypot container now carries its own copy of the shared RFC 5424
formatter. Services that previously rolled their own ad-hoc syslog
lines can now import syslog_line / write_syslog_file for a consistent
SD-element format that the collector already knows how to parse.
Adds the server-side wiring and frontend UI to surface files captured
by the SSH honeypot for a given attacker.
- New repository method get_attacker_artifacts (abstract + SQLModel
impl) that joins the attacker's IP to `file_captured` log rows.
- New route GET /attackers/{uuid}/artifacts.
- New router /artifacts/{decky}/{service}/{stored_as} that streams a
quarantined file back to an authenticated viewer.
- AttackerDetail grows an ArtifactDrawer panel with per-file metadata
(sha256, size, orig_path) and a download action.
- ssh service fragment now sets NODE_NAME=decky_name so logs and the
host-side artifacts bind-mount share the same decky identifier.
The /opt/emit_capture.py, /opt/syslog_bridge.py, and
/usr/libexec/udev/journal-relay files were plaintext and world-readable
to any attacker root-shelled into the SSH honeypot — revealing the full
capture logic on a single cat.
Pack all three into /entrypoint.sh as XOR+gzip+base64 blobs at build
time (_build_stealth.py), then decode in-memory at container start and
exec the capture loop from a bash -c string. No .py files under /opt,
no journal-relay file under /usr/libexec/udev, no argv_zap name
anywhere. The LD_PRELOAD shim is installed as
/usr/lib/x86_64-linux-gnu/libudev-shared.so.1 — sits next to the real
libudev.so.1 and blends into the multiarch layout.
A 1-byte random XOR key is chosen at image build so a bare
'base64 -d | gunzip' probe on the visible entrypoint returns binary
noise instead of readable Python.
Docker-dependent tests live under tests/docker/ behind a new 'docker'
pytest marker (excluded from the default run, same pattern as fuzz /
live / bench).
The named pipe at /run/systemd/journal/syslog-relay had two problems
beyond its argv leak: any root-in-container process could (a) `cat`
the pipe and watch the live SIEM feed, and (b) write to it and inject
forged log lines. Since an attacker with a shell is already root
inside the honeypot, file permissions can't fix it.
Point rsyslog's auth/user actions directly at /proc/1/fd/1 — the
container-stdout fd Docker attached to PID 1 — and delete the
mkfifo + cat relay from the entrypoint. No pipe on disk, nothing to
read, nothing to inject, and one fewer cloaked process in `ps`.
Two leaks remained after the inotifywait argv fix:
1. The bash running journal-relay showed its argv[1] (the script path)
in /proc/PID/cmdline, producing a line like
'journal-relay /usr/libexec/udev/journal-relay'
Apply argv_zap.so to that bash too.
2. argv_zap previously hardcoded PR_SET_NAME to 'kmsg-watch', which was
wrong for any caller other than inotifywait. The comm name now comes
from ARGV_ZAP_COMM so each caller can pick its own (kmsg-watch for
inotifywait, journal-relay for the watcher bash).
3. The capture.sh header started with 'SSH honeypot file-catcher' —
fatal if an attacker runs 'cat' on it. Rewritten as a plausible
systemd-journal relay helper; stray 'attacker' / 'honeypot' words
in mid-script comments stripped too.
A lived-in Linux box ships with iputils-ping, ca-certificates, and nmap
available. Their absence is a cheap tell, and they're handy for letting
the attacker move laterally in ways we want to observe. iproute2 (ip a)
was already installed for attribution — noted here for completeness.
The kmsg-watch (inotifywait) process was the last honest giveaway in
`ps aux` — its watch paths and event flags betrayed the honeypot. The
argv_zap.so shim hooks __libc_start_main, heap-copies argv for the real
main, then memsets the contiguous argv[1..] region to NUL so the kernel's
cmdline reader returns just argv[0].
gcc is installed and purged in the same Docker layer to keep the image
slim. The shim also calls prctl(PR_SET_NAME) so /proc/self/comm mirrors
the argv[0] disguise.
exec -a replaces argv[0] so ps shows 'journal-relay /usr/libexec/udev/journal-relay'
instead of '/bin/bash /usr/libexec/udev/journal-relay' — no interpreter
hint on the watcher process.
inotify | while spawns a subshell for the tail of the pipeline, so
two bash processes (the script itself and the while-loop subshell)
showed up under /usr/libexec/udev/journal-relay in ps aux. Enable
lastpipe so the while loop runs in the main shell — ps now shows
one bash plus the inotify child, matching a simple udev helper.
Rename the container-side logging module decnet_logging → syslog_bridge
(canonical at templates/syslog_bridge.py, synced into each template by
the deployer). Drop the stale per-template copies; setuptools find was
picking them up anyway. Swap useradd/USER/chown "decnet" for "logrelay"
so no obvious token appears in the rendered container image.
Apply the same cloaking pattern to the telnet template that SSH got:
syslog pipe moves to /run/systemd/journal/syslog-relay and the relay
is cat'd via exec -a "systemd-journal-fwd". rsyslog.d conf rename
99-decnet.conf → 50-journal-forward.conf. SSH capture script:
/var/decnet/captured → /var/lib/systemd/coredump (real systemd path),
logger tag decnet-capture → systemd-journal. Compose volume updated
to match the new in-container quarantine path.
SD element ID shifts decnet@55555 → relay@55555; synced across
collector, parser, sniffer, prober, formatter, tests, and docs so the
host-side pipeline still matches what containers emit.
Rename the rsyslog→stdout pipe from /var/run/decnet-logs (dead giveaway)
to /run/systemd/journal/syslog-relay, and launch the relay via
exec -a "systemd-journal-fwd" so ps shows a plausible systemd forwarder
instead of a bare cat. Casual ps/ls inspection now shows nothing
with "decnet" in the name.
Old ps output was a dead giveaway: two "decnet-capture" bash procs
and a raw "inotifywait". Install script at /usr/libexec/udev/journal-relay
and invoke inotifywait through a /usr/libexec/udev/kmsg-watch symlink so
both now render as plausible udev/journal helpers under casual inspection.
fuser and /proc fd walks race scp/wget/sftp — by close_write the writer
has already closed the fd, so pid-chain attribution always resolved to
unknown for non-interactive drops. Fall back to the ss snapshot: one
established session → ss-only, multiple → ss-ambiguous (still record
src_ip from the first, analysts cross-check concurrent_sessions).
inotifywait watches writable paths in the SSH decky and mirrors any
file close_write/moved_to into a per-decky host-mounted quarantine dir.
Each artifact carries a .meta.json with attacker attribution resolved
by walking the writer PID's PPid chain to the sshd session leader,
then cross-referencing ss and utmp for source IP/user/login time.
Also emits an RFC 5424 syslog line per capture for SIEM correlation.
Commit-by-commit evidence of the perf work: each CSV is the raw
Locust output for the commit hash in its filename, plus the four
fb69a06 variants (single worker, tracing on/off, single-core pinned,
12 workers) referenced in the README baseline table.
Some pyinstrument frame trees contain branches where an identifier is
missing (typically at the very top or with certain async boundaries),
which crashed the aggregator with a KeyError mid-run. Short-circuit
on None frames and missing identifiers so a single ugly HTML no
longer kills the summary of the other few hundred.
asyncmy needs cryptography for caching_sha2_password (the MySQL 8
default auth plugin). Without it, connection handshake fails the
moment the server negotiates the modern plugin.
Capture Locust numbers from the fb69a06 branch across five
configurations so future regressions have something to measure against.
- 500u tracing-on single-worker: ~960 RPS / p99 2.9 s
- 1500u tracing-on single-worker: ~880 RPS / p99 9.5 s
- 1500u tracing-off single-worker: ~990 RPS / p99 8.4 s
- 1500u tracing-off pinned to one core: ~46 RPS / p99 122 s
- 1500u tracing-off 12 workers: ~1585 RPS / p99 4.2 s
Also note MySQL max_connections math (pool_size * max_overflow *
workers = 720) to explain why the default 151 needs bumping, and the
Python 3.14 GC segfault so nobody repeats that mistake.
Previous attempt (shield + sync invalidate fallback) didn't work
because shield only protects against cancellation from *other* tasks.
When the caller task itself is cancelled mid-query, its next await
re-raises CancelledError as soon as the shielded coroutine yields —
rollback inside session.close() never completes, the aiomysql
connection is orphaned, and the pool logs 'non-checked-in connection'
when GC finally reaches it.
Hand exception-path cleanup to loop.create_task() so the new task
isn't subject to the caller's pending cancellation. close() (and the
invalidate() fallback for a dead connection) runs to completion.
Success path is unchanged — still awaits close() inline so callers
see commit visibility and pool release before proceeding.
Under high-concurrency MySQL load, uvicorn cancels request tasks when
clients disconnect. If cancellation lands mid-query, session.close()
tries to ROLLBACK on a connection that aiomysql has already marked as
closed — raising InterfaceError("Cancelled during execution") and
leaving the connection checked-out until GC, which the pool then
warns about as a 'non-checked-in connection'.
The old fallback tried sync.rollback() + sync.close(), but those still
go through the async driver and fail the same way on a dead connection.
Replace them with session.sync_session.invalidate(), which just flips
the pool's internal record — no I/O, so it can't be cancelled — and
tells the pool to drop the connection immediately instead of waiting
for garbage collection.
Locust @task(2) hammers /auth/login in steady state on top of the
on_start burst. After caching the uuid-keyed user lookup and every
other read endpoint, login alone accounted for 47% of total
_execute at 500c/u — pure DB queueing on SELECT users WHERE
username=?.
5s TTL, positive hits only (misses bypass so a freshly-created
user can log in immediately). Password verify still runs against
the cached hash, so security is unchanged — the only staleness
window is: a changed password accepts the old password for up to
5s until invalidate_user_cache fires (it's called on every write).
The per-request SELECT users WHERE uuid=? in require_role was the
hidden tax behind every authed endpoint — it kept _execute at ~60%
across the profile even after the page caches landed. Even /health
(with its DB and Docker probes cached) was still 52% _execute from
this one query.
- dependencies.py: 10s TTL cache on get_user_by_uuid, well below JWT
expiry. invalidate_user_cache(uuid) is called on password change,
role change, and user delete.
- api_get_config.py: 5s TTL cache on the admin branch's list_users()
(previously fetched every /config call). Invalidated on user
create/update/delete.
- api_change_pass.py + api_manage_users.py: invalidation hooks on
all user-mutating endpoints.
Round-2 follow-up: profile at 500c/u showed _execute still dominating
the uncached read endpoints (/bounty 76%, /logs/histogram 73%,
/deckies 56%). Same router-level TTL pattern as /stats — 5s window,
asyncio.Lock to collapse concurrent calls into one DB hit.
- /bounty: cache default unfiltered page (limit=50, offset=0,
bounty_type=None, search=None). Filtered requests bypass.
- /logs/histogram: cache default (interval_minutes=15, no filters).
Filtered / non-default interval requests bypass.
- /deckies: cache full response (endpoint takes no params).
- /config: bump _STATE_TTL from 1.0 to 5.0 — admin writes are rare,
1s was too short for bursts to coalesce at high concurrency.
SQLite is a local file — a SELECT 1 per session checkout is pure
overhead. Env var DECNET_DB_POOL_PRE_PING stays for anyone running
on a network-mounted volume. MySQL backend keeps its current default.
Popen moved inside the try so a missing uvicorn falls through to the
existing error message instead of crashing the CLI. test_cli was still
patching the old subprocess.run entrypoint; switched both api command
tests to patch subprocess.Popen / os.killpg to match the current path.
Every /stats call ran SELECT count(*) FROM logs + SELECT count(DISTINCT
attacker_ip) FROM logs; every /logs and /attackers call ran an
unfiltered count for the paginator. At 500 concurrent users these
serialize through aiosqlite's worker threads and dominate wall time.
Cache at the router layer (repo stays dialect-agnostic):
- /stats response: 5s TTL
- /logs total (only when no filters): 2s TTL
- /attackers total (only when no filters): 2s TTL
Filtered paths bypass the cache. Pattern reused from api_get_config
and api_get_health (asyncio.Lock + time.monotonic window + lazy lock).
require_role._check previously chained from get_current_user, which
already loaded the user — then looked it up again. Inline the decode +
single user fetch + must_change_password + role check so every
authenticated request costs one SELECT users WHERE uuid=? instead of
two.
Only database, docker, and ingestion_worker now count as critical
(→ 503 unhealthy). attacker/sniffer/collector failures drop overall
status to degraded (still 200) so the dashboard doesn't panic when a
non-essential worker isn't running.
The ingester now accumulates up to DECNET_BATCH_SIZE rows (default 100)
or DECNET_BATCH_MAX_WAIT_MS (default 250ms) before flushing through
repo.add_logs — one transaction, one COMMIT per batch instead of per
row. Under attacker traffic this collapses N commits into ⌈N/100⌉ and
takes most of the SQLite writer-lock contention off the hot path.
Flush semantics are cancel-safe: _position only advances after a batch
commits successfully, and the flush helper bails without touching the
DB if the enclosing task is being cancelled (lifespan teardown).
Un-flushed lines stay in the file and are re-read on next startup.
Tests updated to assert on add_logs (bulk) instead of the per-row
add_log that the ingester no longer uses, plus a new test that 250
lines flush in ≤5 calls.
Adds BaseRepository.add_logs (default: loops add_log for backwards
compatibility) and a real single-session/single-commit implementation
on SQLModelRepository. Introduces DECNET_BATCH_SIZE (default 100) and
DECNET_BATCH_MAX_WAIT_MS (default 250) so the ingester can flush on
either a size or a time bound when it adopts the new method.
Ingester wiring is deferred to a later pass — the single-log path was
deadlocking tests when flushed during lifespan teardown, so this change
ships the DB primitive alone.
A module-level asyncio.Lock binds to the loop it was first awaited on.
Under pytest-anyio (and xdist) each test spins up a new loop; any later
test that hit /health or /config would wait on a lock owned by a dead
loop and the whole worker would hang.
Create the lock on first use and drop it in the test-reset helpers so a
fresh loop always gets a fresh lock.
Under CPU saturation the sync docker.from_env()/ping() calls could miss
their socket timeout, cache _docker_healthy=False, and return 503 for
the full 5s TTL window. Both calls now run on a thread so the event
loop keeps serving other requests while Docker is being probed.
With --workers > 1, SIGINT from the terminal raced uvicorn's supervisor:
some workers got signaled directly, the supervisor respawned them, and
the result behaved like a forkbomb. Start uvicorn in its own session and
signal the whole process group (SIGTERM → 10s grace → SIGKILL) when we
catch KeyboardInterrupt.
Forwards straight to uvicorn's --workers. Default stays at 1 so the
single-worker efficiency direction is preserved; raising it is available
for threat-actor load scenarios where the honeypot needs to soak real
attack traffic without queueing on one event loop.
Previously every user did login → change-pass → re-login in on_start
regardless of whether the server actually required a password change.
With bcrypt at ~250ms/call that's 3 bcrypt-bound requests per user.
At 2500 users the on_start queue was ~10k bcrypt ops — users never
escaped warmup, so @task endpoints never fired.
Login already returns must_change_password; only run the change-pass
+ re-login dance when the server says we have to. Cuts on_start from
3 requests to 1 for every user after the first DB initialization.
stdlib json was FastAPI's default. Every response body, every SSE frame,
and every add_log/state/payload write paid the stdlib encode cost.
- pyproject.toml: add orjson>=3.10 as a core dep.
- decnet/web/api.py: default_response_class=ORJSONResponse on the
FastAPI app, so every endpoint return goes through orjson without
touching call sites. Explicit JSONResponse sites in the validation
exception handlers migrated to ORJSONResponse for consistency.
- health endpoint's explicit JSONResponse → ORJSONResponse.
- SSE stream (api_stream_events.py): 6 json.dumps call sites →
orjson.dumps(...).decode() — the per-event frames that fire on every
sse tick.
- sqlmodel_repo.py: encode sites on the log-insert path switched to
orjson (fields, payload, state value). Parser sites (json.loads)
left as-is for now — not on the measured hot path.
Locust hit /health and /config on every @task(3), so each request was
firing repo.get_total_logs() and two repo.get_state() calls against
aiosqlite — filling the driver queue for data that changes on the order
of seconds, not milliseconds.
Both caches follow the shape already used by the existing Docker cache:
- asyncio.Lock with double-checked TTL so concurrent callers collapse
into one DB hit per 1s window.
- _reset_* helpers called from tests/api/conftest.py::setup_db so the
module-level cache can't leak across tests.
tests/test_health_config_cache.py asserts 50 concurrent callers
produce exactly 1 repo call, and the cache expires after TTL.
Creating a new docker.from_env() client per /health request opened a
fresh unix-socket connection each time. Under load that's wasteful and
hammers dockerd.
Keep a module-level client + last-check timestamp; actually ping every
5 seconds, return cached state in between. Reset helper provided for
tests.
- aiomysql → asyncmy on both sides of the URL/import (faster, maintained).
- Pool sizing now reads DECNET_DB_POOL_SIZE / MAX_OVERFLOW / RECYCLE /
PRE_PING for both SQLite and MySQL engines so stress runs can bump
without code edits.
- MySQL initialize() now wraps schema DDL in a GET_LOCK advisory lock so
concurrent uvicorn workers racing create_all() don't hit 'Table was
skipped since its definition is being modified by concurrent DDL'.
- sqlite & mysql repo get_log_histogram use the shared _session() helper
instead of session_factory() for consistency with the rest of the repo.
- SSE stream_events docstring updated to asyncmy.
verify_password / get_password_hash are CPU-bound and take ~250ms each
at rounds=12. Called directly from async endpoints, they stall every
other coroutine for that window — the single biggest single-worker
bottleneck on the login path.
Adds averify_password / ahash_password that wrap the sync versions in
asyncio.to_thread. Sync versions stay put because _ensure_admin_user and
tests still use them.
5 call sites updated: login, change-password, create-user, reset-password.
tests/test_auth_async.py asserts parallel averify runs concurrently (~1x
of a single verify, not 2x).
_ensure_admin_user was strict insert-if-missing: once a stale hash landed
in decnet.db (e.g. from a deploy that used a different DECNET_ADMIN_PASSWORD),
login silently 401'd because changing the env var later had no effect.
Now on startup: if the admin still has must_change_password=True (they
never finalized their own password), re-sync the hash from the current
env var. Once the admin sets a real password, we leave it alone.
Found via locustfile.py login storm — see tests/test_admin_seed.py.
Note: this commit also bundles uncommitted pool-management work already
present in sqlmodel_repo.py from prior sessions.
Parses every HTML in profiles/, reattributes [self]/[await] synthetic
leaves to their parent function, and reports per-endpoint wall-time
(mean/p50/p95/max) plus top hot functions by cumulative self-time.
Makes post-locust profile dirs actually readable — otherwise they're
just a pile of hundred-plus HTML files.
When decnet.system.log is root-owned (e.g. created by a pre-fix 'sudo
decnet deploy') and a subsequent non-root process tries to log, the
InodeAwareRotatingFileHandler raised PermissionError out of emit(),
which propagated up through logger.debug/info and killed the collector's
log stream loop ('log stream ended ... reason=[Errno 13]').
Now matches stdlib behaviour: wrap _open() in try/except OSError and
defer to handleError() on failure. Adds a regression test.
Also: scripts/profile/view.sh 'pyinstrument' keyword was matching
memray-flamegraph-*.html files. Exclude the memray-* prefix.
Reads the memray usage CSV and emits a verdict based on tail-drop-from-
peak: CLIMB-AND-DROP, MOSTLY-RELEASED, or SUSTAINED-AT-PEAK. Deliberately
ignores net-growth-vs-baseline since any active workload grows vs. a cold
interpreter — that metric is misleading as a leak signal.
Mirrors the inode-check fix from 935a9a5 (collector worker) for the
stdlib-handler-based log paths. Both decnet.system.log (config.py) and
decnet.log (logging/file_handler.py) now use a subclass that stats the
target path before each emit and reopens on inode/device mismatch —
matching the behavior of stdlib WatchedFileHandler while preserving
size-based rotation.
Previously: rm decnet.system.log → handler kept writing to the orphaned
inode until maxBytes triggered; all lines between were lost.
'sudo decnet deploy' needs root for MACVLAN, but the log files it creates
(decnet.log and decnet.system.log) end up owned by root. A subsequent
non-root 'decnet api' then crashes on PermissionError appending to them.
New decnet.privdrop helper reads SUDO_UID/SUDO_GID and chowns files/dirs
back to the invoking user. Best-effort: no-op when not root, not under
sudo, path missing, or chown fails. Applied at both log-file creation
sites (config.py system log, logging/file_handler.py syslog file).
The API's lifespan unconditionally spawned a MACVLAN sniffer task, which
duplicated the standalone 'decnet sniffer --daemon' process that
'decnet deploy' always starts — causing two workers to sniff the same
interface, double events, and wasted CPU.
Mirror the existing DECNET_EMBED_PROFILER pattern: sniffer is OFF by
default, opt in explicitly. Static regression tests guard against
accidental removal of the gate.
Without it, 'Total number of frames seen: 0' in memray stats and flamegraphs
render empty / C-only. Also added --follow-fork so uvicorn workers spawned
as child processes are tracked.
Dispatches by extension: .prof -> snakeviz, memray .bin -> memray flamegraph
(overridable via VIEW=table|tree|stats|summary|leaks), .svg/.html -> xdg-open.
Positional arg can be a file path or a type keyword (cprofile, memray, pyspy,
pyinstrument).
Root cause of 'No python processes found in process <pid>': py-spy needs
per-release ABI knowledge and 0.4.1 (latest PyPI) predates 3.14. Wrapper
now detects the interpreter and points users at pyinstrument/memray/cProfile.
The builder in decnet/web/db/mysql/database.py emits 'mysql+asyncmy://' URLs
(asyncmy is the declared dep in pyproject.toml). Tests were stale from a
prior aiomysql era.
New `profile` optional-deps group, opt-in Pyinstrument ASGI middleware
gated by DECNET_PROFILE_REQUESTS, bench marker + tests/perf/ micro-benchmarks
for repository hot paths, and scripts/profile/ helpers for py-spy/cProfile/memray.
Root cause: test_schemathesis.py mutates decnet.web.auth.SECRET_KEY at
module-level import time, poisoning JWT verification for all other tests
in the same process — even when fuzz tests are deselected.
- Add pytest_ignore_collect hook in tests/api/conftest.py to skip
collecting test_schemathesis.py unless -m fuzz is selected
- Add --dist loadscope to addopts so xdist groups by module (protects
module-scoped fixtures in live tests)
- Remove now-unnecessary xdist_group markers from live test classes
- Add 403 response to all RBAC-gated endpoints (schemathesis UndefinedStatusCode)
- Add 400 response to all endpoints accepting JSON bodies (malformed input)
- Add required 'title' field to schemathesis.toml for schemathesis 4.15+
- Add xdist_group markers to live tests with module-scoped fixtures to
prevent xdist from distributing them across workers (fixture isolation)
Extends tracing to every remaining module: all 23 API route handlers,
correlation engine, sniffer (fingerprint/p0f/syslog), prober (jarm/hassh/tcpfp),
profiler behavioral analysis, logging subsystem, engine, and mutator.
Bridges the ingester→SSE trace gap by persisting trace_id/span_id columns on
the logs table and creating OTEL span links in the SSE endpoint. Adds log-trace
correlation via _TraceContextFilter injecting otel_trace_id into Python LogRecords.
Includes development/docs/TRACING.md with full span reference (76 spans),
pipeline propagation architecture, quick start guide, and troubleshooting.
Collector now creates a span per event and injects W3C trace context
into JSON records. Ingester extracts that context and creates child
spans, connecting the full event journey: collector -> ingester ->
db.add_log + extract_bounty -> db.add_bounty.
Profiler now creates per-IP spans inside update_profiles with rich
attributes (event_count, is_traversal, bounty_count, command_count).
Traces in Jaeger now show the complete execution map from capture
through ingestion and profiling.
Replace brittle explicit method-by-method proxy with __getattr__-based
dynamic proxy that forwards all args/kwargs to the inner repo. Fixes
TypeError on get_logs_after_id() where concrete repo accepts extra
kwargs beyond the ABC signature.
Pin DECNET_DEVELOPER_TRACING=false in conftest.py so .env.local
settings don't leak into the test suite.
Gated by DECNET_DEVELOPER_TRACING env var (default off, zero overhead).
When enabled, traces flow through FastAPI routes, background workers
(collector, ingester, profiler, sniffer, prober), engine/mutator
operations, and all DB calls via TracedRepository proxy.
Includes Jaeger docker-compose for local dev and 18 unit tests.
resp.read(4096) blocks until 4096 bytes accumulate, which stalls SSE
events (~100-500 bytes each) in the proxy buffer indefinitely. Switch
to read1() which returns bytes immediately available without waiting
for more. Also disable the 120s socket timeout for SSE connections.
The collector spawned one permanent thread per Docker container via
asyncio.to_thread(), saturating the default asyncio executor. This
starved short-lived to_thread(load_state) calls in get_deckies() and
get_stats_summary(), causing the SSE stream and deckies endpoints to
hang indefinitely while other DB-only endpoints worked fine.
Give the collector and sniffer their own ThreadPoolExecutor so they
never compete with the default pool.
decnet deploy spawns a standalone profiler daemon AND api.py was also starting
attacker_profile_worker as an asyncio task inside the web server. Both instances
shared the same attacker_worker_cursor key in the state table, causing a race
where one instance could skip events already claimed by the other or overwrite
the cursor mid-batch.
Default is now OFF (embedded profiler disabled). The standalone daemon started
by decnet deploy is the single authoritative instance. Set DECNET_EMBED_PROFILER=true
only when running decnet api in isolation without a full deploy.
The active prober emits tcpfp_fingerprint events with TTL, window, MSS etc.
from the attacker's SYN-ACK. These were invisible to the behavioral profiler
for two reasons:
1. target_ip (prober's field name for attacker IP) was not in _IP_FIELDS in
collector/worker.py or correlation/parser.py, so the profiler re-parsed
raw_lines and got attacker_ip=None, never attributing prober events to
the attacker profile.
2. sniffer_rollup only handled tcp_syn_fingerprint (passive sniffer) and
ignored tcpfp_fingerprint (active prober). Prober events use different
field names: window_size/window_scale/sack_ok vs window/wscale/has_sack.
Changes:
- Add target_ip to _IP_FIELDS in collector and parser
- Add _PROBER_TCPFP_EVENT and _INITIAL_TTL table to behavioral.py
- sniffer_rollup now processes tcpfp_fingerprint: maps field names, derives
OS from TTL via _os_from_ttl, computes hop_distance = initial_ttl - observed
- Expand prober DEFAULT_TCPFP_PORTS to [22,80,443,8080,8443,445,3389] for
better SYN-ACK coverage on attacker machines
- Add 4 tests covering prober OS detection, hop distance, and field mapping
Templates for http, https, k8s, and docker_api log the client IP as
remote_addr (Flask's request.remote_addr) instead of src_ip. The collector
and correlation parser only checked src_ip/src/client_ip/remote_ip/ip, so
every request event from those services was stored with attacker_ip="Unknown"
and never associated with any attacker profile.
Adding remote_addr to _IP_FIELDS in both collector/worker.py and
correlation/parser.py fixes attribution. The profiler cursor was also reset
to 0 so the worker performs a cold rebuild and re-ingests existing events with
the corrected field mapping.
templates/decnet_logging.py calls str(v) on all SD-PARAM values, turning a
headers dict into Python repr ('{'User-Agent': ...}') rather than JSON.
detect_tools_from_headers() called json.loads() on that string and silently
swallowed the error, returning [] for every HTTP event. Same bug prevented
the ingester from extracting User-Agent bounty fingerprints.
- templates/http/server.py: wrap headers dict in json.dumps() before passing
to syslog_line so the value is a valid JSON string in the syslog record
- behavioral.py: add ast.literal_eval fallback for existing DB rows that were
stored with the old Python repr format
- ingester.py: parse headers as JSON string in _extract_bounty so User-Agent
fingerprints are stored correctly going forward
- tests: add test_json_string_headers and test_python_repr_headers_fallback
to exercise both formats in detect_tools_from_headers
Replaces the single persistent open() with inode-based reopen logic.
If decnet.log or decnet.json is deleted or renamed by logrotate, the
next write detects the stale inode, closes the old handle, and creates
a fresh file — preventing silent data loss to orphaned inodes.
- Ingester now loads byte-offset from DB on startup (key: ingest_worker_position)
and saves it after each batch — prevents full re-read on every API restart
- On file truncation/rotation the saved offset is reset to 0
- Profiler worker now loads last_log_id from DB on startup — every restart
becomes an incremental update instead of a full cold rebuild
- Updated all affected tests to mock get_state/set_state; added new tests
covering position restore, set_state call, truncation reset, and cursor
restore/cold-start paths
Cold start fetched all logs in one bulk query then processed them in a tight
synchronous loop with no yields, blocking the asyncio event loop for seconds
on datasets of 30K+ rows. This stalled every concurrent await — including the
SSE stream generator's initial DB calls — causing the dashboard to show
INITIALIZING SENSORS indefinitely.
Changes:
- Drop _cold_start() and get_all_logs_raw(); uninitialized state now runs the
same cursor loop as incremental, starting from last_log_id=0
- Yield to the event loop after every _BATCH_SIZE rows (asyncio.sleep(0))
- Add SSE keepalive comment as first yield so the connection flushes before
any DB work begins
- Add Cache-Control/X-Accel-Buffering headers to StreamingResponse
Existing MySQL databases hit a DataError when the commands/fingerprints
JSON blobs exceed 64 KiB (TEXT limit). _BIG_TEXT emits MEDIUMTEXT only
at CREATE TABLE time; create_all() is a no-op on existing columns.
Add MySQLRepository._migrate_column_types() that queries
information_schema and issues ALTER TABLE … MODIFY COLUMN … MEDIUMTEXT
for the five affected columns (commands, fingerprints, services, deckies,
state.value) whenever they are still TEXT. Called from an overridden
initialize() after _migrate_attackers_table() and before create_all().
Add tests/test_mysql_migration.py covering: ALTER issued for TEXT columns,
no-op for already-MEDIUMTEXT, idempotency, DEFAULT clause correctness,
and initialize() call order.
- test_mysql_backend_live.py: live integration tests for MySQL connections
- test_mysql_histogram_sql.py: dialect-specific histogram query tests
- test_mysql_url_builder.py: MySQL connection string construction
- mysql_spinup.sh: Docker spinup script for local MySQL testing
- templates/sniffer/decnet_logging.py: add logging configuration for sniffer integration
- templates/ssh/decnet_logging.py: add SSH service logging template
- development/DEVELOPMENT.md: document new MySQL backend, p0f, profiler, config API features
- pyproject.toml: update dependencies for MySQL, p0f, profiler functionality
- decnet/profiler/: analyze attacker behavior timings, command sequences, service probing patterns
- Enables detection of coordinated attacks vs random scanning
- Feeds into attacker scoring and risk assessment
- Implement MySQLRepository extending BaseRepository
- Add SQLAlchemy/SQLModel ORM abstraction layer (sqlmodel_repo.py)
- Support connection pooling and tuning via DECNET_DB_URL env var
- Cross-compatible with SQLite backend via factory pattern
- Prepared for production deployment with MySQL SIEM/ELK integration
- Add @require_role() decorators to all GET/POST/PUT endpoints
- Centralize role-based access control per memory: RBAC null-role bug required server-side gating
- Admin (manage_admins), Editor (write ops), Viewer (read ops), Public endpoints
- Removes client-side role checks as per memory: server-side UI gating is mandatory
- Refactor deploy command to support service randomization and selective service deployment
- Add --services flag to filter deployed services by name
- Improve status and teardown command output formatting
- Update help text for clarity
- Extract dialect-agnostic methods to BaseRepository
- Keep only SQLite-specific SQL and initialization in SQLiteRepository
- Reduces duplication for upcoming MySQL backend
- Maintains 100% backward compatibility
- Add `get_repository()` factory function to select DB implementation at runtime via DECNET_DB_TYPE env var
- Extract BaseRepository abstract interface from SQLiteRepository
- Update dependencies to use factory-based repository injection
- Add DECNET_DB_TYPE env var support (defaults to sqlite)
- Refactor models and repository base class for cross-dialect compatibility
Connection-lifecycle events (connect, disconnect, accept, close) fire once
per TCP connection. During a portscan or credential-stuffing run this
firehoses the SQLite ingester with tiny WAL writes and starves all reads
until the queue drains.
The collector now deduplicates these events by
(attacker_ip, decky, service, event_type) over a 1-second window before
writing to the .json ingestion stream. The raw .log file is untouched, so
rsyslog/SIEM still see every event for forensic fidelity.
Tunable via DECNET_COLLECTOR_RL_WINDOW_SEC and DECNET_COLLECTOR_RL_EVENT_TYPES.
Remove unused imports (ruff F401), suppress B324 false positives on
spec-mandated MD5 in HASSH/JA3/JA3S fingerprinting, drop unused
record_version assignment in JARM parser, and pin pip>=26.0 in dev
deps to address CVE-2025-8869 and CVE-2026-1703.
The live test modules set DECNET_CONTRACT_TEST=true at module level,
which persisted across xdist workers and caused the mutate endpoint
to short-circuit before the mock was reached. Clear the env var in
affected tests with monkeypatch.delenv.
21 live tests covering all background workers against real resources:
collector (real Docker daemon), ingester (real filesystem + DB),
attacker worker (real DB profiles), sniffer (real network interfaces),
API lifespan (real health endpoint), and cross-service cascade isolation.
9 tests covering auth enforcement, component reporting, status
transitions, degraded mode, and real DB/Docker state validation.
Runs with -m live alongside other live service tests.
23 tests verifying that each background worker degrades gracefully
when its dependencies are unavailable, and that failures don't cascade:
- Collector: Docker unavailable, no state file, empty fleet
- Ingester: missing log file, unset env var, malformed JSON, fatal DB
- Attacker: DB errors, empty database
- Sniffer: missing interface, no state, scapy crash, non-decky traffic
- API lifespan: all workers failing, DB init failure, sniffer import fail
- Cascade: collector→ingester, ingester→attacker, sniffer→collector, DB→sniffer
Replace per-decky sniffer containers with a single host-side sniffer
that monitors all traffic on the MACVLAN interface. Runs as a background
task in the FastAPI lifespan alongside the collector, fully fault-isolated
so failures never crash the API.
- Add fleet_singleton flag to BaseService; sniffer marked as singleton
- Composer skips fleet_singleton services in compose generation
- Fleet builder excludes singletons from random service assignment
- Extract TLS fingerprinting engine from templates/sniffer/server.py
into decnet/sniffer/ package (parameterized for fleet-wide use)
- Sniffer worker maps packets to deckies via IP→name state mapping
- Original templates/sniffer/server.py preserved for future use
All info sections (Timeline, Services, Deckies, Commands, Fingerprints)
now have clickable headers with a chevron toggle to expand/collapse
content. Pagination controls in Commands stay clickable without
triggering the collapse. All sections default to open.
Replace flat fingerprint card list with a structured section that
groups fingerprints by type under two categories: Active Probes
(JARM, HASSH, TCP/IP) and Passive Fingerprints (TLS, certificates,
latency, etc.). Each group shows its icon, label, and count.
AttackerDetail: dedicated render components for JARM (hash + target),
HASSHServer (hash, banner, expandable KEX/encryption algorithms), and
TCP/IP stack (TTL, window, MSS as bold stats, DF/SACK/TS as tags,
options order string).
Bounty: add fingerprint field labels and priority keys so prober
bounties display structured rows instead of raw JSON. Add FINGERPRINTS
filter option to the type dropdown.
Extends the prober with two new active probe types alongside JARM:
- HASSHServer: SSH server fingerprinting via KEX_INIT algorithm ordering
(MD5 hash of kex;enc_s2c;mac_s2c;comp_s2c, pure stdlib)
- TCP/IP stack: OS/tool fingerprinting via SYN-ACK analysis using scapy
(TTL, window size, DF bit, MSS, TCP options ordering, SHA256 hash)
Worker probe cycle now runs three phases per IP with independent
per-type port tracking. Ingester extracts bounties for all three
fingerprint types.
Reverts commits 8c249f6, a6c7cfd, 7ff5703. The SSH log relay approach
requires container redeployment and doesn't retroactively fix existing
attacker profiles. Rolling back to reassess the approach.
New log_relay.py replaces raw 'cat' on the rsyslog pipe. Intercepts
sshd and bash lines and re-emits them as structured RFC 5424 events:
login_success, session_opened, disconnect, connection_closed, command.
Parsers updated to accept non-nil PROCID (sshd uses PID).
The SSH honeypot logs commands via PROMPT_COMMAND logger as:
<14>1 ... bash - - - CMD uid=0 pwd=/root cmd=ls
These lines had service=bash and event_type=-, so the attacker worker
never recognized them as commands. Both the collector and correlation
parsers now detect the CMD pattern and normalize to service=ssh,
event_type=command, with uid/pwd/command in fields.
New GET /attackers/{uuid}/commands?limit=&offset=&service= endpoint
serves commands with server-side pagination and optional service filter.
AttackerDetail frontend fetches commands from this endpoint with
page controls. Service badge filter now drives both the API query
and the local fingerprint filter.
Clicking a service badge in the attacker detail view now filters the
commands and fingerprints sections on that page instead of navigating
away. Click again to clear. Header shows filtered/total counts.
API now accepts ?service=https to filter attackers by targeted service.
Service badges are clickable in both the attacker list and detail views,
navigating to a filtered view. Active filter shows as a dismissable tag.
Same (src_ip, event_type, fingerprint) tuple is now suppressed within a
5-minute window (configurable via DEDUP_TTL env var). Prevents the bounty
vault from filling up with identical JA3/JA4 rows from repeated connections.
TLS-wrapped variant of the HTTP honeypot. Auto-generates a self-signed
certificate on startup if none is provided. Supports all the same persona
options (fake_app, server_header, custom_body, etc.) plus TLS_CERT,
TLS_KEY, and TLS_CN configuration.
EHLO/HELO require a domain or address-literal argument. Previously
the server accepted bare EHLO with no argument and responded 250,
which deviates from the spec and makes the honeypot easier to
fingerprint.
The collector kept streaming stale container IDs after a redeploy,
causing new service logs to never reach decnet.log. Now _kill_api()
also matches and SIGTERMs any running decnet.cli collect process.
Every service's _log() called print() then write_syslog_file() which also
calls print(), causing every log line to appear twice in Docker logs. The
collector streamed both copies, doubling ingested events. Removed the
redundant print() from all 22 service server.py files.
Two bugs fixed:
- data_received only split on CRLF, so clients sending bare LF (telnet, nc,
some libraries) got no responses at all. Now splits on LF and strips
trailing CR, matching real Postfix behavior.
- AUTH PLAIN without inline credentials set state to "await_plain" but no
handler existed for that state, causing the next line to be dispatched as
a normal command. Added the missing state handler.
Migrate Attacker model from IP-based to UUID-based primary key with
auto-migration for old schema. Add GET /attackers (paginated, search,
sort) and GET /attackers/{uuid} API routes. Rewrite Attackers.tsx as
a card grid with full threat info and create AttackerDetail.tsx as a
dedicated detail page with back navigation, stats, commands table,
and fingerprints.
- Modify Rfc5424Formatter to read decnet_component from LogRecord
and use it as RFC 5424 APP-NAME field (falls back to 'decnet')
- Add get_logger(component) factory in decnet/logging/__init__.py
with _ComponentFilter that injects decnet_component on each record
- Wire all five layers to their component tag:
cli -> 'cli', engine -> 'engine', api -> 'api' (api.py, ingester,
routers), mutator -> 'mutator', collector -> 'collector'
- Add structured INFO/DEBUG/WARNING/ERROR log calls throughout each
layer per the defined vocabulary; DEBUG calls are suppressed unless
DECNET_DEVELOPER=true
- Add tests/test_logging.py covering factory, filter, formatter
component-awareness, fallback behaviour, and level gating
2026-04-13 07:39:01 -04:00
1900 changed files with 257815 additions and 8811 deletions
DECNET is a honeypot/deception network framework. It deploys fake machines (called **deckies**) with realistic services (RDP, SMB, SSH, FTP, etc.) to lure and profile attackers. All attacker interactions are aggregated to an isolated logging network (ELK stack / SIEM).
## Deployment Models
**UNIHOST** — one real host spins up _n_ deckies via a container orchestrator. Simpler, single-machine deployment.
**SWARM (MULTIHOST)** — _n_ real hosts each running deckies. Orchestrated via Ansible/sshpass or similar tooling.
## Core Technology Choices
- **Containers**: Docker Compose is the starting point but other orchestration frameworks should be evaluated if they serve the project better. `debian:bookworm-slim` is the default base image; mixing in Ubuntu, CentOS, or other distros is encouraged to make the decoy network look heterogeneous.
- **Networking**: Deckies need to appear as real machines on the LAN (own MACs/IPs). MACVLAN and IPVLAN are candidates; the right driver depends on the host environment. WSL has known limitations — bare metal or a VM is preferred for testing.
- **Log pipeline**: Logstash → ELK stack → SIEM (isolated network, not reachable from decoy network)
## Architecture Constraints
- The decoy network must be reachable from the outside (attacker-facing).
- The logging/aggregation network must be isolated from the decoy network.
- A publicly accessible real server acts as the bridge between the two networks.
- Deckies should differ in exposed services and OS fingerprints to appear as a heterogeneous network.
- **IMPORTANT**: The system now strictly enforces dependency injection for storage. Do not import `SQLiteRepository` directly in new features; instead, use `get_repository()` from the factory or the FastAPI `get_repo` dependency.
## Development and testing
- For every new feature, pytests must me made.
- Pytest is the main testing framework in use.
- NEVER pass broken code to the user.
- Broken means: not running, not passing 100% tests, etc.
- After tests pass with 100%, always git commit your changes.
- NEVER add "Co-Authored-By" or any Claude attribution lines to git commit messages.
DECNET is a high-fidelity honeypot framework designed to deploy heterogeneous fleets of fake machines (called **deckies**) that appear as real hosts on a local network.
## Project Overview
- **Core Purpose:** To lure, profile, and log attacker interactions within a controlled, deceptive environment.
- **Key Technology:** Linux-native container networking (MACVLAN/IPvlan) combined with Docker to give each decoy its own MAC address, IP, and realistic TCP/IP stack behavior.
- **Main Components:**
- **Deckies:** Group of containers sharing a network namespace (one base container + multiple service containers).
- **Archetypes:** Pre-defined machine profiles (e.g., `windows-workstation`, `linux-server`) that bundle services and OS fingerprints.
- **Services:** Modular honeypot plugins (SSH, SMB, RDP, etc.) built as `BaseService` subclasses.
- **OS Fingerprinting:** Sysctl-based TCP/IP stack tuning to spoof OS detection (nmap).
- **Logging Pipeline:** RFC 5424 syslog forwarding to an isolated SIEM/ELK stack.
- **Always use typed variables**. If any non-types variables are found, they must be corrected.
- The correct way is `x: int = 1`, never `x : int = 1`.
- If assignment is present, always use a space between the type and the equal sign `x: int = 1`.
- **Never** use lowercase L (l), uppercase o (O) or uppercase i (i) in single-character names.
- **Internal vars are to be declared with an underscore** (_internal_variable_name).
- **Internal to internal vars are to be declared with double underscore** (__internal_variable_name).
- Always use snake_case for code.
- Always use PascalCase for classes and generics.
- **Testing:** New features MUST include a `pytest` case. 100% test pass rate is mandatory before merging.
- **Plugin System:**
- New services go in `decnet/services/<name>.py`.
- Subclass `decnet.services.base.BaseService`.
- The registry uses auto-discovery; no manual registration required.
- **Configuration:**
- Use Pydantic models in `decnet/config.py` for any new settings.
- INI file parsing is handled in `decnet/ini_loader.py`.
- **State Management:**
- Runtime state is persisted in `decnet-state.json`.
- Do not modify this file manually.
- **General Development Guidelines**:
- **Never** commit broken code, or before running `pytest`s or `bandit` at the project level.
- **No matter how small** the changes, they must be committed.
- **If new features are addedd** new tests must be added, too.
- **Never present broken code to the user**. Test, validate, then present.
- **Extensive testing** for every function must be created.
- **Always develop in the `dev` branch, never in `main`.**
- **Test in the `testing` branch.**
- **IMPORTANT**: The system now strictly enforces dependency injection for storage. Do not import `SQLiteRepository` directly in new features; instead, use `get_repository()` from the factory or the FastAPI `get_repo` dependency.
## Directory Structure
-`decnet/`: Main source code.
-`services/`: Honeypot service implementations.
-`logging/`: Syslog formatting and forwarding logic.
-`correlation/`: (In Progress) Logic for grouping attacker events.
-`templates/`: Dockerfiles and entrypoint scripts for services.
-`tests/`: Pytest suite.
-`pyproject.toml`: Dependency and entry point definitions.
Default operator-supplied email seed for IMAP/POP3 deckies. Drop `*.eml` and/or `*.json` files here; the IMAP/POP3 services bind-mount this dir read-only at `/var/spool/decnet-emails/seed` when no per-decky `email_seed` is configured. Entries concatenate onto the hardcoded bait baseline (additive to realism-engine output, never replacing).
JSON shape: list of dicts with required `from_addr`, `to_addr`, `subject`, `body`; optional `from_name`, `date`, `flags`. See `decnet/templates/imap/server.py` for the loader.
deckies:Optional[int]=typer.Option(None,"--deckies","-n",help="Number of deckies to deploy (required without --config)",min=1),
interface:Optional[str]=typer.Option(None,"--interface","-i",help="Host NIC (auto-detected if omitted)"),
subnet:Optional[str]=typer.Option(None,"--subnet",help="LAN subnet CIDR (auto-detected if omitted)"),
ip_start:Optional[str]=typer.Option(None,"--ip-start",help="First decky IP (auto if omitted)"),
services:Optional[str]=typer.Option(None,"--services",help="Comma-separated services, e.g. ssh,smb,rdp"),
randomize_services:bool=typer.Option(False,"--randomize-services",help="Assign random services to each decky"),
distro:Optional[str]=typer.Option(None,"--distro",help="Comma-separated distro slugs, e.g. debian,ubuntu22,rocky9"),
randomize_distros:bool=typer.Option(False,"--randomize-distros",help="Assign a random distro to each decky"),
log_file:Optional[str]=typer.Option(DECNET_INGEST_LOG_FILE,"--log-file",help="Host path for the collector to write RFC 5424 logs (e.g. /var/log/decnet/decnet.log)"),
port:int=typer.Option(8765,"--port",help="Port for the worker agent"),
host:str=typer.Option("0.0.0.0","--host",help="Bind address for the worker agent"),# nosec B104
agent_dir:Optional[str]=typer.Option(None,"--agent-dir",help="Worker cert bundle dir (default: ~/.decnet/agent, expanded under the running user's HOME — set this when running as sudo/root)"),
daemon:bool=typer.Option(False,"--daemon","-d",help="Detach to background as a daemon process"),
no_forwarder:bool=typer.Option(False,"--no-forwarder",help="Do not auto-spawn the log forwarder alongside the agent"),
)->None:
"""Run the DECNET SWARM worker agent (requires a cert bundle in ~/.decnet/agent/).
By default, `decnet agent` auto-spawns `decnet forwarder` as a fully-
detached sibling process so worker logs start flowing to the master
without a second manual invocation. The forwarder survives agent
restarts and crashes — if it dies on its own, restart it manually
with `decnet forwarder --daemon …`. Pass --no-forwarder to skip.
deckies:Optional[int]=typer.Option(None,"--deckies","-n",help="Number of deckies to deploy (required without --config)",min=1),
interface:Optional[str]=typer.Option(None,"--interface","-i",help="Host NIC (auto-detected if omitted)"),
subnet:Optional[str]=typer.Option(None,"--subnet",help="LAN subnet CIDR (auto-detected if omitted)"),
ip_start:Optional[str]=typer.Option(None,"--ip-start",help="First decky IP (auto if omitted)"),
services:Optional[str]=typer.Option(None,"--services",help="Comma-separated services, e.g. ssh,smb,rdp"),
randomize_services:bool=typer.Option(False,"--randomize-services",help="Assign random services to each decky"),
distro:Optional[str]=typer.Option(None,"--distro",help="Comma-separated distro slugs, e.g. debian,ubuntu22,rocky9"),
randomize_distros:bool=typer.Option(False,"--randomize-distros",help="Assign a random distro to each decky"),
log_file:Optional[str]=typer.Option(DECNET_INGEST_LOG_FILE,"--log-file",help="Host path for the collector to write RFC 5424 logs (e.g. /var/log/decnet/decnet.log)"),
console.print("[red]Failed to start swarmctl. Ensure 'uvicorn' is installed in the current environment.[/]")
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
