3
Module Reference Workers
anti edited this page 2026-04-19 00:21:15 -04:00
This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Module Reference — Workers

Every async background worker and host-side process that DECNET runs outside the FastAPI request path. These are the modules under decnet/collector/, decnet/profiler/, decnet/sniffer/, decnet/prober/, decnet/mutator/, decnet/correlation/, decnet/engine/, and decnet/logging/.

Citation format: decnet/<package>/<module>.py::<symbol>.

See also: Design Overview, Logging and Syslog, Environment Variables.


Collector — decnet/collector/

Tails docker logs for every decky service container, parses RFC 5424 lines, and forwards structured events to the ingester. Lives in the API lifespan as an asyncio task.

decnet/collector/__init__.py

Re-exports is_service_container, is_service_event, log_collector_worker, parse_rfc5424.

decnet/collector/worker.py

  • decnet/collector/worker.py::log_collector_worker — async entry point. Walks the Docker API for containers matching DECNET service naming, launches a _stream_container task for each, and subscribes to Docker events so newly-started containers are attached mid-flight. Uses a dedicated ThreadPoolExecutor(max_workers=64, thread_name_prefix="decnet-collector") so blocking docker logs reads never starve the API event loop.
  • decnet/collector/worker.py::parse_rfc5424 — regex-based RFC 5424 parser. Extracts TIMESTAMP, HOSTNAME (decky), APP-NAME (service), MSGID (event_type), the [relay@55555 …] SD block, and a free-text MSG tail. Falls back to a _MSG_KV_RE scanner for key=value pairs inside MSG when services skip the SD block.
  • decnet/collector/worker.py::is_service_container / is_service_event — gatekeepers that reject non-DECNET containers and non-service log lines.
  • decnet/collector/worker.py::_stream_container — per-container reader with inode-aware reopen. If the log file gets rotated underneath us, the next read reopens.
  • decnet/collector/worker.py::_should_ingest — rate-limit dedup. Events whose event_type appears in DECNET_COLLECTOR_RL_EVENT_TYPES (default: connect,disconnect,connection,accept,close) are keyed by (attacker_ip, decky, service, event_type) and suppressed within DECNET_COLLECTOR_RL_WINDOW_SEC seconds (default 5). Protects the DB from connection-flood amplification.
  • _RFC5424_RE, _SD_BLOCK_RE, _MSG_KV_RE — compiled regexes.
  • _IP_FIELDS = ("src_ip", "src", "client_ip", "remote_ip", "remote_addr", "target_ip", "ip") — the canonical priority order for extracting the attacker IP from the SD params.

Reads: Docker container logs. Writes: calls repo.insert_log(...) via the ingester queue. See Logging and Syslog for the end-to-end pipeline.


Profiler — decnet/profiler/

Periodic attacker-profile builder. Walks new log rows, updates the in-memory CorrelationEngine, and upserts per-IP attacker records + behavioural aggregates.

decnet/profiler/__init__.py

Re-exports attacker_profile_worker.

decnet/profiler/worker.py

  • decnet/profiler/worker.py::attacker_profile_worker — async loop while True: _incremental_update(); await asyncio.sleep(interval). Default interval=30.
  • decnet/profiler/worker.py::_WorkerState — dataclass with engine: CorrelationEngine, last_log_id: int, initialized: bool. Persisted across restarts via repo.set_state("attacker_worker_cursor", …) keyed by _STATE_KEY.
  • decnet/profiler/worker.py::_incremental_update — cursor-driven batch read. Calls repo.get_logs_after_id(last_log_id, limit=_BATCH_SIZE) where _BATCH_SIZE = 500, feeds each line into engine.ingest, advances the cursor, then calls _update_profiles.
  • decnet/profiler/worker.py::_update_profiles — for every attacker IP currently known to the engine, builds a _first_contact_deckies map + a command history from _extract_commands_from_events, and upserts both the attacker record (repo.upsert_attacker) and the behavioural aggregates (repo.upsert_attacker_behavior).
  • decnet/profiler/worker.py::_build_record — assembles the per-attacker dict (bounties map, traversal path, decky count, first/last seen, command count).
  • _COMMAND_EVENT_TYPES = {"command", "exec", "query", "input", "shell_input", "execute", "run", "sql_query", "redis_command"} — event-type allow-list used when mining commands.
  • _COMMAND_FIELDS = ("command", "query", "input", "line", "sql", "cmd") — SD-param keys probed inside each matching event.

Reads: repo.get_logs_after_id, repo.get_state. Writes: repo.upsert_attacker, repo.upsert_attacker_behavior, repo.set_state.


Sniffer — decnet/sniffer/

Fleet-singleton passive TLS fingerprinter. Runs in the API lifespan but sniffs the MACVLAN/IPvlan host interface directly with scapy. Extracts JA3/JA3S/JA4/JA4S/JA4L, TCP SYN OS fingerprints, flow timing, and x509 certificate metadata from the wire.

See also: templates/sniffer/ for the container variant (fleet-wide when the singleton is declared via INI). The Python worker here is the host-side flavour.

decnet/sniffer/__init__.py

Thin re-export of sniffer_worker.

decnet/sniffer/worker.py

  • decnet/sniffer/worker.py::sniffer_worker — async entry point. Selects the interface (DECNET_SNIFFER_IFACE env override, else HOST_MACVLAN_IFACE then HOST_IPVLAN_IFACE from decnet.network). If neither exists (fleet not deployed), the worker logs a warning and returns — the API continues. Builds a stop-event, spawns a dedicated 2-worker ThreadPoolExecutor, and runs the blocking scapy sniff() loop via loop.run_in_executor. Fully fault-isolated.
  • decnet/sniffer/worker.py::_sniff_loop — the blocking thread target. Builds the initial ip_to_decky map, instantiates a SnifferEngine, launches a daemon refresh thread that calls _load_ip_to_decky() every 60 s so late-joining deckies are captured, then enters scapy.sniff(filter="tcp", prn=engine.on_packet, store=False, stop_filter=…).
  • decnet/sniffer/worker.py::_load_ip_to_decky — pulls the {ip: decky_name} map from decnet-state.json via decnet.config.load_state.
  • decnet/sniffer/worker.py::_interface_existsip link show <iface> probe, returns bool.
  • _IP_MAP_REFRESH_INTERVAL = 60.0 — seconds between refreshes.

decnet/sniffer/fingerprint.py

Stateful engine + TLS parser. 1166 lines — the bulk is a hand-written TLS 1.01.3 ClientHello / ServerHello / Certificate parser (deliberately avoids cryptography/scapy.layers.tls to keep the fingerprint exactly reproducible).

  • decnet/sniffer/fingerprint.py::SnifferEngine — per-flow session tracker. Keys flows by (src_ip, src_port, dst_ip, dst_port), maintains _sessions, _session_ts, _tcp_syn, _tcp_rtt, _flows, and _dedup_cache. Public: update_ip_map(...), on_packet(pkt).
  • decnet/sniffer/fingerprint.py::SnifferEngine._resolve_decky — maps a packet's src/dst IP to a known decky name; returns None (and the packet is dropped) if neither side is a decky.
  • decnet/sniffer/fingerprint.py::SnifferEngine._dedup_key_for — per-event-type dedup strategy. Notable rule: tcp_flow_timing dedups on (dst_ip, dst_port) only — the attacker's ephemeral source port is deliberately excluded so a port scanner rotating sources still only emits one timing event per window.
  • TLS parsers: _parse_client_hello, _parse_server_hello, _parse_certificate, plus DER helpers _der_read_tag_len, _der_read_sequence, _der_read_oid, _der_extract_cn, _der_extract_name_str, _parse_x509_der, _extract_sans, _parse_san_sequence.
  • Fingerprint hashes: _ja3, _ja3s, _ja4, _ja4s, _ja4l. _ja4_version, _ja4_alpn_tag, _sha256_12 are helpers.
  • GREASE handling: _is_grease, _filter_grease drop the RFC 8701 sentinel values before hashing so Chrome/Firefox JA3s are stable.
  • TCP SYN: _extract_tcp_fingerprint pulls MSS / window scale / option order out of the SYN options list and hands it to decnet.sniffer.p0f::guess_os.

decnet/sniffer/p0f.py

Mini-p0f OS classifier using the passive TCP SYN fingerprint.

  • decnet/sniffer/p0f.py::initial_ttl — map an observed TTL to the most likely hop-origin value (64, 128, 255).
  • decnet/sniffer/p0f.py::hop_distance — estimate hop count.
  • decnet/sniffer/p0f.py::guess_os — score the observed SYN against the built-in signature table; returns the best-match OS string or "unknown".
  • _match_signature — internal scorer used by guess_os.

decnet/sniffer/syslog.py

Lightweight RFC 5424 emitter used by the sniffer (not decnet.logging.syslog_formatter — this one writes directly to a file and a sibling .json for the live-logs stream).

  • decnet/sniffer/syslog.py::syslog_line — build a line with <PRI>1 TS HOSTNAME APP-NAME - MSGID [relay@55555 …] MSG.
  • decnet/sniffer/syslog.py::write_event — append a line to .log + a JSON record to .json. Tested by tests/test_sniffer_emit_capture.py.
  • _sd_escape, _sd_element — per-SD-param escaping helpers matching the main formatter.

Prober — decnet/prober/

Standalone active-probe daemon. Tails the collector JSON log for new attacker IPs, then hits each with JARM (TLS), HASSH (SSH), and raw TCP SYN probes.

decnet/prober/__init__.py

Re-exports prober_worker, jarm_hash, hassh_server, tcp_fingerprint.

decnet/prober/worker.py

  • decnet/prober/worker.py::prober_worker — async entry point, 5-minute default cycle. Parameters: log_file, interval=300, timeout=5.0, ports/ssh_ports/tcpfp_ports. Writes a prober_startup event, then loops: _discover_attackers → _probe_cycle → asyncio.sleep(interval).
  • decnet/prober/worker.py::_discover_attackers — resumes from a byte offset into the collector JSON log, returns newly-seen attacker IPs.
  • decnet/prober/worker.py::_probe_cycle — fan-out across phases. Each attacker is probed at most once per (phase, port) pair; the probed: dict[str, dict[str, set[int]]] cache is the durable state.
  • decnet/prober/worker.py::_jarm_phase — calls jarm_hash(ip, port, timeout) for each port in DEFAULT_PROBE_PORTS = [443, 8443, 8080, 4443, 50050, 2222, 993, 995, 8888, 9001].
  • decnet/prober/worker.py::_hassh_phase — calls hassh_server for DEFAULT_SSH_PORTS = [22, 2222, 22222, 2022].
  • decnet/prober/worker.py::_tcpfp_phase — calls tcp_fingerprint for DEFAULT_TCPFP_PORTS = [22, 80, 443, 8080, 8443, 445, 3389].
  • decnet/prober/worker.py::_write_event — dual-write RFC 5424 line + JSON record (same shape as decnet.sniffer.syslog.write_event; see the module docstring's tech-debt note about extracting a shared sink).
  • _parse_to_json, _syslog_line, _sd_escape, _sd_element — inline formatting helpers.

decnet/prober/jarm.py

Raw-socket JARM probe. Builds 10 hand-crafted ClientHellos per the JARM spec, concatenates the 10 ServerHello fingerprints, and returns the 62-character JARM hash.

  • decnet/prober/jarm.py::jarm_hash — public entry.
  • _build_client_hello, _parse_server_hello, _send_probe, _compute_jarm — the four stages.
  • Extension builders: _ext, _ext_sni, _ext_supported_groups, _ext_ec_point_formats, _ext_signature_algorithms, _ext_supported_versions_13, _ext_psk_key_exchange_modes, _ext_key_share, _ext_alpn, _ext_session_ticket, _ext_encrypt_then_mac, _ext_extended_master_secret, _ext_padding. _middle_out produces the ciphersuite ordering used by probes 59. _version_to_str maps the two-byte TLS version field to the human string.

decnet/prober/hassh.py

SSH server HASSH fingerprint. Reads the SSH banner, sends nothing, parses the server's SSH_MSG_KEXINIT, and computes md5(kex;encryption;mac;compression).

  • decnet/prober/hassh.py::hassh_server — public entry.
  • _ssh_connect, _read_banner, _read_ssh_packet, _recv_exact — socket IO.
  • _parse_kex_init — extract the four algorithm lists.
  • _compute_hassh — MD5 concatenation per the HASSH spec.

decnet/prober/tcpfp.py

Raw-socket SYN probe for active OS fingerprinting (complements the passive sniffer fingerprint).

  • decnet/prober/tcpfp.py::tcp_fingerprint — public entry. Sends a SYN, parses the SYN-ACK, computes a short fingerprint string + SHA-256 hash, sends an RST to avoid the half-open state.
  • _send_syn, _send_rst, _parse_synack, _extract_options_order, _compute_fingerprint — internal stages.

Mutator — decnet/mutator/

Rotates the service set of a decky without tearing down the fleet. Each mutation picks a new service pool from the decky's archetype (or the global catalogue) and re-writes the compose file.

decnet/mutator/__init__.py

Empty package marker.

decnet/mutator/engine.py

  • decnet/mutator/engine.py::mutate_decky — Intra-Archetype Shuffle for a single decky. Reads the deployment state from repo.get_state("deployment"), picks a fresh random service subset from the decky's archetype (via decnet.archetypes.get_archetype) or decnet.fleet.all_service_names, calls decnet.composer.write_compose to regenerate the file, then _compose_with_retry("up", "-d", …) to roll the container set.
  • decnet/mutator/engine.py::mutate_all — iterates every decky in the deployment; a decky is "due" when time.time() - decky.last_mutated >= decky.mutate_interval * 60. force=True bypasses the schedule.
  • decnet/mutator/engine.py::run_watch_loop — infinite mutate_all(force=False) poll with poll_interval_secs=10 default. Started by the API lifespan when the CLI deploys with mutation enabled.

Reads: repo.get_state("deployment"). Writes: compose file on disk, repo.set_state to persist last_mutated timestamps.


Correlation — decnet/correlation/

In-memory cross-decky correlation engine. Used by both the standalone decnet correlate CLI and the live profiler. Independent of the DB — pure stream-ingest.

decnet/correlation/__init__.py

Re-exports CorrelationEngine, AttackerTraversal, TraversalHop, LogEvent, parse_line.

decnet/correlation/parser.py

  • decnet/correlation/parser.py::LogEvent — dataclass (timestamp, decky, service, event_type, attacker_ip, fields, raw).
  • decnet/correlation/parser.py::parse_line — parse one RFC 5424 line into a LogEvent. Returns None for blank / non-DECNET / missing-hostname lines.
  • _parse_sd_params — extract k="v" pairs from the [relay@55555 …] SD block with proper RFC 5424 unescaping.
  • _extract_attacker_ip — probes _IP_FIELDS = ("src_ip", "src", "client_ip", "remote_ip", "remote_addr", "target_ip", "ip") in priority order.
  • _RFC5424_RE, _SD_BLOCK_RE, _PARAM_RE — compiled regexes.

decnet/correlation/graph.py

  • decnet/correlation/graph.py::TraversalHop — single event in a traversal: (timestamp, decky, service, event_type).
  • decnet/correlation/graph.py::AttackerTraversal — all hops for one attacker IP, sorted chronologically. Properties: first_seen, last_seen, duration_seconds, deckies (unique, first-contact-ordered), decky_count, path ("decky-01 → decky-03 → decky-07"). to_dict() for JSON serialisation.

decnet/correlation/engine.py

  • decnet/correlation/engine.py::CorrelationEngine — stateful aggregator. _events: dict[str, list[LogEvent]] keyed by attacker IP.
    • ingest(line) — parse + index one line.
    • ingest_file(path) — whole-file variant with OTEL summary span.
    • traversals(min_deckies=2) — return attackers that touched at least min_deckies distinct deckies, sorted by first-seen.
    • all_attackers(){ip: event_count} sorted desc.
    • report_table(min_deckies) — rich Table for CLI output.
    • report_json(min_deckies) — serialisable dict.
    • traversal_syslog_lines(min_deckies) — emit format_rfc5424(service="correlator", hostname="decnet-correlator", event_type="traversal_detected", severity=SEVERITY_WARNING, …) lines so the SIEM can ingest correlator findings alongside raw service events.
  • _fmt_duration — humanise seconds → 45s / 3.2m / 1.5h.

Reads: RFC 5424 lines (string or file). Writes: nothing — pure computation; the profiler embeds a CorrelationEngine and persists its results via repo.upsert_attacker.


Engine — decnet/engine/

Docker-compose orchestration. Wraps the docker compose CLI with retry logic and network setup/teardown.

decnet/engine/__init__.py

Re-exports COMPOSE_FILE, _compose_with_retry, deploy, status, teardown.

decnet/engine/deployer.py

  • decnet/engine/deployer.py::COMPOSE_FILE — module-level Path("decnet-compose.yml").
  • decnet/engine/deployer.py::deploy — create the MACVLAN / IPvlan network, set up the host-side virtual iface via decnet.network.setup_host_macvlan / setup_host_ipvlan, sync the canonical templates/syslog_bridge.py into every active service template (_sync_logging_helper), write the compose file (decnet.composer.write_compose), save deployment state, build (docker compose build), and up -d. parallel=True enables DOCKER_BUILDKIT=1 for concurrent image builds. no_cache=True forces a full rebuild.
  • decnet/engine/deployer.py::teardown — either decky_id=None (full teardown: compose down, teardown host iface, remove docker network, clear state) or decky_id="decky-xx" (stop + compose rm only that decky's containers).
  • decnet/engine/deployer.py::status — render a rich table of running/absent containers per decky.
  • decnet/engine/deployer.py::_compose_with_retry — exponential-backoff retry (3 attempts, 5 s → 10 s → 20 s). Errors matching _PERMANENT_ERRORS = ("manifest unknown", "manifest for", "not found", "pull access denied", "repository does not exist") short-circuit to fail-fast.
  • _compose — no-retry variant used by teardown.
  • _sync_logging_helper — copies templates/syslog_bridge.py into every service's dockerfile_context() directory so the canonical copy is what gets baked into images. Only re-copies when the target differs byte-for-byte.
  • _print_status — compact post-deploy summary.

Reads: Docker daemon, decnet-state.json. Writes: decnet-compose.yml, host iface state, docker networks/containers.


Logging — decnet/logging/

Application-level logging helpers. Separate from the container-side templates/syslog_bridge.py used inside deckies, though both produce RFC 5424.

decnet/logging/__init__.py

  • decnet/logging/__init__.py::get_logger — returns logging.getLogger(f"decnet.{component}") with a _ComponentFilter attached so Rfc5424Formatter emits the component as APP-NAME. Idempotent — calling twice for the same component doesn't stack filters.
  • decnet/logging/__init__.py::enable_trace_context — attach the _TraceContextFilter to the root decnet logger. Called once from decnet.telemetry.setup_tracing() so every LogRecord carries otel_trace_id / otel_span_id drawn from the active OTEL span (or "0" when no span is active — cheap string comparison downstream).
  • _ComponentFilter — injects record.decnet_component so Rfc5424Formatter can promote it to APP-NAME.
  • _TraceContextFilter — pulls trace ids from opentelemetry.trace.get_current_span(). All errors are swallowed (logging must never crash the caller).
  • _trace_filter_installed — module-level idempotency flag for enable_trace_context.

decnet/logging/syslog_formatter.py

Canonical RFC 5424 emitter used by decnet.telemetry and the correlator.

  • decnet/logging/syslog_formatter.py::format_rfc5424<PRI>1 TIMESTAMP HOSTNAME APP-NAME - MSGID [SD] MSG. Enforces RFC 5424 length limits: HOSTNAME ≤ 255, APP-NAME ≤ 48, MSGID ≤ 32. Facility is always local0 (16). PEN is relay@55555.
  • SEVERITY_INFO = 6, SEVERITY_WARNING = 4, SEVERITY_ERROR = 3 — public severity constants.
  • FACILITY_LOCAL0 = 16, NILVALUE = "-", _SD_ID = "relay@55555".
  • _pri, _truncate, _sd_escape (RFC 5424 §6.3.3 escapes), _sd_element — internal formatters.

decnet/logging/file_handler.py

Module-level singleton RotatingFileHandler for write_syslog (used by worker-side code that needs to write directly to decnet.log without the full logging pipeline, e.g. sniffer/prober).

  • decnet/logging/file_handler.py::write_syslog — append one pre-formatted RFC 5424 line to the log.
  • decnet/logging/file_handler.py::get_log_path — return the Path of the target file.
  • _init_file_handler, _get_logger — internal singleton init.

decnet/logging/inode_aware_handler.py

  • decnet/logging/inode_aware_handler.py::InodeAwareRotatingFileHandler — subclass of RotatingFileHandler that before every emit stats the configured path and compares inode+device to the open file; on mismatch it closes and reopens. Solves the "logrotate without copytruncate silently drops lines" failure mode that plain RotatingFileHandler has. Matches the pattern already used in decnet/collector/worker.py::_reopen_if_needed. Cost: one os.stat per log record.

decnet/logging/forwarder.py

Shared helpers for the LOG_TARGET env var used by service plugins.

  • decnet/logging/forwarder.py::parse_log_target"ip:port"(host, port); raises ValueError on bad format.
  • decnet/logging/forwarder.py::probe_log_target — TCP connect with timeout, returns bool. Non-fatal: the CLI uses it to warn before deployment but never to block.

See Module Reference — Core for top-level modules (cli, composer, telemetry, etc.) and Module Reference — Web for the FastAPI surface and DB layer.


Swarm — decnet/swarm/

Master-side orchestration of multi-host deployments: HTTP clients for the worker daemons, the PKI that signs their certs, the tar helper that packages the working tree for a remote update, and the syslog-over-TLS forwarder/listener pair. Everything in this package runs either on the master or on every worker that needs to talk back to it — there is no third role.

See PKI and mTLS for the cert-chain details, cert layout, and why CN is not actually validated at the handler level.

decnet/swarm/__init__.py

Re-exports AgentClient, UpdaterClient, MasterIdentity, ensure_master_identity, and the pki submodule. Importing decnet.swarm is enough for CLI-level callers; nothing else is considered public.

decnet/swarm/client.py

Async HTTP client for the worker-side agent daemon (port 8765). One instance per worker target; httpx.AsyncClient is re-used across calls.

  • decnet/swarm/client.py::AgentClient.__init__ — accepts either a host dict (from swarm_hosts DB rows) or a raw address string, resolves the master's own cert bundle via MasterIdentity, and builds the mTLS ssl.SSLContext. agent_port defaults to 8765. verify_hostname=False by default — we pin by CA chain, not DNS, because workers enroll with whatever SANs the operator chose.
  • decnet/swarm/client.py::AgentClient.deployPOST /deploy with a serialized DecnetConfig + dry_run + no_cache. Read timeout is bumped to 600 s because docker compose build can be very slow on underpowered workers.
  • decnet/swarm/client.py::AgentClient.teardownPOST /teardown with optional decky_id.
  • decnet/swarm/client.py::AgentClient.healthGET /health. The master never gets to this handler without a valid cert (uvicorn rejects the handshake) — this is a real liveness probe, not an auth endpoint.
  • decnet/swarm/client.py::AgentClient.statusGET /status.
  • mTLS wiring (in __init__): ctx.load_cert_chain(...), ctx.load_verify_locations(cafile=...), ctx.verify_mode = ssl.CERT_REQUIRED, ctx.check_hostname = self._verify_hostname.

decnet/swarm/updater_client.py

Sibling client for the self-updater daemon (port 8766). Same mTLS pattern as AgentClient but targets a different port and uses multipart/form-data for tarball uploads.

  • decnet/swarm/updater_client.py::UpdaterClient.__init__updater_port=8766, same MasterIdentity bundle as AgentClient. The master uses one cert for both; the TLS layer doesn't care which daemon answers.
  • decnet/swarm/updater_client.py::UpdaterClient.healthGET /health.
  • decnet/swarm/updater_client.py::UpdaterClient.updatePOST /update with tarball: bytes + sha: str as multipart fields. 180 s read timeout covers tarball upload + pip install + probe-with-retry.
  • decnet/swarm/updater_client.py::UpdaterClient.update_selfPOST /update-self; sends confirm_self=true to pass the server-side safety check (see Remote-Updates). Tolerates the mid-response disconnect that os.execv causes by catching RemoteProtocolError and treating it as "success pending /health poll".
  • decnet/swarm/updater_client.py::UpdaterClient.rollbackPOST /rollback, 404 if no prev/ slot.

decnet/swarm/pki.py

The one place in the codebase that holds a private key. Everything else consumes IssuedCert bundles it produces.

  • decnet/swarm/pki.py::DEFAULT_CA_DIR = ~/.decnet/ca; decnet/swarm/pki.py::DEFAULT_AGENT_DIR = ~/.decnet/agent.
  • CA_KEY_BITS = 4096, WORKER_KEY_BITS = 2048, CA_VALIDITY_DAYS = 3650, WORKER_VALIDITY_DAYS = 825.
  • decnet/swarm/pki.py::CABundle(key_pem: bytes, cert_pem: bytes) dataclass for the CA private key + self-signed cert.
  • decnet/swarm/pki.py::IssuedCert(key_pem, cert_pem, ca_cert_pem, fingerprint_sha256: str) for a signed leaf bundle. fingerprint_sha256 is what the DB stores for out-of-band enrollment audit.
  • decnet/swarm/pki.py::generate_ca — RSA-4096, self-signed, BasicConstraints(ca=True, path_length=0), KeyUsage(key_cert_sign=True, crl_sign=True), signed with SHA-256, 10-year validity.
  • decnet/swarm/pki.py::issue_worker_cert — RSA-2048 leaf, CN = caller-supplied worker_name (hostname for agent certs, updater@hostname for updater certs), SANs built from the list the caller passes (IPs parsed as IPAddress, everything else as DNSName), ExtKeyUsage(serverAuth, clientAuth) — both flags because the worker is a server to the master and a client when it forwards logs.
  • decnet/swarm/pki.py::write_worker_bundle — writes worker.key (mode 0600), worker.crt, ca.crt into the bundle dir. Updater bundles write to ~/.decnet/updater/ with updater.key / updater.crt names instead.
  • decnet/swarm/pki.py::load_worker_bundle — loads an IssuedCert off disk; used by the agent/updater at startup.
  • decnet/swarm/pki.py::fingerprintsha256(cert_pem_der).hexdigest(). Cheap, deterministic, stable across cert re-encodings.

decnet/swarm/tar_tree.py

Builds the working-tree tarball that decnet swarm update ships to the updater.

  • decnet/swarm/tar_tree.py::DEFAULT_EXCLUDES — filter tuple: .venv/, __pycache__/, .git/, wiki-checkout/, *.pyc, *.pyo, *.db*, *.log, .pytest_cache/, .mypy_cache/, .tox/, *.egg-info/, decnet-state.json, master.log, master.json, decnet.db*. These are enforced regardless of .gitignore so untracked dev artefacts never leak onto workers.
  • decnet/swarm/tar_tree.py::_is_excludedfnmatch the relative path and every leading subpath so a pattern like .git/ excludes everything underneath.

decnet/swarm/log_forwarder.py

Worker → master half of the RFC 5425 syslog-over-TLS pipeline. Wakes up periodically, reads new lines from the local log file, frames them octet-counted per RFC 5425, and writes them over an mTLS connection to port 6514 on the master.

  • decnet/swarm/log_forwarder.py::ForwarderConfig — dataclass: log_path, master_host, master_port=6514, agent_dir=~/.decnet/agent, optional state_db for byte-offset persistence.
  • Plaintext syslog across hosts is forbidden by project policy — see Syslog over TLS notes. Loopback only may use plaintext.

decnet/swarm/log_listener.py

Master-side RFC 5425 receiver. One mTLS-protected TCP socket on 6514; accepts connections from any worker whose cert is signed by the DECNET CA.

  • decnet/swarm/log_listener.py::ListenerConfiglog_path, json_path, bind_host="0.0.0.0", bind_port=6514, ca_dir=~/.decnet/ca.
  • decnet/swarm/log_listener.py::build_listener_ssl_context — server-side ssl.SSLContext: master presents ca/master/worker.crt, requires the peer to present a DECNET CA-signed cert. The CN on the peer cert is the authoritative worker identity — the RFC 5424 HOSTNAME field is untrusted input and is never used for authentication.

Agent — decnet/agent/

Worker-side daemon. FastAPI app behind uvicorn with mTLS on port 8765. Accepts deploy / teardown / status requests from the master and executes them locally.

See Remote-Updates for the lifecycle management around this process — the agent is not self-supervising.

decnet/agent/__init__.py

Empty package marker.

decnet/agent/app.py

  • decnet/agent/app.py::DeployRequest — pydantic body model: {config: DecnetConfig, dry_run: bool, no_cache: bool}.
  • decnet/agent/app.py::TeardownRequest{decky_id: str | None}.
  • decnet/agent/app.py::MutateRequest{decky_id: str, services: list[str]} (reserved; handler returns 501).
  • GET /health — returns {"status": "ok", "marker": "..."}. mTLS still required — the master's liveness probe carries its cert.
  • GET /status — awaits executor.status(); returns the worker's current deployment snapshot.
  • POST /deploy — calls executor.deploy(config, dry_run, no_cache). Returns {"status": "deployed", "deckies": int} on success, HTTPException(500) with the caught exception's message on failure.
  • POST /teardown — calls executor.teardown(decky_id).
  • POST /mutate — stub. Returns 501. Per-decky mutation is currently performed as a full /deploy with an updated DecnetConfig to avoid duplicating mutation logic worker-side.
  • FastAPI app itself is built with docs_url=None, redoc_url=None, openapi_url=None — no interactive docs on workers.

decnet/agent/server.py

uvicorn launcher. Not the app process itself — spawns uvicorn as a subprocess so signals land on a predictable PID and the tls-related flags live in one place.

  • Requires ~/.decnet/agent/{worker.key, worker.crt, ca.crt}. Missing bundle → prints an instructional error, exits 2 (operator likely forgot swarm enroll).
  • Spawns python -m uvicorn decnet.agent.app:app --host HOST --port PORT --ssl-keyfile <worker.key> --ssl-certfile <worker.crt> --ssl-ca-certs <ca.crt> --ssl-cert-reqs 2. The 2 is ssl.CERT_REQUIRED — no cert = TCP reset before any handler runs.

decnet/agent/executor.py

Thin async shim between the FastAPI handlers and the existing unihost orchestration code.

  • decnet/agent/executor.py::deploy — async wrapper around decnet.engine.deployer.deploy. Runs the blocking work off the event loop. If the worker's local NIC/subnet differs from what the master serialised, the config is relocalised before deploy (see engine.deployer for the rewriting rules).
  • decnet/agent/executor.py::teardown — async wrapper around decnet.engine.deployer.teardown.
  • decnet/agent/executor.py::status — calls decnet.config.load_state() and returns the snapshot dict verbatim.

Reads: decnet-state.json, Docker daemon. Writes: whatever the engine writes (compose file, docker networks/containers, state file).


Updater — decnet/updater/

Worker-side self-update daemon. FastAPI app behind uvicorn with mTLS on port 8766. Runs from /opt/decnet/venv/ initially, and from /opt/decnet/updater/venv/ after the first successful --include-self push. Never modified by a normal /update.

This is the daemon that owns the agent's lifecycle during a push — see Remote-Updates for the operator-facing view and PKI and mTLS for the cert story.

decnet/updater/__init__.py

Empty package marker.

decnet/updater/app.py

  • decnet/updater/app.py::_Config — module-level holder for the three paths the handlers need (install_dir, updater_install_dir, agent_dir). Defaults come from DECNET_UPDATER_INSTALL_DIR / DECNET_UPDATER_UPDATER_DIR / DECNET_UPDATER_AGENT_DIR, which server.py sets before spawning uvicorn.
  • decnet/updater/app.py::configure — injected-paths setter used by the server launcher. Must run before serving.
  • GET /health — returns {"status": "ok", "role": "updater", "releases": [...]}. The role field is the only thing that distinguishes this from the agent's /health to a caller that doesn't track ports.
  • GET /releases{"releases": [...]}; each release is {slot, sha, installed_at}.
  • POST /update — multipart: tarball: UploadFile, sha: str. Delegates to executor.run_update. Returns 500 on generic UpdateError, 409 if the update was already rolled back (operator should read the response body for stderr + probe transcripts).
  • POST /update-self — multipart: tarball, sha, confirm_self: str. The confirm_self.lower() != "true" guard is non-negotiable; there is no auto-rollback on this path.
  • POST /rollback — no body. 404 if there's no prev/ slot (fresh install), 500 on other failure.
  • FastAPI app built with docs_url=None, redoc_url=None, openapi_url=None.

decnet/updater/server.py

Same shape as the agent's server launcher — spawns uvicorn with mTLS flags. Reads ~/.decnet/updater/{updater.key, updater.crt, ca.crt}.

Before spawning uvicorn, exports:

  • DECNET_UPDATER_INSTALL_DIR — release root (/opt/decnet by default).
  • DECNET_UPDATER_UPDATER_DIR — updater's own install root (<install_dir>/updater).
  • DECNET_UPDATER_AGENT_DIR — agent bundle dir (for the local mTLS health probe after an update).
  • DECNET_UPDATER_BUNDLE_DIR — the updater's own cert bundle (~/.decnet/updater/).
  • DECNET_UPDATER_HOST, DECNET_UPDATER_PORT — needed so run_update_self can rebuild the operator-visible decnet updater ... command line when it os.execvs into the new binary.

decnet/updater/executor.py

The heart of the update pipeline. Every seam is named _foo and monkeypatched by tests so the test suite never shells out.

  • decnet/updater/executor.py::DEFAULT_INSTALL_DIR = /opt/decnet.
  • decnet/updater/executor.py::UpdateError(RuntimeError) — carries stderr: str (pip output) and rolled_back: bool.
  • decnet/updater/executor.py::Release(slot: str, sha: str | None, installed_at: datetime | None) dataclass, what /releases returns.
  • decnet/updater/executor.py::list_releases — scans install_dir/releases/*/release.json; returns them oldest-first.
  • decnet/updater/executor.py::run_update — the big one. Extracts the tarball into active.new/, runs _run_pip, rotates, _stop_agent, _spawn_agent, _probe_agent. On probe failure: flip symlink back to prev, restart agent, re-probe, raise UpdateError(rolled_back=True).
  • decnet/updater/executor.py::run_rollback — symbolic wrapper around the swap-and-restart path, for manual use via POST /rollback.
  • decnet/updater/executor.py::run_update_self — separate pipeline targeting updater_install_dir. Does not call _stop_agent/_spawn_agent; ends in os.execv so the process image is replaced. Rebuilds the argv from env vars (see server.py above) — sys.argv[1:] is the uvicorn subprocess invocation and cannot be reused.
  • decnet/updater/executor.py::_run_pip — on first use, bootstraps <install_dir>/venv/ with the full dep tree; subsequent calls use --force-reinstall --no-deps so the near-no-op case is cheap.
  • decnet/updater/executor.py::_spawn_agentsubprocess.Popen([<venv>/bin/decnet, "agent", "--daemon"], start_new_session=True, cwd=install_dir). Writes the new PID to agent.pid. cwd=install_dir is what lets a persistent <install_dir>/.env.local take effect.
  • decnet/updater/executor.py::_stop_agent — SIGTERM the PID in agent.pid, wait up to AGENT_RESTART_GRACE_S, SIGKILL the survivor. Falls back to _discover_agent_pids when no pidfile exists (manually-started agents) so restart is reliable regardless of how the agent was originally launched.
  • decnet/updater/executor.py::_discover_agent_pids — scans /proc/*/cmdline for any process whose argv contains decnet + agent. Skips its own PID. Returns an int list.
  • decnet/updater/executor.py::_probe_agent — mTLS GET https://127.0.0.1:8765/health up to 10 times with 1 s backoff. Uses a bare ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) rather than ssl.create_default_context() — on Python 3.13 the default context enables VERIFY_X509_STRICT, which rejects CA certs without AKI (which generate_ca doesn't emit).
  • decnet/updater/executor.py::_shared_venv — returns <install_dir>/venv. Central so every caller agrees on one path.

Reads: nothing persistent before the first update; afterwards, the release directories under install_dir/releases/. Writes: the release directories, agent.pid, agent.spawn.log, and the current symlink.

decnet/updater/routes/

Reserved for handler splits once the app grows. All routes currently live in app.py.