From e2612e87ab928daf8277b37862a0baa0fee6dc23 Mon Sep 17 00:00:00 2001 From: anti Date: Sat, 18 Apr 2026 06:16:32 -0400 Subject: [PATCH] wiki: recover Module-Reference Web/Services/Workers --- Module-Reference-Services.md | 88 ++++++++++++++ Module-Reference-Web.md | 161 ++++++++++++++++++++++++++ Module-Reference-Workers.md | 215 +++++++++++++++++++++++++++++++++++ 3 files changed, 464 insertions(+) create mode 100644 Module-Reference-Services.md create mode 100644 Module-Reference-Web.md create mode 100644 Module-Reference-Workers.md diff --git a/Module-Reference-Services.md b/Module-Reference-Services.md new file mode 100644 index 0000000..23947d3 --- /dev/null +++ b/Module-Reference-Services.md @@ -0,0 +1,88 @@ +# Module Reference: Services + +Code-level view of every service plugin under `decnet/services/`. For the user-facing description of what each honeypot does (personas, credentials, bait), see [Services catalog](Services-Catalog). For writing your own plugin, see [Writing a Service Plugin](Writing-a-Service-Plugin). + +Citation format: `decnet/services/::`. + +--- + +## Core + +### `services/base.py::BaseService` + +Abstract contract every plugin implements. Class attributes: + +- `name: str` — unique slug (e.g. `"ssh"`). +- `ports: list[int]` — ports the service listens on inside the container. +- `default_image: str` — Docker image tag, or `"build"` when a Dockerfile context is needed. +- `fleet_singleton: bool = False` — `True` means the service runs once fleet-wide, not per-decky (used by the sniffer). + +Methods: + +- `compose_fragment(decky_name, log_target, service_cfg) -> dict` — **abstract**; returns the docker-compose service dict. Networking keys are injected by the composer and **must not** be set here. +- `dockerfile_context() -> Path | None` — return the build-context path if a custom image is needed; `None` means `default_image` is pulled directly. + +### `services/registry.py` + +Plugin auto-discovery. + +- `registry.py::_load_plugins` — walks `decnet/services/`, imports every module, and collects all `BaseService` subclasses. +- `registry.py::register_custom_service` — register an out-of-tree plugin at runtime (used by tests and third-party integrations). +- `registry.py::get_service(name)` — fetch one plugin by slug. +- `registry.py::all_services()` — return `{slug: instance}` for every registered plugin. + +--- + +## Service plugins + +All rows are subclasses of `BaseService`. The `Image` column is `build` for every in-tree plugin — each one ships a Dockerfile context under `templates//`. See [Service Personas](Service-Personas) and [Archetypes](Archetypes) for how they're composed into deckies. + +| Slug | Ports | Image | Class | One-line description | +|------|-------|-------|-------|----------------------| +| `conpot` | 502, 161 (UDP), 80 | build | `ConpotService` | ICS/SCADA honeypot covering Modbus, SNMP, and HTTP. | +| `docker_api` | 2375, 2376 | build | `DockerAPIService` | Fake Docker Engine API (unauthenticated + TLS ports). | +| `elasticsearch` | 9200 | build | `ElasticsearchService` | Exposed Elasticsearch REST endpoint with index bait. | +| `ftp` | 21 | build | `FTPService` | vsftpd honeypot with captured-upload persistence. | +| `http` | 80, 443 | build | `HTTPService` | Realistic HTTP/HTTPS web server with persona templates. | +| `https` | 443 | build | `HTTPSService` | TLS-terminated web persona (paired with `http` or standalone). | +| `imap` | 143, 993 | build | `IMAPService` | Dovecot IMAP/IMAPS with fake mailboxes. | +| `k8s` | 6443, 8080 | build | `KubernetesAPIService` | Mock Kubernetes API server surface. | +| `ldap` | 389, 636 | build | `LDAPService` | OpenLDAP with a seeded fake directory. | +| `llmnr` | 5355, 5353 | build | `LLMNRService` | LLMNR/mDNS/NBNS poisoning detector. | +| `mongodb` | 27017 | build | `MongoDBService` | Exposed MongoDB with unauthenticated collections. | +| `mqtt` | 1883 | build | `MQTTService` | Mosquitto broker with discoverable topics. | +| `mssql` | 1433 | build | `MSSQLService` | SQL Server persona emulating login + TDS banner. | +| `mysql` | 3306 | build | `MySQLService` | MySQL server with weak creds and seeded schema. | +| `pop3` | 110, 995 | build | `POP3Service` | Dovecot POP3/POP3S counterpart to `imap`. | +| `postgres` | 5432 | build | `PostgresService` | PostgreSQL server with bait databases. | +| `rdp` | 3389 | build | `RDPService` | RDP (xrdp) honeypot with credential capture. | +| `redis` | 6379 | build | `RedisService` | Unauthenticated Redis with bait keys. | +| `sip` | 5060 | build | `SIPService` | SIP/VoIP registrar bait. | +| `smb` | 445, 139 | build | `SMBService` | Samba server with seeded fake shares. | +| `smtp` | 25, 587 | build | `SMTPService` | Postfix SMTP endpoint with auth capture. | +| `smtp_relay` | 25, 587 | build | `SMTPRelayService` | Open-relay bait — accepts any RCPT TO and delivers. | +| `sniffer` | — | build | `SnifferService` | Fleet-wide passive MACVLAN sniffer (`fleet_singleton=True`). | +| `snmp` | 161 (UDP) | build | `SNMPService` | SNMP agent with fake MIB tree. | +| `ssh` | 22 | build | `SSHService` | Interactive OpenSSH (replaces Cowrie) for realistic fingerprints. | +| `telnet` | 23 | build | `TelnetService` | Real `busybox telnetd` with rsyslog logging pipeline. | +| `tftp` | 69 (UDP) | build | `TFTPService` | TFTP server bait for firmware-fetch probing. | +| `vnc` | 5900 | build | `VNCService` | VNC server with weak auth. | + +--- + +## How they're assembled + +- `engine/deployer.py` reads the archetype-selected slugs and calls each plugin's `compose_fragment()` to build `docker-compose.yml`. See [Module Reference: Workers](Module-Reference-Workers) for the deployer internals. +- The canonical `syslog_bridge.py` is copied into every active build context by `engine/deployer.py::_sync_logging_helper` so each container forwards logs identically. See [Logging](Logging-and-Syslog). + +--- + +## See also + +- [Services catalog](Services-Catalog) — user view: personas, creds, what each honeypot looks like to an attacker. +- [Writing a Service Plugin](Writing-a-Service-Plugin) — step-by-step guide. +- [Archetypes](Archetypes) — how slugs are bundled into OS personas. +- [Design overview](Design-Overview) +- [REST API](REST-API-Reference) +- [Logging](Logging-and-Syslog) +- [Developer guide](Developer-Guide) diff --git a/Module-Reference-Web.md b/Module-Reference-Web.md new file mode 100644 index 0000000..c8e5a17 --- /dev/null +++ b/Module-Reference-Web.md @@ -0,0 +1,161 @@ +# Module Reference: Web + +Code-level reference for every module under `decnet/web/`. For the user-facing API see [REST API](REST-API-Reference); for the dashboard UX see [Web Dashboard](Web-Dashboard); architecture is in [Design overview](Design-Overview). + +Citation format: `decnet/web/::`. + +--- + +## `decnet/web/api.py` + +FastAPI application factory and lifespan wiring. Instantiates the `FastAPI` app, mounts every router under `decnet/web/router/`, attaches exception handlers, and owns the background-task handles for the ingester, profiler, prober, sniffer, and correlation workers. + +- `api.py::get_background_tasks` — returns the live `dict` of worker task handles so the `/health` endpoint can introspect liveness. +- `api.py::lifespan` — async context manager run by FastAPI on startup/shutdown: initializes the repository, seeds the admin user, spawns worker tasks, and cancels them on exit. +- `api.py::validation_exception_handler` — maps request-validation errors to the specific HTTP codes the contract tests require. +- `api.py::pydantic_validation_exception_handler` — handles Pydantic errors raised during manual model instantiation (e.g. state hydration). + +--- + +## `decnet/web/auth.py` + +Password hashing and JWT issuance primitives. No business logic — just stateless helpers shared by the login router and dependency injectors. + +- `auth.py::verify_password` — sync bcrypt verify. +- `auth.py::get_password_hash` — sync bcrypt hash. +- `auth.py::averify_password` — async verify (offloaded to a threadpool). +- `auth.py::ahash_password` — async hash (threadpool). +- `auth.py::create_access_token` — issue a signed JWT with configured TTL and subject claim. + +--- + +## `decnet/web/ingester.py` + +Background worker that tails the DECNET ingest log (`DECNET_INGEST_LOG_FILE.json`) and writes rows into the `logs` and `bounties` tables. Checkpoints file offset via the `State` table so restarts are idempotent. + +- `ingester.py::log_ingestion_worker` — asyncio task loop: tail + parse + flush batches. +- `ingester.py::_flush_batch` — commits a batch of log rows and returns the new file position (bulk-insert, one transaction). +- `ingester.py::_extract_bounty` — inspects a parsed log row and returns a bounty payload (creds, uploaded files, etc.) or `None`. +- Const: `_INGEST_STATE_KEY` — key under which the tail offset is stored in `State`. + +--- + +## `decnet/web/dependencies.py` + +FastAPI dependencies for repository injection, authentication, RBAC, and user-cache management. Every router imports from here — never directly from the repo module. + +- `dependencies.py::get_repo` — repository DI entrypoint; returns the configured `BaseRepository` singleton. +- `dependencies.py::invalidate_user_cache` — drop a single username (or all) from the auth caches (`_get_user_cached`, `get_user_by_username_cached`). +- `dependencies.py::get_user_by_username_cached` — TTL-cached lookup used on the login hot path. +- `dependencies.py::get_stream_user` — auth for SSE endpoints; accepts Bearer header **or** `?token=` query param (EventSource cannot set headers). +- `dependencies.py::get_current_user` — Bearer-token auth; enforces the `must_change_password` flag. +- `dependencies.py::get_current_user_unchecked` — same, but skips the change-password gate (used by the change-password endpoint itself). +- `dependencies.py::require_role` — factory returning an RBAC dependency enforcing role membership (`admin`, `analyst`, `viewer`). +- `dependencies.py::require_stream_role` — same, for SSE endpoints that take a query-param token. +- Private helpers: `_reset_user_cache`, `_get_user_cached`, `_decode_token` — cache reset (tests), cached user fetch by UUID, JWT decode. +- Consts: `_USER_TTL`, `_USERNAME_TTL` — cache TTLs in seconds. + +--- + +## `decnet/web/db/` + +### `db/factory.py` +Repository factory. Selects a `BaseRepository` implementation based on the `DECNET_DB_TYPE` env var (`sqlite` or `mysql`). Returns a module-level singleton. + +- `db/factory.py::get_repository` — construct and cache the active repository. + +### `db/repository.py` +Abstract base class declaring every storage operation the dashboard and workers rely on. + +- `db/repository.py::BaseRepository` — abstract interface. + Notable methods: `initialize`, `add_log`, `add_logs`, `get_logs`, `get_total_logs`, `get_stats_summary`, `get_deckies`, `get_user_by_username`, `get_user_by_uuid`, `create_user`, `update_user_password`, `list_users`, `delete_user`, `update_user_role`, `purge_logs_and_bounties`, `add_bounty`, `get_bounties`, `get_total_bounties`, `get_state`, `set_state`, `get_max_log_id`, `get_logs_after_id`, `get_all_bounties_by_ip`, `get_bounties_for_ips`, `upsert_attacker`, `upsert_attacker_behavior`, `get_attacker_behavior`, `get_behaviors_for_ips`, `get_attacker_by_uuid`, `get_attackers`, `get_total_attackers`, `get_attacker_commands`, `get_attacker_artifacts`. + +### `db/sqlmodel_repo.py` +Shared SQLModel/SQLAlchemy-async concrete implementation. Both the SQLite and MySQL backends subclass this and override only dialect-specific bits (DDL introspection, `get_log_histogram`). + +- `db/sqlmodel_repo.py::SQLModelRepository` — concrete async repository. +- `db/sqlmodel_repo.py::_detach_close` — runs session close in a fresh asyncio task so cancellation in the caller never leaves sessions hanging. +- `db/sqlmodel_repo.py::_safe_session` — session context manager with cancellation-safe cleanup. +- Key methods: `initialize`, `reinitialize`, `add_log`, `add_logs`, `get_logs`, `get_log_histogram` (override per backend), `get_stats_summary`, `upsert_attacker`, `upsert_attacker_behavior`, `get_attacker_commands`, `get_attacker_artifacts`. Many private helpers for filter composition, JSON-field equality, and row normalization. + +### `db/sqlite/database.py` +- `get_async_engine`, `get_sync_engine`, `init_db`, `get_session` — SQLite engine and session factories. `init_db` creates tables synchronously (used by CLI/tests). + +### `db/sqlite/repository.py` +- `SQLiteRepository` — `SQLModelRepository` subclass using `aiosqlite`. Overrides `_migrate_attackers_table`, `_json_field_equals`, `get_log_histogram` for SQLite dialect. + +### `db/mysql/database.py` +MySQL async engine factory. + +- `build_mysql_url` — compose an async SQLAlchemy URL using `asyncmy`. +- `resolve_url` — explicit arg → `DECNET_DB_URL` env → built from components. +- `get_async_engine` — create the `AsyncEngine`. +- Consts: `DEFAULT_POOL_SIZE`, `DEFAULT_MAX_OVERFLOW`, `DEFAULT_POOL_RECYCLE`, `DEFAULT_POOL_PRE_PING`. + +### `db/mysql/repository.py` +- `MySQLRepository` — `SQLModelRepository` subclass using `asyncmy`. Adds MySQL migrations (`_migrate_attackers_table`, `_migrate_column_types` to upgrade `TEXT` → `MEDIUMTEXT` on large-JSON columns) and dialect-specific histogram SQL. + +### `db/models.py` +SQLModel table classes and Pydantic request/response DTOs. + +- Tables: `User`, `Log`, `Bounty`, `State`, `Attacker`, `AttackerBehavior` (1:1 with `Attacker` by UUID). +- Request models: `LoginRequest`, `ChangePasswordRequest`, `MutateIntervalRequest`, `DeployIniRequest`, `CreateUserRequest`, `UpdateUserRoleRequest`, `ResetUserPasswordRequest`, `DeploymentLimitRequest`, `GlobalMutationIntervalRequest`. +- Response models: `Token`, `LogsResponse`, `BountyResponse`, `AttackersResponse`, `StatsResponse`, `UserResponse`, `ConfigResponse`, `AdminConfigResponse`, `ComponentHealth`, `HealthResponse`. +- Helpers: `_normalize_null` — collapses SQL `NULL` / empty-string variance. + +--- + +## `decnet/web/router/` + +All route modules follow one pattern: declare a module-level `APIRouter`, attach a single endpoint function. The aggregator `router/__init__.py` imports and mounts them. Response caches (`functools.lru_cache` with TTL) are used on read endpoints where invalidation is explicit. + +### `router/logs/` +- `api_get_logs.py::get_logs` — paginated log feed with filters. Cached total-count via `_get_total_logs_cached` / `_reset_total_cache`. +- `api_get_histogram.py::get_logs_histogram` — binned log timeseries for the dashboard chart. + +### `router/stats/` +- `api_get_stats.py::get_stats` — dashboard summary: event counts, attackers, bounties, deckies. Cached. + +### `router/stream/` +- `api_stream_events.py::stream_events` — Server-Sent Events endpoint streaming new logs by id. Uses `get_stream_user` (query-param token). `_build_trace_links` builds OTEL span links from persisted `trace_id` / `span_id`. + +### `router/bounty/` +- `api_get_bounties.py::get_bounties` — paginated bounty (captured credentials/files) feed. TTL-cached defaults. + +### `router/attackers/` +- `api_get_attackers.py::get_attackers` — paginated attacker profile list. Cached total. +- `api_get_attacker_detail.py::get_attacker_detail` — single attacker profile joined with its `AttackerBehavior` row. +- `api_get_attacker_commands.py::get_attacker_commands` — paginated executed-commands list per attacker, optional service filter. +- `api_get_attacker_artifacts.py::get_attacker_artifacts` — captured file-drop artifacts for an attacker, newest first. + +### `router/artifacts/` +- `api_get_artifact.py::get_artifact` — download a single captured artifact. `_resolve_artifact_path` validates inputs, resolves the on-disk path, and asserts it stays under `ARTIFACTS_ROOT` (path-traversal guard). Regex guards `_DECKY_RE`, `_STORED_AS_RE` sanitize path components. + +### `router/fleet/` +- `api_deploy_deckies.py::api_deploy_deckies` — server-side deploy trigger (admin only). See [CLI Reference](CLI-Reference) for the same operation from shell. +- `api_get_deckies.py::get_deckies` — running decky inventory. Cached. +- `api_mutate_decky.py::api_mutate_decky` — force-mutate a single decky. See [Mutation and Randomization](Mutation-and-Randomization). +- `api_mutate_interval.py::api_update_mutate_interval` — set a decky's per-machine mutation interval. `_parse_duration` converts `5d`/`12h` strings to minutes. + +### `router/config/` +- `api_get_config.py::api_get_config` — read the active config and user list (cached via `_get_list_users_cached` and `_get_state_cached`). `invalidate_list_users_cache` is called after any user mutation. +- `api_update_config.py::api_update_deployment_limit`, `api_update_global_mutation_interval` — write config knobs. +- `api_manage_users.py::api_create_user`, `api_delete_user`, `api_update_user_role`, `api_reset_user_password` — admin-only user CRUD. +- `api_reinit.py::api_reinit` — destructive reinit endpoint (drops logs/bounties/attackers, keeps users). Admin only. + +### `router/auth/` +- `api_login.py::login` — username/password → JWT. +- `api_change_pass.py::change_password` — enforced when `must_change_password` is set; uses `get_current_user_unchecked`. + +### `router/health/` +- `api_get_health.py::get_health` — aggregated health: DB liveness (`_check_database_cached`), Docker daemon reachable, background-worker statuses from `api.get_background_tasks()`. Cache reset helpers (`_reset_db_cache`, `_reset_docker_cache`) exist for tests. + +--- + +## See also + +- [Design overview](Design-Overview) +- [REST API](REST-API-Reference) +- [Logging](Logging-and-Syslog) +- [Services catalog](Services-Catalog) +- [Developer guide](Developer-Guide) diff --git a/Module-Reference-Workers.md b/Module-Reference-Workers.md new file mode 100644 index 0000000..2401c7d --- /dev/null +++ b/Module-Reference-Workers.md @@ -0,0 +1,215 @@ +# Module Reference: Workers + +Code-level reference for every worker/subsystem module outside `decnet/web/`. For the pipeline overview see [Design overview](Design-Overview); for log format see [Logging](Logging-and-Syslog). + +Each subsection lists: **entrypoint** (function spawned by the lifespan or CLI), **loop** (what it does on each tick), **reads** (input source), **writes** (output sink). + +Citation format: `decnet/::`. + +--- + +## `decnet/collector/` — Host-side Docker log collector + +Streams logs out of every DECNET-owned container on the host, parses the RFC 5424 line format, and appends to the shared JSON ingest file that `decnet/web/ingester.py` tails. + +### `collector/worker.py` + +- **Entrypoint:** `worker.py::log_collector_worker` — asyncio task started by the API lifespan. +- **Loop:** enumerate running service containers via the Docker SDK; per container, spawn `_stream_container` which opens a log stream and appends parsed events to the ingest file. Rate-limits duplicate events. +- **Reads:** Docker API container log streams; `decnet-state.json` (for the canonical service container name set). +- **Writes:** JSON ingest log file (consumed by `web/ingester.py`). + +Key functions: +- `worker.py::parse_rfc5424` — parse a DECNET RFC 5424 line into a structured dict (SD params, IP fields, msg k/v). +- `worker.py::is_service_container`, `is_service_event` — gate which containers to ingest. +- `worker.py::_stream_container` — per-container streaming loop; `_reopen_if_needed` handles inode changes (log rotation). +- `worker.py::_should_ingest` — dedup + rate-limit filter. +- Consts: `_RFC5424_RE`, `_SD_BLOCK_RE`, `_PARAM_RE`, `_IP_FIELDS`, `_MSG_KV_RE`. + +--- + +## `decnet/profiler/` — Attacker profile builder + +Incrementally joins raw log rows into per-attacker profiles in the `attackers` + `attacker_behavior` tables. + +### `profiler/worker.py` + +- **Entrypoint:** `worker.py::attacker_profile_worker` — long-running asyncio task. +- **Loop:** every N seconds, fetch logs with `id > last_id` (`_incremental_update`), bucket by attacker IP, call `_update_profiles` to upsert `Attacker` and `AttackerBehavior` rows. State key `_STATE_KEY` persists the last-seen log id. +- **Reads:** `logs` table (incremental via `get_logs_after_id`). +- **Writes:** `attackers`, `attacker_behavior`, `state` tables. + +Helpers: `_build_record`, `_first_contact_deckies`, `_extract_commands_from_events`. Consts: `_BATCH_SIZE`, `_COMMAND_EVENT_TYPES`, `_COMMAND_FIELDS`. + +### `profiler/behavioral.py` + +Stateless analyzers used by `worker.py::_update_profiles`. + +- `behavioral.py::timing_stats` — inter-arrival time mean/stddev/CV. +- `behavioral.py::classify_behavior` — coarse behavior bucket (scanner / interactive / automated). +- `behavioral.py::guess_tools` — match (mean IAT, CV) against C2 beacon profiles. +- `behavioral.py::detect_tools_from_headers` — scan HTTP `request` events for tool-identifying headers. +- `behavioral.py::phase_sequence` — derive recon→exfil transitions. +- `behavioral.py::sniffer_rollup` — roll up `tcp_syn_fingerprint` + `tcp_flow_timing` events. +- `behavioral.py::build_behavior_record` — compose the dict persisted to `attacker_behavior`. +- Helpers: `_os_from_ttl`, `_int_or_none`, `guess_tool` (deprecated, kept for BC). + +--- + +## `decnet/sniffer/` — Fleet-wide passive MACVLAN sniffer + +Runs as a `fleet_singleton` container on the MACVLAN (or falls back to the `ipvlan` host iface) and emits TLS/TCP fingerprints for every flow touching a decky. + +### `sniffer/worker.py` + +- **Entrypoint:** `worker.py::sniffer_worker` — started as `asyncio.create_task` by the API lifespan. +- **Loop:** `worker.py::_sniff_loop` (runs in a dedicated thread via `asyncio.to_thread`) drives `scapy.sniff` on the MACVLAN interface; each packet goes to `SnifferEngine.on_packet`. +- **Reads:** live packets on the sniffer interface; `decnet-state.json` → `_load_ip_to_decky`. +- **Writes:** RFC 5424 syslog lines via `sniffer/syslog.py::write_event` (both raw syslog and parsed JSON). + +Helpers: `_interface_exists`, `_load_ip_to_decky`. + +### `sniffer/fingerprint.py::SnifferEngine` + +Stateful TLS + TCP fingerprint engine. Tracks sessions, TCP RTTs, flow aggregation, dedup. Emits `tls_handshake`, `tcp_syn_fingerprint`, `tcp_flow_timing` events. + +Notable methods: `on_packet` (main dispatch), `_update_flow`, `_flush_flow`, `flush_all_flows`, `_flush_idle_flows`, `_resolve_decky`, `_is_duplicate`. Private parsers: `_parse_client_hello`, `_parse_server_hello`, `_parse_certificate`, `_parse_x509_der`, `_extract_sans`; fingerprint algorithms: `_ja3`, `_ja3s`, `_ja4`, `_ja4s`, `_ja4l`. + +### `sniffer/p0f.py` + +Passive OS fingerprinting (p0f-lite). + +- `initial_ttl` — round observed TTL up to the nearest initial-TTL bucket. +- `hop_distance` — estimate hop count. +- `guess_os` — coarse OS bucket from SYN characteristics. +- `_match_signature` — predicate evaluator. + +### `sniffer/syslog.py` + +RFC 5424 emitter for the sniffer. + +- `syslog.py::syslog_line` — build a line. +- `syslog.py::write_event` — append a syslog line to the raw log and its parsed JSON to the json log. +- Helpers: `_sd_escape`, `_sd_element`. Consts: `_FACILITY_LOCAL0`, `_SD_ID`, `_NILVALUE`, `SEVERITY_INFO`, `SEVERITY_WARNING`, `_MAX_HOSTNAME`, `_MAX_APPNAME`, `_MAX_MSGID`. + +--- + +## `decnet/prober/` — Active attacker fingerprinting + +Standalone worker process (separate from the API) that actively probes IPs observed in the logs to collect JARM, HASSHServer, and TCP/IP fingerprints without ever impersonating DECNET. + +### `prober/worker.py` + +- **Entrypoint:** `worker.py::prober_worker` — main loop for the standalone prober process. +- **Loop:** tail the JSON ingest log (`_discover_attackers`), cycle through unique attacker IPs (`_probe_cycle`), run `_jarm_phase`, `_hassh_phase`, `_tcpfp_phase`, emit results as RFC 5424 events via `_syslog_line` + `_write_event`. +- **Reads:** JSON ingest log; configured probe targets. +- **Writes:** RFC 5424 events into the ingest file so the regular collector pipeline picks them up. + +### `prober/jarm.py` + +Pure-stdlib JARM TLS fingerprinting (10 ClientHello probes, reassembled into the 62-char JARM hash). + +- `jarm.py::jarm_hash` — public entrypoint. +- Helpers: `_build_client_hello`, `_send_probe`, `_parse_server_hello`, `_compute_jarm`; per-extension builders (`_ext_*`). + +### `prober/hassh.py` + +HASSHServer SSH fingerprinting (MD5 of `kex;enc_s2c;mac_s2c;comp_s2c` name-lists). + +- `hassh.py::hassh_server` — public entrypoint. +- Helpers: `_ssh_connect`, `_read_banner`, `_read_ssh_packet`, `_parse_kex_init`, `_compute_hassh`, `_recv_exact`. + +### `prober/tcpfp.py` + +TCP/IP stack fingerprinting via crafted SYN + SYN-ACK analysis (scapy). + +- `tcpfp.py::tcp_fingerprint` — public entrypoint. +- Helpers: `_send_syn`, `_send_rst`, `_parse_synack`, `_extract_options_order`, `_compute_fingerprint`. + +--- + +## `decnet/mutator/` — Persona mutation engine + +Periodically swaps a decky's services/personas (intra-archetype shuffle) so the same IP/MAC shows different attack surfaces over time. See [Mutation and Randomization](Mutation-and-Randomization). + +### `mutator/engine.py` + +- **Entrypoint:** `engine.py::run_watch_loop` — infinite loop checking for deckies past their mutation interval; spawned by the CLI or the API. +- **Loop:** `mutate_all` iterates due deckies → `mutate_decky` runs intra-archetype shuffle (rewrite compose, `docker compose up -d`). +- **Reads:** `decnet-state.json`, repository (per-decky mutation interval). +- **Writes:** rewritten `docker-compose.yml`, updated state, new containers. + +--- + +## `decnet/correlation/` — Cross-decky attacker traversal + +Builds attacker-traversal graphs by correlating events across multiple deckies. + +### `correlation/engine.py::CorrelationEngine` + +- **Entrypoint:** `ingest_file(path)` (batch) or `ingest(line)` (stream) — no background task by default; callable from CLI and tests. +- **Loop:** per-line: parse with `parser.parse_line`, append `TraversalHop` to per-IP `AttackerTraversal`. +- **Reads:** RFC 5424 syslog lines (file or stream). +- **Writes:** nothing by default; exposes `traversals(min_deckies)`, `all_attackers()`, `report_table()`, `report_json()`, `traversal_syslog_lines()`. Helper `_fmt_duration`. + +### `correlation/graph.py` +Data classes. `TraversalHop` = a single event; `AttackerTraversal` = all activity from one IP across ≥ 2 deckies with `first_seen`, `last_seen`, `duration_seconds`, `deckies` (ordered), `decky_count`, `path` (`decky-01 → decky-03 → decky-07`), `to_dict`. + +### `correlation/parser.py` +Stateless RFC 5424 parser shared by `correlation/engine.py` and ad-hoc tooling. + +- `parser.py::parse_line` — parse one line into `LogEvent`. +- `parser.py::LogEvent` — parsed event record. +- Helpers: `_parse_sd_params`, `_extract_attacker_ip`. Consts: `_RFC5424_RE`, `_SD_BLOCK_RE`, `_PARAM_RE`, `_IP_FIELDS`. + +--- + +## `decnet/engine/` — Deploy/teardown orchestration + +### `engine/deployer.py` + +- **Entrypoint:** `deployer.py::deploy`, `teardown`, `status` — called from the CLI and the fleet router. +- **Loop (`deploy`):** render `docker-compose.yml` from archetypes + service plugins → sync the canonical `syslog_bridge.py` into every build context (`_sync_logging_helper`) → invoke `docker compose up -d` via `_compose` with retry on transient failures (`_compose_with_retry`). +- **Reads:** CLI args / INI config / archetype definitions / service plugin `compose_fragment`. +- **Writes:** `docker-compose.yml`, built images, running containers, `decnet-state.json`. + +Consts: `COMPOSE_FILE`, `_CANONICAL_LOGGING`, `_PERMANENT_ERRORS`. Helper: `_print_status`. + +See [Deployment Modes](Deployment-Modes) and [Teardown and State](Teardown-and-State). + +--- + +## `decnet/logging/` — Application logging helpers + +Standard-library `logging` integration that makes DECNET's own Python logs RFC 5424-compliant and mergeable with service logs. + +### `logging/__init__.py` +- `__init__.py::get_logger(component)` — return a named logger that self-identifies as `component` in RFC 5424. +- `__init__.py::enable_trace_context` — install the OTEL trace-context filter on the root `decnet` logger. +- `__init__.py::_ComponentFilter`, `_TraceContextFilter` — inject `decnet_component`, `otel_trace_id`, `otel_span_id` onto LogRecords so the formatter can emit them as SD params. + +### `logging/syslog_formatter.py` +- `syslog_formatter.py::format_rfc5424` — return a single RFC 5424-compliant syslog line (no trailing newline). +- Helpers: `_pri`, `_truncate`, `_sd_escape`, `_sd_element`. Consts: `FACILITY_LOCAL0`, `NILVALUE`, `_SD_ID`, `SEVERITY_INFO`, `SEVERITY_WARNING`, `SEVERITY_ERROR`, `_MAX_HOSTNAME`, `_MAX_APPNAME`, `_MAX_MSGID`. + +### `logging/file_handler.py` +- `file_handler.py::write_syslog` — write a single RFC 5424 line to the rotating log file. +- `file_handler.py::get_log_path` — return the configured log file path. +- `file_handler.py::_init_file_handler`, `_get_logger`. Consts: `_LOG_FILE_ENV`, `_DEFAULT_LOG_FILE`, `_MAX_BYTES`, `_BACKUP_COUNT`. + +### `logging/inode_aware_handler.py` +- `inode_aware_handler.py::InodeAwareRotatingFileHandler` — RotatingFileHandler that detects external deletion/rotation (logrotate) and reopens the target. Methods: `_should_reopen`, `emit`. + +### `logging/forwarder.py` +- `forwarder.py::parse_log_target` — parse `"ip:port"` into `(host, port)`. +- `forwarder.py::probe_log_target` — TCP-connect probe for reachability. + +--- + +## See also + +- [Design overview](Design-Overview) +- [REST API](REST-API-Reference) +- [Logging](Logging-and-Syslog) +- [Services catalog](Services-Catalog) +- [Developer guide](Developer-Guide)