1
Module Reference Core
anti edited this page 2026-04-18 06:17:53 -04:00
This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Module Reference — Core

Every Python module in decnet/ (top level, excluding sub-packages). This is the code view. For user-facing material see Design overview, REST API, and Logging.

Citation format used throughout: decnet/<module>.py::<symbol>.


decnet/__init__.py

Empty package marker. No exports.


decnet/cli.py

Typer-based command-line entry point for DECNET. Defines every decnet <verb> subcommand, process-group control for uvicorn workers, double-fork daemonisation, and a microservice registry used by decnet status / decnet redeploy to check and relaunch the supervisor processes (Collector, Mutator, Prober, Profiler, Sniffer, API).

  • decnet/cli.py::app — the root typer.Typer instance registered by the decnet console script.
  • decnet/cli.py::console — module-level rich.console.Console used for all user-facing output.
  • decnet/cli.py::_daemonize — Unix double-fork that redirects stdio to /dev/null; called by every command that accepts --daemon.
  • decnet/cli.py::_kill_all_services — iterates the service registry and sends SIGTERM to every running DECNET microservice PID; invoked by teardown --all.
  • decnet/cli.py::api — starts uvicorn for decnet.web.api:app in a new session group so Ctrl+C can tear down the worker tree.
  • decnet/cli.py::deploy — main deploy command. Resolves interface/subnet/IP allocation, optionally parses an INI, builds the DecnetConfig, invokes decnet.engine.deploy, then spawns the mutator/collector/prober/profiler/sniffer/api children as requested.
  • decnet/cli.py::probe — runs decnet.prober.prober_worker (JARM/HASSH/TCP fingerprint loop).
  • decnet/cli.py::collect — runs decnet.collector.log_collector_worker to stream Docker container logs to the RFC 5424 file.
  • decnet/cli.py::mutate — trigger mutation manually (--decky, --all) or in watch mode (--watch).
  • decnet/cli.py::status — prints decky table via decnet.engine.status plus the DECNET-service table.
  • decnet/cli.py::teardown — stops containers (--all / --id) and kills services when --all.
  • decnet/cli.py::list_servicesdecnet services — prints registered service plugins.
  • decnet/cli.py::list_distrosdecnet distros — prints distro profiles.
  • decnet/cli.py::list_archetypesdecnet archetypes — prints archetype profiles.
  • decnet/cli.py::correlate — runs CorrelationEngine over a log file or stdin; emits table/json/syslog.
  • decnet/cli.py::redeploy — health-checks each service in the registry and relaunches any that are down.
  • decnet/cli.py::serve_webdecnet web — serves the Vite dist/ build and reverse-proxies /api/* to the API port (with SSE-aware read1()/disabled socket timeout).
  • decnet/cli.py::profiler_cmd — standalone decnet profiler worker.
  • decnet/cli.py::sniffer_cmd — standalone decnet sniffer worker.
  • decnet/cli.py::db_resetdecnet db-reset — MySQL-only destructive wipe (dry-run unless --i-know-what-im-doing).
  • decnet/cli.py::_db_reset_mysql_async — coroutine that inspects row counts and optionally TRUNCATEs or DROPs the DECNET tables. Extracted from the CLI wrapper so tests can drive it without Typer.
  • decnet/cli.py::_is_running — scans psutil.process_iter for a cmdline matching a predicate; returns PID or None.
  • decnet/cli.py::_service_registry — returns the authoritative (name, match_fn, launch_args) triples used for status/redeploy.
  • decnet/cli.py::_DB_RESET_TABLES — drop order tuple for the MySQL reset; attacker_behavior must be dropped before attackers because of the FK.

decnet/models.py

Centralised Pydantic domain models — no web or DB imports. Used by the CLI, the INI loader, the fleet builder and the web API alike so that core logic stays isolated from adapters.

  • decnet/models.py::validate_ini_string — structural validator for INI text (≤ 512 KB, non-empty, must parse and contain at least one section). Raises ValueError with phrasing the tests assert against.
  • decnet/models.py::IniContentAnnotated[str, BeforeValidator(validate_ini_string)] alias for fields that accept raw INI.
  • decnet/models.py::DeckySpec — strict INI spec for a single decky (name, ip, services, archetype, service_config, nmap_os, mutate_interval).
  • decnet/models.py::CustomServiceSpec[custom-*] INI section (name, image, exec_cmd, ports).
  • decnet/models.py::IniConfig — the parsed INI as a whole; enforces at_least_one_decky.
  • decnet/models.py::DeckyConfig — runtime deployment record for one decky (base_image, build_base, hostname, nmap_os, mutate_interval, last_mutated, last_login_attempt).
  • decnet/models.py::DecnetConfig — root config for a whole fleet (mode, interface, subnet, gateway, deckies, log_file, ipvlan, mutate_interval).

decnet/ini_loader.py

INI parser for DECNET deployment files. Two-pass: first collects decky sections and [custom-*] service definitions, then walks again to attach [decky.service] per-service persona subsections — including expanding into group-01, group-02, … when the INI uses amount=N.

  • decnet/ini_loader.py::load_ini — read and parse an INI file, returning IniConfig.
  • decnet/ini_loader.py::load_ini_from_string — normalise CRLF/CR to LF, validate, parse, return IniConfig. Used by the web API update-config endpoint.
  • decnet/ini_loader.py::_parse_configparser — shared logic that walks a configparser.ConfigParser and emits IniConfig; implements the two-pass strategy and the amount= fan-out.

decnet/composer.py

Generates a docker-compose.yml dict from a DecnetConfig. Each decky has one base container that owns the MACVLAN IP and runs sleep infinity; every service container attaches via network_mode: "service:<base>" so all services on that decky share a single externally-visible IP. Service containers' Docker logs are captured by the json-file driver (10 MB × 5 rotation) and streamed by the host-side collector — no bind mounts.

  • decnet/composer.py::generate_compose — build and return the compose dict (services, networks). Injects get_os_sysctls(decky.nmap_os) into the base container so its namespace matches the claimed TTL/TCP-option profile; adds NET_ADMIN for the sysctl writes; seeds build.args.BASE_IMAGE so per-service Dockerfiles render the correct distro.
  • decnet/composer.py::write_compose — dump the dict to YAML at a path and return the path.
  • decnet/composer.py::_DOCKER_LOGGING — the json-file rotation options (max-size: 10m, max-file: 5).

decnet/network.py

Host networking primitives: interface/subnet autodetection, IP allocation inside a subnet (skipping reserved + host + gateway), Docker MACVLAN and IPvlan network create/remove, and the host-side decnet_macvlan0 / decnet_ipvlan0 shim interfaces that fix the MACVLAN hairpin problem so the host can reach deckies it just spawned.

  • decnet/network.py::MACVLAN_NETWORK_NAME"decnet_lan" — Docker network name used everywhere.
  • decnet/network.py::HOST_MACVLAN_IFACE / HOST_IPVLAN_IFACE — host-side shim interface names.
  • decnet/network.py::detect_interface — parse ip route show default and return the outbound dev.
  • decnet/network.py::detect_subnet — parse ip addr show <iface> + default route; returns (cidr, gateway).
  • decnet/network.py::get_host_ip — extract the host's IPv4 on a given interface.
  • decnet/network.py::allocate_ips — yield count IPs from a subnet, skipping net/broadcast/gateway/host and honouring ip_start.
  • decnet/network.py::create_macvlan_network / create_ipvlan_network — create the Docker driver network (no-op if it already exists).
  • decnet/network.py::remove_macvlan_network — remove the Docker network.
  • decnet/network.py::setup_host_macvlan / setup_host_ipvlan — build the host shim interface, assign a /32, add route to the decky range. Idempotent. Requires root.
  • decnet/network.py::teardown_host_macvlan / teardown_host_ipvlan — inverse of the above.
  • decnet/network.py::ips_to_range — compute the tightest CIDR that covers a given list of IPs. Used for --ip-range on MACVLAN.
  • decnet/network.py::_require_root / _run — internal helpers (euid check, subprocess.run wrapper).

decnet/config.py

Installs the RFC 5424 root logger used by every DECNET process, defines the state-file persistence helpers (decnet-state.json), and re-exports DeckyConfig/DecnetConfig from models.py. Imported very early so that merely doing import decnet.config configures logging side-effectfully.

  • decnet/config.py::Rfc5424Formatter — logging formatter emitting <PRI>1 TS HOST APP PID MSGID - MSG with microsecond UTC timestamp; honours record.decnet_component to override the APP-NAME field.
  • decnet/config.py::_configure_logging — idempotent root-logger setup: StreamHandler (stderr) plus InodeAwareRotatingFileHandler to DECNET_SYSTEM_LOGS; skips the file handler under pytest; chowns the file back to the sudo-invoking user.
  • decnet/config.py::STATE_FILEPath pointing at <repo>/decnet-state.json.
  • decnet/config.py::DEFAULT_MUTATE_INTERVAL30 minutes.
  • decnet/config.py::random_hostname — thin re-export of decnet.distros.random_hostname.
  • decnet/config.py::save_state / load_state / clear_state — persist the DecnetConfig + compose path across CLI invocations.
  • decnet/config.py::_SYSLOG_SEVERITY / _FACILITY_LOCAL0 — RFC 5424 §6.2.1 mapping tables used by the formatter.

decnet/env.py

Environment-variable loader. Resolves .env.local first then .env, exposes every DECNET_* setting as a module-level constant, and refuses to boot when required vars are missing or set to known-bad defaults. See Environment Variables for the full catalogue.

  • decnet/env.py::_port — parse an env var as an integer port (165535); raises with a clear message.
  • decnet/env.py::_require_env — required-var guard; rejects empty values and the known-bad set {admin, secret, password, changeme, fallback-secret-key-change-me}; enforces ≥ 32-byte DECNET_JWT_SECRET outside developer mode. Pytest is exempt.
  • decnet/env.py::DECNET_SYSTEM_LOGS — system-log file path (default decnet.system.log).
  • decnet/env.py::DECNET_EMBED_PROFILER / DECNET_EMBED_SNIFFER — opt-in flags to run those workers inside the API process instead of as standalone daemons.
  • decnet/env.py::DECNET_PROFILE_REQUESTS / DECNET_PROFILE_DIR — Pyinstrument ASGI middleware toggle.
  • decnet/env.py::DECNET_API_HOST / DECNET_API_PORT — bind address for the FastAPI app.
  • decnet/env.py::DECNET_JWT_SECRET — HS256 secret for bearer tokens (required, ≥ 32 bytes outside dev).
  • decnet/env.py::DECNET_INGEST_LOG_FILE — canonical syslog path (default /var/log/decnet/decnet.log).
  • decnet/env.py::DECNET_BATCH_SIZE / DECNET_BATCH_MAX_WAIT_MS — ingester batch knobs.
  • decnet/env.py::DECNET_WEB_HOST / DECNET_WEB_PORT — dashboard bind.
  • decnet/env.py::DECNET_ADMIN_USER / DECNET_ADMIN_PASSWORD — bootstrap admin credentials.
  • decnet/env.py::DECNET_DEVELOPER / DECNET_DEVELOPER_TRACING / DECNET_OTEL_ENDPOINT — dev-mode and OTEL knobs.
  • decnet/env.py::DECNET_DB_TYPE / DECNET_DB_URL / DECNET_DB_HOST / DECNET_DB_PORT / DECNET_DB_NAME / DECNET_DB_USER / DECNET_DB_PASSWORD — database selection (see Database Drivers).
  • decnet/env.py::DECNET_CORS_ORIGINS — parsed list; defaults to the configured web host/port.

decnet/privdrop.py

Drops root ownership on files created under sudo. When the deploy path (which needs root for MACVLAN) writes log files, they would otherwise be owned by root and block a later non-root decnet api from appending. SUDO_UID / SUDO_GID are used to chown them back to the invoking user.

  • decnet/privdrop.py::chown_to_invoking_user — best-effort chown on a path. No-op when not root, not launched via sudo, or the path doesn't exist. Failures are swallowed.
  • decnet/privdrop.py::chown_tree_to_invoking_user — recursive variant for freshly-created parent directories.
  • decnet/privdrop.py::_sudo_ids — read SUDO_UID/SUDO_GID from the environment, return (uid, gid) or None.

decnet/distros.py

Distro registry that backs the heterogeneous-network look. Each DistroProfile pairs a Docker image (used for the base/IP-holder container) with a build_base used as FROM ${BASE_IMAGE} inside service Dockerfiles. Non-Debian distros pin their build_base to debian:bookworm-slim because the service Dockerfiles assume apt-get.

  • decnet/distros.py::DistroProfile — frozen dataclass (slug, image, display_name, hostname_style, build_base).
  • decnet/distros.py::DISTROS — registry of built-in profiles: debian, ubuntu22, ubuntu20, rocky9, centos7, alpine, fedora, kali, arch.
  • decnet/distros.py::random_hostname — generate a plausible hostname; shape depends on the profile's hostname_style (genericSRV-WORD-NN, rhelwordNN.localdomain, minimalword-NN, rollingword-word).
  • decnet/distros.py::get_distro — lookup by slug or raise ValueError.
  • decnet/distros.py::random_distro — pick a profile at random.
  • decnet/distros.py::all_distros — copy of the registry dict.

decnet/custom_service.py

Runtime wrapper for [custom-*] INI sections. Instantiated by the CLI/INI path and registered via register_custom_service(); not part of auto-discovery.

  • decnet/custom_service.py::CustomServiceBaseService subclass that emits a compose fragment for an arbitrary user image. Ports come from the INI; exec_cmd becomes the container command; LOG_TARGET is injected when present.
  • decnet/custom_service.py::CustomService.compose_fragment — build the dict for compose, including NODE_NAME env var for log tagging.
  • decnet/custom_service.py::CustomService.dockerfile_context — always None — custom services never build.

decnet/os_fingerprint.py

Namespace-scoped Linux sysctls applied to each decky's base container so that TCP/IP stack behaviour matches the claimed nmap OS family. Primary discriminator is net.ipv4.ip_default_ttl (64 Linux/BSD, 128 Windows, 255 Cisco/embedded). Secondary: tcp_timestamps, tcp_window_scaling, tcp_sack, tcp_ecn, ip_no_pmtu_disc, tcp_fin_timeout, tcp_syn_retries, icmp_ratelimit, icmp_ratemask.

  • decnet/os_fingerprint.py::OS_SYSCTLS — profile dict for linux, windows, bsd, embedded, cisco.
  • decnet/os_fingerprint.py::get_os_sysctls — return the dict for a slug, falling back to linux on unknown input.
def get_os_sysctls(nmap_os: str) -> dict[str, str]:
    return dict(OS_SYSCTLS.get(nmap_os, OS_SYSCTLS[_DEFAULT_OS]))
  • decnet/os_fingerprint.py::all_os_families — list of all registered slugs.
  • decnet/os_fingerprint.py::_REQUIRED_SYSCTLS — frozenset used by tests to assert every profile carries the full key set.

decnet/archetypes.py

Pre-packaged machine identities: a service list, preferred distro pool, and nmap OS family. Lets users say archetype=windows-workstation instead of hand-picking smb, rdp.

  • decnet/archetypes.py::Archetype — frozen dataclass (slug, display_name, description, services, preferred_distros, nmap_os).
  • decnet/archetypes.py::ARCHETYPES — registry: windows-workstation, windows-server, domain-controller, linux-server, web-server, database-server, mail-server, file-server, printer, iot-device, industrial-control, voip-server, monitoring-node, devops-host, deaddeck.
  • decnet/archetypes.py::get_archetype / all_archetypes / random_archetype — standard lookup trio.

decnet/fleet.py

Shared builder functions for constructing a list of DeckyConfig from either CLI flags or a parsed IniConfig. Lives outside cli.py so that the web API and the mutator can import it without dragging Typer.

  • decnet/fleet.py::all_service_names — sorted list of registered services that are not fleet_singleton.
  • decnet/fleet.py::resolve_distros — decide the distro slug per decky (explicit list → randomised → archetype pool → all distros round-robin).
  • decnet/fleet.py::build_deckies — CLI-style builder. Picks services from services_explicit, archetype, or random (with a dedup set so the first 20 attempts avoid duplicate service combos).
  • decnet/fleet.py::build_deckies_from_ini — INI-style builder. Honours explicit ip=, auto-allocates the rest from the subnet (skipping network/broadcast/gateway/host/other-explicit IPs), resolves archetypes, and cascades mutate_interval (CLI > decky > global).

decnet/telemetry.py

OpenTelemetry integration that is strictly opt-in via DECNET_DEVELOPER_TRACING=true. When disabled, every public export is a zero-cost no-op: @traced returns the unwrapped function, get_tracer() returns a _NoOpTracer, the repository wrapper returns the original repo, and injection/extraction of trace context is a nop.

  • decnet/telemetry.py::setup_tracing — initialise the TracerProvider, install FastAPIInstrumentor, enable log↔trace correlation. Called once from the FastAPI lifespan.
  • decnet/telemetry.py::shutdown_tracing — best-effort provider flush+shutdown.
  • decnet/telemetry.py::get_tracerget_tracer("db"), get_tracer("ingester"), … cached per component.
  • decnet/telemetry.py::traced — decorator that wraps async or sync functions in a span. Three call forms: @traced, @traced("name"), @traced(name="name").
  • decnet/telemetry.py::wrap_repository — dynamic __getattr__ proxy that wraps every async repository method in a db.<method> span.
  • decnet/telemetry.py::inject_context / extract_context — W3C trace-context propagation into/out of JSON log records so the collector → ingester → profiler pipeline shows up as one trace in Jaeger. extract_context pops _trace from the dict so it never reaches the DB.
  • decnet/telemetry.py::start_span_with_context — helper to continue a trace from an extracted context.
  • decnet/telemetry.py::_init_provider — lazy OTEL SDK import + TracerProvider construction with an OTLP gRPC exporter to DECNET_OTEL_ENDPOINT.
  • decnet/telemetry.py::_NoOpTracer / _NoOpSpan — stand-ins used when tracing is disabled.
  • decnet/telemetry.py::_wrap — internal async-aware wrapper used by traced.