wiki: replace module-reference stubs with Unit 16 authored versions
@@ -1,88 +1,104 @@
|
|||||||
# Module Reference: Services
|
# Module Reference — Services
|
||||||
|
|
||||||
Code-level view of every service plugin under `decnet/services/`. For the user-facing description of what each honeypot does (personas, credentials, bait), see [Services catalog](Services-Catalog). For writing your own plugin, see [Writing a Service Plugin](Writing-a-Service-Plugin).
|
Every module under `decnet/services/`. This is the *code* view — for the user-facing description of each honeypot (what it emulates, what the captured attacker interaction looks like, which personas are supported) see [Services Catalog](Services-Catalog). For bring-your-own definitions see [Custom Services](Custom-Services).
|
||||||
|
|
||||||
Citation format: `decnet/services/<file>::<symbol>`.
|
Citation format: `decnet/services/<module>.py::<symbol>`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Core
|
## Framework
|
||||||
|
|
||||||
### `services/base.py::BaseService`
|
### `decnet/services/__init__.py`
|
||||||
|
|
||||||
Abstract contract every plugin implements. Class attributes:
|
Empty package marker.
|
||||||
|
|
||||||
- `name: str` — unique slug (e.g. `"ssh"`).
|
### `decnet/services/base.py`
|
||||||
- `ports: list[int]` — ports the service listens on inside the container.
|
|
||||||
- `default_image: str` — Docker image tag, or `"build"` when a Dockerfile context is needed.
|
|
||||||
- `fleet_singleton: bool = False` — `True` means the service runs once fleet-wide, not per-decky (used by the sniffer).
|
|
||||||
|
|
||||||
Methods:
|
Defines the contract every plugin implements. The registry auto-discovers every `BaseService` subclass at import time, so adding a service is just dropping a new file here.
|
||||||
|
|
||||||
- `compose_fragment(decky_name, log_target, service_cfg) -> dict` — **abstract**; returns the docker-compose service dict. Networking keys are injected by the composer and **must not** be set here.
|
- `decnet/services/base.py::BaseService` — abstract base with class attributes `name`, `ports`, `default_image`, `fleet_singleton` and the abstract `compose_fragment(decky_name, log_target=None, service_cfg=None)`.
|
||||||
- `dockerfile_context() -> Path | None` — return the build-context path if a custom image is needed; `None` means `default_image` is pulled directly.
|
- `decnet/services/base.py::BaseService.compose_fragment` — return the docker-compose service dict. Networking keys (`networks`, `ipv4_address`) are injected by the composer and must NOT be returned here. Include `image` / `build`, `environment`, `volumes`, `restart`, and service-specific options.
|
||||||
|
- `decnet/services/base.py::BaseService.dockerfile_context` — return the build-context directory, or `None` when `default_image` is used directly.
|
||||||
|
|
||||||
### `services/registry.py`
|
### `decnet/services/registry.py`
|
||||||
|
|
||||||
Plugin auto-discovery.
|
Plugin registry. All modules under `decnet.services.` are imported lazily on first call; each `BaseService` subclass in the package is instantiated once and keyed by its `.name`.
|
||||||
|
|
||||||
- `registry.py::_load_plugins` — walks `decnet/services/`, imports every module, and collects all `BaseService` subclasses.
|
- `decnet/services/registry.py::_load_plugins` — walk the package with `pkgutil.iter_modules`, import everything except `base` and `registry`, instantiate every `BaseService.__subclasses__()`.
|
||||||
- `registry.py::register_custom_service` — register an out-of-tree plugin at runtime (used by tests and third-party integrations).
|
- `decnet/services/registry.py::register_custom_service` — register a dynamically-built `CustomService` (used for `[custom-*]` INI sections; see [Custom Services](Custom-Services)).
|
||||||
- `registry.py::get_service(name)` — fetch one plugin by slug.
|
- `decnet/services/registry.py::get_service` — lookup a service by name, raise `KeyError` on miss.
|
||||||
- `registry.py::all_services()` — return `{slug: instance}` for every registered plugin.
|
- `decnet/services/registry.py::all_services` — return a copy of the registry dict.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Service plugins
|
## Plugin catalogue
|
||||||
|
|
||||||
All rows are subclasses of `BaseService`. The `Image` column is `build` for every in-tree plugin — each one ships a Dockerfile context under `templates/<slug>/`. See [Service Personas](Service-Personas) and [Archetypes](Archetypes) for how they're composed into deckies.
|
All plugins except `sniffer` are per-decky (`fleet_singleton=False`). All plugins set `default_image="build"`, meaning the corresponding `templates/<name>/` Dockerfile is built at deploy time with `BASE_IMAGE` injected from the decky's distro. The `NODE_NAME` env var is always set to the decky name; the `LOG_TARGET` env var is set when the composer has a `log_target` (syslog forward address). Persona keys listed are those consulted by `compose_fragment(service_cfg=...)` from the INI `[decky.service]` subsection.
|
||||||
|
|
||||||
| Slug | Ports | Image | Class | One-line description |
|
| Service | Ports | Template | Persona keys |
|
||||||
|------|-------|-------|-------|----------------------|
|
| --- | --- | --- | --- |
|
||||||
| `conpot` | 502, 161 (UDP), 80 | build | `ConpotService` | ICS/SCADA honeypot covering Modbus, SNMP, and HTTP. |
|
| `ssh` | 22 | `templates/ssh` | `password`, `hostname` |
|
||||||
| `docker_api` | 2375, 2376 | build | `DockerAPIService` | Fake Docker Engine API (unauthenticated + TLS ports). |
|
| `smb` | 445, 139 | `templates/smb` | service-specific |
|
||||||
| `elasticsearch` | 9200 | build | `ElasticsearchService` | Exposed Elasticsearch REST endpoint with index bait. |
|
| `rdp` | 3389 | `templates/rdp` | service-specific |
|
||||||
| `ftp` | 21 | build | `FTPService` | vsftpd honeypot with captured-upload persistence. |
|
| `http` | 80, 443 | `templates/http` | `server_header`, `response_code`, `fake_app`, `extra_headers`, `custom_body`, `files` |
|
||||||
| `http` | 80, 443 | build | `HTTPService` | Realistic HTTP/HTTPS web server with persona templates. |
|
| `https` | 443 | `templates/https` | service-specific |
|
||||||
| `https` | 443 | build | `HTTPSService` | TLS-terminated web persona (paired with `http` or standalone). |
|
| `ftp` | 21 | `templates/ftp` | service-specific |
|
||||||
| `imap` | 143, 993 | build | `IMAPService` | Dovecot IMAP/IMAPS with fake mailboxes. |
|
| `mysql` | 3306 | `templates/mysql` | service-specific |
|
||||||
| `k8s` | 6443, 8080 | build | `KubernetesAPIService` | Mock Kubernetes API server surface. |
|
| `postgres` | 5432 | `templates/postgres` | service-specific |
|
||||||
| `ldap` | 389, 636 | build | `LDAPService` | OpenLDAP with a seeded fake directory. |
|
| `redis` | 6379 | `templates/redis` | service-specific |
|
||||||
| `llmnr` | 5355, 5353 | build | `LLMNRService` | LLMNR/mDNS/NBNS poisoning detector. |
|
| `mongodb` | 27017 | `templates/mongodb` | service-specific |
|
||||||
| `mongodb` | 27017 | build | `MongoDBService` | Exposed MongoDB with unauthenticated collections. |
|
| `mssql` | 1433 | `templates/mssql` | service-specific |
|
||||||
| `mqtt` | 1883 | build | `MQTTService` | Mosquitto broker with discoverable topics. |
|
| `elasticsearch` | 9200 | `templates/elasticsearch` | service-specific |
|
||||||
| `mssql` | 1433 | build | `MSSQLService` | SQL Server persona emulating login + TDS banner. |
|
| `telnet` | 23 | `templates/telnet` | service-specific |
|
||||||
| `mysql` | 3306 | build | `MySQLService` | MySQL server with weak creds and seeded schema. |
|
| `vnc` | 5900 | `templates/vnc` | service-specific |
|
||||||
| `pop3` | 110, 995 | build | `POP3Service` | Dovecot POP3/POP3S counterpart to `imap`. |
|
| `snmp` | 161 | `templates/snmp` | service-specific |
|
||||||
| `postgres` | 5432 | build | `PostgresService` | PostgreSQL server with bait databases. |
|
| `mqtt` | 1883 | `templates/mqtt` | service-specific |
|
||||||
| `rdp` | 3389 | build | `RDPService` | RDP (xrdp) honeypot with credential capture. |
|
| `sip` | 5060 | `templates/sip` | service-specific |
|
||||||
| `redis` | 6379 | build | `RedisService` | Unauthenticated Redis with bait keys. |
|
| `ldap` | 389, 636 | `templates/ldap` | service-specific |
|
||||||
| `sip` | 5060 | build | `SIPService` | SIP/VoIP registrar bait. |
|
| `llmnr` | 5355, 5353 | `templates/llmnr` | service-specific |
|
||||||
| `smb` | 445, 139 | build | `SMBService` | Samba server with seeded fake shares. |
|
| `k8s` | 6443, 8080 | `templates/k8s` | service-specific |
|
||||||
| `smtp` | 25, 587 | build | `SMTPService` | Postfix SMTP endpoint with auth capture. |
|
| `docker_api` | 2375, 2376 | `templates/docker_api` | service-specific |
|
||||||
| `smtp_relay` | 25, 587 | build | `SMTPRelayService` | Open-relay bait — accepts any RCPT TO and delivers. |
|
| `conpot` | 502, 161, 80 | `templates/conpot` | service-specific |
|
||||||
| `sniffer` | — | build | `SnifferService` | Fleet-wide passive MACVLAN sniffer (`fleet_singleton=True`). |
|
| `tftp` | 69 | `templates/tftp` | service-specific |
|
||||||
| `snmp` | 161 (UDP) | build | `SNMPService` | SNMP agent with fake MIB tree. |
|
| `imap` | 143, 993 | `templates/imap` | service-specific |
|
||||||
| `ssh` | 22 | build | `SSHService` | Interactive OpenSSH (replaces Cowrie) for realistic fingerprints. |
|
| `pop3` | 110, 995 | `templates/pop3` | service-specific |
|
||||||
| `telnet` | 23 | build | `TelnetService` | Real `busybox telnetd` with rsyslog logging pipeline. |
|
| `smtp` | 25, 587 | `templates/smtp` | service-specific |
|
||||||
| `tftp` | 69 (UDP) | build | `TFTPService` | TFTP server bait for firmware-fetch probing. |
|
| `smtp_relay` | 25, 587 | `templates/smtp` (shared, with `SMTP_OPEN_RELAY=1`) | service-specific |
|
||||||
| `vnc` | 5900 | build | `VNCService` | VNC server with weak auth. |
|
| `sniffer` | — | `templates/sniffer` | — (fleet singleton) |
|
||||||
|
|
||||||
|
Per-service notes:
|
||||||
|
|
||||||
|
- `decnet/services/ssh.py::SSHService` — real OpenSSH server (not Cowrie) so that fingerprinting cannot trivially identify the honeypot. Auth, sudo, and interactive commands are forwarded to stdout as RFC 5424 by the container's rsyslog bridge. Bind-mounts a per-decky quarantine directory at `/var/lib/decnet/artifacts/<decky>/ssh` onto the in-container path `/var/lib/systemd/coredump` so attacker drops (`scp`/`sftp`/`wget`) are captured out of band while looking benign from inside.
|
||||||
|
- `decnet/services/smb.py::SMBService` — SMB 1/2/3 honeypot on 445 + 139 (NetBIOS).
|
||||||
|
- `decnet/services/rdp.py::RDPService` — Remote Desktop Protocol on 3389.
|
||||||
|
- `decnet/services/http.py::HTTPService` — HTTP/HTTPS on 80 + 443 with optional `server_header`, `response_code`, `fake_app` (e.g. `wordpress`), `extra_headers` (dict or JSON string), `custom_body`, and an optional `files=<path>` bind mount to serve arbitrary content out of `/opt/html_files`.
|
||||||
|
- `decnet/services/https.py::HTTPSService` — standalone HTTPS emulator on 443; shares no code with `http`.
|
||||||
|
- `decnet/services/ftp.py::FTPService` — FTP control channel on 21.
|
||||||
|
- `decnet/services/mysql.py::MySQLService` — MySQL wire protocol on 3306.
|
||||||
|
- `decnet/services/postgres.py::PostgresService` — PostgreSQL startup handshake on 5432.
|
||||||
|
- `decnet/services/redis.py::RedisService` — RESP protocol on 6379.
|
||||||
|
- `decnet/services/mongodb.py::MongoDBService` — MongoDB wire protocol on 27017.
|
||||||
|
- `decnet/services/mssql.py::MSSQLService` — TDS pre-login on 1433.
|
||||||
|
- `decnet/services/elasticsearch.py::ElasticsearchService` — HTTP REST facade on 9200 emulating ES cluster endpoints.
|
||||||
|
- `decnet/services/telnet.py::TelnetService` — Telnet negotiation + login prompt on 23.
|
||||||
|
- `decnet/services/vnc.py::VNCService` — RFB 3.x handshake on 5900, emits VNC client-version fingerprints.
|
||||||
|
- `decnet/services/snmp.py::SNMPService` — SNMP v1/v2c/v3 on 161/UDP.
|
||||||
|
- `decnet/services/mqtt.py::MQTTService` — MQTT broker handshake on 1883.
|
||||||
|
- `decnet/services/sip.py::SIPService` — SIP REGISTER / OPTIONS on 5060.
|
||||||
|
- `decnet/services/ldap.py::LDAPService` — LDAPv3 bind on 389 + 636.
|
||||||
|
- `decnet/services/llmnr.py::LLMNRService` — link-local multicast name resolution on 5355 + 5353 (mDNS bundled).
|
||||||
|
- `decnet/services/k8s.py::KubernetesAPIService` — fake Kubernetes API on 6443 + 8080.
|
||||||
|
- `decnet/services/docker_api.py::DockerAPIService` — fake Docker daemon socket exposed over TCP on 2375 + 2376.
|
||||||
|
- `decnet/services/conpot.py::ConpotService` — upstream Conpot ICS/SCADA honeypot: Modbus (502), SNMP (161 UDP), HTTP (80).
|
||||||
|
- `decnet/services/tftp.py::TFTPService` — TFTP RRQ/WRQ on 69/UDP.
|
||||||
|
- `decnet/services/imap.py::IMAPService` — IMAP4 capability + login on 143 + 993.
|
||||||
|
- `decnet/services/pop3.py::POP3Service` — POP3 USER/PASS on 110 + 995.
|
||||||
|
- `decnet/services/smtp.py::SMTPService` — SMTP EHLO/AUTH on 25 + 587.
|
||||||
|
- `decnet/services/smtp_relay.py::SMTPRelayService` — reuses the `smtp` template with `SMTP_OPEN_RELAY=1` to offer the open-relay persona beloved of spam bots.
|
||||||
|
- `decnet/services/sniffer.py::SnifferService` — **fleet singleton**. Passive MACVLAN sniffer: no inbound ports, `cap_add: [NET_RAW, NET_ADMIN]`, extracts JA3/JA3S/JA4/JA4L TLS fingerprints plus connection metadata from the wire. Feeds the bounty table via the ingester.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## How they're assembled
|
## `decnet/custom_service.py`
|
||||||
|
|
||||||
- `engine/deployer.py` reads the archetype-selected slugs and calls each plugin's `compose_fragment()` to build `docker-compose.yml`. See [Module Reference: Workers](Module-Reference-Workers) for the deployer internals.
|
See the [Core reference](Module-Reference-Core#decnetcustom_servicepy). `CustomService` is a `BaseService` subclass instantiated from `[custom-*]` INI sections and registered via `register_custom_service()`.
|
||||||
- The canonical `syslog_bridge.py` is copied into every active build context by `engine/deployer.py::_sync_logging_helper` so each container forwards logs identically. See [Logging](Logging-and-Syslog).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## See also
|
|
||||||
|
|
||||||
- [Services catalog](Services-Catalog) — user view: personas, creds, what each honeypot looks like to an attacker.
|
|
||||||
- [Writing a Service Plugin](Writing-a-Service-Plugin) — step-by-step guide.
|
|
||||||
- [Archetypes](Archetypes) — how slugs are bundled into OS personas.
|
|
||||||
- [Design overview](Design-Overview)
|
|
||||||
- [REST API](REST-API-Reference)
|
|
||||||
- [Logging](Logging-and-Syslog)
|
|
||||||
- [Developer guide](Developer-Guide)
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Module Reference: Web
|
# Module Reference — Web
|
||||||
|
|
||||||
Code-level reference for every module under `decnet/web/`. For the user-facing API see [REST API](REST-API-Reference); for the dashboard UX see [Web Dashboard](Web-Dashboard); architecture is in [Design overview](Design-Overview).
|
Every Python module under `decnet/web/`. See [REST API Reference](REST-API-Reference) for the user-facing endpoint catalogue and [Database Drivers](Database-Drivers) for SQLite vs MySQL details.
|
||||||
|
|
||||||
Citation format: `decnet/web/<path>::<symbol>`.
|
Citation format: `decnet/web/<path>::<symbol>`.
|
||||||
|
|
||||||
@@ -8,154 +8,255 @@ Citation format: `decnet/web/<path>::<symbol>`.
|
|||||||
|
|
||||||
## `decnet/web/api.py`
|
## `decnet/web/api.py`
|
||||||
|
|
||||||
FastAPI application factory and lifespan wiring. Instantiates the `FastAPI` app, mounts every router under `decnet/web/router/`, attaches exception handlers, and owns the background-task handles for the ingester, profiler, prober, sniffer, and correlation workers.
|
FastAPI application factory. Wires the CORS middleware, the Pyinstrument profiling middleware (opt-in), the tiered validation-error exception handlers, and the lifespan that initialises the repository and optionally spawns the in-process ingestion / collector / profiler / sniffer background tasks. Defaults to running those workers as standalone daemons; the `DECNET_EMBED_*` flags are the escape hatch.
|
||||||
|
|
||||||
- `api.py::get_background_tasks` — returns the live `dict` of worker task handles so the `/health` endpoint can introspect liveness.
|
- `decnet/web/api.py::app` — the `FastAPI` instance. Docs are hidden unless `DECNET_DEVELOPER=true`.
|
||||||
- `api.py::lifespan` — async context manager run by FastAPI on startup/shutdown: initializes the repository, seeds the admin user, spawns worker tasks, and cancels them on exit.
|
- `decnet/web/api.py::lifespan` — async context manager that (1) warns on low `ulimit -n`, (2) retries DB init up to 5 times, (3) sets up OTEL tracing, (4) starts background tasks unless `DECNET_CONTRACT_TEST=true`, (5) cancels them on shutdown.
|
||||||
- `api.py::validation_exception_handler` — maps request-validation errors to the specific HTTP codes the contract tests require.
|
- `decnet/web/api.py::get_background_tasks` — expose the four task handles to the health endpoint.
|
||||||
- `api.py::pydantic_validation_exception_handler` — handles Pydantic errors raised during manual model instantiation (e.g. state hydration).
|
- `decnet/web/api.py::validation_exception_handler` — tiered mapping for `RequestValidationError`: 400 for structural schema violations, 409 for semantic INI content violations (empty / syntax / missing sections), 422 otherwise.
|
||||||
|
- `decnet/web/api.py::pydantic_validation_exception_handler` — catches manual model-instantiation `ValidationError` so bad DB rows never surface as 500s.
|
||||||
|
- `decnet/web/api.py::PyinstrumentMiddleware` — mounted when `DECNET_PROFILE_REQUESTS=true`; writes per-request HTML flamegraphs to `DECNET_PROFILE_DIR`.
|
||||||
|
- `decnet/web/api.py::ingestion_task` / `collector_task` / `attacker_task` / `sniffer_task` — module-level task handles used by the health endpoint.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/web/auth.py`
|
## `decnet/web/auth.py`
|
||||||
|
|
||||||
Password hashing and JWT issuance primitives. No business logic — just stateless helpers shared by the login router and dependency injectors.
|
JWT + bcrypt primitives. HS256, 1440-minute (24 h) default expiry, bcrypt password length capped at 72 bytes to match the bcrypt spec, `cost=12` work factor, and `asyncio.to_thread` wrappers so the ~250 ms bcrypt calls never block the event loop.
|
||||||
|
|
||||||
- `auth.py::verify_password` — sync bcrypt verify.
|
- `decnet/web/auth.py::SECRET_KEY` — alias for `DECNET_JWT_SECRET`.
|
||||||
- `auth.py::get_password_hash` — sync bcrypt hash.
|
- `decnet/web/auth.py::ALGORITHM` — `"HS256"`.
|
||||||
- `auth.py::averify_password` — async verify (offloaded to a threadpool).
|
- `decnet/web/auth.py::ACCESS_TOKEN_EXPIRE_MINUTES` — `1440`.
|
||||||
- `auth.py::ahash_password` — async hash (threadpool).
|
- `decnet/web/auth.py::verify_password` — sync bcrypt compare (72-byte truncation).
|
||||||
- `auth.py::create_access_token` — issue a signed JWT with configured TTL and subject claim.
|
- `decnet/web/auth.py::get_password_hash` — bcrypt hash with `rounds=12`.
|
||||||
|
- `decnet/web/auth.py::averify_password` / `ahash_password` — `asyncio.to_thread` wrappers.
|
||||||
|
- `decnet/web/auth.py::create_access_token` — sign a dict with `exp` (default 15 minutes if no `expires_delta`) and `iat` claims.
|
||||||
|
|
||||||
---
|
```python
|
||||||
|
def create_access_token(data, expires_delta=None):
|
||||||
## `decnet/web/ingester.py`
|
payload = data.copy()
|
||||||
|
expire = datetime.now(timezone.utc) + (expires_delta or timedelta(minutes=15))
|
||||||
Background worker that tails the DECNET ingest log (`DECNET_INGEST_LOG_FILE.json`) and writes rows into the `logs` and `bounties` tables. Checkpoints file offset via the `State` table so restarts are idempotent.
|
payload.update({"exp": expire, "iat": datetime.now(timezone.utc)})
|
||||||
|
return jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)
|
||||||
- `ingester.py::log_ingestion_worker` — asyncio task loop: tail + parse + flush batches.
|
```
|
||||||
- `ingester.py::_flush_batch` — commits a batch of log rows and returns the new file position (bulk-insert, one transaction).
|
|
||||||
- `ingester.py::_extract_bounty` — inspects a parsed log row and returns a bounty payload (creds, uploaded files, etc.) or `None`.
|
|
||||||
- Const: `_INGEST_STATE_KEY` — key under which the tail offset is stored in `State`.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/web/dependencies.py`
|
## `decnet/web/dependencies.py`
|
||||||
|
|
||||||
FastAPI dependencies for repository injection, authentication, RBAC, and user-cache management. Every router imports from here — never directly from the repo module.
|
FastAPI dependency injection for the repository, auth decoding, and RBAC. Holds two in-memory caches (user-by-uuid with 10 s TTL, user-by-username with 5 s TTL) which collapse the "SELECT users WHERE uuid=?" that used to run once per authed request. Cache misses on username are intentionally **not** cached so that a just-created user can log in immediately.
|
||||||
|
|
||||||
- `dependencies.py::get_repo` — repository DI entrypoint; returns the configured `BaseRepository` singleton.
|
- `decnet/web/dependencies.py::get_repo` — returns the process-wide `BaseRepository` singleton (constructs it on first call via `get_repository()`).
|
||||||
- `dependencies.py::invalidate_user_cache` — drop a single username (or all) from the auth caches (`_get_user_cached`, `get_user_by_username_cached`).
|
- `decnet/web/dependencies.py::repo` — module-level singleton convenience handle.
|
||||||
- `dependencies.py::get_user_by_username_cached` — TTL-cached lookup used on the login hot path.
|
- `decnet/web/dependencies.py::oauth2_scheme` — `OAuth2PasswordBearer(tokenUrl="/api/v1/auth/login")`.
|
||||||
- `dependencies.py::get_stream_user` — auth for SSE endpoints; accepts Bearer header **or** `?token=` query param (EventSource cannot set headers).
|
- `decnet/web/dependencies.py::invalidate_user_cache` — drop a single uuid (or everything); callers: password change, role change, user create/delete.
|
||||||
- `dependencies.py::get_current_user` — Bearer-token auth; enforces the `must_change_password` flag.
|
- `decnet/web/dependencies.py::get_user_by_username_cached` — cached read for the login hot path.
|
||||||
- `dependencies.py::get_current_user_unchecked` — same, but skips the change-password gate (used by the change-password endpoint itself).
|
- `decnet/web/dependencies.py::get_stream_user` — SSE-only dependency that accepts Bearer header **or** `?token=` query param (EventSource can't set headers).
|
||||||
- `dependencies.py::require_role` — factory returning an RBAC dependency enforcing role membership (`admin`, `analyst`, `viewer`).
|
- `decnet/web/dependencies.py::get_current_user` — standard auth dependency that enforces `must_change_password`.
|
||||||
- `dependencies.py::require_stream_role` — same, for SSE endpoints that take a query-param token.
|
- `decnet/web/dependencies.py::get_current_user_unchecked` — same decode but skips `must_change_password`; used by the change-password endpoint itself.
|
||||||
- Private helpers: `_reset_user_cache`, `_get_user_cached`, `_decode_token` — cache reset (tests), cached user fetch by UUID, JWT decode.
|
- `decnet/web/dependencies.py::require_role` / `require_stream_role` — factories that return a dependency enforcing role membership and returning the full user dict. Inlines decode + lookup + `must_change_password` check to avoid a double DB hit.
|
||||||
- Consts: `_USER_TTL`, `_USERNAME_TTL` — cache TTLs in seconds.
|
- `decnet/web/dependencies.py::require_admin` — `require_role("admin")`.
|
||||||
|
- `decnet/web/dependencies.py::require_viewer` — `require_role("viewer", "admin")`.
|
||||||
|
- `decnet/web/dependencies.py::require_stream_viewer` — SSE variant of `require_viewer`.
|
||||||
|
- `decnet/web/dependencies.py::_decode_token` / `_get_user_cached` / `_reset_user_cache` — internal helpers (decode Bearer JWT, lookup with lock + TTL, test-only reset).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/web/db/`
|
## `decnet/web/ingester.py`
|
||||||
|
|
||||||
### `db/factory.py`
|
Background worker that tails `<DECNET_INGEST_LOG_FILE>.json` line-by-line, batches parsed records, and inserts them into the repository via `repo.add_logs`. After each batch, it calls `_extract_bounty` on every record to harvest credentials, HTTP user-agents, VNC client versions, SSH client banners, JA3/JA4/JA4L TLS fingerprints, TLS certificate details, and prober JARM / HASSH / TCP-FP hashes into the bounty table. File position is persisted via `repo.set_state("ingest_worker_position", {...})` so restarts resume exactly where they stopped.
|
||||||
Repository factory. Selects a `BaseRepository` implementation based on the `DECNET_DB_TYPE` env var (`sqlite` or `mysql`). Returns a module-level singleton.
|
|
||||||
|
|
||||||
- `db/factory.py::get_repository` — construct and cache the active repository.
|
- `decnet/web/ingester.py::log_ingestion_worker` — main loop. Detects truncation (current size < saved position → reset to 0), flushes when `DECNET_BATCH_SIZE` rows are buffered or `DECNET_BATCH_MAX_WAIT_MS` elapses, extracts OTEL parent context with `extract_context(_log_data)` so the ingester span chains off the collector's, and embeds the ingester span's trace/span IDs into the row.
|
||||||
|
- `decnet/web/ingester.py::_flush_batch` — commit a batch and advance the saved position. Bails out early on cancellation so session teardown can't stall the worker; un-flushed lines stay uncommitted and are re-read on next start.
|
||||||
### `db/repository.py`
|
- `decnet/web/ingester.py::_extract_bounty` — 11 bounty extractors keyed on `service` / `event_type` / field presence (credentials, http_useragent, vnc_client_version, ja3, ja4l, tls_resumption, tls_certificate, jarm, hassh_server, tcpfp).
|
||||||
Abstract base class declaring every storage operation the dashboard and workers rely on.
|
- `decnet/web/ingester.py::_INGEST_STATE_KEY` — `"ingest_worker_position"`.
|
||||||
|
|
||||||
- `db/repository.py::BaseRepository` — abstract interface.
|
|
||||||
Notable methods: `initialize`, `add_log`, `add_logs`, `get_logs`, `get_total_logs`, `get_stats_summary`, `get_deckies`, `get_user_by_username`, `get_user_by_uuid`, `create_user`, `update_user_password`, `list_users`, `delete_user`, `update_user_role`, `purge_logs_and_bounties`, `add_bounty`, `get_bounties`, `get_total_bounties`, `get_state`, `set_state`, `get_max_log_id`, `get_logs_after_id`, `get_all_bounties_by_ip`, `get_bounties_for_ips`, `upsert_attacker`, `upsert_attacker_behavior`, `get_attacker_behavior`, `get_behaviors_for_ips`, `get_attacker_by_uuid`, `get_attackers`, `get_total_attackers`, `get_attacker_commands`, `get_attacker_artifacts`.
|
|
||||||
|
|
||||||
### `db/sqlmodel_repo.py`
|
|
||||||
Shared SQLModel/SQLAlchemy-async concrete implementation. Both the SQLite and MySQL backends subclass this and override only dialect-specific bits (DDL introspection, `get_log_histogram`).
|
|
||||||
|
|
||||||
- `db/sqlmodel_repo.py::SQLModelRepository` — concrete async repository.
|
|
||||||
- `db/sqlmodel_repo.py::_detach_close` — runs session close in a fresh asyncio task so cancellation in the caller never leaves sessions hanging.
|
|
||||||
- `db/sqlmodel_repo.py::_safe_session` — session context manager with cancellation-safe cleanup.
|
|
||||||
- Key methods: `initialize`, `reinitialize`, `add_log`, `add_logs`, `get_logs`, `get_log_histogram` (override per backend), `get_stats_summary`, `upsert_attacker`, `upsert_attacker_behavior`, `get_attacker_commands`, `get_attacker_artifacts`. Many private helpers for filter composition, JSON-field equality, and row normalization.
|
|
||||||
|
|
||||||
### `db/sqlite/database.py`
|
|
||||||
- `get_async_engine`, `get_sync_engine`, `init_db`, `get_session` — SQLite engine and session factories. `init_db` creates tables synchronously (used by CLI/tests).
|
|
||||||
|
|
||||||
### `db/sqlite/repository.py`
|
|
||||||
- `SQLiteRepository` — `SQLModelRepository` subclass using `aiosqlite`. Overrides `_migrate_attackers_table`, `_json_field_equals`, `get_log_histogram` for SQLite dialect.
|
|
||||||
|
|
||||||
### `db/mysql/database.py`
|
|
||||||
MySQL async engine factory.
|
|
||||||
|
|
||||||
- `build_mysql_url` — compose an async SQLAlchemy URL using `asyncmy`.
|
|
||||||
- `resolve_url` — explicit arg → `DECNET_DB_URL` env → built from components.
|
|
||||||
- `get_async_engine` — create the `AsyncEngine`.
|
|
||||||
- Consts: `DEFAULT_POOL_SIZE`, `DEFAULT_MAX_OVERFLOW`, `DEFAULT_POOL_RECYCLE`, `DEFAULT_POOL_PRE_PING`.
|
|
||||||
|
|
||||||
### `db/mysql/repository.py`
|
|
||||||
- `MySQLRepository` — `SQLModelRepository` subclass using `asyncmy`. Adds MySQL migrations (`_migrate_attackers_table`, `_migrate_column_types` to upgrade `TEXT` → `MEDIUMTEXT` on large-JSON columns) and dialect-specific histogram SQL.
|
|
||||||
|
|
||||||
### `db/models.py`
|
|
||||||
SQLModel table classes and Pydantic request/response DTOs.
|
|
||||||
|
|
||||||
- Tables: `User`, `Log`, `Bounty`, `State`, `Attacker`, `AttackerBehavior` (1:1 with `Attacker` by UUID).
|
|
||||||
- Request models: `LoginRequest`, `ChangePasswordRequest`, `MutateIntervalRequest`, `DeployIniRequest`, `CreateUserRequest`, `UpdateUserRoleRequest`, `ResetUserPasswordRequest`, `DeploymentLimitRequest`, `GlobalMutationIntervalRequest`.
|
|
||||||
- Response models: `Token`, `LogsResponse`, `BountyResponse`, `AttackersResponse`, `StatsResponse`, `UserResponse`, `ConfigResponse`, `AdminConfigResponse`, `ComponentHealth`, `HealthResponse`.
|
|
||||||
- Helpers: `_normalize_null` — collapses SQL `NULL` / empty-string variance.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/web/router/`
|
## `decnet/web/db/factory.py`
|
||||||
|
|
||||||
All route modules follow one pattern: declare a module-level `APIRouter`, attach a single endpoint function. The aggregator `router/__init__.py` imports and mounts them. Response caches (`functools.lru_cache` with TTL) are used on read endpoints where invalidation is explicit.
|
Selects the repository implementation based on `DECNET_DB_TYPE` and wraps it in the optional tracing proxy.
|
||||||
|
|
||||||
### `router/logs/`
|
- `decnet/web/db/factory.py::get_repository` — instantiate `SQLiteRepository` or `MySQLRepository`, wrap with `decnet.telemetry.wrap_repository` (no-op when tracing is off), return a `BaseRepository`.
|
||||||
- `api_get_logs.py::get_logs` — paginated log feed with filters. Cached total-count via `_get_total_logs_cached` / `_reset_total_cache`.
|
|
||||||
- `api_get_histogram.py::get_logs_histogram` — binned log timeseries for the dashboard chart.
|
|
||||||
|
|
||||||
### `router/stats/`
|
|
||||||
- `api_get_stats.py::get_stats` — dashboard summary: event counts, attackers, bounties, deckies. Cached.
|
|
||||||
|
|
||||||
### `router/stream/`
|
|
||||||
- `api_stream_events.py::stream_events` — Server-Sent Events endpoint streaming new logs by id. Uses `get_stream_user` (query-param token). `_build_trace_links` builds OTEL span links from persisted `trace_id` / `span_id`.
|
|
||||||
|
|
||||||
### `router/bounty/`
|
|
||||||
- `api_get_bounties.py::get_bounties` — paginated bounty (captured credentials/files) feed. TTL-cached defaults.
|
|
||||||
|
|
||||||
### `router/attackers/`
|
|
||||||
- `api_get_attackers.py::get_attackers` — paginated attacker profile list. Cached total.
|
|
||||||
- `api_get_attacker_detail.py::get_attacker_detail` — single attacker profile joined with its `AttackerBehavior` row.
|
|
||||||
- `api_get_attacker_commands.py::get_attacker_commands` — paginated executed-commands list per attacker, optional service filter.
|
|
||||||
- `api_get_attacker_artifacts.py::get_attacker_artifacts` — captured file-drop artifacts for an attacker, newest first.
|
|
||||||
|
|
||||||
### `router/artifacts/`
|
|
||||||
- `api_get_artifact.py::get_artifact` — download a single captured artifact. `_resolve_artifact_path` validates inputs, resolves the on-disk path, and asserts it stays under `ARTIFACTS_ROOT` (path-traversal guard). Regex guards `_DECKY_RE`, `_STORED_AS_RE` sanitize path components.
|
|
||||||
|
|
||||||
### `router/fleet/`
|
|
||||||
- `api_deploy_deckies.py::api_deploy_deckies` — server-side deploy trigger (admin only). See [CLI Reference](CLI-Reference) for the same operation from shell.
|
|
||||||
- `api_get_deckies.py::get_deckies` — running decky inventory. Cached.
|
|
||||||
- `api_mutate_decky.py::api_mutate_decky` — force-mutate a single decky. See [Mutation and Randomization](Mutation-and-Randomization).
|
|
||||||
- `api_mutate_interval.py::api_update_mutate_interval` — set a decky's per-machine mutation interval. `_parse_duration` converts `5d`/`12h` strings to minutes.
|
|
||||||
|
|
||||||
### `router/config/`
|
|
||||||
- `api_get_config.py::api_get_config` — read the active config and user list (cached via `_get_list_users_cached` and `_get_state_cached`). `invalidate_list_users_cache` is called after any user mutation.
|
|
||||||
- `api_update_config.py::api_update_deployment_limit`, `api_update_global_mutation_interval` — write config knobs.
|
|
||||||
- `api_manage_users.py::api_create_user`, `api_delete_user`, `api_update_user_role`, `api_reset_user_password` — admin-only user CRUD.
|
|
||||||
- `api_reinit.py::api_reinit` — destructive reinit endpoint (drops logs/bounties/attackers, keeps users). Admin only.
|
|
||||||
|
|
||||||
### `router/auth/`
|
|
||||||
- `api_login.py::login` — username/password → JWT.
|
|
||||||
- `api_change_pass.py::change_password` — enforced when `must_change_password` is set; uses `get_current_user_unchecked`.
|
|
||||||
|
|
||||||
### `router/health/`
|
|
||||||
- `api_get_health.py::get_health` — aggregated health: DB liveness (`_check_database_cached`), Docker daemon reachable, background-worker statuses from `api.get_background_tasks()`. Cache reset helpers (`_reset_db_cache`, `_reset_docker_cache`) exist for tests.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## See also
|
## `decnet/web/db/models.py`
|
||||||
|
|
||||||
- [Design overview](Design-Overview)
|
SQLModel ORM tables and Pydantic request/response models. Long-text columns use a `_BIG_TEXT` variant that maps to `MEDIUMTEXT` on MySQL (16 MiB) and plain `TEXT` on SQLite.
|
||||||
- [REST API](REST-API-Reference)
|
|
||||||
- [Logging](Logging-and-Syslog)
|
- `decnet/web/db/models.py::_BIG_TEXT` — `Text().with_variant(MEDIUMTEXT(), "mysql")`.
|
||||||
- [Services catalog](Services-Catalog)
|
- `decnet/web/db/models.py::NullableDatetime` / `NullableString` — normalise the strings `"null"`, `"undefined"`, `""` to `None`.
|
||||||
- [Developer guide](Developer-Guide)
|
- `decnet/web/db/models.py::User` — `uuid`, `username`, `password_hash`, `role`, `must_change_password`.
|
||||||
|
- `decnet/web/db/models.py::Log` — `timestamp`, `decky`, `service`, `event_type`, `attacker_ip`, `raw_line`, `fields` (JSON text), `msg`, `trace_id`, `span_id`.
|
||||||
|
- `decnet/web/db/models.py::Bounty` — `decky`, `service`, `attacker_ip`, `bounty_type`, `payload` (JSON text).
|
||||||
|
- `decnet/web/db/models.py::State` — key/value store. `value` is `_BIG_TEXT` because DecnetConfig blobs can be large.
|
||||||
|
- `decnet/web/db/models.py::Attacker` — aggregated attacker profile row. All JSON blobs (`services`, `deckies`, `fingerprints`, `commands`) are `_BIG_TEXT`.
|
||||||
|
- `decnet/web/db/models.py::AttackerBehavior` — separate table (FK to `attackers.uuid`) holding timing + behavioral profile: `os_guess`, `hop_distance`, `tcp_fingerprint`, `retransmit_count`, `behavior_class`, `beacon_interval_s`, `beacon_jitter_pct`, `tool_guesses`, `timing_stats`, `phase_sequence`.
|
||||||
|
- `decnet/web/db/models.py::Token` / `LoginRequest` / `ChangePasswordRequest` — auth DTOs.
|
||||||
|
- `decnet/web/db/models.py::LogsResponse` / `BountyResponse` / `AttackersResponse` / `StatsResponse` — paginated list envelopes.
|
||||||
|
- `decnet/web/db/models.py::MutateIntervalRequest` — `r"^[1-9]\d*[mdMyY]$"` — minutes/days/Months/years.
|
||||||
|
- `decnet/web/db/models.py::DeployIniRequest` — wraps `IniContent` from `decnet.models`.
|
||||||
|
- `decnet/web/db/models.py::CreateUserRequest` / `UpdateUserRoleRequest` / `ResetUserPasswordRequest` — user-management DTOs (password length 8–72, role literal).
|
||||||
|
- `decnet/web/db/models.py::DeploymentLimitRequest` / `GlobalMutationIntervalRequest` — global config DTOs.
|
||||||
|
- `decnet/web/db/models.py::UserResponse` / `ConfigResponse` / `AdminConfigResponse` — response DTOs.
|
||||||
|
- `decnet/web/db/models.py::ComponentHealth` / `HealthResponse` — `/health` envelope.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/db/repository.py`
|
||||||
|
|
||||||
|
Abstract base class. Every method is `@abstractmethod` so any repo must explicitly implement the whole contract; `add_logs` provides a default per-row fallback.
|
||||||
|
|
||||||
|
- `decnet/web/db/repository.py::BaseRepository` — contract for storage: `initialize`, `add_log` / `add_logs`, `get_logs` / `get_total_logs` / `get_max_log_id` / `get_logs_after_id`, `get_stats_summary`, `get_deckies`, user CRUD (`get_user_by_username` / `get_user_by_uuid` / `create_user` / `update_user_password` / `list_users` / `delete_user` / `update_user_role`), `purge_logs_and_bounties`, bounty CRUD + `get_all_bounties_by_ip` / `get_bounties_for_ips`, `get_state` / `set_state`, attacker upsert + queries (`upsert_attacker`, `upsert_attacker_behavior`, `get_attacker_behavior`, `get_behaviors_for_ips`, `get_attacker_by_uuid`, `get_attackers`, `get_total_attackers`, `get_attacker_commands`, `get_attacker_artifacts`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/db/sqlmodel_repo.py`
|
||||||
|
|
||||||
|
Concrete portable SQLModel/SQLAlchemy-async implementation. Subclasses only override `__init__`, `_migrate_attackers_table`, `_json_field_equals`, and `get_log_histogram`. Everything else (filters, bounty dedup, attacker upsert, JSON blob deserialisation) is shared.
|
||||||
|
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository` — the portable base.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.initialize` — migrate legacy `attackers`, create all tables, seed the admin user.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.reinitialize` — re-create schema without dropping. Used by the `DELETE /config/reinit` flow.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository._ensure_admin_user` — seed admin from env; self-heal password drift when `must_change_password` is still true.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository._apply_filters` — shared `where` builder: `start_time` / `end_time`, `decky:xxx` / `service:xxx` / `event:xxx` / `attacker:xxx` shortcuts, arbitrary `key:value` that translates to `JSON_EXTRACT(fields, '$.key') = :val`, plus a `%search%` `OR` across `raw_line` / `decky` / `service` / `attacker_ip`.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository._json_field_equals` — SQL builder for JSON-field equality; overridden per dialect (SQLite `json_extract`, MySQL `JSON_UNQUOTE(JSON_EXTRACT(...))`).
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.add_log` / `add_logs` — single and bulk insert; `add_logs` is one session, one commit.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.get_logs` / `get_logs_after_id` / `get_total_logs` / `get_max_log_id` — paginated read paths; `get_logs_after_id` drives the SSE stream.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.get_stats_summary` — counts (logs, unique IPs) joined with the in-memory `load_state()` deckies.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.get_deckies` — returns the deckies from the state file (not the DB — the deploy state is the source of truth).
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.add_bounty` — deduplicates on `(bounty_type, attacker_ip, payload)` before insert.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.get_all_bounties_by_ip` / `get_bounties_for_ips` — grouped reads used by the profiler.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.upsert_attacker` / `upsert_attacker_behavior` / `get_attacker_behavior` / `get_behaviors_for_ips` / `get_attacker_by_uuid` / `get_attackers` / `get_total_attackers` / `get_attacker_commands` / `get_attacker_artifacts` — attacker-table operations.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository.purge_logs_and_bounties` — wipes logs, bounty, attacker_behavior, attackers; returns row counts.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::SQLModelRepository._deserialize_attacker` / `_deserialize_behavior` — decode JSON-text columns into Python dicts/lists.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::_safe_session` — cancellation-safe session context manager. Success path inline-closes so the caller observes cleanup; exception path hands `close()` off to a fresh task so a client disconnect can't orphan a MySQL connection.
|
||||||
|
- `decnet/web/db/sqlmodel_repo.py::_detach_close` — fire-and-forget session close on a brand-new task; invalidates the connection if close fails.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/db/sqlite/database.py`
|
||||||
|
|
||||||
|
Async + sync engine factories for SQLite (`aiosqlite`). On every connection, sets `PRAGMA journal_mode=WAL`, `PRAGMA synchronous=NORMAL`, and `PRAGMA busy_timeout=30000`.
|
||||||
|
|
||||||
|
- `decnet/web/db/sqlite/database.py::get_async_engine` — `create_async_engine("sqlite+aiosqlite:///<path>")` with pool knobs from env + the pragma listener.
|
||||||
|
- `decnet/web/db/sqlite/database.py::get_sync_engine` — sync twin used for DDL.
|
||||||
|
- `decnet/web/db/sqlite/database.py::init_db` — synchronously create all tables. Forces WAL first.
|
||||||
|
- `decnet/web/db/sqlite/database.py::get_session` — async-generator session dependency.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/db/sqlite/repository.py`
|
||||||
|
|
||||||
|
SQLite concrete subclass. Default DB path is `<repo root>/decnet.db`.
|
||||||
|
|
||||||
|
- `decnet/web/db/sqlite/repository.py::SQLiteRepository` — inherits `SQLModelRepository`.
|
||||||
|
- `decnet/web/db/sqlite/repository.py::SQLiteRepository.__init__` — build async engine and session factory.
|
||||||
|
- `decnet/web/db/sqlite/repository.py::SQLiteRepository._migrate_attackers_table` — via `PRAGMA table_info(attackers)`; drops the pre-UUID schema.
|
||||||
|
- `decnet/web/db/sqlite/repository.py::SQLiteRepository._json_field_equals` — `json_extract(fields, '$.key') = :val`.
|
||||||
|
- `decnet/web/db/sqlite/repository.py::SQLiteRepository.get_log_histogram` — `datetime((strftime('%s', timestamp) / N) * N, 'unixepoch')` for per-`interval_minutes` bucketing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/db/mysql/database.py`
|
||||||
|
|
||||||
|
MySQL async engine factory (driver: `asyncmy`). Precedence: explicit `url` argument > `DECNET_DB_URL` > component vars. Password is URL-encoded; empty passwords are allowed only under pytest.
|
||||||
|
|
||||||
|
- `decnet/web/db/mysql/database.py::build_mysql_url` — compose `mysql+asyncmy://user:pass@host:port/db`.
|
||||||
|
- `decnet/web/db/mysql/database.py::resolve_url` — pick URL source per the above precedence.
|
||||||
|
- `decnet/web/db/mysql/database.py::get_async_engine` — `create_async_engine` with `pool_size`, `max_overflow`, hourly `pool_recycle` (idle-reaper sidestep), and `pool_pre_ping=true` by default.
|
||||||
|
- `decnet/web/db/mysql/database.py::DEFAULT_POOL_SIZE` / `DEFAULT_MAX_OVERFLOW` / `DEFAULT_POOL_RECYCLE` / `DEFAULT_POOL_PRE_PING` — env-driven defaults.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/db/mysql/repository.py`
|
||||||
|
|
||||||
|
MySQL concrete subclass.
|
||||||
|
|
||||||
|
- `decnet/web/db/mysql/repository.py::MySQLRepository.__init__` — build async engine and session factory from a resolved URL.
|
||||||
|
- `decnet/web/db/mysql/repository.py::MySQLRepository.initialize` — acquires the `GET_LOCK('decnet_schema_init', 30)` advisory lock so concurrent uvicorn workers don't race on DDL, then runs `_migrate_attackers_table`, `_migrate_column_types`, `SQLModel.metadata.create_all`, `_ensure_admin_user`, releases lock.
|
||||||
|
- `decnet/web/db/mysql/repository.py::MySQLRepository._migrate_attackers_table` — uses `information_schema.COLUMNS` to drop pre-UUID `attackers`.
|
||||||
|
- `decnet/web/db/mysql/repository.py::MySQLRepository._migrate_column_types` — `ALTER TABLE … MODIFY COLUMN …` to widen `TEXT → MEDIUMTEXT` on columns that pre-date `_BIG_TEXT` (`attackers.commands` / `fingerprints` / `services` / `deckies`, `state.value`).
|
||||||
|
- `decnet/web/db/mysql/repository.py::MySQLRepository._json_field_equals` — `JSON_UNQUOTE(JSON_EXTRACT(fields, '$.key')) = :val`.
|
||||||
|
- `decnet/web/db/mysql/repository.py::MySQLRepository.get_log_histogram` — `FROM_UNIXTIME((UNIX_TIMESTAMP(timestamp) DIV N) * N)`; normalises the returned `datetime` to an ISO string for parity with SQLite.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/router/__init__.py`
|
||||||
|
|
||||||
|
Aggregates every sub-router (auth, logs, bounty, stats, fleet, attackers, config, health, stream, artifacts) into a single `api_router` which the app mounts under `/api/v1`.
|
||||||
|
|
||||||
|
- `decnet/web/router/__init__.py::api_router` — the combined `APIRouter`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## `decnet/web/router/auth/api_login.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/auth/api_login.py::login` — `POST /auth/login`. Verifies the bcrypt hash via `averify_password`, issues a 24-hour JWT, returns `Token`.
|
||||||
|
|
||||||
|
## `decnet/web/router/auth/api_change_pass.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/auth/api_change_pass.py::change_password` — `POST /auth/change-password`. Uses `get_current_user_unchecked` so a forced-change user can still call it; verifies the old password, hashes and stores the new one, clears `must_change_password`, invalidates the auth cache.
|
||||||
|
|
||||||
|
## `decnet/web/router/logs/api_get_logs.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/logs/api_get_logs.py` — `GET /logs` (`require_viewer`). Paginated log reads with the shared `_apply_filters` search syntax.
|
||||||
|
|
||||||
|
## `decnet/web/router/logs/api_get_histogram.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/logs/api_get_histogram.py` — `GET /logs/histogram` (`require_viewer`). Dialect-aware bucketed count series.
|
||||||
|
|
||||||
|
## `decnet/web/router/bounty/api_get_bounties.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/bounty/api_get_bounties.py::get_bounties` — `GET /bounty` (`require_viewer`). Cached default (no filters) via `_get_bounty_default_cached`.
|
||||||
|
|
||||||
|
## `decnet/web/router/stats/api_get_stats.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/stats/api_get_stats.py::get_stats` — `GET /stats` (`require_viewer`). Cached via `_get_stats_cached`.
|
||||||
|
|
||||||
|
## `decnet/web/router/fleet/api_get_deckies.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/fleet/api_get_deckies.py::get_deckies` — `GET /deckies` (`require_viewer`). Cached via `_get_deckies_cached`; reads the state file.
|
||||||
|
|
||||||
|
## `decnet/web/router/fleet/api_mutate_decky.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/fleet/api_mutate_decky.py::api_mutate_decky` — `POST /deckies/{decky_name}/mutate` (`require_admin`). Triggers `decnet.mutator.mutate_decky` immediately.
|
||||||
|
|
||||||
|
## `decnet/web/router/fleet/api_mutate_interval.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/fleet/api_mutate_interval.py::api_update_mutate_interval` — `PUT /deckies/{decky_name}/mutate-interval` (`require_admin`). Expects `<n>[mdMyY]`.
|
||||||
|
|
||||||
|
## `decnet/web/router/fleet/api_deploy_deckies.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/fleet/api_deploy_deckies.py::api_deploy_deckies` — `POST /deckies/deploy` (`require_admin`). Accepts `ini_content` (validated by `IniContent`), delegates to `decnet.engine.deploy`.
|
||||||
|
|
||||||
|
## `decnet/web/router/stream/api_stream_events.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/stream/api_stream_events.py::stream_events` — `GET /stream` (`require_stream_viewer`). Server-Sent-Events loop that polls `get_logs_after_id` and flushes new rows as `data:` frames. Honours `Last-Event-ID` and `?token=`.
|
||||||
|
|
||||||
|
## `decnet/web/router/attackers/*`
|
||||||
|
|
||||||
|
- `decnet/web/router/attackers/api_get_attackers.py::get_attackers` — `GET /attackers` (`require_viewer`). Filtering by service / search, sort by `recent` / `active` / `traversals`. Cached default via `_get_total_attackers_cached`.
|
||||||
|
- `decnet/web/router/attackers/api_get_attacker_detail.py::get_attacker_detail` — `GET /attackers/{uuid}` (`require_viewer`). Joins the behavior row.
|
||||||
|
- `decnet/web/router/attackers/api_get_attacker_commands.py::get_attacker_commands` — `GET /attackers/{uuid}/commands` (`require_viewer`). Paginated per-service slicing over the JSON `commands` column.
|
||||||
|
- `decnet/web/router/attackers/api_get_attacker_artifacts.py::get_attacker_artifacts` — `GET /attackers/{uuid}/artifacts` (`require_viewer`). `file_captured` log rows, newest first.
|
||||||
|
|
||||||
|
## `decnet/web/router/config/*`
|
||||||
|
|
||||||
|
- `decnet/web/router/config/api_get_config.py::api_get_config` — `GET /config` (`require_viewer`). Returns the caller's role, the deployment limit, the global mutation interval, and — for admins — the full user list. Caches user list + state via `_get_list_users_cached` / `_get_state_cached`.
|
||||||
|
- `decnet/web/router/config/api_update_config.py::api_update_deployment_limit` — `PUT /config/deployment-limit` (`require_admin`).
|
||||||
|
- `decnet/web/router/config/api_update_config.py::api_update_global_mutation_interval` — `PUT /config/global-mutation-interval` (`require_admin`).
|
||||||
|
- `decnet/web/router/config/api_manage_users.py::api_create_user` — `POST /config/users` (`require_admin`).
|
||||||
|
- `decnet/web/router/config/api_manage_users.py::api_delete_user` — `DELETE /config/users/{user_uuid}` (`require_admin`). Invalidates auth cache.
|
||||||
|
- `decnet/web/router/config/api_manage_users.py::api_update_user_role` — `PUT /config/users/{user_uuid}/role` (`require_admin`).
|
||||||
|
- `decnet/web/router/config/api_manage_users.py::api_reset_user_password` — `PUT /config/users/{user_uuid}/reset-password` (`require_admin`). Sets `must_change_password=True`.
|
||||||
|
- `decnet/web/router/config/api_reinit.py::api_reinit` — `DELETE /config/reinit` (`require_admin`). Calls `purge_logs_and_bounties` then `reinitialize`.
|
||||||
|
|
||||||
|
## `decnet/web/router/health/api_get_health.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/health/api_get_health.py::get_health` — `GET /health` (`require_viewer`). Returns `HealthResponse` (`healthy|degraded|unhealthy`) aggregated from: database roundtrip (cached via `_check_database_cached`), each background task's `.done()` status.
|
||||||
|
|
||||||
|
## `decnet/web/router/artifacts/api_get_artifact.py`
|
||||||
|
|
||||||
|
- `decnet/web/router/artifacts/api_get_artifact.py::get_artifact` — `GET /artifacts/{decky}/{stored_as}` (`require_admin`). Streams back a previously-captured attacker file drop; resolves the path inside the decky's artifact directory and refuses traversal attempts.
|
||||||
|
|||||||
@@ -1,215 +1,269 @@
|
|||||||
# Module Reference: Workers
|
# Module Reference — Workers
|
||||||
|
|
||||||
Code-level reference for every worker/subsystem module outside `decnet/web/`. For the pipeline overview see [Design overview](Design-Overview); for log format see [Logging](Logging-and-Syslog).
|
Every async background worker and host-side process that DECNET runs outside the FastAPI request path. These are the modules under `decnet/collector/`, `decnet/profiler/`, `decnet/sniffer/`, `decnet/prober/`, `decnet/mutator/`, `decnet/correlation/`, `decnet/engine/`, and `decnet/logging/`.
|
||||||
|
|
||||||
Each subsection lists: **entrypoint** (function spawned by the lifespan or CLI), **loop** (what it does on each tick), **reads** (input source), **writes** (output sink).
|
Citation format: `decnet/<package>/<module>.py::<symbol>`.
|
||||||
|
|
||||||
Citation format: `decnet/<path>::<symbol>`.
|
See also: [Design Overview](Design-Overview), [Logging and Syslog](Logging-and-Syslog), [Environment Variables](Environment-Variables).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/collector/` — Host-side Docker log collector
|
## Collector — `decnet/collector/`
|
||||||
|
|
||||||
Streams logs out of every DECNET-owned container on the host, parses the RFC 5424 line format, and appends to the shared JSON ingest file that `decnet/web/ingester.py` tails.
|
Tails `docker logs` for every decky service container, parses RFC 5424 lines, and forwards structured events to the ingester. Lives in the API lifespan as an asyncio task.
|
||||||
|
|
||||||
### `collector/worker.py`
|
### `decnet/collector/__init__.py`
|
||||||
|
|
||||||
- **Entrypoint:** `worker.py::log_collector_worker` — asyncio task started by the API lifespan.
|
Re-exports `is_service_container`, `is_service_event`, `log_collector_worker`, `parse_rfc5424`.
|
||||||
- **Loop:** enumerate running service containers via the Docker SDK; per container, spawn `_stream_container` which opens a log stream and appends parsed events to the ingest file. Rate-limits duplicate events.
|
|
||||||
- **Reads:** Docker API container log streams; `decnet-state.json` (for the canonical service container name set).
|
|
||||||
- **Writes:** JSON ingest log file (consumed by `web/ingester.py`).
|
|
||||||
|
|
||||||
Key functions:
|
### `decnet/collector/worker.py`
|
||||||
- `worker.py::parse_rfc5424` — parse a DECNET RFC 5424 line into a structured dict (SD params, IP fields, msg k/v).
|
|
||||||
- `worker.py::is_service_container`, `is_service_event` — gate which containers to ingest.
|
- `decnet/collector/worker.py::log_collector_worker` — async entry point. Walks the Docker API for containers matching DECNET service naming, launches a `_stream_container` task for each, and subscribes to Docker `events` so newly-started containers are attached mid-flight. Uses a dedicated `ThreadPoolExecutor(max_workers=64, thread_name_prefix="decnet-collector")` so blocking `docker logs` reads never starve the API event loop.
|
||||||
- `worker.py::_stream_container` — per-container streaming loop; `_reopen_if_needed` handles inode changes (log rotation).
|
- `decnet/collector/worker.py::parse_rfc5424` — regex-based RFC 5424 parser. Extracts TIMESTAMP, HOSTNAME (decky), APP-NAME (service), MSGID (event_type), the `[relay@55555 …]` SD block, and a free-text MSG tail. Falls back to a `_MSG_KV_RE` scanner for `key=value` pairs inside MSG when services skip the SD block.
|
||||||
- `worker.py::_should_ingest` — dedup + rate-limit filter.
|
- `decnet/collector/worker.py::is_service_container` / `is_service_event` — gatekeepers that reject non-DECNET containers and non-service log lines.
|
||||||
- Consts: `_RFC5424_RE`, `_SD_BLOCK_RE`, `_PARAM_RE`, `_IP_FIELDS`, `_MSG_KV_RE`.
|
- `decnet/collector/worker.py::_stream_container` — per-container reader with inode-aware reopen. If the log file gets rotated underneath us, the next read reopens.
|
||||||
|
- `decnet/collector/worker.py::_should_ingest` — rate-limit dedup. Events whose `event_type` appears in `DECNET_COLLECTOR_RL_EVENT_TYPES` (default: `connect,disconnect,connection,accept,close`) are keyed by `(attacker_ip, decky, service, event_type)` and suppressed within `DECNET_COLLECTOR_RL_WINDOW_SEC` seconds (default 5). Protects the DB from connection-flood amplification.
|
||||||
|
- `_RFC5424_RE`, `_SD_BLOCK_RE`, `_MSG_KV_RE` — compiled regexes.
|
||||||
|
- `_IP_FIELDS = ("src_ip", "src", "client_ip", "remote_ip", "remote_addr", "target_ip", "ip")` — the canonical priority order for extracting the attacker IP from the SD params.
|
||||||
|
|
||||||
|
Reads: Docker container logs. Writes: calls `repo.insert_log(...)` via the ingester queue. See [Logging and Syslog](Logging-and-Syslog) for the end-to-end pipeline.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/profiler/` — Attacker profile builder
|
## Profiler — `decnet/profiler/`
|
||||||
|
|
||||||
Incrementally joins raw log rows into per-attacker profiles in the `attackers` + `attacker_behavior` tables.
|
Periodic attacker-profile builder. Walks new log rows, updates the in-memory `CorrelationEngine`, and upserts per-IP attacker records + behavioural aggregates.
|
||||||
|
|
||||||
### `profiler/worker.py`
|
### `decnet/profiler/__init__.py`
|
||||||
|
|
||||||
- **Entrypoint:** `worker.py::attacker_profile_worker` — long-running asyncio task.
|
Re-exports `attacker_profile_worker`.
|
||||||
- **Loop:** every N seconds, fetch logs with `id > last_id` (`_incremental_update`), bucket by attacker IP, call `_update_profiles` to upsert `Attacker` and `AttackerBehavior` rows. State key `_STATE_KEY` persists the last-seen log id.
|
|
||||||
- **Reads:** `logs` table (incremental via `get_logs_after_id`).
|
|
||||||
- **Writes:** `attackers`, `attacker_behavior`, `state` tables.
|
|
||||||
|
|
||||||
Helpers: `_build_record`, `_first_contact_deckies`, `_extract_commands_from_events`. Consts: `_BATCH_SIZE`, `_COMMAND_EVENT_TYPES`, `_COMMAND_FIELDS`.
|
### `decnet/profiler/worker.py`
|
||||||
|
|
||||||
### `profiler/behavioral.py`
|
- `decnet/profiler/worker.py::attacker_profile_worker` — async loop `while True: _incremental_update(); await asyncio.sleep(interval)`. Default `interval=30`.
|
||||||
|
- `decnet/profiler/worker.py::_WorkerState` — dataclass with `engine: CorrelationEngine`, `last_log_id: int`, `initialized: bool`. Persisted across restarts via `repo.set_state("attacker_worker_cursor", …)` keyed by `_STATE_KEY`.
|
||||||
|
- `decnet/profiler/worker.py::_incremental_update` — cursor-driven batch read. Calls `repo.get_logs_after_id(last_log_id, limit=_BATCH_SIZE)` where `_BATCH_SIZE = 500`, feeds each line into `engine.ingest`, advances the cursor, then calls `_update_profiles`.
|
||||||
|
- `decnet/profiler/worker.py::_update_profiles` — for every attacker IP currently known to the engine, builds a `_first_contact_deckies` map + a command history from `_extract_commands_from_events`, and upserts both the attacker record (`repo.upsert_attacker`) and the behavioural aggregates (`repo.upsert_attacker_behavior`).
|
||||||
|
- `decnet/profiler/worker.py::_build_record` — assembles the per-attacker dict (bounties map, traversal path, decky count, first/last seen, command count).
|
||||||
|
- `_COMMAND_EVENT_TYPES = {"command", "exec", "query", "input", "shell_input", "execute", "run", "sql_query", "redis_command"}` — event-type allow-list used when mining commands.
|
||||||
|
- `_COMMAND_FIELDS = ("command", "query", "input", "line", "sql", "cmd")` — SD-param keys probed inside each matching event.
|
||||||
|
|
||||||
Stateless analyzers used by `worker.py::_update_profiles`.
|
Reads: `repo.get_logs_after_id`, `repo.get_state`. Writes: `repo.upsert_attacker`, `repo.upsert_attacker_behavior`, `repo.set_state`.
|
||||||
|
|
||||||
- `behavioral.py::timing_stats` — inter-arrival time mean/stddev/CV.
|
|
||||||
- `behavioral.py::classify_behavior` — coarse behavior bucket (scanner / interactive / automated).
|
|
||||||
- `behavioral.py::guess_tools` — match (mean IAT, CV) against C2 beacon profiles.
|
|
||||||
- `behavioral.py::detect_tools_from_headers` — scan HTTP `request` events for tool-identifying headers.
|
|
||||||
- `behavioral.py::phase_sequence` — derive recon→exfil transitions.
|
|
||||||
- `behavioral.py::sniffer_rollup` — roll up `tcp_syn_fingerprint` + `tcp_flow_timing` events.
|
|
||||||
- `behavioral.py::build_behavior_record` — compose the dict persisted to `attacker_behavior`.
|
|
||||||
- Helpers: `_os_from_ttl`, `_int_or_none`, `guess_tool` (deprecated, kept for BC).
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/sniffer/` — Fleet-wide passive MACVLAN sniffer
|
## Sniffer — `decnet/sniffer/`
|
||||||
|
|
||||||
Runs as a `fleet_singleton` container on the MACVLAN (or falls back to the `ipvlan` host iface) and emits TLS/TCP fingerprints for every flow touching a decky.
|
Fleet-singleton passive TLS fingerprinter. Runs in the API lifespan but sniffs the MACVLAN/IPvlan host interface directly with scapy. Extracts JA3/JA3S/JA4/JA4S/JA4L, TCP SYN OS fingerprints, flow timing, and x509 certificate metadata from the wire.
|
||||||
|
|
||||||
### `sniffer/worker.py`
|
See also: `templates/sniffer/` for the container variant (fleet-wide when the singleton is declared via INI). The Python worker here is the host-side flavour.
|
||||||
|
|
||||||
- **Entrypoint:** `worker.py::sniffer_worker` — started as `asyncio.create_task` by the API lifespan.
|
### `decnet/sniffer/__init__.py`
|
||||||
- **Loop:** `worker.py::_sniff_loop` (runs in a dedicated thread via `asyncio.to_thread`) drives `scapy.sniff` on the MACVLAN interface; each packet goes to `SnifferEngine.on_packet`.
|
|
||||||
- **Reads:** live packets on the sniffer interface; `decnet-state.json` → `_load_ip_to_decky`.
|
|
||||||
- **Writes:** RFC 5424 syslog lines via `sniffer/syslog.py::write_event` (both raw syslog and parsed JSON).
|
|
||||||
|
|
||||||
Helpers: `_interface_exists`, `_load_ip_to_decky`.
|
Thin re-export of `sniffer_worker`.
|
||||||
|
|
||||||
### `sniffer/fingerprint.py::SnifferEngine`
|
### `decnet/sniffer/worker.py`
|
||||||
|
|
||||||
Stateful TLS + TCP fingerprint engine. Tracks sessions, TCP RTTs, flow aggregation, dedup. Emits `tls_handshake`, `tcp_syn_fingerprint`, `tcp_flow_timing` events.
|
- `decnet/sniffer/worker.py::sniffer_worker` — async entry point. Selects the interface (`DECNET_SNIFFER_IFACE` env override, else `HOST_MACVLAN_IFACE` then `HOST_IPVLAN_IFACE` from `decnet.network`). If neither exists (fleet not deployed), the worker logs a warning and returns — the API continues. Builds a stop-event, spawns a dedicated 2-worker `ThreadPoolExecutor`, and runs the blocking scapy `sniff()` loop via `loop.run_in_executor`. Fully fault-isolated.
|
||||||
|
- `decnet/sniffer/worker.py::_sniff_loop` — the blocking thread target. Builds the initial `ip_to_decky` map, instantiates a `SnifferEngine`, launches a daemon refresh thread that calls `_load_ip_to_decky()` every 60 s so late-joining deckies are captured, then enters `scapy.sniff(filter="tcp", prn=engine.on_packet, store=False, stop_filter=…)`.
|
||||||
|
- `decnet/sniffer/worker.py::_load_ip_to_decky` — pulls the `{ip: decky_name}` map from `decnet-state.json` via `decnet.config.load_state`.
|
||||||
|
- `decnet/sniffer/worker.py::_interface_exists` — `ip link show <iface>` probe, returns bool.
|
||||||
|
- `_IP_MAP_REFRESH_INTERVAL = 60.0` — seconds between refreshes.
|
||||||
|
|
||||||
Notable methods: `on_packet` (main dispatch), `_update_flow`, `_flush_flow`, `flush_all_flows`, `_flush_idle_flows`, `_resolve_decky`, `_is_duplicate`. Private parsers: `_parse_client_hello`, `_parse_server_hello`, `_parse_certificate`, `_parse_x509_der`, `_extract_sans`; fingerprint algorithms: `_ja3`, `_ja3s`, `_ja4`, `_ja4s`, `_ja4l`.
|
### `decnet/sniffer/fingerprint.py`
|
||||||
|
|
||||||
### `sniffer/p0f.py`
|
Stateful engine + TLS parser. 1166 lines — the bulk is a hand-written TLS 1.0–1.3 ClientHello / ServerHello / Certificate parser (deliberately avoids `cryptography`/`scapy.layers.tls` to keep the fingerprint exactly reproducible).
|
||||||
|
|
||||||
Passive OS fingerprinting (p0f-lite).
|
- `decnet/sniffer/fingerprint.py::SnifferEngine` — per-flow session tracker. Keys flows by `(src_ip, src_port, dst_ip, dst_port)`, maintains `_sessions`, `_session_ts`, `_tcp_syn`, `_tcp_rtt`, `_flows`, and `_dedup_cache`. Public: `update_ip_map(...)`, `on_packet(pkt)`.
|
||||||
|
- `decnet/sniffer/fingerprint.py::SnifferEngine._resolve_decky` — maps a packet's src/dst IP to a known decky name; returns `None` (and the packet is dropped) if neither side is a decky.
|
||||||
|
- `decnet/sniffer/fingerprint.py::SnifferEngine._dedup_key_for` — per-event-type dedup strategy. Notable rule: `tcp_flow_timing` dedups on `(dst_ip, dst_port)` only — the attacker's ephemeral source port is deliberately excluded so a port scanner rotating sources still only emits one timing event per window.
|
||||||
|
- TLS parsers: `_parse_client_hello`, `_parse_server_hello`, `_parse_certificate`, plus DER helpers `_der_read_tag_len`, `_der_read_sequence`, `_der_read_oid`, `_der_extract_cn`, `_der_extract_name_str`, `_parse_x509_der`, `_extract_sans`, `_parse_san_sequence`.
|
||||||
|
- Fingerprint hashes: `_ja3`, `_ja3s`, `_ja4`, `_ja4s`, `_ja4l`. `_ja4_version`, `_ja4_alpn_tag`, `_sha256_12` are helpers.
|
||||||
|
- GREASE handling: `_is_grease`, `_filter_grease` drop the RFC 8701 sentinel values before hashing so Chrome/Firefox JA3s are stable.
|
||||||
|
- TCP SYN: `_extract_tcp_fingerprint` pulls MSS / window scale / option order out of the SYN options list and hands it to `decnet.sniffer.p0f::guess_os`.
|
||||||
|
|
||||||
- `initial_ttl` — round observed TTL up to the nearest initial-TTL bucket.
|
### `decnet/sniffer/p0f.py`
|
||||||
- `hop_distance` — estimate hop count.
|
|
||||||
- `guess_os` — coarse OS bucket from SYN characteristics.
|
|
||||||
- `_match_signature` — predicate evaluator.
|
|
||||||
|
|
||||||
### `sniffer/syslog.py`
|
Mini-p0f OS classifier using the passive TCP SYN fingerprint.
|
||||||
|
|
||||||
RFC 5424 emitter for the sniffer.
|
- `decnet/sniffer/p0f.py::initial_ttl` — map an observed TTL to the most likely hop-origin value (64, 128, 255).
|
||||||
|
- `decnet/sniffer/p0f.py::hop_distance` — estimate hop count.
|
||||||
|
- `decnet/sniffer/p0f.py::guess_os` — score the observed SYN against the built-in signature table; returns the best-match OS string or `"unknown"`.
|
||||||
|
- `_match_signature` — internal scorer used by `guess_os`.
|
||||||
|
|
||||||
- `syslog.py::syslog_line` — build a line.
|
### `decnet/sniffer/syslog.py`
|
||||||
- `syslog.py::write_event` — append a syslog line to the raw log and its parsed JSON to the json log.
|
|
||||||
- Helpers: `_sd_escape`, `_sd_element`. Consts: `_FACILITY_LOCAL0`, `_SD_ID`, `_NILVALUE`, `SEVERITY_INFO`, `SEVERITY_WARNING`, `_MAX_HOSTNAME`, `_MAX_APPNAME`, `_MAX_MSGID`.
|
Lightweight RFC 5424 emitter used by the sniffer (not `decnet.logging.syslog_formatter` — this one writes directly to a file and a sibling `.json` for the live-logs stream).
|
||||||
|
|
||||||
|
- `decnet/sniffer/syslog.py::syslog_line` — build a line with `<PRI>1 TS HOSTNAME APP-NAME - MSGID [relay@55555 …] MSG`.
|
||||||
|
- `decnet/sniffer/syslog.py::write_event` — append a line to `.log` + a JSON record to `.json`. Tested by `tests/test_sniffer_emit_capture.py`.
|
||||||
|
- `_sd_escape`, `_sd_element` — per-SD-param escaping helpers matching the main formatter.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/prober/` — Active attacker fingerprinting
|
## Prober — `decnet/prober/`
|
||||||
|
|
||||||
Standalone worker process (separate from the API) that actively probes IPs observed in the logs to collect JARM, HASSHServer, and TCP/IP fingerprints without ever impersonating DECNET.
|
Standalone active-probe daemon. Tails the collector JSON log for new attacker IPs, then hits each with JARM (TLS), HASSH (SSH), and raw TCP SYN probes.
|
||||||
|
|
||||||
### `prober/worker.py`
|
### `decnet/prober/__init__.py`
|
||||||
|
|
||||||
- **Entrypoint:** `worker.py::prober_worker` — main loop for the standalone prober process.
|
Re-exports `prober_worker`, `jarm_hash`, `hassh_server`, `tcp_fingerprint`.
|
||||||
- **Loop:** tail the JSON ingest log (`_discover_attackers`), cycle through unique attacker IPs (`_probe_cycle`), run `_jarm_phase`, `_hassh_phase`, `_tcpfp_phase`, emit results as RFC 5424 events via `_syslog_line` + `_write_event`.
|
|
||||||
- **Reads:** JSON ingest log; configured probe targets.
|
|
||||||
- **Writes:** RFC 5424 events into the ingest file so the regular collector pipeline picks them up.
|
|
||||||
|
|
||||||
### `prober/jarm.py`
|
### `decnet/prober/worker.py`
|
||||||
|
|
||||||
Pure-stdlib JARM TLS fingerprinting (10 ClientHello probes, reassembled into the 62-char JARM hash).
|
- `decnet/prober/worker.py::prober_worker` — async entry point, 5-minute default cycle. Parameters: `log_file`, `interval=300`, `timeout=5.0`, `ports`/`ssh_ports`/`tcpfp_ports`. Writes a `prober_startup` event, then loops: `_discover_attackers → _probe_cycle → asyncio.sleep(interval)`.
|
||||||
|
- `decnet/prober/worker.py::_discover_attackers` — resumes from a byte offset into the collector JSON log, returns newly-seen attacker IPs.
|
||||||
|
- `decnet/prober/worker.py::_probe_cycle` — fan-out across phases. Each attacker is probed at most once per (phase, port) pair; the `probed: dict[str, dict[str, set[int]]]` cache is the durable state.
|
||||||
|
- `decnet/prober/worker.py::_jarm_phase` — calls `jarm_hash(ip, port, timeout)` for each port in `DEFAULT_PROBE_PORTS = [443, 8443, 8080, 4443, 50050, 2222, 993, 995, 8888, 9001]`.
|
||||||
|
- `decnet/prober/worker.py::_hassh_phase` — calls `hassh_server` for `DEFAULT_SSH_PORTS = [22, 2222, 22222, 2022]`.
|
||||||
|
- `decnet/prober/worker.py::_tcpfp_phase` — calls `tcp_fingerprint` for `DEFAULT_TCPFP_PORTS = [22, 80, 443, 8080, 8443, 445, 3389]`.
|
||||||
|
- `decnet/prober/worker.py::_write_event` — dual-write RFC 5424 line + JSON record (same shape as `decnet.sniffer.syslog.write_event`; see the module docstring's tech-debt note about extracting a shared sink).
|
||||||
|
- `_parse_to_json`, `_syslog_line`, `_sd_escape`, `_sd_element` — inline formatting helpers.
|
||||||
|
|
||||||
- `jarm.py::jarm_hash` — public entrypoint.
|
### `decnet/prober/jarm.py`
|
||||||
- Helpers: `_build_client_hello`, `_send_probe`, `_parse_server_hello`, `_compute_jarm`; per-extension builders (`_ext_*`).
|
|
||||||
|
|
||||||
### `prober/hassh.py`
|
Raw-socket JARM probe. Builds 10 hand-crafted ClientHellos per the JARM spec, concatenates the 10 ServerHello fingerprints, and returns the 62-character JARM hash.
|
||||||
|
|
||||||
HASSHServer SSH fingerprinting (MD5 of `kex;enc_s2c;mac_s2c;comp_s2c` name-lists).
|
- `decnet/prober/jarm.py::jarm_hash` — public entry.
|
||||||
|
- `_build_client_hello`, `_parse_server_hello`, `_send_probe`, `_compute_jarm` — the four stages.
|
||||||
|
- Extension builders: `_ext`, `_ext_sni`, `_ext_supported_groups`, `_ext_ec_point_formats`, `_ext_signature_algorithms`, `_ext_supported_versions_13`, `_ext_psk_key_exchange_modes`, `_ext_key_share`, `_ext_alpn`, `_ext_session_ticket`, `_ext_encrypt_then_mac`, `_ext_extended_master_secret`, `_ext_padding`. `_middle_out` produces the ciphersuite ordering used by probes 5–9. `_version_to_str` maps the two-byte TLS version field to the human string.
|
||||||
|
|
||||||
- `hassh.py::hassh_server` — public entrypoint.
|
### `decnet/prober/hassh.py`
|
||||||
- Helpers: `_ssh_connect`, `_read_banner`, `_read_ssh_packet`, `_parse_kex_init`, `_compute_hassh`, `_recv_exact`.
|
|
||||||
|
|
||||||
### `prober/tcpfp.py`
|
SSH server HASSH fingerprint. Reads the SSH banner, sends nothing, parses the server's `SSH_MSG_KEXINIT`, and computes `md5(kex;encryption;mac;compression)`.
|
||||||
|
|
||||||
TCP/IP stack fingerprinting via crafted SYN + SYN-ACK analysis (scapy).
|
- `decnet/prober/hassh.py::hassh_server` — public entry.
|
||||||
|
- `_ssh_connect`, `_read_banner`, `_read_ssh_packet`, `_recv_exact` — socket IO.
|
||||||
|
- `_parse_kex_init` — extract the four algorithm lists.
|
||||||
|
- `_compute_hassh` — MD5 concatenation per the HASSH spec.
|
||||||
|
|
||||||
- `tcpfp.py::tcp_fingerprint` — public entrypoint.
|
### `decnet/prober/tcpfp.py`
|
||||||
- Helpers: `_send_syn`, `_send_rst`, `_parse_synack`, `_extract_options_order`, `_compute_fingerprint`.
|
|
||||||
|
Raw-socket SYN probe for active OS fingerprinting (complements the passive sniffer fingerprint).
|
||||||
|
|
||||||
|
- `decnet/prober/tcpfp.py::tcp_fingerprint` — public entry. Sends a SYN, parses the SYN-ACK, computes a short fingerprint string + SHA-256 hash, sends an RST to avoid the half-open state.
|
||||||
|
- `_send_syn`, `_send_rst`, `_parse_synack`, `_extract_options_order`, `_compute_fingerprint` — internal stages.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/mutator/` — Persona mutation engine
|
## Mutator — `decnet/mutator/`
|
||||||
|
|
||||||
Periodically swaps a decky's services/personas (intra-archetype shuffle) so the same IP/MAC shows different attack surfaces over time. See [Mutation and Randomization](Mutation-and-Randomization).
|
Rotates the service set of a decky without tearing down the fleet. Each mutation picks a new service pool from the decky's archetype (or the global catalogue) and re-writes the compose file.
|
||||||
|
|
||||||
### `mutator/engine.py`
|
### `decnet/mutator/__init__.py`
|
||||||
|
|
||||||
- **Entrypoint:** `engine.py::run_watch_loop` — infinite loop checking for deckies past their mutation interval; spawned by the CLI or the API.
|
Empty package marker.
|
||||||
- **Loop:** `mutate_all` iterates due deckies → `mutate_decky` runs intra-archetype shuffle (rewrite compose, `docker compose up -d`).
|
|
||||||
- **Reads:** `decnet-state.json`, repository (per-decky mutation interval).
|
### `decnet/mutator/engine.py`
|
||||||
- **Writes:** rewritten `docker-compose.yml`, updated state, new containers.
|
|
||||||
|
- `decnet/mutator/engine.py::mutate_decky` — Intra-Archetype Shuffle for a single decky. Reads the deployment state from `repo.get_state("deployment")`, picks a fresh random service subset from the decky's archetype (via `decnet.archetypes.get_archetype`) or `decnet.fleet.all_service_names`, calls `decnet.composer.write_compose` to regenerate the file, then `_compose_with_retry("up", "-d", …)` to roll the container set.
|
||||||
|
- `decnet/mutator/engine.py::mutate_all` — iterates every decky in the deployment; a decky is "due" when `time.time() - decky.last_mutated >= decky.mutate_interval * 60`. `force=True` bypasses the schedule.
|
||||||
|
- `decnet/mutator/engine.py::run_watch_loop` — infinite `mutate_all(force=False)` poll with `poll_interval_secs=10` default. Started by the API lifespan when the CLI deploys with mutation enabled.
|
||||||
|
|
||||||
|
Reads: `repo.get_state("deployment")`. Writes: compose file on disk, `repo.set_state` to persist `last_mutated` timestamps.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/correlation/` — Cross-decky attacker traversal
|
## Correlation — `decnet/correlation/`
|
||||||
|
|
||||||
Builds attacker-traversal graphs by correlating events across multiple deckies.
|
In-memory cross-decky correlation engine. Used by both the standalone `decnet correlate` CLI and the live profiler. Independent of the DB — pure stream-ingest.
|
||||||
|
|
||||||
### `correlation/engine.py::CorrelationEngine`
|
### `decnet/correlation/__init__.py`
|
||||||
|
|
||||||
- **Entrypoint:** `ingest_file(path)` (batch) or `ingest(line)` (stream) — no background task by default; callable from CLI and tests.
|
Re-exports `CorrelationEngine`, `AttackerTraversal`, `TraversalHop`, `LogEvent`, `parse_line`.
|
||||||
- **Loop:** per-line: parse with `parser.parse_line`, append `TraversalHop` to per-IP `AttackerTraversal`.
|
|
||||||
- **Reads:** RFC 5424 syslog lines (file or stream).
|
|
||||||
- **Writes:** nothing by default; exposes `traversals(min_deckies)`, `all_attackers()`, `report_table()`, `report_json()`, `traversal_syslog_lines()`. Helper `_fmt_duration`.
|
|
||||||
|
|
||||||
### `correlation/graph.py`
|
### `decnet/correlation/parser.py`
|
||||||
Data classes. `TraversalHop` = a single event; `AttackerTraversal` = all activity from one IP across ≥ 2 deckies with `first_seen`, `last_seen`, `duration_seconds`, `deckies` (ordered), `decky_count`, `path` (`decky-01 → decky-03 → decky-07`), `to_dict`.
|
|
||||||
|
|
||||||
### `correlation/parser.py`
|
- `decnet/correlation/parser.py::LogEvent` — dataclass `(timestamp, decky, service, event_type, attacker_ip, fields, raw)`.
|
||||||
Stateless RFC 5424 parser shared by `correlation/engine.py` and ad-hoc tooling.
|
- `decnet/correlation/parser.py::parse_line` — parse one RFC 5424 line into a `LogEvent`. Returns `None` for blank / non-DECNET / missing-hostname lines.
|
||||||
|
- `_parse_sd_params` — extract `k="v"` pairs from the `[relay@55555 …]` SD block with proper RFC 5424 unescaping.
|
||||||
|
- `_extract_attacker_ip` — probes `_IP_FIELDS = ("src_ip", "src", "client_ip", "remote_ip", "remote_addr", "target_ip", "ip")` in priority order.
|
||||||
|
- `_RFC5424_RE`, `_SD_BLOCK_RE`, `_PARAM_RE` — compiled regexes.
|
||||||
|
|
||||||
- `parser.py::parse_line` — parse one line into `LogEvent`.
|
### `decnet/correlation/graph.py`
|
||||||
- `parser.py::LogEvent` — parsed event record.
|
|
||||||
- Helpers: `_parse_sd_params`, `_extract_attacker_ip`. Consts: `_RFC5424_RE`, `_SD_BLOCK_RE`, `_PARAM_RE`, `_IP_FIELDS`.
|
- `decnet/correlation/graph.py::TraversalHop` — single event in a traversal: `(timestamp, decky, service, event_type)`.
|
||||||
|
- `decnet/correlation/graph.py::AttackerTraversal` — all hops for one attacker IP, sorted chronologically. Properties: `first_seen`, `last_seen`, `duration_seconds`, `deckies` (unique, first-contact-ordered), `decky_count`, `path` (`"decky-01 → decky-03 → decky-07"`). `to_dict()` for JSON serialisation.
|
||||||
|
|
||||||
|
### `decnet/correlation/engine.py`
|
||||||
|
|
||||||
|
- `decnet/correlation/engine.py::CorrelationEngine` — stateful aggregator. `_events: dict[str, list[LogEvent]]` keyed by attacker IP.
|
||||||
|
- `ingest(line)` — parse + index one line.
|
||||||
|
- `ingest_file(path)` — whole-file variant with OTEL summary span.
|
||||||
|
- `traversals(min_deckies=2)` — return attackers that touched at least `min_deckies` distinct deckies, sorted by first-seen.
|
||||||
|
- `all_attackers()` — `{ip: event_count}` sorted desc.
|
||||||
|
- `report_table(min_deckies)` — rich `Table` for CLI output.
|
||||||
|
- `report_json(min_deckies)` — serialisable dict.
|
||||||
|
- `traversal_syslog_lines(min_deckies)` — emit `format_rfc5424(service="correlator", hostname="decnet-correlator", event_type="traversal_detected", severity=SEVERITY_WARNING, …)` lines so the SIEM can ingest correlator findings alongside raw service events.
|
||||||
|
- `_fmt_duration` — humanise seconds → `45s` / `3.2m` / `1.5h`.
|
||||||
|
|
||||||
|
Reads: RFC 5424 lines (string or file). Writes: nothing — pure computation; the profiler embeds a `CorrelationEngine` and persists its results via `repo.upsert_attacker`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/engine/` — Deploy/teardown orchestration
|
## Engine — `decnet/engine/`
|
||||||
|
|
||||||
### `engine/deployer.py`
|
Docker-compose orchestration. Wraps the `docker compose` CLI with retry logic and network setup/teardown.
|
||||||
|
|
||||||
- **Entrypoint:** `deployer.py::deploy`, `teardown`, `status` — called from the CLI and the fleet router.
|
### `decnet/engine/__init__.py`
|
||||||
- **Loop (`deploy`):** render `docker-compose.yml` from archetypes + service plugins → sync the canonical `syslog_bridge.py` into every build context (`_sync_logging_helper`) → invoke `docker compose up -d` via `_compose` with retry on transient failures (`_compose_with_retry`).
|
|
||||||
- **Reads:** CLI args / INI config / archetype definitions / service plugin `compose_fragment`.
|
|
||||||
- **Writes:** `docker-compose.yml`, built images, running containers, `decnet-state.json`.
|
|
||||||
|
|
||||||
Consts: `COMPOSE_FILE`, `_CANONICAL_LOGGING`, `_PERMANENT_ERRORS`. Helper: `_print_status`.
|
Re-exports `COMPOSE_FILE`, `_compose_with_retry`, `deploy`, `status`, `teardown`.
|
||||||
|
|
||||||
See [Deployment Modes](Deployment-Modes) and [Teardown and State](Teardown-and-State).
|
### `decnet/engine/deployer.py`
|
||||||
|
|
||||||
|
- `decnet/engine/deployer.py::COMPOSE_FILE` — module-level `Path("decnet-compose.yml")`.
|
||||||
|
- `decnet/engine/deployer.py::deploy` — create the MACVLAN / IPvlan network, set up the host-side virtual iface via `decnet.network.setup_host_macvlan` / `setup_host_ipvlan`, sync the canonical `templates/syslog_bridge.py` into every active service template (`_sync_logging_helper`), write the compose file (`decnet.composer.write_compose`), save deployment state, build (`docker compose build`), and `up -d`. `parallel=True` enables `DOCKER_BUILDKIT=1` for concurrent image builds. `no_cache=True` forces a full rebuild.
|
||||||
|
- `decnet/engine/deployer.py::teardown` — either `decky_id=None` (full teardown: `compose down`, teardown host iface, remove docker network, clear state) or `decky_id="decky-xx"` (stop + `compose rm` only that decky's containers).
|
||||||
|
- `decnet/engine/deployer.py::status` — render a rich table of running/absent containers per decky.
|
||||||
|
- `decnet/engine/deployer.py::_compose_with_retry` — exponential-backoff retry (3 attempts, 5 s → 10 s → 20 s). Errors matching `_PERMANENT_ERRORS = ("manifest unknown", "manifest for", "not found", "pull access denied", "repository does not exist")` short-circuit to fail-fast.
|
||||||
|
- `_compose` — no-retry variant used by teardown.
|
||||||
|
- `_sync_logging_helper` — copies `templates/syslog_bridge.py` into every service's `dockerfile_context()` directory so the canonical copy is what gets baked into images. Only re-copies when the target differs byte-for-byte.
|
||||||
|
- `_print_status` — compact post-deploy summary.
|
||||||
|
|
||||||
|
Reads: Docker daemon, `decnet-state.json`. Writes: `decnet-compose.yml`, host iface state, docker networks/containers.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## `decnet/logging/` — Application logging helpers
|
## Logging — `decnet/logging/`
|
||||||
|
|
||||||
Standard-library `logging` integration that makes DECNET's own Python logs RFC 5424-compliant and mergeable with service logs.
|
Application-level logging helpers. Separate from the container-side `templates/syslog_bridge.py` used inside deckies, though both produce RFC 5424.
|
||||||
|
|
||||||
### `logging/__init__.py`
|
### `decnet/logging/__init__.py`
|
||||||
- `__init__.py::get_logger(component)` — return a named logger that self-identifies as `component` in RFC 5424.
|
|
||||||
- `__init__.py::enable_trace_context` — install the OTEL trace-context filter on the root `decnet` logger.
|
|
||||||
- `__init__.py::_ComponentFilter`, `_TraceContextFilter` — inject `decnet_component`, `otel_trace_id`, `otel_span_id` onto LogRecords so the formatter can emit them as SD params.
|
|
||||||
|
|
||||||
### `logging/syslog_formatter.py`
|
- `decnet/logging/__init__.py::get_logger` — returns `logging.getLogger(f"decnet.{component}")` with a `_ComponentFilter` attached so Rfc5424Formatter emits the component as APP-NAME. Idempotent — calling twice for the same component doesn't stack filters.
|
||||||
- `syslog_formatter.py::format_rfc5424` — return a single RFC 5424-compliant syslog line (no trailing newline).
|
- `decnet/logging/__init__.py::enable_trace_context` — attach the `_TraceContextFilter` to the root `decnet` logger. Called once from `decnet.telemetry.setup_tracing()` so every LogRecord carries `otel_trace_id` / `otel_span_id` drawn from the active OTEL span (or `"0"` when no span is active — cheap string comparison downstream).
|
||||||
- Helpers: `_pri`, `_truncate`, `_sd_escape`, `_sd_element`. Consts: `FACILITY_LOCAL0`, `NILVALUE`, `_SD_ID`, `SEVERITY_INFO`, `SEVERITY_WARNING`, `SEVERITY_ERROR`, `_MAX_HOSTNAME`, `_MAX_APPNAME`, `_MAX_MSGID`.
|
- `_ComponentFilter` — injects `record.decnet_component` so Rfc5424Formatter can promote it to APP-NAME.
|
||||||
|
- `_TraceContextFilter` — pulls trace ids from `opentelemetry.trace.get_current_span()`. All errors are swallowed (logging must never crash the caller).
|
||||||
|
- `_trace_filter_installed` — module-level idempotency flag for `enable_trace_context`.
|
||||||
|
|
||||||
### `logging/file_handler.py`
|
### `decnet/logging/syslog_formatter.py`
|
||||||
- `file_handler.py::write_syslog` — write a single RFC 5424 line to the rotating log file.
|
|
||||||
- `file_handler.py::get_log_path` — return the configured log file path.
|
|
||||||
- `file_handler.py::_init_file_handler`, `_get_logger`. Consts: `_LOG_FILE_ENV`, `_DEFAULT_LOG_FILE`, `_MAX_BYTES`, `_BACKUP_COUNT`.
|
|
||||||
|
|
||||||
### `logging/inode_aware_handler.py`
|
Canonical RFC 5424 emitter used by `decnet.telemetry` and the correlator.
|
||||||
- `inode_aware_handler.py::InodeAwareRotatingFileHandler` — RotatingFileHandler that detects external deletion/rotation (logrotate) and reopens the target. Methods: `_should_reopen`, `emit`.
|
|
||||||
|
|
||||||
### `logging/forwarder.py`
|
- `decnet/logging/syslog_formatter.py::format_rfc5424` — `<PRI>1 TIMESTAMP HOSTNAME APP-NAME - MSGID [SD] MSG`. Enforces RFC 5424 length limits: HOSTNAME ≤ 255, APP-NAME ≤ 48, MSGID ≤ 32. Facility is always `local0 (16)`. PEN is `relay@55555`.
|
||||||
- `forwarder.py::parse_log_target` — parse `"ip:port"` into `(host, port)`.
|
- `SEVERITY_INFO = 6`, `SEVERITY_WARNING = 4`, `SEVERITY_ERROR = 3` — public severity constants.
|
||||||
- `forwarder.py::probe_log_target` — TCP-connect probe for reachability.
|
- `FACILITY_LOCAL0 = 16`, `NILVALUE = "-"`, `_SD_ID = "relay@55555"`.
|
||||||
|
- `_pri`, `_truncate`, `_sd_escape` (RFC 5424 §6.3.3 escapes), `_sd_element` — internal formatters.
|
||||||
|
|
||||||
|
### `decnet/logging/file_handler.py`
|
||||||
|
|
||||||
|
Module-level singleton `RotatingFileHandler` for `write_syslog` (used by worker-side code that needs to write directly to `decnet.log` without the full logging pipeline, e.g. sniffer/prober).
|
||||||
|
|
||||||
|
- `decnet/logging/file_handler.py::write_syslog` — append one pre-formatted RFC 5424 line to the log.
|
||||||
|
- `decnet/logging/file_handler.py::get_log_path` — return the `Path` of the target file.
|
||||||
|
- `_init_file_handler`, `_get_logger` — internal singleton init.
|
||||||
|
|
||||||
|
### `decnet/logging/inode_aware_handler.py`
|
||||||
|
|
||||||
|
- `decnet/logging/inode_aware_handler.py::InodeAwareRotatingFileHandler` — subclass of `RotatingFileHandler` that before every emit stats the configured path and compares inode+device to the open file; on mismatch it closes and reopens. Solves the "logrotate without copytruncate silently drops lines" failure mode that plain `RotatingFileHandler` has. Matches the pattern already used in `decnet/collector/worker.py::_reopen_if_needed`. Cost: one `os.stat` per log record.
|
||||||
|
|
||||||
|
### `decnet/logging/forwarder.py`
|
||||||
|
|
||||||
|
Shared helpers for the `LOG_TARGET` env var used by service plugins.
|
||||||
|
|
||||||
|
- `decnet/logging/forwarder.py::parse_log_target` — `"ip:port"` → `(host, port)`; raises `ValueError` on bad format.
|
||||||
|
- `decnet/logging/forwarder.py::probe_log_target` — TCP connect with timeout, returns bool. Non-fatal: the CLI uses it to warn before deployment but never to block.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## See also
|
See [Module Reference — Core](Module-Reference-Core) for top-level modules (cli, composer, telemetry, etc.) and [Module Reference — Web](Module-Reference-Web) for the FastAPI surface and DB layer.
|
||||||
|
|
||||||
- [Design overview](Design-Overview)
|
|
||||||
- [REST API](REST-API-Reference)
|
|
||||||
- [Logging](Logging-and-Syslog)
|
|
||||||
- [Services catalog](Services-Catalog)
|
|
||||||
- [Developer guide](Developer-Guide)
|
|
||||||
|
|||||||
Reference in New Issue
Block a user