Compare commits
22 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| a9a86c94ad | |||
| 6b997c5de8 | |||
| 4798a9eb9c | |||
| 65d33bc611 | |||
| e9cc09a50f | |||
| a7256276b0 | |||
| 402c1ef7a2 | |||
| f715ac6bcd | |||
| 082d3fec19 | |||
| 3ed6d5dfc6 | |||
| 2ca6533666 | |||
| bf66e875a5 | |||
| b0bf31a31e | |||
| d7a2b5b9cf | |||
| c918538f35 | |||
| beaa604811 | |||
| 7b0ff127c3 | |||
| 419172ecfb | |||
| fcc9a9aad1 | |||
| 1a765854ec | |||
| a5e11f7d86 | |||
| 74096b6df0 |
4
.gitignore
vendored
4
.gitignore
vendored
@@ -84,3 +84,7 @@ testfail
|
||||
|
||||
# Internal design/dev notes — not for publication
|
||||
/development/
|
||||
decnet.tar
|
||||
|
||||
# cloak base-image build context: decnet subtree synced in at deploy time
|
||||
decnet/templates/_shared/cloak/decnet/
|
||||
|
||||
90
CHANGELOG.md
90
CHANGELOG.md
@@ -5,6 +5,94 @@ All notable changes to DECNET are documented here.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [1.2.1] - 2026-06-20
|
||||
|
||||
OS fingerprint **cloak** — make a decky read as its claimed OS under *active*
|
||||
fingerprinting (`nmap -O`), not just passively. sysctl profiles only reach global
|
||||
packet fields; the cloak owns the SYN-ACK *shape* and stack *behaviours* sysctl
|
||||
can't reach. Verified live against real `nmap -O`: a `windows` decky reads as
|
||||
**Windows 10 (95%)** and a `windows_server` decky as **Windows Server 2012/2016
|
||||
(94%)** — up from a Linux 2.6 classification — with client handshakes intact.
|
||||
|
||||
### Added
|
||||
- `decnet.cloak` — egress TCP/IP masquerading library, run inside the decky base
|
||||
container (`python -m decnet.cloak`, `CAP_NET_ADMIN`/`CAP_NET_RAW`):
|
||||
- **NFQUEUE mangler** — rewrites the egress packet shape sysctl cannot set
|
||||
per-container: SYN-ACK (TCP option order, window, IP-ID; preserves the
|
||||
kernel's live timestamp; recomputes `dataofs`/checksums), RST (IP-ID + a
|
||||
nonzero ack on bare RSTs → nmap `CI`, `T4`/`T6` `A=O`), and ICMP echo-reply
|
||||
(`code=0` → `IE.CD=Z`). One shared IP-ID counter across all three reads as a
|
||||
shared sequence (`SS=S`).
|
||||
- **T2/T3 probe-response synthesizer** — answers the nmap probes Linux drops
|
||||
but Windows replies to (null-flags / SYN+FIN+PSH+URG to an open port).
|
||||
Injects at L2 (reflecting the probe's MACs) so its replies bypass the OUTPUT
|
||||
chain and coexist with the mangler's RST rule.
|
||||
- Profiles live in `os_fingerprint.OS_MANGLE`, keyed by the same `nmap_os`
|
||||
slug; pure packet-shaping logic is unit-tested offline (scapy/netfilterqueue
|
||||
lazy-imported, Linux-only).
|
||||
- `windows_server` nmap_os family — Windows Server stack deltas (ECN negotiated
|
||||
`CC=Y`, randomized IP-ID `TI=RD`); the `windows-server` and `domain-controller`
|
||||
archetypes now use it (workstation stays `windows`).
|
||||
- Cloak base image (`templates/_shared/cloak/Dockerfile`, `FROM` the per-decky
|
||||
distro) and `deployer._sync_cloak_sources`, which ships the light `decnet`
|
||||
subtree into the build context. Base containers stay netns-safe — the cloak runs
|
||||
best-effort behind `exec sleep infinity`, so a cloak crash never tears down the
|
||||
decky or the netns its service containers share.
|
||||
|
||||
### Fixed
|
||||
- **OS fingerprint timestamps bug**: the `windows` sysctl profile disabled TCP
|
||||
timestamps, fingerprinting as an ancient stack. Modern Windows 10/11 run
|
||||
timestamps **on** (`nmap SEQ.TS=A`) — corrected, and the single
|
||||
highest-weighted field in the nmap match.
|
||||
|
||||
## [1.2.0] - 2026-06-18
|
||||
|
||||
Prefork worker consolidation — share the import floor across *separate* processes
|
||||
(own GIL, full isolation) via copy-on-write, for the heavy/isolation-critical
|
||||
workers the in-process supervisor can't co-host.
|
||||
|
||||
### Added
|
||||
- `decnet.prefork` — prefork supervisor primitive: a master imports the base
|
||||
floor once, then forks one child per worker (own process/GIL, CoW-shared
|
||||
floor), reaps and restarts with backoff, and shuts down gracefully. CoW
|
||||
viability measured on CPython 3.14 (idle child ~1 MB private, ~71 MB shared;
|
||||
`gc.freeze()` unnecessary thanks to PEP 683 immortal objects).
|
||||
- `decnet fleet <name>` — prefork master that imports the shared base floor once
|
||||
then forks one child per worker. First fleet `heavy` = profiler + ttp (DB-only,
|
||||
process-isolated heavy tier); systemd unit `decnet-fleet-heavy.service`
|
||||
Conflicts= the units it replaces and carries no extra privilege.
|
||||
Verified live: fleet footprint ≈412 MB Pss (master 67 + profiler 81 + ttp 264)
|
||||
vs 661 MB standalone — profiler's RSS collapsed 353→110 MB (base floor now
|
||||
CoW-shared). ttp barely moved: its bulk is the privately-parsed ATT&CK bundle,
|
||||
which it alone consumes — so master-warming it was confirmed pointless and
|
||||
dropped. Lesson: prefork pays for base-floor-bound workers, not state-bound ones.
|
||||
- **(Pro) Scan-based topology creation** — the MazeNET *New Topology* wizard
|
||||
gains a third option alongside Blank and Seed-based: import an Nmap XML scan
|
||||
and mirror its live hosts and services as decoys. Parses entirely in-browser
|
||||
(native `DOMParser`, no new dependency), resolves discovered service
|
||||
names/ports to DECNET services against the live catalog, groups hosts one LAN
|
||||
per /24, and builds the topology through the existing CRUD APIs (blank → LANs
|
||||
→ deckies → edges) — no new backend. Hosts with no recognizable service
|
||||
(e.g. `nmap -sn`) default to a bare SSH decoy. The XML parser is hardened
|
||||
against XXE/SSRF and entity-expansion DoS, and scan values render as inert
|
||||
text (no XSS). Professional-tier; tree-shaken out of the community build
|
||||
(`decnet/pro` `v1.2.0`).
|
||||
|
||||
### Changed
|
||||
- MITRE ATT&CK Enterprise bundle pinned 19.0 → **19.1**. The bundle and its
|
||||
LICENSE now resolve from `decnet/data/` (hash-pinned in `attack_version.py`,
|
||||
fetched on demand via `python -m decnet.ttp.attack_stix fetch`, gitignored —
|
||||
not committed).
|
||||
|
||||
### Removed
|
||||
- Per-worker systemd unit templates superseded by consolidation:
|
||||
`decnet-{reconciler,enrich,orchestrator,mutator}` (→ `supervise batch`),
|
||||
`decnet-{clusterer,campaign-clusterer,attribution,reuse-correlator}`
|
||||
(→ `supervise cpu`), and `decnet-{profiler,ttp}` (→ `fleet heavy`).
|
||||
`decnet.target` now pulls in the 3 consolidated units. The underlying CLI
|
||||
commands remain for manual/standalone runs; a worker can be re-extracted to its
|
||||
own unit by editing the group/fleet spec.
|
||||
|
||||
## [1.1.1] - 2026-06-18
|
||||
|
||||
### Fixed
|
||||
@@ -66,5 +154,7 @@ own unit.
|
||||
|
||||
Initial 1.0 release. See tag `v1.0.0`.
|
||||
|
||||
[1.2.0]: https://git.resacachile.cl/anti/DECNET/compare/v1.1.1...v1.2.0
|
||||
[1.1.1]: https://git.resacachile.cl/anti/DECNET/compare/v1.1.0...v1.1.1
|
||||
[1.1.0]: https://git.resacachile.cl/anti/DECNET/compare/v1.0.0...v1.1.0
|
||||
[1.0.0]: https://git.resacachile.cl/anti/DECNET/releases/tag/v1.0.0
|
||||
|
||||
BIN
decnet.tar
BIN
decnet.tar
Binary file not shown.
@@ -47,7 +47,7 @@ ARCHETYPES: dict[str, Archetype] = {
|
||||
description="Windows domain member: SMB, RDP, and LDAP directory",
|
||||
services=["smb", "rdp", "ldap"],
|
||||
preferred_distros=["debian", "ubuntu22"],
|
||||
nmap_os="windows",
|
||||
nmap_os="windows_server",
|
||||
),
|
||||
"domain-controller": Archetype(
|
||||
slug="domain-controller",
|
||||
@@ -55,7 +55,7 @@ ARCHETYPES: dict[str, Archetype] = {
|
||||
description="Active Directory DC: LDAP, SMB, RDP, LLMNR",
|
||||
services=["ldap", "smb", "rdp", "llmnr"],
|
||||
preferred_distros=["debian", "ubuntu22"],
|
||||
nmap_os="windows",
|
||||
nmap_os="windows_server",
|
||||
),
|
||||
"linux-server": Archetype(
|
||||
slug="linux-server",
|
||||
|
||||
@@ -20,6 +20,7 @@ import os
|
||||
from typing import Any, cast
|
||||
|
||||
from decnet.bus.base import BaseBus
|
||||
from decnet.paths import resolve_runtime_path
|
||||
|
||||
|
||||
def get_bus(**kwargs: Any) -> BaseBus:
|
||||
@@ -58,14 +59,12 @@ def _default_socket_path() -> str:
|
||||
``RuntimeDirectory=decnet`` sets it up with the right perms; the home
|
||||
fallback keeps dev boxes usable without systemd.
|
||||
"""
|
||||
explicit = os.environ.get("DECNET_BUS_SOCKET")
|
||||
if explicit:
|
||||
return explicit
|
||||
|
||||
runtime_dir = "/run/decnet"
|
||||
if os.path.isdir(runtime_dir) and os.access(runtime_dir, os.W_OK):
|
||||
return f"{runtime_dir}/bus.sock"
|
||||
return os.path.expanduser("~/.decnet/bus.sock")
|
||||
return resolve_runtime_path(
|
||||
"bus.sock",
|
||||
env_var="DECNET_BUS_SOCKET",
|
||||
runtime_dir="/run/decnet",
|
||||
user_fallback="~/.decnet/bus.sock",
|
||||
)
|
||||
|
||||
|
||||
def _maybe_wrap_telemetry(bus: BaseBus) -> BaseBus:
|
||||
|
||||
@@ -26,6 +26,7 @@ from . import (
|
||||
canary,
|
||||
db,
|
||||
deploy,
|
||||
fleet,
|
||||
forwarder,
|
||||
geoip,
|
||||
init,
|
||||
@@ -62,7 +63,7 @@ for _mod in (
|
||||
swarm,
|
||||
deploy, lifecycle, workers, inventory,
|
||||
web, profiler, orchestrator, realism, reconciler, sniffer, db,
|
||||
topology, bus, geoip, init, webhook, canary, ttp, supervise,
|
||||
topology, bus, geoip, init, webhook, canary, ttp, supervise, fleet,
|
||||
):
|
||||
_mod.register(app)
|
||||
|
||||
|
||||
94
decnet/cli/fleet.py
Normal file
94
decnet/cli/fleet.py
Normal file
@@ -0,0 +1,94 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""``decnet fleet <name>`` — prefork supervisor (DECNET 1.2).
|
||||
|
||||
Imports the shared base floor ONCE in the master, then forks one child process
|
||||
per worker (see :mod:`decnet.prefork`). Children share the floor via copy-on-write
|
||||
(measured ~71 MB shared / ~1 MB private per idle child on CPython 3.14) while
|
||||
keeping their OWN process and GIL — unlike ``decnet supervise``, which co-hosts
|
||||
workers as asyncio tasks in one shared-GIL process.
|
||||
|
||||
Use ``fleet`` for workers that must stay process-isolated (heavy resident state,
|
||||
sustained CPU) but shouldn't each re-import the world; use ``supervise`` for cheap
|
||||
co-resident IO workers.
|
||||
|
||||
CONSOLIDATION COSTS (same shape as ``supervise``):
|
||||
* Forked children inherit the master's privileges — a fleet's systemd unit
|
||||
carries the UNION of its members' caps. So group by privilege profile, not
|
||||
convenience. The ``heavy`` fleet is DB-only (no docker socket, no raw net).
|
||||
* To share via CoW the master pre-imports each worker's module BEFORE forking,
|
||||
so its RSS is large — but that RSS is the shared floor, not per-child cost.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import typer
|
||||
|
||||
from . import utils as _utils
|
||||
from .utils import console, log
|
||||
|
||||
_FLEETS = ("heavy",)
|
||||
|
||||
|
||||
def _build_fleet(name: str) -> dict:
|
||||
"""Return ``{worker_name: entry_thunk}`` for *name*.
|
||||
|
||||
Imports happen here, in the MASTER, before :func:`run_fleet` forks — that is
|
||||
what lets children share the imported code/objects via copy-on-write. Each
|
||||
thunk blocks running one worker; ``repo`` is initialized inside the child
|
||||
(post-fork) so every child opens its own pool, never a fork-inherited one.
|
||||
"""
|
||||
import asyncio
|
||||
|
||||
if name == "heavy":
|
||||
from decnet.profiler import attacker_profile_worker
|
||||
from decnet.ttp.worker import run_ttp_worker_loop
|
||||
from decnet.web.dependencies import repo
|
||||
|
||||
# Importing the worker modules here (in the master) is what lets children
|
||||
# share their code via CoW. Heavy per-worker runtime state (ATT&CK bundle,
|
||||
# ML) still loads lazily in each child — warming it in the master to share
|
||||
# it too is a future optimization, gated on a live RSS measurement showing
|
||||
# the big object graph actually CoW-shares rather than refcount-dirtying.
|
||||
def _profiler() -> None:
|
||||
async def _go() -> None:
|
||||
await repo.initialize()
|
||||
await attacker_profile_worker(repo, interval=60)
|
||||
asyncio.run(_go())
|
||||
|
||||
def _ttp() -> None:
|
||||
async def _go() -> None:
|
||||
await repo.initialize()
|
||||
await run_ttp_worker_loop(repo, poll_interval_secs=60.0)
|
||||
asyncio.run(_go())
|
||||
|
||||
return {"profiler": _profiler, "ttp": _ttp}
|
||||
|
||||
raise ValueError(f"unknown fleet: {name}")
|
||||
|
||||
|
||||
def register(app: typer.Typer) -> None:
|
||||
@app.command(name="fleet")
|
||||
def fleet_cmd(
|
||||
name: str = typer.Argument(
|
||||
..., help=f"Worker fleet to fork. One of: {', '.join(_FLEETS)}"
|
||||
),
|
||||
daemon: bool = typer.Option(
|
||||
False, "--daemon", "-d", help="Detach to background as a daemon process"
|
||||
),
|
||||
) -> None:
|
||||
"""Prefork a worker fleet: shared base floor (CoW), one child process per worker."""
|
||||
from decnet.prefork import run_fleet
|
||||
|
||||
if name not in _FLEETS:
|
||||
console.print(
|
||||
f"[red]unknown fleet {name!r}; known fleets: {', '.join(_FLEETS)}[/]"
|
||||
)
|
||||
raise typer.Exit(2)
|
||||
|
||||
if daemon:
|
||||
log.info("fleet %s daemonizing", name)
|
||||
_utils._daemonize()
|
||||
|
||||
log.info("fleet %s starting", name)
|
||||
console.print(f"[bold cyan]Fleet starting[/] {name} (prefork)")
|
||||
specs = _build_fleet(name)
|
||||
run_fleet(specs)
|
||||
30
decnet/cloak/__init__.py
Normal file
30
decnet/cloak/__init__.py
Normal file
@@ -0,0 +1,30 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
DECNET cloak — egress TCP/IP fingerprint masquerading for deckies.
|
||||
|
||||
sysctls (decnet/os_fingerprint.py) own GLOBAL packet fields. The cloak owns the
|
||||
SYN-ACK *shape* and stack *behaviours* sysctl can't reach, so a decky reads as
|
||||
its claimed nmap_os under active fingerprinting (nmap -O):
|
||||
|
||||
- mangler : NFQUEUE rewrite of egress SYN-ACK (window, TCP option order,
|
||||
IP-ID generation) to match the MangleProfile for the slug.
|
||||
- responder : raw-socket synthesis of replies to probes the Linux kernel
|
||||
drops but the target OS answers (nmap T2/T3).
|
||||
|
||||
Both run INSIDE the decky's network namespace (CAP_NET_ADMIN), launched by the
|
||||
base container — never a sidecar (that would double container count per decky).
|
||||
Driven by os_fingerprint.get_os_mangle(nmap_os); a slug with no profile is a
|
||||
no-op (the real Linux stack already approximates it).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from decnet.cloak.mangler import build_synack_options, next_ipid
|
||||
from decnet.cloak.responder import ProbeKind, build_reply_fields, classify_probe
|
||||
|
||||
__all__ = [
|
||||
"build_synack_options",
|
||||
"next_ipid",
|
||||
"classify_probe",
|
||||
"build_reply_fields",
|
||||
"ProbeKind",
|
||||
]
|
||||
51
decnet/cloak/__main__.py
Normal file
51
decnet/cloak/__main__.py
Normal file
@@ -0,0 +1,51 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Cloak entrypoint — run inside the decky base container (CAP_NET_ADMIN).
|
||||
|
||||
python -m decnet.cloak
|
||||
|
||||
Config via env (set by the composer when nmap_os has a mangle profile):
|
||||
DECNET_NMAP_OS nmap_os slug (e.g. "windows", "windows_server")
|
||||
DECNET_OPEN_PORTS comma-separated TCP ports the decky serves (for T2/T3)
|
||||
DECKY_IP this decky's IP (BPF scope for the responder)
|
||||
|
||||
Starts the mangler and responder, each in its own thread. A slug with no mangle
|
||||
profile exits 0 immediately — harmless to launch unconditionally.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import threading
|
||||
|
||||
from decnet.cloak import mangler, responder
|
||||
from decnet.logging import get_logger
|
||||
from decnet.os_fingerprint import get_os_mangle
|
||||
|
||||
log = get_logger("cloak")
|
||||
|
||||
|
||||
def _open_ports() -> frozenset[int]:
|
||||
raw = os.environ.get("DECNET_OPEN_PORTS", "")
|
||||
return frozenset(int(p) for p in raw.split(",") if p.strip().isdigit())
|
||||
|
||||
|
||||
def main() -> int:
|
||||
nmap_os = os.environ.get("DECNET_NMAP_OS", "linux")
|
||||
if get_os_mangle(nmap_os) is None:
|
||||
log.info("cloak: no mangle profile for %r — exiting", nmap_os)
|
||||
return 0
|
||||
|
||||
# Responder runs in a daemon thread; the mangler runs in the MAIN thread so
|
||||
# its SIGTERM/SIGINT iptables-teardown handlers can be installed (signal only
|
||||
# works in the main thread).
|
||||
threading.Thread(
|
||||
target=responder.run, args=(nmap_os, _open_ports()),
|
||||
name="cloak-responder", daemon=True,
|
||||
).start()
|
||||
log.info("cloak: started for nmap_os=%r", nmap_os)
|
||||
mangler.run(nmap_os)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
180
decnet/cloak/mangler.py
Normal file
180
decnet/cloak/mangler.py
Normal file
@@ -0,0 +1,180 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Egress mangler — rewrites the TCP/IP shape & behaviours sysctl can't reach.
|
||||
|
||||
Touches only the fingerprint-relevant egress packets:
|
||||
- SYN-ACK : window, TCP option order, IP-ID (nmap OPS/WIN/TI)
|
||||
- RST : IP-ID + a nonzero ack on bare RSTs (nmap CI, T4/T6 A=O)
|
||||
- ICMP echo-reply : code=0 + IP-ID (nmap IE.CD, II)
|
||||
A single shared IP-ID counter across all three reads as a shared sequence (SS=S).
|
||||
|
||||
Split so the packet-shaping logic is pure and unit-testable without scapy, root,
|
||||
or a live NFQUEUE:
|
||||
|
||||
- build_synack_options() / next_ipid() / _rst_needs_ack() : pure, tested offline.
|
||||
- _rewrite() : mutates a scapy packet (lazy import).
|
||||
- run() : the NFQUEUE loop (needs CAP_NET_ADMIN).
|
||||
|
||||
scapy/netfilterqueue are imported lazily inside the runtime functions, mirroring
|
||||
decnet/prober/tcpfp.py, so importing this module is cheap and side-effect-free.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import signal
|
||||
import subprocess # nosec B404 — fixed-arg iptables, no shell
|
||||
import sys
|
||||
import threading
|
||||
from typing import Any
|
||||
|
||||
from decnet.logging import get_logger
|
||||
from decnet.os_fingerprint import MangleProfile, get_os_mangle
|
||||
|
||||
log = get_logger("cloak.mangler")
|
||||
|
||||
_QUEUE = 0
|
||||
# Only the fingerprint-relevant egress packets are queued (never bulk data):
|
||||
# SYN-bearing → SYN-ACK (OPS/WIN/options/TI)
|
||||
# RST-bearing → T4-T7 RST shape (CI IP-ID, T4/T6 ack)
|
||||
# ICMP echo-reply → IE.CD code + II IP-ID
|
||||
# --queue-bypass: a dead handler never blackholes the decky.
|
||||
def _nfq_rule(match: list[str]) -> list[str]:
|
||||
return ["OUTPUT", *match, "-j", "NFQUEUE", "--queue-num", str(_QUEUE), "--queue-bypass"]
|
||||
|
||||
|
||||
_RULES = [
|
||||
_nfq_rule(["-p", "tcp", "--tcp-flags", "SYN", "SYN"]),
|
||||
_nfq_rule(["-p", "tcp", "--tcp-flags", "RST", "RST"]),
|
||||
_nfq_rule(["-p", "icmp", "--icmp-type", "echo-reply"]),
|
||||
]
|
||||
|
||||
|
||||
def _rst_needs_ack(flags: int) -> bool:
|
||||
"""A bare RST (RST set, ACK clear) — the T4/T6 case. Windows fills a nonzero
|
||||
ack (nmap A=O); Linux leaves it 0 (A=Z). R+ACK RSTs (T5/T7) already match."""
|
||||
return bool(flags & 0x04) and not (flags & 0x10)
|
||||
|
||||
|
||||
def next_ipid(prev: int, mode: str) -> int:
|
||||
"""Next IP-ID for *mode*: 'incr' (TI=I), 'random' (TI=RD), 'keep' (unchanged).
|
||||
|
||||
'keep' returns -1 as a sentinel meaning "do not touch the kernel's value".
|
||||
"""
|
||||
if mode == "incr":
|
||||
return (prev + 1) & 0xFFFF
|
||||
if mode == "random":
|
||||
# Not for security — only to read as randomized to nmap (TI=RD).
|
||||
return int.from_bytes(os.urandom(2), "big") or 1
|
||||
return -1
|
||||
|
||||
|
||||
def build_synack_options(
|
||||
orig_options: list[tuple[str, Any]], profile: MangleProfile
|
||||
) -> list[tuple[str, Any]]:
|
||||
"""Build the SYN-ACK TCP option list for *profile*, preserving the kernel's
|
||||
live Timestamp value (so nmap's SEQ.TS increment-rate test still passes).
|
||||
|
||||
*orig_options* is a scapy-style ``[(name, value), ...]`` list.
|
||||
"""
|
||||
ts = next((v for n, v in orig_options if n == "Timestamp"), None)
|
||||
out: list[tuple[str, Any]] = []
|
||||
for code in profile.option_order:
|
||||
if code == "MSS":
|
||||
out.append(("MSS", profile.mss))
|
||||
elif code == "WScale":
|
||||
out.append(("WScale", profile.wscale))
|
||||
elif code == "SAckOK":
|
||||
out.append(("SAckOK", b""))
|
||||
elif code == "NOP":
|
||||
out.append(("NOP", None))
|
||||
elif code == "TS":
|
||||
if ts is not None: # only if sysctl kept timestamps on
|
||||
out.append(("Timestamp", ts))
|
||||
return out
|
||||
|
||||
|
||||
def _is_synack(flags: int) -> bool:
|
||||
return bool(flags & 0x02) and bool(flags & 0x10) # SYN & ACK
|
||||
|
||||
|
||||
def _iptables(action: str) -> None:
|
||||
for rule in _RULES:
|
||||
subprocess.run(["iptables", action, *rule], check=True) # nosec B603 B607
|
||||
|
||||
|
||||
def run(nmap_os: str) -> int:
|
||||
"""Install the NFQUEUE rules and rewrite egress SYN-ACK / RST / ICMP for *nmap_os*."""
|
||||
profile = get_os_mangle(nmap_os)
|
||||
if profile is None:
|
||||
log.info("cloak.mangler: no profile for %r — nothing to do", nmap_os)
|
||||
return 0
|
||||
|
||||
from netfilterqueue import NetfilterQueue # type: ignore
|
||||
from scapy.all import ICMP, IP, TCP # type: ignore
|
||||
|
||||
# ONE shared IP-ID counter across SYN-ACK / RST / ICMP — keeps TCP and ICMP
|
||||
# IDs close, which is what nmap reads as a shared sequence (SS=S, Windows).
|
||||
ipid = [0x0400]
|
||||
|
||||
def _bump_ipid(p: Any) -> None:
|
||||
nid = next_ipid(ipid[0], profile.ipid)
|
||||
if nid >= 0:
|
||||
ipid[0] = nid
|
||||
p[IP].id = nid
|
||||
|
||||
def _rewrite(pkt: Any) -> None:
|
||||
try:
|
||||
p = IP(pkt.get_payload())
|
||||
touched = False
|
||||
tcp_synack = False
|
||||
if p.haslayer(TCP):
|
||||
f = int(p[TCP].flags)
|
||||
if _is_synack(f):
|
||||
p[TCP].window = profile.window
|
||||
p[TCP].options = build_synack_options(p[TCP].options, profile)
|
||||
_bump_ipid(p)
|
||||
touched = tcp_synack = True
|
||||
elif f & 0x04: # RST (T4-T7)
|
||||
_bump_ipid(p)
|
||||
if _rst_needs_ack(f):
|
||||
p[TCP].ack = (int(p[TCP].seq) + 1) & 0xFFFFFFFF # A=O
|
||||
touched = True
|
||||
if touched:
|
||||
del p[TCP].chksum
|
||||
if tcp_synack: # options length changed → recompute offset
|
||||
del p[TCP].dataofs
|
||||
elif p.haslayer(ICMP) and int(p[ICMP].type) == 0: # echo-reply
|
||||
p[ICMP].code = 0 # IE.CD=Z (Windows); Linux echoes the code
|
||||
_bump_ipid(p)
|
||||
del p[ICMP].chksum
|
||||
touched = True
|
||||
if touched:
|
||||
del p[IP].chksum, p[IP].len
|
||||
pkt.set_payload(bytes(p))
|
||||
except Exception: # nosec B110 — never drop a packet on a rewrite bug
|
||||
log.exception("cloak.mangler: rewrite failed; passing packet through")
|
||||
pkt.accept()
|
||||
|
||||
_iptables("-A")
|
||||
nfq = NetfilterQueue()
|
||||
nfq.bind(_QUEUE, _rewrite)
|
||||
|
||||
def _cleanup(*_: Any) -> None:
|
||||
try:
|
||||
_iptables("-D")
|
||||
finally:
|
||||
sys.exit(0)
|
||||
|
||||
# signal.signal() only works in the main thread; the `finally` below still
|
||||
# removes the rule on a normal exit, and on container stop the netns (and
|
||||
# its iptables rules) are torn down regardless.
|
||||
if threading.current_thread() is threading.main_thread():
|
||||
signal.signal(signal.SIGTERM, _cleanup)
|
||||
signal.signal(signal.SIGINT, _cleanup)
|
||||
log.info("cloak.mangler: rewriting SYN-ACK/RST/ICMP -> %s (window=%#x ipid=%s)",
|
||||
nmap_os, profile.window, profile.ipid)
|
||||
try:
|
||||
nfq.run()
|
||||
finally:
|
||||
_iptables("-D")
|
||||
return 0
|
||||
92
decnet/cloak/responder.py
Normal file
92
decnet/cloak/responder.py
Normal file
@@ -0,0 +1,92 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Probe-response synthesizer — answers the nmap probes the Linux kernel drops.
|
||||
|
||||
nmap's T2 (null-flags) and T3 (SYN+FIN+PSH+URG) to an OPEN port get no reply
|
||||
from Linux (R=N), but Windows replies RST+ACK. We sniff the probe and inject the
|
||||
target-OS-shaped reply ourselves; the kernel stays silent, so nothing races us.
|
||||
|
||||
Pure classification/reply logic is separated from the scapy sniff/send loop so it
|
||||
is unit-testable without root or a live capture.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import enum
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
from decnet.logging import get_logger
|
||||
from decnet.os_fingerprint import get_os_mangle
|
||||
|
||||
log = get_logger("cloak.responder")
|
||||
|
||||
_NULL = 0x00
|
||||
_T3 = 0x2B # SYN+FIN+PSH+URG
|
||||
|
||||
|
||||
class ProbeKind(enum.Enum):
|
||||
T2 = "T2"
|
||||
T3 = "T3"
|
||||
|
||||
|
||||
def classify_probe(flags: int, dport: int, open_ports: frozenset[int]) -> ProbeKind | None:
|
||||
"""Identify an nmap T2/T3 probe by flag combo + open destination port.
|
||||
|
||||
Returns None for anything else (legit traffic, probes to closed ports, and
|
||||
T1/T4-T7 which the real stack already answers).
|
||||
"""
|
||||
if dport not in open_ports:
|
||||
return None
|
||||
if flags == _NULL:
|
||||
return ProbeKind.T2
|
||||
if flags == _T3:
|
||||
return ProbeKind.T3
|
||||
return None
|
||||
|
||||
|
||||
def build_reply_fields(probe_seq: int, kind: ProbeKind) -> dict[str, Any]:
|
||||
"""Windows T2/T3 reply fields: seq 0, RST+ACK, window 0, DF=1.
|
||||
|
||||
ack differs by probe (nmap): T2 A=S (ack == probe seq); T3 A=O (other — we
|
||||
use probe seq + 1 so it reads as 'other', never zero or the probe seq).
|
||||
"""
|
||||
ack = probe_seq if kind is ProbeKind.T2 else (probe_seq + 1) & 0xFFFFFFFF
|
||||
return {"seq": 0, "ack": ack, "flags": "RA", "window": 0, "df": True}
|
||||
|
||||
|
||||
def run(nmap_os: str, open_ports: frozenset[int], decky_ip: str | None = None) -> int:
|
||||
"""Sniff for T2/T3 probes to *open_ports* and inject Windows-shaped replies."""
|
||||
profile = get_os_mangle(nmap_os)
|
||||
if profile is None or not profile.respond_t2t3:
|
||||
log.info("cloak.responder: nothing to do for %r", nmap_os)
|
||||
return 0
|
||||
|
||||
from scapy.all import IP, TCP, Ether, sendp, sniff # type: ignore
|
||||
|
||||
ip = decky_ip or os.environ.get("DECKY_IP", "")
|
||||
ipid = [0x0800]
|
||||
|
||||
def _on(pkt: Any) -> None:
|
||||
if not pkt.haslayer(TCP) or not pkt.haslayer(Ether):
|
||||
return
|
||||
kind = classify_probe(int(pkt[TCP].flags), int(pkt[TCP].dport), open_ports)
|
||||
if kind is None:
|
||||
return
|
||||
f = build_reply_fields(int(pkt[TCP].seq), kind)
|
||||
ipid[0] = (ipid[0] + 1) & 0xFFFF
|
||||
# Inject at L2 (reflecting the probe's MACs) so the reply BYPASSES the
|
||||
# OUTPUT chain — otherwise the mangler's RST rule would re-process and
|
||||
# drop our own RST. The reply is already in final Windows shape.
|
||||
reply = (
|
||||
Ether(src=pkt[Ether].dst, dst=pkt[Ether].src)
|
||||
/ IP(src=pkt[IP].dst, dst=pkt[IP].src, id=ipid[0], flags="DF", ttl=128)
|
||||
/ TCP(sport=int(pkt[TCP].dport), dport=int(pkt[TCP].sport),
|
||||
seq=f["seq"], ack=f["ack"], flags=f["flags"], window=f["window"])
|
||||
)
|
||||
sendp(reply, iface=pkt.sniffed_on, verbose=0)
|
||||
|
||||
bpf = f"tcp and dst host {ip}" if ip else "tcp"
|
||||
log.info("cloak.responder: answering T2/T3 on %d ports (filter=%r)",
|
||||
len(open_ports), bpf)
|
||||
sniff(filter=bpf, prn=_on, store=0)
|
||||
return 0
|
||||
@@ -21,7 +21,7 @@ import yaml
|
||||
|
||||
from decnet.config import DecnetConfig
|
||||
from decnet.network import MACVLAN_NETWORK_NAME
|
||||
from decnet.os_fingerprint import get_os_sysctls
|
||||
from decnet.os_fingerprint import get_os_mangle, get_os_sysctls
|
||||
from decnet.services.registry import get_service
|
||||
|
||||
_DOCKER_LOGGING = {
|
||||
@@ -32,6 +32,26 @@ _DOCKER_LOGGING = {
|
||||
},
|
||||
}
|
||||
|
||||
# Build context for the cloak base image (decnet subtree synced in by
|
||||
# deployer._sync_cloak_sources before build).
|
||||
_CLOAK_CONTEXT = Path(__file__).parent / "templates" / "_shared" / "cloak"
|
||||
|
||||
# Netns-safe: run the cloak best-effort in the background, but keep `sleep
|
||||
# infinity` as PID 1 in the foreground so a cloak crash never tears down the
|
||||
# base container (and with it the netns every service container shares).
|
||||
_CLOAK_COMMAND = ["sh", "-c", "python3 -m decnet.cloak & exec sleep infinity"]
|
||||
|
||||
|
||||
def _decky_open_tcp_ports(services: list[str]) -> list[int]:
|
||||
"""Sorted, de-duped TCP ports a decky's services listen on (for the cloak
|
||||
responder's T2/T3 classification — DECNET_OPEN_PORTS)."""
|
||||
ports: set[int] = set()
|
||||
for svc_name in services:
|
||||
svc = get_service(svc_name)
|
||||
if svc is not None:
|
||||
ports.update(svc.ports)
|
||||
return sorted(ports)
|
||||
|
||||
|
||||
def generate_compose(config: DecnetConfig) -> dict:
|
||||
"""Build and return the full docker-compose data structure."""
|
||||
@@ -60,6 +80,25 @@ def generate_compose(config: DecnetConfig) -> dict:
|
||||
base["sysctls"] = get_os_sysctls(decky.nmap_os)
|
||||
base["cap_add"] = ["NET_ADMIN"]
|
||||
|
||||
# sysctls reach only global packet fields. nmap_os families with an
|
||||
# egress mangle profile (windows*) additionally run the cloak in the
|
||||
# base container to rewrite SYN-ACK shape + synthesize T2/T3 replies, so
|
||||
# they read as the claimed OS under active fingerprinting (nmap -O).
|
||||
if get_os_mangle(decky.nmap_os) is not None:
|
||||
base.pop("image", None)
|
||||
base["build"] = {
|
||||
"context": str(_CLOAK_CONTEXT),
|
||||
"args": {"BASE_IMAGE": decky.build_base},
|
||||
}
|
||||
base["command"] = _CLOAK_COMMAND
|
||||
base["cap_add"] = ["NET_ADMIN", "NET_RAW"] # NET_RAW: responder send/sniff
|
||||
ports = _decky_open_tcp_ports(decky.services)
|
||||
base["environment"] = {
|
||||
"DECNET_NMAP_OS": decky.nmap_os,
|
||||
"DECNET_OPEN_PORTS": ",".join(str(p) for p in ports),
|
||||
"DECKY_IP": decky.ip,
|
||||
}
|
||||
|
||||
services[base_key] = base
|
||||
|
||||
# --- Service containers: share base network namespace ---
|
||||
|
||||
7
decnet/data/.gitignore
vendored
Normal file
7
decnet/data/.gitignore
vendored
Normal file
@@ -0,0 +1,7 @@
|
||||
# MITRE ATT&CK STIX bundle + license live here but are NOT committed:
|
||||
# ~51MB and MITRE-licensed (fetched on demand, hash-pinned in attack_version.py).
|
||||
#
|
||||
# Populate locally / in CI with:
|
||||
# DECNET_ATTACK_CACHE_DIR=decnet/data python -m decnet.ttp.attack_stix fetch
|
||||
/enterprise-attack-*.json
|
||||
/LICENSE.txt
|
||||
@@ -5,8 +5,10 @@ Deploy, teardown, and status via Docker SDK + subprocess docker compose.
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess # nosec B404
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
@@ -63,6 +65,20 @@ _CANONICAL_NTLMSSP = Path(__file__).parent.parent / "templates" / "_shared" / "n
|
||||
_NTLMSSP_SERVICES = {"smb", "rdp"}
|
||||
_CANONICAL_CADDY_MODULES_DIR = Path(__file__).parent.parent / "templates" / "_caddy_modules"
|
||||
_CADDY_SERVICES = {"http", "https"}
|
||||
# Cloak base image: the decnet package root + the 8 light files shipped into the
|
||||
# cloak build context so `python -m decnet.cloak` runs in the base container.
|
||||
_DECNET_SRC = Path(__file__).parent.parent
|
||||
_CANONICAL_CLOAK_DIR = _DECNET_SRC / "templates" / "_shared" / "cloak"
|
||||
_CLOAK_SHIP_FILES = (
|
||||
"__init__.py",
|
||||
"config_ini.py",
|
||||
"logging/__init__.py",
|
||||
"os_fingerprint.py",
|
||||
"cloak/__init__.py",
|
||||
"cloak/__main__.py",
|
||||
"cloak/mangler.py",
|
||||
"cloak/responder.py",
|
||||
)
|
||||
|
||||
|
||||
def _sync_logging_helper(config: DecnetConfig) -> None:
|
||||
@@ -85,6 +101,26 @@ def _sync_logging_helper(config: DecnetConfig) -> None:
|
||||
shutil.copy2(src, dest)
|
||||
|
||||
|
||||
def _sync_cloak_sources(config: DecnetConfig) -> None:
|
||||
"""Ship the light decnet subtree into the cloak base-image build context.
|
||||
|
||||
Only when at least one decky has an egress mangle profile (windows*). Copies
|
||||
the 8 files in _CLOAK_SHIP_FILES into <cloak ctx>/decnet/ preserving package
|
||||
structure so the image's `python -m decnet.cloak` resolves. The dest tree is
|
||||
gitignored. Mirrors the _sync_*_sources copy-if-changed idiom.
|
||||
"""
|
||||
from decnet.os_fingerprint import get_os_mangle
|
||||
if not any(get_os_mangle(d.nmap_os) is not None for d in config.deckies):
|
||||
return
|
||||
dest_root = _CANONICAL_CLOAK_DIR / "decnet"
|
||||
for rel in _CLOAK_SHIP_FILES:
|
||||
src = _DECNET_SRC / rel
|
||||
dest = dest_root / rel
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not dest.exists() or dest.read_bytes() != src.read_bytes():
|
||||
shutil.copy2(src, dest)
|
||||
|
||||
|
||||
def _sync_auth_helper_sources(config: DecnetConfig) -> None:
|
||||
"""Copy auth-helper.c into SSH/Telnet build contexts as auth-helper/.
|
||||
|
||||
@@ -677,6 +713,7 @@ def deploy(config: DecnetConfig, dry_run: bool = False, no_cache: bool = False,
|
||||
_sync_auth_helper_sources(config)
|
||||
_sync_ntlmssp_sources(config)
|
||||
_sync_caddy_modules(config)
|
||||
_sync_cloak_sources(config)
|
||||
|
||||
compose_path = write_compose(config, COMPOSE_FILE)
|
||||
console.print(f"[bold cyan]Compose file written[/] → {compose_path}")
|
||||
@@ -833,7 +870,16 @@ def _teardown_order(lans: list[dict]) -> list[str]:
|
||||
|
||||
|
||||
def _topology_compose_path(topology_id: str) -> Path:
|
||||
return Path(f"decnet-topology-{topology_id[:8]}-compose.yml")
|
||||
# Anchor to a stable absolute dir so write and teardown agree
|
||||
# regardless of process CWD — a CWD-relative path littered the
|
||||
# install tree and let teardown's unlink() miss orphaned files.
|
||||
base = Path(os.environ.get("DECNET_RUN_DIR", "/var/lib/decnet/topologies"))
|
||||
try:
|
||||
base.mkdir(parents=True, exist_ok=True)
|
||||
except OSError:
|
||||
base = Path(tempfile.gettempdir()) / "decnet-topologies"
|
||||
base.mkdir(parents=True, exist_ok=True)
|
||||
return base / f"decnet-topology-{topology_id[:8]}-compose.yml"
|
||||
|
||||
|
||||
def _topology_compose_project(topology_id: str) -> str:
|
||||
|
||||
@@ -35,6 +35,8 @@ Windows (64240) from the kernel's default tcp_rmem settings.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
OS_SYSCTLS: dict[str, dict[str, str]] = {
|
||||
"linux": {
|
||||
"net.ipv4.ip_default_ttl": "64",
|
||||
@@ -49,9 +51,12 @@ OS_SYSCTLS: dict[str, dict[str, str]] = {
|
||||
"net.ipv4.icmp_ratemask": "6168",
|
||||
},
|
||||
"windows": {
|
||||
# Windows 10/11 workstation. NOTE: modern Windows runs TCP timestamps
|
||||
# ON (nmap SEQ.TS=A) — an earlier value of 0 here fingerprinted as an
|
||||
# ancient Windows/Linux stack. ECN off → nmap ECN.CC=N (workstation).
|
||||
"net.ipv4.ip_default_ttl": "128",
|
||||
"net.ipv4.tcp_syn_retries": "2",
|
||||
"net.ipv4.tcp_timestamps": "0",
|
||||
"net.ipv4.tcp_timestamps": "1",
|
||||
"net.ipv4.tcp_window_scaling": "1",
|
||||
"net.ipv4.tcp_sack": "1",
|
||||
"net.ipv4.tcp_ecn": "0",
|
||||
@@ -60,6 +65,22 @@ OS_SYSCTLS: dict[str, dict[str, str]] = {
|
||||
"net.ipv4.icmp_ratelimit": "0",
|
||||
"net.ipv4.icmp_ratemask": "0",
|
||||
},
|
||||
"windows_server": {
|
||||
# Windows Server 2016/2019. Same NT stack as the workstation; the only
|
||||
# stack-visible deltas nmap reads are ECN negotiated (CC=Y → tcp_ecn=1)
|
||||
# and randomized IP-ID (SEQ.TI=RD, applied by the cloak mangler, not a
|
||||
# sysctl). Everything else == "windows".
|
||||
"net.ipv4.ip_default_ttl": "128",
|
||||
"net.ipv4.tcp_syn_retries": "2",
|
||||
"net.ipv4.tcp_timestamps": "1",
|
||||
"net.ipv4.tcp_window_scaling": "1",
|
||||
"net.ipv4.tcp_sack": "1",
|
||||
"net.ipv4.tcp_ecn": "1",
|
||||
"net.ipv4.ip_no_pmtu_disc": "0",
|
||||
"net.ipv4.tcp_fin_timeout": "30",
|
||||
"net.ipv4.icmp_ratelimit": "0",
|
||||
"net.ipv4.icmp_ratemask": "0",
|
||||
},
|
||||
"bsd": {
|
||||
"net.ipv4.ip_default_ttl": "64",
|
||||
"net.ipv4.tcp_syn_retries": "6",
|
||||
@@ -112,3 +133,47 @@ def all_os_families() -> list[str]:
|
||||
"""Return all registered nmap OS family slugs."""
|
||||
return list(OS_SYSCTLS.keys())
|
||||
|
||||
|
||||
# ─── Egress mangle profiles (cloak) ──────────────────────────────────────────
|
||||
#
|
||||
# sysctls above reach only GLOBAL fields (TTL, timestamps on/off, ECN). The
|
||||
# SYN-ACK *shape* nmap also scores — exact window, TCP option order, IP-ID
|
||||
# generation — cannot be set per-container by sysctl. The cloak mangler
|
||||
# (decnet/cloak) rewrites those on egress, driven by these profiles, keyed by
|
||||
# the SAME nmap_os slug. A slug ABSENT here needs no mangling (its real Linux
|
||||
# stack already approximates the target, e.g. "linux"/"bsd").
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MangleProfile:
|
||||
"""How the cloak rewrites a decky's egress to match an nmap_os family."""
|
||||
|
||||
window: int # TCP advertised window on SYN-ACK
|
||||
mss: int # MSS option value
|
||||
wscale: int # window-scale shift
|
||||
# Ordered TCP option layout to emit on SYN-ACK. "TS" is kept only if the
|
||||
# kernel emitted a Timestamp (sysctl tcp_timestamps=1) so its live,
|
||||
# incrementing value survives the rewrite (nmap SEQ.TS rate test).
|
||||
option_order: tuple[str, ...]
|
||||
ipid: str # "incr" (TI=I) | "random" (TI=RD) | "keep"
|
||||
respond_t2t3: bool # synthesize Windows T2/T3 replies
|
||||
|
||||
|
||||
_WIN_OPTS = ("MSS", "NOP", "WScale", "SAckOK", "TS")
|
||||
|
||||
OS_MANGLE: dict[str, MangleProfile] = {
|
||||
"windows": MangleProfile(
|
||||
window=0x2000, mss=1460, wscale=8,
|
||||
option_order=_WIN_OPTS, ipid="incr", respond_t2t3=True,
|
||||
),
|
||||
"windows_server": MangleProfile(
|
||||
window=0x2000, mss=1460, wscale=8,
|
||||
option_order=_WIN_OPTS, ipid="random", respond_t2t3=True,
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def get_os_mangle(nmap_os: str) -> MangleProfile | None:
|
||||
"""Return the cloak mangle profile for *nmap_os*, or None if it needs none."""
|
||||
return OS_MANGLE.get(nmap_os)
|
||||
|
||||
|
||||
41
decnet/paths.py
Normal file
41
decnet/paths.py
Normal file
@@ -0,0 +1,41 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""Shared runtime filesystem path resolution.
|
||||
|
||||
DECNET writes runtime state under a system dir provisioned by ``decnet
|
||||
init`` / systemd (``/var/lib/decnet`` for state, ``/run/decnet`` for
|
||||
sockets). On dev boxes without systemd, or in CI, that dir may be absent
|
||||
or read-only, so callers fall back to a per-user location.
|
||||
|
||||
:func:`resolve_runtime_path` centralises the writable-dir probe that the
|
||||
vectorstore and bus backends previously copy-pasted verbatim.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
|
||||
def resolve_runtime_path(
|
||||
filename: str,
|
||||
*,
|
||||
env_var: str,
|
||||
runtime_dir: str,
|
||||
user_fallback: str,
|
||||
) -> str:
|
||||
"""Resolve a runtime file path. Creates nothing.
|
||||
|
||||
Precedence:
|
||||
1. ``$env_var`` if set (used verbatim).
|
||||
2. ``runtime_dir/filename`` if ``runtime_dir`` exists and is writable.
|
||||
3. ``user_fallback`` (``~`` expanded).
|
||||
|
||||
``runtime_dir`` is *probed*, never created: it is meant to be
|
||||
provisioned with the right ownership and perms by init/systemd, so
|
||||
creating it here with whatever perms the current process happens to
|
||||
have would be worse than falling back to the user path.
|
||||
"""
|
||||
explicit = os.environ.get(env_var)
|
||||
if explicit:
|
||||
return explicit
|
||||
if os.path.isdir(runtime_dir) and os.access(runtime_dir, os.W_OK):
|
||||
return os.path.join(runtime_dir, filename)
|
||||
return os.path.expanduser(user_fallback)
|
||||
140
decnet/prefork.py
Normal file
140
decnet/prefork.py
Normal file
@@ -0,0 +1,140 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""Prefork supervisor — import the base floor ONCE in a master, then fork one
|
||||
child process per worker. Children share the ~70 MB import floor via
|
||||
copy-on-write.
|
||||
|
||||
Measured on CPython 3.14 (development/cow_probe.py): an idle forked child keeps
|
||||
~71 MB shared and dirties only ~1 MB private; a working child dirties ~26 MB
|
||||
(its own heap, not the floor). PEP 683 immortal objects keep module/code pages
|
||||
clean, so the classic refcount-dirties-CoW problem does not bite and gc.freeze()
|
||||
is unnecessary on 3.14.
|
||||
|
||||
Contrast with :mod:`decnet.supervisor` (asyncio tasks in ONE process, shared
|
||||
GIL): use that for cheap co-resident IO workers. Use prefork for workers that
|
||||
must keep their OWN process / GIL — CPU-heavy or isolation-critical — but
|
||||
shouldn't each re-import the world.
|
||||
|
||||
Each worker spec is a zero-arg callable that BLOCKS running the worker (e.g.
|
||||
``lambda: asyncio.run(profiler_worker(repo))``). It executes in the forked
|
||||
child; the master only forks, reaps, and restarts.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
|
||||
log = logging.getLogger("decnet.prefork")
|
||||
|
||||
WorkerEntry = Callable[[], None]
|
||||
|
||||
|
||||
def run_fleet(
|
||||
specs: dict[str, WorkerEntry],
|
||||
*,
|
||||
max_backoff: float = 30.0,
|
||||
poll_interval: float = 0.2,
|
||||
stop_after: float | None = None,
|
||||
) -> None:
|
||||
"""Fork one child per worker and supervise them until SIGTERM/SIGINT.
|
||||
|
||||
A dead child is re-forked after exponential backoff (in-process
|
||||
``Restart=on-failure``). Backoff is tracked per worker and scheduled
|
||||
non-blockingly, so one worker's restart delay never stalls reaping of
|
||||
another. On shutdown, children get SIGTERM, then SIGKILL after a grace
|
||||
period.
|
||||
|
||||
``stop_after`` (seconds) is a test hook: cleanly shut the fleet down after
|
||||
that long instead of waiting for a signal.
|
||||
"""
|
||||
if not specs:
|
||||
return
|
||||
|
||||
children: dict[int, str] = {} # pid -> name
|
||||
backoff: dict[str, float] = {n: 1.0 for n in specs}
|
||||
due: dict[str, float] = {} # name -> earliest restart time
|
||||
stopping = {"flag": False}
|
||||
|
||||
def _request_stop(_signum: int, _frame: object) -> None:
|
||||
stopping["flag"] = True
|
||||
|
||||
signal.signal(signal.SIGTERM, _request_stop)
|
||||
signal.signal(signal.SIGINT, _request_stop)
|
||||
|
||||
def spawn(name: str) -> None:
|
||||
pid = os.fork()
|
||||
if pid == 0: # ---- child ----
|
||||
# Restore default signal handling so the worker's own asyncio
|
||||
# handlers (or KeyboardInterrupt) work as if launched standalone.
|
||||
signal.signal(signal.SIGTERM, signal.SIG_DFL)
|
||||
signal.signal(signal.SIGINT, signal.SIG_DFL)
|
||||
try:
|
||||
specs[name]()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
except BaseException: # noqa: BLE001 — last-resort child logging
|
||||
log.exception("prefork: worker %s raised", name)
|
||||
os._exit(1)
|
||||
os._exit(0)
|
||||
children[pid] = name # ---- parent ----
|
||||
log.info("prefork: spawned %s pid=%d", name, pid)
|
||||
|
||||
log.info("prefork: master pid=%d forking %d workers: %s",
|
||||
os.getpid(), len(specs), ", ".join(specs))
|
||||
for name in specs:
|
||||
spawn(name)
|
||||
|
||||
deadline = (time.monotonic() + stop_after) if stop_after is not None else None
|
||||
while not stopping["flag"]:
|
||||
if deadline is not None and time.monotonic() >= deadline:
|
||||
break
|
||||
now = time.monotonic()
|
||||
# Restart any workers whose backoff has elapsed.
|
||||
for name in [n for n, t in due.items() if now >= t]:
|
||||
del due[name]
|
||||
spawn(name)
|
||||
# Reap without blocking so concurrent crashes are all handled.
|
||||
try:
|
||||
pid, status = os.waitpid(-1, os.WNOHANG)
|
||||
except ChildProcessError:
|
||||
pid = 0
|
||||
if pid == 0:
|
||||
time.sleep(poll_interval)
|
||||
continue
|
||||
name = children.pop(pid, None)
|
||||
if name is None:
|
||||
continue
|
||||
code = os.waitstatus_to_exitcode(status)
|
||||
log.warning("prefork: %s (pid=%d) exited code=%d; restart in %.0fs",
|
||||
name, pid, code, backoff[name])
|
||||
due[name] = time.monotonic() + backoff[name]
|
||||
backoff[name] = min(backoff[name] * 2.0, max_backoff)
|
||||
|
||||
_shutdown(children)
|
||||
|
||||
|
||||
def _shutdown(children: dict[int, str], *, grace: float = 15.0) -> None:
|
||||
"""SIGTERM all children, reap within ``grace``, SIGKILL stragglers."""
|
||||
for pid in list(children):
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
children.pop(pid, None)
|
||||
deadline = time.monotonic() + grace
|
||||
while children and time.monotonic() < deadline:
|
||||
try:
|
||||
pid, _ = os.waitpid(-1, os.WNOHANG)
|
||||
except ChildProcessError:
|
||||
break
|
||||
if pid:
|
||||
children.pop(pid, None)
|
||||
else:
|
||||
time.sleep(0.1)
|
||||
for pid in list(children):
|
||||
try:
|
||||
os.kill(pid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
log.info("prefork: fleet shut down")
|
||||
32
decnet/templates/_shared/cloak/Dockerfile
Normal file
32
decnet/templates/_shared/cloak/Dockerfile
Normal file
@@ -0,0 +1,32 @@
|
||||
# Cloak base image — the IP-holder/netns container for deckies whose nmap_os has
|
||||
# an egress mangle profile (windows, windows_server). Runs `python -m decnet.cloak`
|
||||
# (SYN-ACK mangler + T2/T3 responder) alongside holding the MACVLAN IP.
|
||||
#
|
||||
# FROM the per-decky distro so the base still varies by distro (BASE_IMAGE arg,
|
||||
# set by the composer from decky.build_base — same pattern as service images).
|
||||
# The decnet/ subtree is synced into this context by deployer._sync_cloak_sources
|
||||
# before build (8 light, stdlib-only files; scapy/netfilterqueue are pip'd here).
|
||||
ARG BASE_IMAGE=debian:bookworm-slim
|
||||
FROM ${BASE_IMAGE}
|
||||
|
||||
# Runtime: iptables (NFQUEUE rules), python3, libpcap (scapy BPF sniff in the
|
||||
# responder). Build-only: gcc + headers for the netfilterqueue C extension,
|
||||
# purged after the wheel is built to keep the image lean.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
python3 python3-pip iptables libpcap0.8 \
|
||||
libnetfilter-queue1 libnfnetlink0 \
|
||||
gcc python3-dev libnetfilter-queue-dev libnfnetlink-dev \
|
||||
&& pip3 install --no-cache-dir --break-system-packages \
|
||||
"scapy>=2.6.1" "netfilterqueue>=1.1.0" \
|
||||
&& apt-get purge -y gcc python3-dev libnetfilter-queue-dev libnfnetlink-dev \
|
||||
&& apt-get autoremove -y \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Synced 8-file decnet subtree (decnet/__init__, config_ini, logging/, os_fingerprint,
|
||||
# cloak/). PYTHONPATH=/opt makes `python3 -m decnet.cloak` importable.
|
||||
COPY decnet/ /opt/decnet/
|
||||
ENV PYTHONPATH=/opt
|
||||
|
||||
# The compose `command` drives runtime (netns-safe supervisor: cloak in background,
|
||||
# sleep infinity in foreground so a cloak crash never tears down the netns holder).
|
||||
CMD ["sleep", "infinity"]
|
||||
@@ -16,12 +16,12 @@ from __future__ import annotations
|
||||
|
||||
from typing import Final
|
||||
|
||||
ATTACK_BUNDLE_VERSION: Final[str] = "19.0"
|
||||
ATTACK_BUNDLE_VERSION: Final[str] = "19.1"
|
||||
|
||||
# sha256 of the canonical MITRE-published enterprise-attack-19.0.json
|
||||
# sha256 of the canonical MITRE-published enterprise-attack-19.1.json
|
||||
# from https://github.com/mitre-attack/attack-stix-data.
|
||||
ATTACK_BUNDLE_SHA256: Final[str] = (
|
||||
"df520ea0775a57db7bff760145b02fed89290802913e056b7ed5970b02f3626a"
|
||||
"bdf1ce86a4e604214c5076d37ae4dcb322678afc528df8492e6fdc1b554f5da3"
|
||||
)
|
||||
|
||||
# Raw download URL for the pinned version.
|
||||
|
||||
@@ -26,6 +26,7 @@ import logging
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
from decnet.paths import resolve_runtime_path
|
||||
from decnet.vectorstore.base import BaseVectorStore
|
||||
|
||||
LOG = logging.getLogger(__name__)
|
||||
@@ -65,10 +66,9 @@ def get_vectorstore(**kwargs: Any) -> BaseVectorStore:
|
||||
|
||||
|
||||
def _default_db_path() -> str:
|
||||
explicit = os.environ.get("DECNET_VECTORSTORE_PATH")
|
||||
if explicit:
|
||||
return explicit
|
||||
runtime_dir = "/var/lib/decnet"
|
||||
if os.path.isdir(runtime_dir) and os.access(runtime_dir, os.W_OK):
|
||||
return f"{runtime_dir}/vectors.sqlite"
|
||||
return os.path.expanduser("~/.decnet/vectors.sqlite")
|
||||
return resolve_runtime_path(
|
||||
"vectors.sqlite",
|
||||
env_var="DECNET_VECTORSTORE_PATH",
|
||||
runtime_dir="/var/lib/decnet",
|
||||
user_fallback="~/.decnet/vectors.sqlite",
|
||||
)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
import React, { useCallback, useEffect, useMemo, useRef, useState } from 'react';
|
||||
import { X, Server, Cpu, FileText, Sparkles, Check } from '../../icons';
|
||||
import { X, Server, Cpu, FileText, Sparkles, Check, Crosshair } from '../../icons';
|
||||
import { ScanImport } from '@pro';
|
||||
import api from '../../utils/api';
|
||||
import { useEscapeKey } from '../../hooks/useEscapeKey';
|
||||
import { useFocusTrap } from '../../hooks/useFocusTrap';
|
||||
@@ -28,7 +29,7 @@ interface TopologySummary {
|
||||
status_changed_at: string | null;
|
||||
}
|
||||
|
||||
type Kind = 'blank' | 'seeded';
|
||||
type Kind = 'blank' | 'seeded' | 'scan';
|
||||
|
||||
interface Props {
|
||||
open: boolean;
|
||||
@@ -103,15 +104,19 @@ const CreateTopologyWizard: React.FC<Props> = ({ open, onClose, onCreated }) =>
|
||||
[targetId, hosts],
|
||||
);
|
||||
|
||||
const canNext = step === 0 ? !!targetId : !!kind && name.trim().length > 0;
|
||||
const isAgent = !!targetId && targetId !== LOCAL_CARD_ID;
|
||||
const targetHostUuid = isAgent ? targetId : null;
|
||||
const mode = isAgent ? 'agent' : 'unihost';
|
||||
|
||||
// Scan import owns its own name/preview/create sub-flow inside the pro panel,
|
||||
// so the wizard's name gate and CREATE button don't apply to it.
|
||||
const canNext =
|
||||
step === 0 ? !!targetId : kind === 'scan' || (!!kind && name.trim().length > 0);
|
||||
|
||||
const handleCreate = async () => {
|
||||
if (!targetId || !kind) return;
|
||||
if (!targetId || !kind || kind === 'scan') return;
|
||||
setSubmitting(true);
|
||||
setErr(null);
|
||||
const isAgent = targetId !== LOCAL_CARD_ID;
|
||||
const targetHostUuid = isAgent ? targetId : null;
|
||||
const mode = isAgent ? 'agent' : 'unihost';
|
||||
try {
|
||||
if (kind === 'blank') {
|
||||
const { data } = await api.post<TopologySummary>('/topologies/blank', {
|
||||
@@ -234,6 +239,22 @@ const CreateTopologyWizard: React.FC<Props> = ({ open, onClose, onCreated }) =>
|
||||
Runs the MazeNET generator with depth/branching/deckies parameters. Seed is optional — omit for a fresh roll.
|
||||
</div>
|
||||
</div>
|
||||
{ScanImport && (
|
||||
<div
|
||||
onClick={() => setKind('scan')}
|
||||
className={`ctw-card ${kind === 'scan' ? 'selected' : ''}`}
|
||||
>
|
||||
<div className="ctw-card-head">
|
||||
<Crosshair size={16} className="ctw-violet" />
|
||||
<span className="ctw-card-name">SCAN-BASED</span>
|
||||
</div>
|
||||
<div className="ctw-card-sub">mirror an Nmap scan</div>
|
||||
<div className="ctw-card-desc">
|
||||
Import an Nmap XML scan and mirror the discovered hosts and services
|
||||
as decoys. Review and pick targets before deploying.
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
|
||||
@@ -285,19 +306,25 @@ const CreateTopologyWizard: React.FC<Props> = ({ open, onClose, onCreated }) =>
|
||||
<div className="ctw-label">
|
||||
Target: <span className="ctw-violet">{targetLabel}</span> · pick a starting point.
|
||||
</div>
|
||||
<div className="ctw-grid-2">{step1Cards}</div>
|
||||
<div className={ScanImport ? 'ctw-grid-3' : 'ctw-grid-2'}>{step1Cards}</div>
|
||||
|
||||
<div className="ctw-field">
|
||||
<label>NAME</label>
|
||||
<input
|
||||
autoFocus
|
||||
type="text"
|
||||
value={name}
|
||||
onChange={(e) => setName(e.target.value)}
|
||||
placeholder="e.g. honeynet-dev"
|
||||
maxLength={64}
|
||||
/>
|
||||
</div>
|
||||
{kind !== 'scan' && (
|
||||
<div className="ctw-field">
|
||||
<label>NAME</label>
|
||||
<input
|
||||
autoFocus
|
||||
type="text"
|
||||
value={name}
|
||||
onChange={(e) => setName(e.target.value)}
|
||||
placeholder="e.g. honeynet-dev"
|
||||
maxLength={64}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{kind === 'scan' && ScanImport && (
|
||||
<ScanImport mode={mode} targetHostUuid={targetHostUuid} onCreated={onCreated} />
|
||||
)}
|
||||
|
||||
{kind === 'seeded' && (
|
||||
<div className="ctw-grid-2">
|
||||
@@ -371,7 +398,7 @@ const CreateTopologyWizard: React.FC<Props> = ({ open, onClose, onCreated }) =>
|
||||
NEXT →
|
||||
</button>
|
||||
)}
|
||||
{step === 1 && (
|
||||
{step === 1 && kind !== 'scan' && (
|
||||
<button className="ctw-btn" disabled={!canNext || submitting} onClick={handleCreate}>
|
||||
{submitting ? 'CREATING…' : 'CREATE'}
|
||||
</button>
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
import { proRoutes } from '@pro';
|
||||
import { proRoutes, ScanImport } from '@pro';
|
||||
|
||||
// In the community build, `@pro` resolves to the stub: no Professional pages,
|
||||
// so App's route map and Layout's nav group both tree-shake to nothing.
|
||||
@@ -7,4 +7,10 @@ describe('pro tier — community build', () => {
|
||||
it('ships no pro routes', () => {
|
||||
expect(proRoutes).toEqual([]);
|
||||
});
|
||||
|
||||
// null tree-shakes the wizard's third "SCAN-BASED" card out of the community
|
||||
// bundle — the scan→topology importer is Professional-only.
|
||||
it('ships no scan importer', () => {
|
||||
expect(ScanImport).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -3,6 +3,10 @@
|
||||
// sets VITE_DECNET_PRO=1 with decnet/pro/web/ present, in which case Vite
|
||||
// aliases `@pro` to the real registry. proRoutes being empty lets the router
|
||||
// and nav tree-shake the pro surface out of the community bundle.
|
||||
import type { ProRoute } from './types';
|
||||
import type { ProRoute, ProScanImport } from './types';
|
||||
|
||||
export const proRoutes: ProRoute[] = [];
|
||||
|
||||
// No scan-based topology creation in the community build — the wizard's third
|
||||
// card tree-shakes out when this is null.
|
||||
export const ScanImport: ProScanImport = null;
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
// SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
// Contract for Professional-tier UI pages. The pro build aliases `@pro` to the
|
||||
// real registry in decnet/pro/web/; the community build resolves it to ./stub.
|
||||
import type { ReactElement, ReactNode } from 'react';
|
||||
import type { ComponentType, ReactElement, ReactNode } from 'react';
|
||||
|
||||
export interface ProRoute {
|
||||
/** Router path, e.g. "/pro/intel". Convention: prefix pro routes with /pro. */
|
||||
@@ -13,3 +13,35 @@ export interface ProRoute {
|
||||
/** Page element rendered at `path`. May be a lazy component (App wraps Suspense). */
|
||||
element: ReactElement;
|
||||
}
|
||||
|
||||
/** Created-topology summary handed back to the wizard. Mirrors the wizard's own
|
||||
* TopologySummary (and GET /topologies rows) structurally so the wizard's
|
||||
* onCreated handler is assignable without a cross-tree type import. */
|
||||
export interface ProTopologySummary {
|
||||
id: string;
|
||||
name: string;
|
||||
mode: string;
|
||||
target_host_uuid: string | null;
|
||||
status: string;
|
||||
version: number;
|
||||
needs_resync?: boolean;
|
||||
created_at: string;
|
||||
status_changed_at: string | null;
|
||||
}
|
||||
|
||||
/** Props the CreateTopologyWizard passes to the pro scan-import panel. The pro
|
||||
* build owns the entire scan→topology flow (file pick, parse, preview, create)
|
||||
* and signals completion through `onCreated`; the community build never sees
|
||||
* this surface. Kept structural — the pro tree implements the shape without
|
||||
* importing it, mirroring how `ProRoute` crosses the trust boundary. */
|
||||
export interface ProScanImportProps {
|
||||
/** "unihost" | "agent" — chosen in the wizard's TARGET step. */
|
||||
mode: string;
|
||||
/** Agent host UUID, or null for local. */
|
||||
targetHostUuid: string | null;
|
||||
/** Fires with the created topology summary; the wizard closes and navigates. */
|
||||
onCreated: (row: ProTopologySummary) => void;
|
||||
}
|
||||
|
||||
/** `null` in the community build (no scan import); a component in the pro build. */
|
||||
export type ProScanImport = ComponentType<ProScanImportProps> | null;
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Attribution Engine v0 (per-(identity, primitive) state machine)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#attribution
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.attribution.log
|
||||
# Subscribes to attacker.observation.> and, for each event, ensures a
|
||||
# stub AttackerIdentity row, runs the per-ValueKind merger over the
|
||||
# full identity-keyed observation series, upserts the derived state in
|
||||
# attribution_state, and publishes attribution.profile.state_changed
|
||||
# only on transition. Periodic tick (default 60s) fires
|
||||
# attribution.profile.multi_actor_suspected when >= 2 primitives flag
|
||||
# the same identity. Closes DEBT-051.
|
||||
ExecStart={{ venv_dir }}/bin/decnet attribution
|
||||
StandardOutput=append:/var/log/decnet/decnet.attribution.log
|
||||
StandardError=append:/var/log/decnet/decnet.attribution.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,52 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Campaign Clusterer (identities → campaigns / operations)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#campaign-clusterer
|
||||
After=network-online.target decnet-bus.service decnet-clusterer.service
|
||||
Wants=network-online.target decnet-bus.service decnet-clusterer.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.campaign-clusterer.log
|
||||
# Subscribes to identity.>; falls back to a 60s slow-tick poll when
|
||||
# the bus is idle or unavailable. Reads AttackerIdentity rows,
|
||||
# projects them into the campaign-level similarity graph
|
||||
# (phase-handoff / shared-infra / temporal overlap / cohort), runs
|
||||
# union-find, writes campaigns rows + sets
|
||||
# attacker_identities.campaign_id, and publishes campaign.formed /
|
||||
# campaign.identity.assigned / campaign.merged / campaign.unmerged
|
||||
# plus the cross-family identity.campaign.assigned for identity-side
|
||||
# subscribers.
|
||||
#
|
||||
# Master-only: gated via MASTER_ONLY_COMMANDS in decnet/cli/gating.py.
|
||||
# Sits one layer above decnet-clusterer (the After=/Wants= ensures the
|
||||
# identity layer is up first; the campaign clusterer then wakes on
|
||||
# identity.> events fired by it).
|
||||
ExecStart={{ venv_dir }}/bin/decnet campaign-clusterer
|
||||
StandardOutput=append:/var/log/decnet/decnet.campaign-clusterer.log
|
||||
StandardError=append:/var/log/decnet/decnet.campaign-clusterer.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,47 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Identity Clusterer (per-IP observations → identities)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#identity-clusterer
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.clusterer.log
|
||||
# Subscribes to attacker.observed and attacker.scored; falls back to a
|
||||
# 60s slow-tick poll when the bus is idle or unavailable. Reads
|
||||
# Attacker rows, projects per-IP observations into the similarity
|
||||
# graph (JA3 / HASSH / payload-hash / C2-endpoint), runs union-find,
|
||||
# writes attacker_identities rows + sets attackers.identity_id, and
|
||||
# publishes identity.formed / identity.observation.linked /
|
||||
# identity.merged / identity.unmerged.
|
||||
#
|
||||
# Master-only: gated via MASTER_ONLY_COMMANDS in decnet/cli/gating.py.
|
||||
ExecStart={{ venv_dir }}/bin/decnet clusterer
|
||||
StandardOutput=append:/var/log/decnet/decnet.clusterer.log
|
||||
StandardError=append:/var/log/decnet/decnet.clusterer.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,47 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Threat-Intel Enrichment (GreyNoise + AbuseIPDB + abuse.ch)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#intel-enrichment
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.enrich.log
|
||||
# Subscribes to attacker.observed and attacker.scored; falls back to a 60s
|
||||
# slow-tick poll when the bus is idle or unavailable. Per attacker IP fans
|
||||
# out across the configured intel providers, writes the aggregate verdict
|
||||
# to attacker_intel, and publishes attacker.intel.enriched.
|
||||
#
|
||||
# Free-tier API keys are read from .env.local:
|
||||
# DECNET_GREYNOISE_API_KEY= (optional, lifts rate limit)
|
||||
# DECNET_ABUSEIPDB_API_KEY= (required for AbuseIPDB lookups)
|
||||
# DECNET_THREATFOX_API_KEY= (optional, lifts rate limit)
|
||||
ExecStart={{ venv_dir }}/bin/decnet enrich
|
||||
StandardOutput=append:/var/log/decnet/decnet.enrich.log
|
||||
StandardError=append:/var/log/decnet/decnet.enrich.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
48
deploy/decnet-fleet-heavy.service.j2
Normal file
48
deploy/decnet-fleet-heavy.service.j2
Normal file
@@ -0,0 +1,48 @@
|
||||
[Unit]
|
||||
Description=DECNET Heavy Fleet (prefork master forking profiler + ttp as CoW-sharing children)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#fleet
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
# Replaces the individual decnet-profiler / decnet-ttp units. Do NOT enable
|
||||
# those alongside this one.
|
||||
Conflicts=decnet-profiler.service decnet-ttp.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.fleet-heavy.log
|
||||
ExecStart={{ venv_dir }}/bin/decnet fleet heavy
|
||||
StandardOutput=append:/var/log/decnet/decnet.fleet-heavy.log
|
||||
StandardError=append:/var/log/decnet/decnet.fleet-heavy.log
|
||||
|
||||
# Prefork master imports the shared base floor once, then forks one child per
|
||||
# worker; children share the floor via copy-on-write. Both members are DB-only
|
||||
# (no docker socket, no raw sockets) so this unit carries NO extra privilege —
|
||||
# the prefork privilege-union cost is nil for this fleet by construction.
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
# Dev installs under /home need ProtectHome=read-only: the ttp child reads
|
||||
# ./rules/ttp/ from the project root (read-only suffices — YAML reads only).
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
# Master forwards SIGTERM to children and reaps; give it room for both to drain.
|
||||
TimeoutStopSec=25
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,41 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Mutator (runtime fleet mutation watch loop)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#mutator
|
||||
After=network-online.target docker.service decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
Requires=docker.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
# Mutator recomposes decky services via docker compose.
|
||||
SupplementaryGroups=docker
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.mutator.log
|
||||
ExecStart={{ venv_dir }}/bin/decnet mutate --watch
|
||||
StandardOutput=append:/var/log/decnet/decnet.mutator.log
|
||||
StandardError=append:/var/log/decnet/decnet.mutator.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,50 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Orchestrator (synthetic life-injection — inter-decky traffic, file plants, email drops)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#orchestrator
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.orchestrator.log
|
||||
# Realism content engine — LLM + persona-pool config used by the
|
||||
# email + (post-stage-6) file-class enrichment paths. See
|
||||
# decnet/realism/llm/factory.py and decnet/realism/personas_pool.py.
|
||||
Environment=DECNET_REALISM_LLM=ollama
|
||||
Environment=DECNET_REALISM_MODEL=llama3.1
|
||||
Environment=DECNET_REALISM_TIMEOUT=60
|
||||
Environment=DECNET_REALISM_PERSONAS=/etc/decnet/email_personas.json
|
||||
ExecStart={{ venv_dir }}/bin/decnet orchestrate
|
||||
StandardOutput=append:/var/log/decnet/decnet.orchestrator.log
|
||||
StandardError=append:/var/log/decnet/decnet.orchestrator.log
|
||||
|
||||
# The orchestrator drives `docker exec` against decky containers, so it
|
||||
# needs membership in the docker group. It does NOT bind to the network,
|
||||
# launch new containers, or write outside its own logs and install dir.
|
||||
SupplementaryGroups=docker
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,38 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Profiler (attacker profiling and scoring)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#profiler
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.profiler.log
|
||||
ExecStart={{ venv_dir }}/bin/decnet profiler
|
||||
StandardOutput=append:/var/log/decnet/decnet.profiler.log
|
||||
StandardError=append:/var/log/decnet/decnet.profiler.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,47 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Fleet Reconciler (converges decnet-state.json ↔ fleet_deckies DB ↔ docker)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#reconciler
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.reconciler.log
|
||||
ExecStart={{ venv_dir }}/bin/decnet reconcile
|
||||
StandardOutput=append:/var/log/decnet/decnet.reconciler.log
|
||||
StandardError=append:/var/log/decnet/decnet.reconciler.log
|
||||
|
||||
# The reconciler queries the docker daemon (via `docker.from_env()`) to
|
||||
# observe per-container state. Membership in the docker group lets it
|
||||
# read /var/run/docker.sock without root. It does NOT exec into
|
||||
# containers, bind to the network, or spawn new containers.
|
||||
SupplementaryGroups=docker
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
# Read-only access to /var/lib/decnet so we can read decnet-state.json.
|
||||
# Read-write access only to install_dir + log dir.
|
||||
ReadOnlyPaths=/var/lib/decnet
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,41 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET Credential-Reuse Correlator (cross-target secret-reuse detection)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#reuse-correlator
|
||||
After=network-online.target decnet-bus.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.reuse-correlator.log
|
||||
# Subscribes to credential.captured and attacker.observed; falls back to
|
||||
# a 60s slow-tick poll when the bus is idle or unavailable. Publishes
|
||||
# credential.reuse.detected once per new/grown finding.
|
||||
ExecStart={{ venv_dir }}/bin/decnet reuse-correlate
|
||||
StandardOutput=append:/var/log/decnet/decnet.reuse-correlator.log
|
||||
StandardError=append:/var/log/decnet/decnet.reuse-correlator.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -1,57 +0,0 @@
|
||||
[Unit]
|
||||
Description=DECNET TTP Tagger (MITRE ATT&CK technique tagging)
|
||||
Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers#ttp-tagger
|
||||
After=network-online.target decnet-bus.service decnet-clusterer.service decnet-enrich.service decnet-reuse-correlator.service
|
||||
Wants=network-online.target decnet-bus.service
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User={{ user }}
|
||||
Group={{ group }}
|
||||
WorkingDirectory={{ install_dir }}
|
||||
EnvironmentFile=-{{ install_dir }}/.env.local
|
||||
Environment=DECNET_SYSTEM_LOGS=/var/log/decnet/decnet.ttp.log
|
||||
# Subscribes to attacker.session.ended (primary), attacker.observed,
|
||||
# attacker.intel.enriched, identity.formed, identity.merged,
|
||||
# credential.reuse.detected, email.received, and canary.> ; falls back
|
||||
# to a 60s slow-tick poll when the bus is idle or unavailable. Each
|
||||
# event is dispatched through the CompositeTagger (RuleEngine +
|
||||
# Behavioral / Intel / Email / CanaryFingerprint / Identity /
|
||||
# Credential lifters), persisted via the idempotent INSERT OR IGNORE
|
||||
# repo write, and ttp.tagged + ttp.rule.fired.<technique_id> are
|
||||
# published only when the insert returned a non-zero rowcount
|
||||
# (loop-prevention invariant — see TTP_TAGGING.md §"Bus topics").
|
||||
#
|
||||
# Master-only: gated via MASTER_ONLY_COMMANDS in decnet/cli/gating.py.
|
||||
# Sits one layer above the identity / intel / reuse-correlator
|
||||
# workers — the After= dependencies ensure their bus topics are live
|
||||
# before the TTP worker subscribes.
|
||||
ExecStart={{ venv_dir }}/bin/decnet ttp
|
||||
StandardOutput=append:/var/log/decnet/decnet.ttp.log
|
||||
StandardError=append:/var/log/decnet/decnet.ttp.log
|
||||
|
||||
CapabilityBoundingSet=
|
||||
AmbientCapabilities=
|
||||
|
||||
# Security Hardening
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=full
|
||||
# Dev installs under /home need ProtectHome=read-only (the worker
|
||||
# reads ./rules/ttp/ from the project root, which lives under /home
|
||||
# on dev boxes — read-only suffices because the FilesystemRuleStore
|
||||
# only reads YAMLs, never writes).
|
||||
ProtectHome=read-only
|
||||
PrivateTmp=yes
|
||||
ProtectKernelTunables=yes
|
||||
ProtectKernelModules=yes
|
||||
ProtectControlGroups=yes
|
||||
RestrictSUIDSGID=yes
|
||||
LockPersonality=yes
|
||||
ReadWritePaths={{ install_dir }} /var/log/decnet
|
||||
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
TimeoutStopSec=15
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -5,23 +5,21 @@ Documentation=https://git.resacachile.cl/anti/DECNET/wiki/Workers
|
||||
# heartbeats to it), then the API + data-plane workers. systemd resolves the
|
||||
# actual ordering via each unit's own After=/Wants= on decnet-bus.service —
|
||||
# this target is a convenience grouping, not an ordering primitive.
|
||||
# Consolidated since 1.1/1.2: the batch (reconcile/enrich/orchestrate/mutate)
|
||||
# and cpu (clusterer/campaign-clusterer/attribution/reuse-correlate) supervisor
|
||||
# groups and the heavy (profiler/ttp) prefork fleet replace their per-worker
|
||||
# units. The standalone workers below kept their own units.
|
||||
Wants=decnet-bus.service \
|
||||
decnet-api.service \
|
||||
decnet-web.service \
|
||||
decnet-collector.service \
|
||||
decnet-profiler.service \
|
||||
decnet-sniffer.service \
|
||||
decnet-prober.service \
|
||||
decnet-mutator.service \
|
||||
decnet-reconciler.service \
|
||||
decnet-reuse-correlator.service \
|
||||
decnet-enrich.service \
|
||||
decnet-clusterer.service \
|
||||
decnet-campaign-clusterer.service \
|
||||
decnet-ttp.service \
|
||||
decnet-webhook.service \
|
||||
decnet-canary.service \
|
||||
decnet-orchestrator.service
|
||||
decnet-supervise-batch.service \
|
||||
decnet-supervise-cpu.service \
|
||||
decnet-fleet-heavy.service
|
||||
After=decnet-bus.service
|
||||
|
||||
[Install]
|
||||
|
||||
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "decnet"
|
||||
version = "1.1.1"
|
||||
version = "1.2.1"
|
||||
description = "Deception network: deploy honeypot deckies that appear as real LAN hosts"
|
||||
readme = "README.md"
|
||||
authors = [{ name = "Samuel Paschuan", email = "samuel.paschuan@xmartlab.com" }]
|
||||
@@ -40,6 +40,9 @@ dependencies = [
|
||||
# `alembic upgrade head` at boot for managed DBs (see db/migrate.py).
|
||||
"alembic>=1.13",
|
||||
"scapy>=2.6.1",
|
||||
# cloak egress mangler (NFQUEUE); Linux-only, lazy-imported so absence on
|
||||
# dev/CI/non-Linux is tolerated (decnet.cloak only needs it at run()).
|
||||
"netfilterqueue>=1.1.0 ; sys_platform == 'linux'",
|
||||
"orjson>=3.10",
|
||||
"cryptography>=48.0.1",
|
||||
"python-multipart>=0.0.31",
|
||||
|
||||
35
tests/cli/test_fleet.py
Normal file
35
tests/cli/test_fleet.py
Normal file
@@ -0,0 +1,35 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""CLI surface for ``decnet fleet`` (DECNET 1.2 prefork). The fork/restart
|
||||
mechanism itself is covered by tests/test_prefork.py."""
|
||||
from __future__ import annotations
|
||||
|
||||
from typer.testing import CliRunner
|
||||
|
||||
from decnet.cli import app
|
||||
from decnet.cli.fleet import _FLEETS, _build_fleet
|
||||
|
||||
runner = CliRunner()
|
||||
|
||||
|
||||
def test_fleet_is_registered():
|
||||
result = runner.invoke(app, ["fleet", "--help"])
|
||||
assert result.exit_code == 0
|
||||
assert "fleet" in result.stdout.lower()
|
||||
|
||||
|
||||
def test_unknown_fleet_exits_2():
|
||||
result = runner.invoke(app, ["fleet", "not-a-fleet"])
|
||||
assert result.exit_code == 2
|
||||
assert "unknown fleet" in result.stdout
|
||||
|
||||
|
||||
def test_heavy_fleet_builds_expected_workers():
|
||||
# _build_fleet imports worker modules + builds thunks but runs nothing
|
||||
# (no fork, no repo.initialize) — safe to call in-process.
|
||||
specs = _build_fleet("heavy")
|
||||
assert set(specs) == {"profiler", "ttp"}
|
||||
assert all(callable(t) for t in specs.values())
|
||||
|
||||
|
||||
def test_heavy_is_known():
|
||||
assert "heavy" in _FLEETS
|
||||
0
tests/cloak/__init__.py
Normal file
0
tests/cloak/__init__.py
Normal file
147
tests/cloak/test_cloak.py
Normal file
147
tests/cloak/test_cloak.py
Normal file
@@ -0,0 +1,147 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Tests for the cloak mangler/responder PURE logic — option layout, IP-ID policy,
|
||||
probe classification, reply fields. No scapy, root, or live NFQUEUE involved
|
||||
(the runtime loops are exercised only on real deckies, not in CI).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from decnet.cloak import (
|
||||
ProbeKind,
|
||||
build_reply_fields,
|
||||
build_synack_options,
|
||||
classify_probe,
|
||||
next_ipid,
|
||||
)
|
||||
from decnet.cloak.mangler import _is_synack, _rst_needs_ack
|
||||
from decnet.os_fingerprint import OS_MANGLE, MangleProfile, get_os_mangle
|
||||
|
||||
WIN = OS_MANGLE["windows"]
|
||||
SRV = OS_MANGLE["windows_server"]
|
||||
|
||||
|
||||
# ── profile wiring ──────────────────────────────────────────────────────────
|
||||
|
||||
def test_get_os_mangle_known():
|
||||
assert isinstance(get_os_mangle("windows"), MangleProfile)
|
||||
assert get_os_mangle("windows_server").ipid == "random"
|
||||
|
||||
|
||||
def test_get_os_mangle_none_for_linux():
|
||||
assert get_os_mangle("linux") is None
|
||||
assert get_os_mangle("nonexistent") is None
|
||||
|
||||
|
||||
def test_windows_workstation_ipid_is_incr():
|
||||
# Win10 workstation = incremental IP-ID (nmap TI=I); server = randomized (RD).
|
||||
assert WIN.ipid == "incr"
|
||||
assert SRV.ipid == "random"
|
||||
|
||||
|
||||
# ── SYN-ACK option building ─────────────────────────────────────────────────
|
||||
|
||||
def test_options_layout_with_timestamp_preserved():
|
||||
orig = [("MSS", 1460), ("SAckOK", b""), ("Timestamp", (111, 222)),
|
||||
("NOP", None), ("WScale", 7)]
|
||||
out = build_synack_options(orig, WIN)
|
||||
names = [n for n, _ in out]
|
||||
assert names == ["MSS", "NOP", "WScale", "SAckOK", "Timestamp"]
|
||||
# the kernel's live timestamp value must survive (SEQ.TS rate test)
|
||||
assert ("Timestamp", (111, 222)) in out
|
||||
# our chosen mss/wscale override whatever the kernel emitted
|
||||
assert ("MSS", WIN.mss) in out
|
||||
assert ("WScale", WIN.wscale) in out
|
||||
|
||||
|
||||
def test_options_drop_timestamp_when_kernel_had_none():
|
||||
"""If timestamps are off (no kernel TS option), emit none — never a fake one."""
|
||||
orig = [("MSS", 1460), ("SAckOK", b""), ("NOP", None), ("WScale", 7)]
|
||||
out = build_synack_options(orig, WIN)
|
||||
assert all(n != "Timestamp" for n, _ in out)
|
||||
|
||||
|
||||
def test_options_length_is_4byte_aligned():
|
||||
"""Sanity: the windows option layout encodes to a multiple of 4 bytes."""
|
||||
from scapy.all import TCP # type: ignore # noqa
|
||||
pytest.importorskip("scapy")
|
||||
orig = [("MSS", 1460), ("Timestamp", (1, 2))]
|
||||
out = build_synack_options(orig, WIN)
|
||||
raw = bytes(TCP(options=out))[20:] # options after the 20-byte base header
|
||||
assert len(raw) % 4 == 0
|
||||
|
||||
|
||||
# ── IP-ID policy ────────────────────────────────────────────────────────────
|
||||
|
||||
def test_next_ipid_incr_wraps():
|
||||
assert next_ipid(5, "incr") == 6
|
||||
assert next_ipid(0xFFFF, "incr") == 0
|
||||
|
||||
|
||||
def test_next_ipid_random_in_range_nonzero():
|
||||
for _ in range(50):
|
||||
v = next_ipid(0, "random")
|
||||
assert 1 <= v <= 0xFFFF
|
||||
|
||||
|
||||
def test_next_ipid_keep_sentinel():
|
||||
assert next_ipid(123, "keep") == -1
|
||||
|
||||
|
||||
# ── SYN-ACK detection ───────────────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.parametrize("flags,expected", [
|
||||
(0x12, True), # SYN+ACK
|
||||
(0x52, True), # SYN+ACK+ECE (ECN SYN-ACK)
|
||||
(0x02, False), # bare SYN
|
||||
(0x10, False), # bare ACK
|
||||
])
|
||||
def test_is_synack(flags, expected):
|
||||
assert _is_synack(flags) is expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("flags,expected", [
|
||||
(0x04, True), # bare RST (T4/T6 ACK-probe response) → fill ack (A=O)
|
||||
(0x14, False), # RST+ACK (T5/T7) → already A=S+, leave
|
||||
(0x12, False), # SYN+ACK
|
||||
])
|
||||
def test_rst_needs_ack(flags, expected):
|
||||
assert _rst_needs_ack(flags) is expected
|
||||
|
||||
|
||||
# ── probe classification ────────────────────────────────────────────────────
|
||||
|
||||
OPEN = frozenset({22, 80, 443})
|
||||
|
||||
|
||||
def test_classify_t2_null_flags_open_port():
|
||||
assert classify_probe(0x00, 80, OPEN) is ProbeKind.T2
|
||||
|
||||
|
||||
def test_classify_t3_synfinpshurg_open_port():
|
||||
assert classify_probe(0x2B, 80, OPEN) is ProbeKind.T3
|
||||
|
||||
|
||||
def test_classify_ignores_closed_port():
|
||||
assert classify_probe(0x00, 9999, OPEN) is None
|
||||
|
||||
|
||||
def test_classify_ignores_normal_traffic():
|
||||
assert classify_probe(0x02, 80, OPEN) is None # SYN — real stack handles
|
||||
assert classify_probe(0x10, 80, OPEN) is None # ACK
|
||||
|
||||
|
||||
# ── reply field shaping ─────────────────────────────────────────────────────
|
||||
|
||||
def test_reply_fields_t2_ack_equals_probe_seq():
|
||||
# T2: A=S (ack == probe seq)
|
||||
f = build_reply_fields(0xDEAD, ProbeKind.T2)
|
||||
assert f == {"seq": 0, "ack": 0xDEAD, "flags": "RA", "window": 0, "df": True}
|
||||
|
||||
|
||||
def test_reply_fields_t3_ack_is_other():
|
||||
# T3: A=O (other — not zero, not the probe seq)
|
||||
f = build_reply_fields(0xDEAD, ProbeKind.T3)
|
||||
assert f["ack"] not in (0, 0xDEAD)
|
||||
assert f["seq"] == 0 and f["flags"] == "RA"
|
||||
130
tests/cloak/test_compose_wiring.py
Normal file
130
tests/cloak/test_compose_wiring.py
Normal file
@@ -0,0 +1,130 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Tests for wiring the cloak into the deploy path:
|
||||
- composer.py: windows* base containers get build+command+caps+env; non-mangled
|
||||
bases stay byte-for-byte unchanged.
|
||||
- composer._decky_open_tcp_ports: service-port enumeration.
|
||||
- deployer._sync_cloak_sources: ships the decnet subtree only when needed.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from decnet.composer import _CLOAK_COMMAND, _decky_open_tcp_ports, generate_compose
|
||||
from decnet.config import DeckyConfig, DecnetConfig
|
||||
|
||||
|
||||
def _decky(nmap_os: str = "linux", services: list[str] | None = None) -> DeckyConfig:
|
||||
return DeckyConfig(
|
||||
name="decky-01",
|
||||
ip="10.0.0.10",
|
||||
services=services or ["ssh"],
|
||||
distro="debian",
|
||||
base_image="debian:bookworm-slim",
|
||||
build_base="debian:bookworm-slim",
|
||||
hostname="test-host",
|
||||
nmap_os=nmap_os,
|
||||
)
|
||||
|
||||
|
||||
def _config(decky: DeckyConfig) -> DecnetConfig:
|
||||
return DecnetConfig(
|
||||
mode="unihost", interface="eth0", subnet="10.0.0.0/24",
|
||||
gateway="10.0.0.1", deckies=[decky],
|
||||
)
|
||||
|
||||
|
||||
def _base(nmap_os: str, services: list[str] | None = None) -> dict:
|
||||
return generate_compose(_config(_decky(nmap_os, services)))["services"]["decky-01"]
|
||||
|
||||
|
||||
# ── port enumeration ────────────────────────────────────────────────────────
|
||||
|
||||
def test_open_ports_union_sorted_deduped():
|
||||
# smb=[445,139], rdp=[3389]
|
||||
assert _decky_open_tcp_ports(["smb", "rdp"]) == [139, 445, 3389]
|
||||
|
||||
|
||||
def test_open_ports_single_and_multiport():
|
||||
assert _decky_open_tcp_ports(["ssh"]) == [22]
|
||||
assert _decky_open_tcp_ports(["imap"]) == [143, 993] # multi-port service
|
||||
|
||||
|
||||
# ── non-mangled base is unchanged ───────────────────────────────────────────
|
||||
|
||||
def test_linux_base_uses_stock_image_and_sleep():
|
||||
base = _base("linux")
|
||||
assert base["image"] == "debian:bookworm-slim"
|
||||
assert base["command"] == ["sleep", "infinity"]
|
||||
assert "build" not in base
|
||||
assert "environment" not in base
|
||||
assert base["cap_add"] == ["NET_ADMIN"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fam", ["embedded", "bsd", "cisco"])
|
||||
def test_other_families_not_cloaked(fam):
|
||||
base = _base(fam)
|
||||
assert "build" not in base
|
||||
assert base["command"] == ["sleep", "infinity"]
|
||||
assert "NET_RAW" not in base["cap_add"]
|
||||
|
||||
|
||||
# ── windows* base gets the cloak ────────────────────────────────────────────
|
||||
|
||||
@pytest.mark.parametrize("fam", ["windows", "windows_server"])
|
||||
def test_windows_base_is_built_cloak_image(fam):
|
||||
base = _base(fam, services=["smb", "rdp"])
|
||||
assert "image" not in base
|
||||
assert base["build"]["args"]["BASE_IMAGE"] == "debian:bookworm-slim"
|
||||
assert base["build"]["context"].endswith("templates/_shared/cloak")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fam", ["windows", "windows_server"])
|
||||
def test_windows_base_runs_cloak_netns_safe(fam):
|
||||
base = _base(fam)
|
||||
# supervisor keeps sleep infinity as PID1 so a cloak crash can't kill the netns
|
||||
assert base["command"] == _CLOAK_COMMAND
|
||||
assert "decnet.cloak" in base["command"][-1]
|
||||
assert "sleep infinity" in base["command"][-1]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("fam", ["windows", "windows_server"])
|
||||
def test_windows_base_caps_include_net_raw(fam):
|
||||
base = _base(fam)
|
||||
assert "NET_ADMIN" in base["cap_add"]
|
||||
assert "NET_RAW" in base["cap_add"]
|
||||
|
||||
|
||||
def test_windows_base_env_carries_profile_and_ports():
|
||||
base = _base("windows_server", services=["smb", "rdp"])
|
||||
env = base["environment"]
|
||||
assert env["DECNET_NMAP_OS"] == "windows_server"
|
||||
assert env["DECNET_OPEN_PORTS"] == "139,445,3389"
|
||||
assert env["DECKY_IP"] == "10.0.0.10"
|
||||
|
||||
|
||||
def test_windows_base_still_has_sysctls():
|
||||
base = _base("windows")
|
||||
assert base["sysctls"]["net.ipv4.ip_default_ttl"] == "128"
|
||||
assert base["sysctls"]["net.ipv4.tcp_timestamps"] == "1"
|
||||
|
||||
|
||||
# ── deployer sync gating ────────────────────────────────────────────────────
|
||||
|
||||
def test_sync_cloak_ships_subtree_only_when_needed(tmp_path, monkeypatch):
|
||||
from decnet.engine import deployer
|
||||
|
||||
dest_root = tmp_path / "cloak"
|
||||
monkeypatch.setattr(deployer, "_CANONICAL_CLOAK_DIR", dest_root)
|
||||
|
||||
# linux-only → no-op
|
||||
deployer._sync_cloak_sources(_config(_decky("linux")))
|
||||
assert not (dest_root / "decnet").exists()
|
||||
|
||||
# windows → ships the subtree, package structure preserved
|
||||
deployer._sync_cloak_sources(_config(_decky("windows")))
|
||||
shipped = dest_root / "decnet"
|
||||
assert (shipped / "__init__.py").is_file()
|
||||
assert (shipped / "os_fingerprint.py").is_file()
|
||||
assert (shipped / "cloak" / "mangler.py").is_file()
|
||||
assert (shipped / "logging" / "__init__.py").is_file()
|
||||
@@ -50,8 +50,20 @@ def test_linux_tcp_timestamps_is_1():
|
||||
assert get_os_sysctls("linux")["net.ipv4.tcp_timestamps"] == "1"
|
||||
|
||||
|
||||
def test_windows_tcp_timestamps_is_0():
|
||||
assert get_os_sysctls("windows")["net.ipv4.tcp_timestamps"] == "0"
|
||||
def test_windows_tcp_timestamps_is_1():
|
||||
# Modern Windows 10/11 runs TCP timestamps ON (nmap SEQ.TS=A). A prior
|
||||
# value of 0 here fingerprinted as an ancient stack — see os_fingerprint.py.
|
||||
assert get_os_sysctls("windows")["net.ipv4.tcp_timestamps"] == "1"
|
||||
|
||||
|
||||
def test_windows_server_tcp_timestamps_is_1():
|
||||
assert get_os_sysctls("windows_server")["net.ipv4.tcp_timestamps"] == "1"
|
||||
|
||||
|
||||
def test_windows_server_tcp_ecn_is_1():
|
||||
# Server negotiates ECN (nmap ECN.CC=Y); workstation does not (CC=N).
|
||||
assert get_os_sysctls("windows_server")["net.ipv4.tcp_ecn"] == "1"
|
||||
assert get_os_sysctls("windows")["net.ipv4.tcp_ecn"] == "0"
|
||||
|
||||
|
||||
def test_embedded_tcp_timestamps_is_0():
|
||||
@@ -237,7 +249,7 @@ def test_all_os_families_non_empty():
|
||||
assert "embedded" in families
|
||||
|
||||
|
||||
@pytest.mark.parametrize("family", ["linux", "windows", "bsd", "embedded", "cisco"])
|
||||
@pytest.mark.parametrize("family", ["linux", "windows", "windows_server", "bsd", "embedded", "cisco"])
|
||||
def test_all_os_profiles_have_required_sysctls(family: str):
|
||||
"""Every OS profile must define the full canonical sysctl set."""
|
||||
from decnet.os_fingerprint import _REQUIRED_SYSCTLS
|
||||
@@ -246,7 +258,7 @@ def test_all_os_profiles_have_required_sysctls(family: str):
|
||||
assert not missing, f"OS profile '{family}' is missing sysctls: {missing}"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("family", ["linux", "windows", "bsd", "embedded", "cisco"])
|
||||
@pytest.mark.parametrize("family", ["linux", "windows", "windows_server", "bsd", "embedded", "cisco"])
|
||||
def test_all_os_sysctl_values_are_strings(family: str):
|
||||
"""Docker Compose requires sysctl values to be strings, never ints."""
|
||||
for _key, _val in get_os_sysctls(family).items():
|
||||
@@ -267,9 +279,13 @@ def test_archetype_nmap_os_is_known(slug, arch):
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("slug", ["windows-workstation", "windows-server", "domain-controller"])
|
||||
def test_windows_archetypes_have_windows_nmap_os(slug):
|
||||
assert ARCHETYPES[slug].nmap_os == "windows"
|
||||
def test_windows_workstation_archetype_nmap_os():
|
||||
assert ARCHETYPES["windows-workstation"].nmap_os == "windows"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("slug", ["windows-server", "domain-controller"])
|
||||
def test_windows_server_archetypes_use_server_nmap_os(slug):
|
||||
assert ARCHETYPES[slug].nmap_os == "windows_server"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("slug", ["printer", "iot-device", "industrial-control"])
|
||||
@@ -403,11 +419,11 @@ def test_compose_linux_sysctls_include_timestamps():
|
||||
assert sysctls.get("net.ipv4.tcp_timestamps") == "1"
|
||||
|
||||
|
||||
def test_compose_windows_sysctls_no_timestamps():
|
||||
"""Windows compose output must have tcp_timestamps disabled (= 0)."""
|
||||
def test_compose_windows_sysctls_timestamps_on():
|
||||
"""Windows compose output must have tcp_timestamps ENABLED (= 1) — Win10/11."""
|
||||
compose = generate_compose(_make_config("windows"))
|
||||
sysctls = compose["services"]["decky-01"]["sysctls"]
|
||||
assert sysctls.get("net.ipv4.tcp_timestamps") == "0"
|
||||
assert sysctls.get("net.ipv4.tcp_timestamps") == "1"
|
||||
|
||||
|
||||
def test_compose_linux_sysctls_full_set():
|
||||
|
||||
@@ -1,152 +0,0 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""Smoke tests for the orchestrator systemd unit + decnet.target wiring.
|
||||
|
||||
These don't exercise systemd (the test host wouldn't have it); they
|
||||
just assert the static contents of ``deploy/decnet-orchestrator.service.j2``
|
||||
and ``deploy/decnet.target`` match what ``decnet init`` will install.
|
||||
|
||||
Anti-regressions for two specific failure modes:
|
||||
|
||||
1. After the realism migration (stage 5), ``decnet-emailgen.service``
|
||||
is gone — the orchestrator covers the email branch. A regression
|
||||
that re-introduces the emailgen unit file or the ``decnet.target``
|
||||
entry would only surface on a fresh host install; cheap to catch
|
||||
here.
|
||||
2. The orchestrator unit must ship the ``DECNET_REALISM_*`` env block
|
||||
so the LLM enrichment + persona-pool path are configurable per
|
||||
host without editing the .j2.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent.parent
|
||||
DEPLOY = REPO / "deploy"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def unit_text() -> str:
|
||||
return (DEPLOY / "decnet-orchestrator.service.j2").read_text()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def target_text() -> str:
|
||||
return (DEPLOY / "decnet.target").read_text()
|
||||
|
||||
|
||||
# ── orchestrator unit ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_orchestrator_unit_exists():
|
||||
assert (DEPLOY / "decnet-orchestrator.service.j2").exists()
|
||||
|
||||
|
||||
def test_orchestrator_unit_uses_orchestrate_subcommand(unit_text):
|
||||
assert "decnet orchestrate" in unit_text
|
||||
|
||||
|
||||
def test_orchestrator_unit_has_docker_supplementary_group(unit_text):
|
||||
"""SSHDriver shells `docker exec` against decky containers — without
|
||||
this group the worker can't reach the docker socket."""
|
||||
assert "SupplementaryGroups=docker" in unit_text
|
||||
|
||||
|
||||
def test_orchestrator_unit_orders_after_bus(unit_text):
|
||||
"""Bus must be up first so heartbeats publish from the start."""
|
||||
assert "After=network-online.target decnet-bus.service" in unit_text
|
||||
assert "Wants=network-online.target decnet-bus.service" in unit_text
|
||||
|
||||
|
||||
def test_orchestrator_unit_has_security_hardening(unit_text):
|
||||
for directive in (
|
||||
"NoNewPrivileges=yes",
|
||||
"ProtectSystem=full",
|
||||
"ProtectHome=read-only",
|
||||
"PrivateTmp=yes",
|
||||
"ProtectKernelTunables=yes",
|
||||
"ProtectKernelModules=yes",
|
||||
"ProtectControlGroups=yes",
|
||||
"RestrictSUIDSGID=yes",
|
||||
"LockPersonality=yes",
|
||||
):
|
||||
assert directive in unit_text, f"missing {directive}"
|
||||
|
||||
|
||||
def test_orchestrator_unit_writes_to_log_dir(unit_text):
|
||||
assert "/var/log/decnet/decnet.orchestrator.log" in unit_text
|
||||
assert "ReadWritePaths={{ install_dir }} /var/log/decnet" in unit_text
|
||||
|
||||
|
||||
def test_orchestrator_unit_restart_on_failure(unit_text):
|
||||
assert "Restart=on-failure" in unit_text
|
||||
|
||||
|
||||
def test_orchestrator_unit_carries_realism_env_block(unit_text):
|
||||
"""Stage 5 + 6 contract: the orchestrator's LLM enrichment and
|
||||
persona-pool path are configured per host via DECNET_REALISM_*
|
||||
env vars. Shipping them in the .j2 means an operator who never
|
||||
drops a .env.local still gets sane defaults."""
|
||||
for var in (
|
||||
"DECNET_REALISM_LLM",
|
||||
"DECNET_REALISM_MODEL",
|
||||
"DECNET_REALISM_TIMEOUT",
|
||||
"DECNET_REALISM_PERSONAS",
|
||||
):
|
||||
assert var in unit_text, f"missing {var} in unit"
|
||||
|
||||
|
||||
def test_orchestrator_unit_does_not_carry_legacy_emailgen_envs(unit_text):
|
||||
"""Pre-v1 clean break per the realism migration: the
|
||||
DECNET_EMAILGEN_* env vars are no longer read. Carrying them in
|
||||
the unit would mislead operators into thinking they still apply."""
|
||||
for legacy in (
|
||||
"DECNET_EMAILGEN_LLM",
|
||||
"DECNET_EMAILGEN_MODEL",
|
||||
"DECNET_EMAILGEN_TIMEOUT",
|
||||
"DECNET_EMAILGEN_PERSONAS",
|
||||
):
|
||||
assert legacy not in unit_text, (
|
||||
f"legacy env {legacy} still referenced; clean-break broken"
|
||||
)
|
||||
|
||||
|
||||
# ── decnet.target ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_target_wants_orchestrator(target_text):
|
||||
assert "decnet-orchestrator.service" in target_text
|
||||
|
||||
|
||||
def test_target_does_not_want_emailgen(target_text):
|
||||
"""Stage 5 of the realism migration deleted decnet-emailgen.service.
|
||||
A fresh `decnet init` against a target file that still mentions it
|
||||
fails systemctl start with `Unit decnet-emailgen.service could not
|
||||
be found`, blocking the whole target. Anti-regression."""
|
||||
assert "decnet-emailgen.service" not in target_text
|
||||
|
||||
|
||||
def test_target_wants_canary(target_text):
|
||||
"""Canary worker is a peer of orchestrator; both are part of the
|
||||
realism + callback story. Bundle check."""
|
||||
assert "decnet-canary.service" in target_text
|
||||
|
||||
|
||||
def test_target_orders_after_bus(target_text):
|
||||
"""Whole target depends on the bus being up."""
|
||||
assert "After=decnet-bus.service" in target_text
|
||||
|
||||
|
||||
# ── unit file no longer exists ───────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_emailgen_unit_template_is_gone():
|
||||
"""The pre-collapse ``deploy/decnet-emailgen.service.j2`` must stay
|
||||
deleted. A future commit that re-creates it (e.g. by accident
|
||||
during a partial revert) would break the realism migration's
|
||||
service-collapse contract."""
|
||||
assert not (DEPLOY / "decnet-emailgen.service.j2").exists(), (
|
||||
"decnet-emailgen.service.j2 reappeared — service collapse undone?"
|
||||
)
|
||||
55
tests/prefork_driver.py
Normal file
55
tests/prefork_driver.py
Normal file
@@ -0,0 +1,55 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""Standalone driver for the prefork supervisor — runnable directly OR via
|
||||
tests/test_prefork.py (which execs it in a subprocess so no fork happens inside
|
||||
the pytest/xdist worker).
|
||||
|
||||
python tests/prefork_driver.py <out_dir>
|
||||
|
||||
Forks two fake workers under decnet.prefork.run_fleet:
|
||||
* "tick" — append a line every 0.2s forever (proves a worker runs & stays up)
|
||||
* "crasher" — write a marker then exit(1) (proves restart-on-crash)
|
||||
Runs for ~2s via stop_after, then shuts the fleet down. Writes results into
|
||||
<out_dir>; the caller asserts on them.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
# Running this file as a script puts its own dir (tests/) on sys.path[0], which
|
||||
# shadows the stdlib `logging` via tests/logging/. Drop it before importing
|
||||
# decnet (still importable — it's installed in the venv).
|
||||
if sys.path and os.path.basename(sys.path[0]) == "tests":
|
||||
sys.path.pop(0)
|
||||
|
||||
from decnet.prefork import run_fleet # noqa: E402
|
||||
|
||||
|
||||
def main(out: str) -> None:
|
||||
tick_log = os.path.join(out, "tick.log")
|
||||
crash_log = os.path.join(out, "crash.log")
|
||||
|
||||
def tick() -> None:
|
||||
while True:
|
||||
with open(tick_log, "a") as f:
|
||||
f.write("t\n")
|
||||
time.sleep(0.2)
|
||||
|
||||
def crasher() -> None:
|
||||
with open(crash_log, "a") as f:
|
||||
f.write("c\n")
|
||||
time.sleep(0.15)
|
||||
os._exit(1)
|
||||
|
||||
# Fast backoff so we observe multiple restarts inside the short window.
|
||||
run_fleet(
|
||||
{"tick": tick, "crasher": crasher},
|
||||
max_backoff=0.2,
|
||||
poll_interval=0.05,
|
||||
stop_after=2.0,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(sys.argv[1] if len(sys.argv) > 1 else ".")
|
||||
32
tests/test_paths.py
Normal file
32
tests/test_paths.py
Normal file
@@ -0,0 +1,32 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""Tests for the shared runtime-path probe."""
|
||||
from __future__ import annotations
|
||||
|
||||
from decnet.paths import resolve_runtime_path
|
||||
|
||||
|
||||
def _resolve(tmp_path, env=None, runtime_dir=None):
|
||||
return resolve_runtime_path(
|
||||
"x.sock",
|
||||
env_var="DECNET_TEST_PATH",
|
||||
runtime_dir=str(runtime_dir if runtime_dir is not None else tmp_path),
|
||||
user_fallback="~/.decnet/x.sock",
|
||||
)
|
||||
|
||||
|
||||
def test_env_override_wins(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("DECNET_TEST_PATH", "/explicit/here.sock")
|
||||
assert _resolve(tmp_path) == "/explicit/here.sock"
|
||||
|
||||
|
||||
def test_writable_runtime_dir(tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("DECNET_TEST_PATH", raising=False)
|
||||
assert _resolve(tmp_path) == str(tmp_path / "x.sock")
|
||||
|
||||
|
||||
def test_falls_back_when_runtime_dir_absent(tmp_path, monkeypatch):
|
||||
monkeypatch.delenv("DECNET_TEST_PATH", raising=False)
|
||||
missing = tmp_path / "nope" # does not exist → not a writable dir
|
||||
result = _resolve(tmp_path, runtime_dir=missing)
|
||||
assert result.endswith("/.decnet/x.sock")
|
||||
assert "~" not in result # tilde expanded
|
||||
40
tests/test_prefork.py
Normal file
40
tests/test_prefork.py
Normal file
@@ -0,0 +1,40 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""Prefork supervisor behaviour, exercised via a subprocess driver so no fork
|
||||
happens inside the pytest/xdist worker (which would be unsafe).
|
||||
|
||||
Proves: workers fork and run, a crashing worker is restarted with backoff, and
|
||||
the fleet shuts down cleanly (stop_after returns, no orphaned children).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
def test_prefork_runs_and_restarts(tmp_path: pathlib.Path):
|
||||
driver = pathlib.Path(__file__).parent / "prefork_driver.py"
|
||||
proc = subprocess.run(
|
||||
[sys.executable, str(driver), str(tmp_path)],
|
||||
capture_output=True, text=True, timeout=30,
|
||||
)
|
||||
assert proc.returncode == 0, f"driver failed:\n{proc.stderr}"
|
||||
|
||||
tick = (tmp_path / "tick.log").read_text().splitlines()
|
||||
crash = (tmp_path / "crash.log").read_text().splitlines()
|
||||
|
||||
# tick ran continuously for ~2s at 0.2s cadence → several lines.
|
||||
assert len(tick) >= 5, f"tick worker did not stay up: {len(tick)} lines"
|
||||
# crasher died fast and was restarted repeatedly → many markers.
|
||||
assert len(crash) >= 3, f"crasher was not restarted: {len(crash)} markers"
|
||||
|
||||
|
||||
def test_empty_fleet_returns(tmp_path: pathlib.Path):
|
||||
# run_fleet([]) must be a no-op, not hang.
|
||||
code = (
|
||||
"from decnet.prefork import run_fleet; run_fleet({}, stop_after=5)"
|
||||
)
|
||||
proc = subprocess.run(
|
||||
[sys.executable, "-c", code], capture_output=True, text=True, timeout=15
|
||||
)
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
@@ -51,7 +51,7 @@ async def repo(tmp_path):
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_dry_run_writes_compose_and_preserves_pending(repo, tmp_path, monkeypatch):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setenv("DECNET_RUN_DIR", str(tmp_path))
|
||||
plan = generate(_cfg())
|
||||
tid = await persist(repo, plan)
|
||||
|
||||
@@ -235,3 +235,22 @@ async def test_deploy_and_teardown_against_real_docker(repo, tmp_path, monkeypat
|
||||
p.unlink()
|
||||
# Sanity: Path roundtrip still resolvable
|
||||
assert isinstance(Path(str(p)), Path)
|
||||
|
||||
|
||||
def test_compose_path_is_absolute_and_cwd_independent(tmp_path, monkeypatch):
|
||||
"""Regression: a CWD-relative compose path littered the install dir and
|
||||
let teardown's unlink() miss orphans. Path must be absolute and stable
|
||||
across CWD changes so write and teardown always agree."""
|
||||
monkeypatch.setenv("DECNET_RUN_DIR", str(tmp_path))
|
||||
tid = "abcdef1234567890"
|
||||
|
||||
monkeypatch.chdir(tmp_path)
|
||||
p1 = _topology_compose_path(tid)
|
||||
sub = tmp_path / "elsewhere"
|
||||
sub.mkdir()
|
||||
monkeypatch.chdir(sub)
|
||||
p2 = _topology_compose_path(tid)
|
||||
|
||||
assert p1.is_absolute()
|
||||
assert p1 == p2, "compose path must not depend on process CWD"
|
||||
assert p1.parent == tmp_path
|
||||
|
||||
@@ -20,7 +20,7 @@ from decnet.clustering import ukc
|
||||
from decnet.ttp import attack_stix
|
||||
from decnet.ttp.impl import intel_lifter
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
|
||||
@@ -19,7 +19,7 @@ from decnet.ttp import attack_stix
|
||||
from decnet.ttp.attack_catalog import technique_name
|
||||
|
||||
_RULES_DIR = Path(__file__).resolve().parents[2] / "rules" / "ttp"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
|
||||
|
||||
@pytest.fixture(scope="module", autouse=True)
|
||||
|
||||
@@ -19,7 +19,7 @@ from decnet.ttp.attack_version import (
|
||||
ATTACK_LICENSE_SHA256,
|
||||
)
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
|
||||
@@ -15,7 +15,7 @@ import pytest
|
||||
|
||||
from decnet.ttp import attack_stix
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
|
||||
@@ -24,7 +24,7 @@ from decnet.ttp.impl._emit import emit_tags
|
||||
from decnet.ttp.impl.rule_engine import CompiledRule
|
||||
from decnet.ttp.store.base import RuleState
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
|
||||
@@ -19,7 +19,7 @@ from decnet.web.router.ttp.api_get_groups_for_technique import (
|
||||
api_groups_for_technique,
|
||||
)
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
|
||||
@@ -31,7 +31,7 @@ from decnet.ttp.data.intel_loader import (
|
||||
load_provider_mapping,
|
||||
)
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
_DATA_DIR = Path(__file__).resolve().parents[2] / "decnet" / "ttp" / "data" / "intel"
|
||||
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from decnet.web.router.attackers.api_export_attacker_misp import (
|
||||
api_export_attacker_misp,
|
||||
)
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
_FAKE_USER: dict = {"uuid": "test-user", "role": "viewer"}
|
||||
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"""Tests for GET /api/v1/attackers/{uuid}/export/stix.
|
||||
|
||||
Tests call the handler directly (no TestClient). The attack_stix bundle
|
||||
is pinned to the repo's enterprise-attack-19.0.json so Sighting and
|
||||
is pinned to the repo's decnet/data/enterprise-attack-19.1.json so Sighting and
|
||||
Relationship target_refs are real MITRE STIX IDs.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
@@ -20,7 +20,7 @@ from decnet.web.router.attackers.api_export_attacker_stix import (
|
||||
api_export_attacker_stix,
|
||||
)
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
_FAKE_USER: dict = {"uuid": "test-user", "role": "viewer"}
|
||||
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from decnet.web.router.attackers.api_export_attackers_misp import (
|
||||
api_export_attackers_misp,
|
||||
)
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
_FAKE_USER: dict = {"uuid": "test-user", "role": "viewer"}
|
||||
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ from decnet.web.router.attackers.api_export_attackers_stix import (
|
||||
api_export_attackers_stix,
|
||||
)
|
||||
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "enterprise-attack-19.0.json"
|
||||
_REPO_BUNDLE = Path(__file__).resolve().parents[2] / "decnet" / "data" / "enterprise-attack-19.1.json"
|
||||
_FAKE_USER: dict = {"uuid": "test-user", "role": "viewer"}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user