Mangler now also rewrites egress RST (IP-ID + nonzero ack on bare RSTs → nmap
CI, T4/T6 A=O) and ICMP echo-reply (code=0 → IE.CD=Z), sharing one IP-ID counter
across SYN-ACK/RST/ICMP (reads as a shared sequence, SS=S). Responder injects at
L2 (reflecting probe MACs) so its own RST replies bypass the OUTPUT/NFQUEUE chain
— otherwise the new RST rule re-processed and dropped them. T3 reply ack now A=O.
Live: windows_server decky reads Microsoft Windows Server 2012 (94%, up from 89%);
T2/T3 R=Y, IE.CD=Z, T4/T6 A=O all confirmed coexisting.
OS fingerprint cloak — sysctl profile fixes (Win timestamps), windows_server
slug, NFQUEUE SYN-ACK mangler + T2/T3 probe-response synthesizer, wired into the
deploy path. Flips real nmap -O to Microsoft Windows / Windows Server.
Base containers whose nmap_os has a mangle profile now build the cloak image
(FROM the per-decky distro), ship the light decnet subtree, and run
'python -m decnet.cloak' alongside holding the MACVLAN IP — netns-safe (cloak
backgrounded behind 'exec sleep infinity' so a cloak crash never tears down the
base/netns). composer injects build/command/NET_RAW/env (DECNET_NMAP_OS,
DECNET_OPEN_PORTS, DECKY_IP); deployer._sync_cloak_sources syncs the subtree;
non-windows deckies are unchanged. Mangler signal-guarded for thread use;
entry runs mangler in main thread, responder as daemon.
Verified live: real path makes nmap -O read 'Microsoft Windows Server 2012/2016'
with handshakes intact.
In-decky-netns NFQUEUE rewriter (window/option-order/IP-ID) and raw-socket
synthesizer for nmap probes Linux drops but the target OS answers (T2/T3),
driven by os_fingerprint.OS_MANGLE. Packet-shaping logic is pure and unit-tested
offline; scapy/netfilterqueue import lazily in the runtime loops. Entry:
python -m decnet.cloak (run by the base container; CAP_NET_ADMIN).
Win10/11 run TCP timestamps ON (nmap SEQ.TS=A); the windows profile had them
OFF, fingerprinting as an ancient stack. Add a windows_server slug (ECN
negotiated, CC=Y) and point the server/DC archetypes at it. Introduce the
OS_MANGLE map (per-slug egress SYN-ACK shape: window, option order, IP-ID
policy) consumed by the new cloak package.
bus.factory and vectorstore.factory carried byte-identical copies of the
'env override -> writable runtime dir -> ~/.decnet fallback' probe. Move
it to decnet.paths.resolve_runtime_path and call it from both.
The mkdir-create variants (deployer topologies dir, _pid_dir candidate
iteration, personas_pool existence-precedence) are deliberately left
inline: they're different policies, not the same probe.
_topology_compose_path returned a CWD-relative Path, so every
deploy/mutate/dry-run wrote decnet-topology-<id8>-compose.yml into the
process CWD (the install dir). Teardown computed the same relative path
against its own CWD, so when it differed the unlink() missed the orphan
and files accumulated forever.
Anchor to $DECNET_RUN_DIR (default /var/lib/decnet/topologies, tempdir
fallback) so write and teardown always agree regardless of CWD.
Adds the @pro ScanImport contract (ProScanImportProps/ProScanImport) and
a null community stub, and slots a third SCAN-BASED card into
CreateTopologyWizard, gated on the pro panel being present so it
tree-shakes out of the community build. The scan->topology importer
itself ships in decnet/pro v1.2.0. CHANGELOG updated under [1.2.0].
Prefork supervisor (decnet.prefork) + 'decnet fleet heavy' (profiler+ttp,
CoW-shared, ~412MB Pss vs 661MB). ATT&CK bundle -> decnet/data/ (19.1).
Removed 10 per-worker unit templates superseded by the supervisor groups
and the heavy fleet.
Prefork worker consolidation (decnet.prefork + decnet fleet heavy),
ATT&CK 19.1 relocation to decnet/data/, and removal of the 10 per-worker
unit templates superseded by the supervisor groups + heavy fleet.
The batch/cpu supervisor groups + heavy fleet replace 10 per-worker units
(reconciler/enrich/orchestrator/mutator/clusterer/campaign-clusterer/
attribution/reuse-correlator/profiler/ttp). Removed their deploy/*.service.j2
templates and rewired decnet.target to the 3 consolidated units. Dropped
test_orchestrator_unit.py (tested a removed unit). CLI commands (decnet ttp,
mutate, …) stay for manual runs; new units' Conflicts= still name the old
units defensively for hosts mid-migration.
Wires the prefork primitive into a CLI command. 'decnet fleet heavy' imports
the shared base floor once in the master, then forks profiler + ttp as
CoW-sharing child processes (own process/GIL, full isolation, shared ~71MB
floor). DB-only tier => systemd unit carries no extra privilege (prefork's
privilege-union cost is nil for this fleet). Unit Conflicts= the profiler/ttp
units it replaces. Heavy per-worker state (ATT&CK/ML) still loads per-child;
warming it in the master to share is deferred until a live RSS measurement
shows the big object graph CoW-shares rather than refcount-dirties.
Bundle pointer moved from repo root to decnet/data/ (with LICENSE.txt),
gitignored + fetched on demand (51MB, MITRE-licensed). Version pin bumped
19.0->19.1 with the new sha256; license unchanged. All _REPO_BUNDLE test
constants repointed. Fixes test-web failures after the repo-root bundle
was deleted.
CoW measurement on CPython 3.14: forked idle child keeps ~71MB shared,
dirties ~1MB private; working child ~26MB. PEP 683 immortal objects keep
code/module pages clean so gc.freeze() is unnecessary (freeze==nofreeze).
prefork.run_fleet: master imports the base floor once, forks one child
per worker (own process/GIL, CoW-shared floor), reaps + restarts with
backoff, graceful SIGTERM->SIGKILL shutdown. Not yet wired to a command
(that lands when 1.2 picks the target worker set).
confidence_max is a ceiling (min(base, ceiling)), not a multiplier — the
ASVS pass fixed this (BUG-8: min(base, base*ceiling) -> min(base, ceiling)),
but 4 lifter clip tests still encoded the old base*ceiling math (0.45/0.4/
0.35) and were masked by the make test-web bundle error fail-fast. All four
now assert the 0.5 ceiling. Separately, test_topics_matches_documented_set
lacked attacker.fingerprinted, which worker.py legitimately subscribes to
(JARM/HASSH/tcpfp/ipv6_leak -> TTP tagging). Located via turbovec + git pickaxe.
(cherry picked from commit f83b467c35649a06fa36f4b350e6666379cd71cb)
Hosts clusterer/campaign-clusterer/attribution/reuse-correlate in one
process. The two O(n^2) connected-components kernels (cluster_observations,
cluster_identities) offload to ONE shared forkserver pool via decnet.offload
.run_kernel, so they run in parallel instead of serialising under the GIL.
- offload.run_kernel: pool when installed + offload_if holds, else inline.
Standalone workers and all tests run inline => behaviour unchanged
(424 clustering/correlation tests green).
- offload_if gates on input size (>=256) to skip pickle cost on small passes.
- forkserver (not fork): supervisor is multithreaded via bus clients.
- attribution/reuse co-located but not offloaded yet (lighter; same run_kernel
path extends to them if profiling shows contention).
- systemd unit Conflicts= the 4 units it replaces; no docker/raw-socket priv.
Controlled unit swap on mothership: single PID hosts all 4 workers,
shared repo + 30s reconcile pass confirmed working, no crashes. Live
delta beat the floor estimate. Batch group complete.
enrich is a batch-group member; its individual unit must also be mutually
exclusive with the supervisor. Unit auto-renders via init.py glob of
deploy/decnet-*.service.j2 — no installer list change needed.
4 live batch workers = 509MB; consolidated startup floor = 118.5MB
(imports 102.5 + repo pool 15.7 once vs 4x + bus 0.3). Side-effect-free
measurement (no mutation tick). Floor is a lower bound; live adds modest
per-worker working set. Exact live number pending controlled unit swap.
Hosts reconcile/enrich/orchestrate/mutate in one process via the
supervision primitive: one import floor, one shared repo/DB pool instead
of 4. Static group registry (membership is architectural, not a knob);
factories lazy-import only the hosted workers. systemd unit Conflicts=
the individual units it replaces and documents the union-of-privileges
cost. Worker code unchanged — any member is extractable by editing _build_specs.
forwarder/listener are role-split swarm singletons, not co-resident with
the herd — drop from grouping. Real master-resident batch group shares
the repo singleton (one DB pool when consolidated). Stage 1 = batch:
reconcile/enrich/orchestrate/mutate. webhook/canary deferred standalone.
HOW to consolidate: supervision-loop primitive (not TaskGroup, whose
all-or-nothing cancel breaks isolation); group by failure domain +
resource profile keeping per-group cgroup limits; every worker remains
config-extractable. Recommend process-groups now (~18->~9 units),
evaluate prefork+gc.freeze CoW on 3.14 as the higher-ceiling follow-on.
Measurement after C2: all 25 CLI modules transitively pull the SQLModel
ORM, and the table chain is a hub (one table loads the whole registry).
Most workers genuinely use the DB, so lazy imports only help the 2-3
DB-less workers. Consolidation (pay the 86MB floor once for the herd)
is the reliable ~600MB win — promoted from fallback to primary.
topology/__init__ eagerly imported generator -> allocator -> repository ->
the full SQLModel ORM. Defer via PEP 562 __getattr__ so importing the
package doesn't drag the DB layer into DB-less workers. Public API
(from decnet.topology import generate) unchanged. Guard test locks it in.
Fleet resident set ~2.57GB across 18 workers; ~1.5GB is the 86MB import
floor paid 18x. Pinned root cause: topology/__init__ eager re-export of
generate drags the full SQLModel ORM (26 tables, ~38MB) into every worker.
The core CLI scans decnet/pro/cli/ and calls each module's register(app),
registered before the master-only gate so pro commands are mode-filtered like
the rest. Lets the Professional tier add commands and standalone daemon entry
points (decnet pro-<cmd> serve, supervised by a systemd unit). No-op in the
Community build (no decnet.pro). Test asserts the shipped pro group registers
when mounted; skips otherwise.
Move the pro mount decnet/services/pro/ -> decnet/pro/ so the Professional tier
can contribute to more than honeypots. The core wires each surface only when
decnet/pro/ is present (absence stays the entitlement gate):
* services — registry scans decnet/pro/services/ (was decnet/services/pro/)
* API routes — decnet/pro/routes.py exposes ROUTERS, mounted under /api/v1
* web pages — Vite aliases @pro to the pro frontend (community -> empty stub),
App.tsx maps proRoutes into <Route>s, Layout renders a
PROFESSIONAL nav group; both tree-shake out of the community build
Frontend gate mirrors the existing VITE_DECNET_DEVELOPER tree-shake pattern.
Tests: registry + router seams (backend), empty-stub contract (frontend).
Newer pytest raises 'Marks cannot be applied to fixtures' instead of ignoring
it. The async test methods already carry @pytest.mark.anyio, which is what
selects the backend; the fixture must not.
Pro-tier honeypots load from an optional decnet/services/pro/ subpackage that
the registry auto-discovers when present; the Community build omits it, so the
directory's absence IS the entitlement gate (no runtime licence check). Recurse
subclasses so a pro service may extend a community one. Exclude pro from the
community wheel and git-ignore the path (it lives in the private
decnet-professional repo).
Add LICENSING.md documenting the dual-license: AGPL-3.0-or-later core plus a
commercial EULA for the Professional tier.
