3 Commits
dev ... testing

Author SHA1 Message Date
DECNET CI
b12d46ff9d ci: auto-merge dev → testing
Some checks failed
CI / Lint (ruff) (push) Has been skipped
CI / SAST (bandit) (push) Has been skipped
CI / Dependency audit (pip-audit) (push) Has been skipped
CI / Merge dev → testing (push) Has been skipped
CI / Test (Standard) (3.11) (push) Successful in 13m19s
CI / Test (Live) (3.11) (push) Successful in 1m17s
CI / Merge testing → main (push) Failing after 11s
2026-04-28 22:17:36 +00:00
DECNET CI
2ce076cd37 ci: auto-merge dev → testing [skip ci] 2026-04-28 22:03:20 +00:00
DECNET CI
e8d97281f7 ci: auto-merge dev → testing [skip ci] 2026-04-20 20:39:35 +00:00
1719 changed files with 10406 additions and 107100 deletions

20
.gitignore vendored
View File

@@ -1,6 +1,5 @@
.venv/ .venv/
.venv*/ .venv*/
docker-compose.yaml
.311/ .311/
.3[0-9][0-9]/ .3[0-9][0-9]/
logs/ logs/
@@ -52,22 +51,3 @@ schem
# pydeps-style dependency graph dumps from local analysis runs. # pydeps-style dependency graph dumps from local analysis runs.
deps.txt deps.txt
# Node modules vendored under decnet/canary/ for the obfuscator helper.
# The package.json is the source of truth; modules are reinstalled at
# build/deploy time.
node_modules/
package-lock.json
# TTP rule-precision corpus pulled from prod sqlite. Real attacker
# payloads — operator-only artifact. The synthetic ``seed_*.jsonl``
# files alongside ARE committed and exercise the harness in CI.
tests/ttp/rule_precision/corpus/*.jsonl
tests/ttp/rule_precision/corpus/seed_*.jsonl
threatfox-api.json
# MITRE ATT&CK STIX bundle — 50 MB, fetched at runtime via attack_stix.py
enterprise-attack-*.json
# pytest failure dump files
testfail

View File

@@ -1,17 +0,0 @@
DECNET - Deception Network
Copyright (C) 2026 Samuel Paschuan <samsam70000@gmail.com>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public
License along with this program. If not, see <https://www.gnu.org/licenses/>.
SPDX-License-Identifier: AGPL-3.0-or-later

141
LICENSE
View File

@@ -1,5 +1,5 @@
GNU AFFERO GENERAL PUBLIC LICENSE GNU GENERAL PUBLIC LICENSE
Version 3, 19 November 2007 Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/> Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies Everyone is permitted to copy and distribute verbatim copies
@@ -7,15 +7,17 @@
Preamble Preamble
The GNU Affero General Public License is a free, copyleft license for The GNU General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure software and other kinds of works.
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast, to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free share and change all versions of a program--to make sure it remains free
software for all its users. software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you price. Our General Public Licenses are designed to make sure that you
@@ -24,34 +26,44 @@ them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things. free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights To protect your rights, we need to prevent others from denying you
with two steps: (1) assert copyright on the software, and (2) offer these rights or asking you to surrender the rights. Therefore, you have
you this License which gives you legal permission to copy, distribute certain responsibilities if you distribute copies of the software, or if
and/or modify the software. you modify it: responsibilities to respect the freedom of others.
A secondary benefit of defending all users' freedom is that For example, if you distribute copies of such a program, whether
improvements made in alternate versions of the program, if they gratis or for a fee, you must pass on to the recipients the same
receive widespread use, become available for other developers to freedoms that you received. You must make sure that they, too, receive
incorporate. Many developers of free software are heartened and or can get the source code. And you must show them these terms so they
encouraged by the resulting cooperation. However, in the case of know their rights.
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to Developers that use the GNU GPL protect your rights with two steps:
ensure that, in such cases, the modified source code becomes available (1) assert copyright on the software, and (2) offer you this License
to the community. It requires the operator of a network server to giving you legal permission to copy, distribute and/or modify it.
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and For the developers' and authors' protection, the GPL clearly explains
published by Affero, was designed to accomplish similar goals. This is that there is no warranty for this free software. For both users' and
a different license, not a version of the Affero GPL, but Affero has authors' sake, the GPL requires that modified versions be marked as
released a new version of the Affero GPL which permits relicensing under changed, so that their problems will not be attributed erroneously to
this license. authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and The precise terms and conditions for copying, distribution and
modification follow. modification follow.
@@ -60,7 +72,7 @@ modification follow.
0. Definitions. 0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License. "This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of "Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks. works, such as semiconductor masks.
@@ -537,45 +549,35 @@ to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program. License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License. 13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work, License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version but the special requirements of the GNU Affero General Public License,
3 of the GNU General Public License. section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License. 14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions the GNU General Public License from time to time. Such new versions will
will be similar in spirit to the present version, but may differ in detail to be similar in spirit to the present version, but may differ in detail to
address new problems or concerns. address new problems or concerns.
Each version is given a distinguishing version number. If the Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published GNU General Public License, you may choose any version ever published
by the Free Software Foundation. by the Free Software Foundation.
If the Program specifies that a proxy can decide which future If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you public statement of acceptance of a version permanently authorizes you
to choose that version for the Program. to choose that version for the Program.
@@ -633,29 +635,40 @@ the "copyright" line and a pointer to where the full notice is found.
Copyright (C) <year> <name of author> Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or the Free Software Foundation, either version 3 of the License, or
(at your option) any later version. (at your option) any later version.
This program is distributed in the hope that it will be useful, This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details. GNU General Public License for more details.
You should have received a copy of the GNU Affero General Public License You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>. along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail. Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer If the program does terminal interaction, make it output a short
network, you should also make sure that it provides a way for users to notice like this when it starts in an interactive mode:
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive <program> Copyright (C) <year> <name of author>
of the code. There are many ways you could offer source, and different This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
solutions will be better for different programs; see section 13 for the This is free software, and you are welcome to redistribute it
specific requirements. under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school, You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary. if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see For more information on this, and how to apply and follow the GNU GPL, see
<https://www.gnu.org/licenses/>. <https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<https://www.gnu.org/licenses/why-not-lgpl.html>.

261
Makefile
View File

@@ -1,261 +0,0 @@
PYTEST := .311/bin/pytest
FAIL_FAST ?= 1
NO_CACHE ?= 0
ARGS :=
# addopts in pyproject.toml already provides -v -q -x -n 4 --dist load.
# Unit suites inherit that; special suites clear it with --override-ini.
UNIT_FLAGS := --timeout=30 --timeout-method=thread
SEQ_FLAGS := --override-ini="addopts=-v -x" -n logical --timeout=120 --timeout-method=thread
FUZZ_FLAGS := --override-ini="addopts=-v -x" -n logical -m fuzz \
--ignore=tests/api/test_schemathesis.py \
--ignore=tests/api/test_schemathesis_agent.py \
--ignore=tests/api/test_schemathesis_swarm.py \
--ignore=tests/api/test_schemathesis_ttp.py
SCHEMA_QUICK ?= 0
SCHEMA_FLAGS := --override-ini="addopts=-v -x" -n 4 -m fuzz --timeout=600 --timeout-method=thread
BENCH_FLAGS := --override-ini="addopts=-v" -p no:xdist --benchmark-only -m bench
# ── Unit suites (xdist, 30s timeout) ─────────────────────────────────────────
.PHONY: test-core
test-core:
$(PYTEST) tests/core tests/config tests/factories tests/fixtures $(UNIT_FLAGS) $(ARGS)
.PHONY: test-web
test-web:
$(PYTEST) tests/web tests/services $(UNIT_FLAGS) $(ARGS)
.PHONY: test-db
test-db:
$(PYTEST) tests/db tests/vectorstore $(UNIT_FLAGS) $(ARGS)
.PHONY: test-bus
test-bus:
$(PYTEST) tests/bus tests/logging tests/telemetry $(UNIT_FLAGS) $(ARGS)
.PHONY: test-ttp
test-ttp:
$(PYTEST) tests/ttp $(UNIT_FLAGS) $(ARGS)
.PHONY: test-intel
test-intel:
$(PYTEST) tests/intel tests/asn tests/geoip $(UNIT_FLAGS) $(ARGS)
.PHONY: test-analysis
test-analysis:
$(PYTEST) tests/clustering tests/correlation $(UNIT_FLAGS) $(ARGS)
.PHONY: test-infra
test-infra:
$(PYTEST) tests/agent tests/collector tests/sniffer tests/profiler $(UNIT_FLAGS) $(ARGS)
.PHONY: test-fleet
test-fleet:
$(PYTEST) tests/fleet tests/swarm tests/topology tests/orchestrator tests/deploy tests/updater $(UNIT_FLAGS) $(ARGS)
.PHONY: test-cli
test-cli:
$(PYTEST) tests/cli tests/engine tests/mutator tests/realism $(UNIT_FLAGS) $(ARGS)
.PHONY: test-features
test-features:
$(PYTEST) tests/canary tests/artifacts tests/webhook tests/decky_io tests/prober $(UNIT_FLAGS) $(ARGS)
# ── Go and React suites ───────────────────────────────────────────────────────
_GO_MODULES := \
decnet/templates/_caddy_modules/decnetfp \
decnet/templates/http/_caddy_modules/decnetfp \
decnet/templates/https/_caddy_modules/decnetfp
.PHONY: test-go
test-go:
@failed=""; \
for mod in $(_GO_MODULES); do \
echo "=== go test: $$mod ==="; \
if (cd "$$mod" && go test ./...); then \
echo "[PASS] $$mod"; \
else \
echo "[FAIL] $$mod"; \
failed="$$failed $$mod"; \
if [ "$(FAIL_FAST)" = "1" ]; then exit 1; fi; \
fi; \
done; \
[ -z "$$failed" ]
.PHONY: test-react
test-react:
cd decnet_web && npm run test:run $(ARGS)
# ── Special suites (sequential, longer timeout) ───────────────────────────────
.PHONY: test-live
test-live:
$(PYTEST) tests/live -m live $(SEQ_FLAGS) $(ARGS)
.PHONY: test-api
test-api:
$(PYTEST) tests/api $(SEQ_FLAGS) $(ARGS)
.PHONY: test-stress
test-stress:
$(PYTEST) tests/stress -m stress $(SEQ_FLAGS) $(ARGS)
.PHONY: test-service
test-service:
$(PYTEST) tests/service_testing $(SEQ_FLAGS) $(ARGS)
.PHONY: test-fuzz
test-fuzz:
$(PYTEST) $(FUZZ_FLAGS) $(ARGS)
.PHONY: test-schema
test-schema:
SCHEMA_QUICK=$(SCHEMA_QUICK) $(PYTEST) \
tests/api/test_schemathesis.py \
tests/api/test_schemathesis_agent.py \
tests/api/test_schemathesis_swarm.py \
tests/api/test_schemathesis_ttp.py \
$(SCHEMA_FLAGS) $(ARGS)
.PHONY: test-bench
test-bench:
$(PYTEST) tests/perf $(BENCH_FLAGS) $(ARGS)
.PHONY: test-docker
test-docker:
DECNET_LIVE_DOCKER=1 $(PYTEST) tests/docker -m docker $(SEQ_FLAGS) $(ARGS)
# ── Static analysis ───────────────────────────────────────────────────────────
.PHONY: test-mypy
test-mypy:
.311/bin/mypy decnet --ignore-missing-imports --no-error-summary
.PHONY: test-bandit
test-bandit:
.311/bin/bandit -r decnet -c pyproject.toml
.PHONY: test-vulture
test-vulture:
.311/bin/vulture decnet --min-confidence 80
.PHONY: test-pip-audit
test-pip-audit:
.311/bin/pip-audit
# ── Composite: all suites ─────────────────────────────────────────────────────
_ALL_SUITES := core web db bus ttp intel analysis infra fleet cli features \
go react \
live api schema stress service fuzz bench docker \
mypy bandit vulture pip-audit
.PHONY: test-all test
test-all test:
@failed=""; \
for suite in $(_ALL_SUITES); do \
echo ""; \
echo "══════════════════════════ $$suite ══════════════════════════"; \
if $(MAKE) --no-print-directory test-$$suite ARGS="$(ARGS)"; then \
echo "[PASS] $$suite"; \
else \
echo "[FAIL] $$suite"; \
failed="$$failed $$suite"; \
if [ "$(FAIL_FAST)" = "1" ]; then \
echo "Stopping at first failure. Use FAIL_FAST=0 to run all suites."; \
exit 1; \
fi; \
fi; \
done; \
if [ -n "$$failed" ]; then \
echo ""; \
echo "Failed:$$failed"; \
exit 1; \
fi; \
echo ""; \
echo "All suites passed."
# ── Decky image pre-build ─────────────────────────────────────────────────────
_DECKY_TEMPLATES := \
conpot docker_api elasticsearch ftp http https imap k8s ldap \
llmnr mongodb mqtt mssql mysql pop3 postgres rdp redis sip smb smtp \
sniffer snmp ssh telnet tftp vnc
.PHONY: build-all
build-all:
@failed=""; \
for svc in $(_DECKY_TEMPLATES); do \
echo ""; \
echo "══════════════════════════ $$svc ══════════════════════════"; \
_nc=""; \
if [ "$(NO_CACHE)" = "1" ]; then _nc="--no-cache"; fi; \
if DOCKER_BUILDKIT=1 docker build $$_nc \
-t decnet/$$svc:latest \
decnet/templates/$$svc; then \
echo "[BUILT] $$svc"; \
else \
echo "[FAIL] $$svc"; \
failed="$$failed $$svc"; \
if [ "$(FAIL_FAST)" = "1" ]; then \
echo "Stopping at first failure. Use FAIL_FAST=0 to build all."; \
exit 1; \
fi; \
fi; \
done; \
if [ -n "$$failed" ]; then \
echo ""; \
echo "Failed:$$failed"; \
exit 1; \
fi; \
echo ""; \
echo "All decky images built."
.PHONY: help
help:
@echo "Unit suites (xdist, 30s timeout):"
@echo " make test-core tests/core + config + factories + fixtures"
@echo " make test-web tests/web + services"
@echo " make test-db tests/db + vectorstore"
@echo " make test-bus tests/bus + logging + telemetry"
@echo " make test-ttp tests/ttp"
@echo " make test-intel tests/intel + asn + geoip"
@echo " make test-analysis tests/clustering + correlation"
@echo " make test-infra tests/agent + collector + sniffer + profiler"
@echo " make test-fleet tests/fleet + swarm + topology + orchestrator + deploy + updater"
@echo " make test-cli tests/cli + engine + mutator + realism"
@echo " make test-features tests/canary + artifacts + webhook + decky_io + prober"
@echo ""
@echo "Go / React suites:"
@echo " make test-go go test ./... in each Caddy module variant"
@echo " make test-react vitest run in decnet_web"
@echo ""
@echo "Special suites (sequential, 120s timeout):"
@echo " make test-live tests/live"
@echo " make test-api tests/api (schemathesis)"
@echo " make test-stress tests/stress"
@echo " make test-service tests/service_testing"
@echo " make test-schema schemathesis contract tests (-m fuzz, xdist logical)"
@echo " make test-schema SCHEMA_QUICK=1 same, capped at 100 examples per test"
@echo " make test-fuzz hypothesis fuzz (all normal dirs, -m fuzz, skips schemathesis files)"
@echo " make test-bench tests/perf"
@echo " make test-docker tests/docker (needs DECNET_LIVE_DOCKER=1)"
@echo ""
@echo "Static analysis:"
@echo " make test-mypy mypy type check on decnet/"
@echo " make test-bandit bandit security scan on decnet/"
@echo " make test-vulture vulture dead code scan (>=80% confidence)"
@echo " make test-pip-audit pip-audit dependency vulnerability scan"
@echo ""
@echo "Composites:"
@echo " make test-all ALL suites (unit + go + react + live + api + schema + fuzz + bench + stress + docker + static analysis)"
@echo " make test-all FAIL_FAST=0 same, report all failures instead of stopping"
@echo ""
@echo "Passthrough: make test-web ARGS='--lf -s'"
@echo ""
@echo "Decky images:"
@echo " make build-all build decnet/<svc>:latest for all 27 decky templates"
@echo " make build-all NO_CACHE=1 same, bypassing Docker layer cache"
@echo " make build-all FAIL_FAST=0 same, continue past failures"

802
README.md

File diff suppressed because it is too large Load Diff

View File

View File

@@ -1,5 +0,0 @@
# bait/
Default operator-supplied email seed for IMAP/POP3 deckies. Drop `*.eml` and/or `*.json` files here; the IMAP/POP3 services bind-mount this dir read-only at `/var/spool/decnet-emails/seed` when no per-decky `email_seed` is configured. Entries concatenate onto the hardcoded bait baseline (additive to realism-engine output, never replacing).
JSON shape: list of dicts with required `from_addr`, `to_addr`, `subject`, `body`; optional `from_name`, `date`, `flags`. See `decnet/templates/imap/server.py` for the loader.

Binary file not shown.

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""DECNET — honeypot deception-network framework. """DECNET — honeypot deception-network framework.
This __init__ runs once, on the first `import decnet.*`. It seeds This __init__ runs once, on the first `import decnet.*`. It seeds

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""DECNET worker agent — runs on every SWARM worker host. """DECNET worker agent — runs on every SWARM worker host.
Exposes an mTLS-protected FastAPI service the master's SWARM controller Exposes an mTLS-protected FastAPI service the master's SWARM controller

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Worker-side FastAPI app. """Worker-side FastAPI app.
Protected by mTLS at the ASGI/uvicorn transport layer: uvicorn is started Protected by mTLS at the ASGI/uvicorn transport layer: uvicorn is started
@@ -26,7 +25,6 @@ from contextlib import asynccontextmanager
from typing import Any, Optional from typing import Any, Optional
from fastapi import FastAPI, HTTPException from fastapi import FastAPI, HTTPException
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
import contextlib import contextlib
@@ -183,7 +181,6 @@ class TeardownRequest(BaseModel):
class MutateRequest(BaseModel): class MutateRequest(BaseModel):
decky_id: str decky_id: str
services: list[str] services: list[str]
dry_run: bool = False
# ------------------------------------------------------------------ routes # ------------------------------------------------------------------ routes
@@ -200,22 +197,15 @@ async def status() -> dict:
@app.post( @app.post(
"/deploy", "/deploy",
status_code=202, responses={500: {"description": "Deployer raised an exception materialising the config"}},
responses={202: {"description": "Deploy accepted; runs in background; lifecycle deltas pushed via heartbeat"}},
) )
async def deploy(req: DeployRequest) -> dict: async def deploy(req: DeployRequest) -> dict:
"""Spawn the deploy in the background and return 202 immediately. try:
await _exec.deploy(req.config, dry_run=req.dry_run, no_cache=req.no_cache)
The master tracks per-decky completion via lifecycle deltas pushed on except Exception as exc:
the next heartbeat (one immediate push on completion, plus the log.exception("agent.deploy failed")
scheduled 30 s ticks as a fallback). Holding the request open across raise HTTPException(status_code=500, detail=str(exc)) from exc
a multi-minute compose build was the previous source of the wizard return {"status": "deployed", "deckies": len(req.config.deckies)}
API-hang."""
asyncio.create_task(
_exec.deploy_async(req.config, dry_run=req.dry_run, no_cache=req.no_cache),
name=f"deploy-{id(req)}",
)
return {"status": "accepted", "deckies": [d.name for d in req.config.deckies]}
@app.post( @app.post(
@@ -317,50 +307,14 @@ async def topology_state() -> dict:
@app.post( @app.post(
"/mutate", "/mutate",
status_code=202, responses={501: {"description": "Worker-side mutate not yet implemented"}},
responses={
202: {"description": "Mutate accepted; runs in background; lifecycle delta pushed via heartbeat"},
404: {"description": "No active deployment, or unknown decky_id (dry_run validation only)"},
},
) )
async def mutate(req: MutateRequest) -> Any: async def mutate(req: MutateRequest) -> dict:
"""Spawn the mutate in the background and return 202 immediately. # TODO: implement worker-side mutate. Currently the master performs
# mutation by re-sending a full /deploy with the updated DecnetConfig;
Master tracks completion via a lifecycle delta pushed on the next # this avoids duplicating mutation logic on the worker for v1. When
heartbeat (immediate push on completion). ``dry_run`` is still # ready, replace the 501 with a real redeploy-of-a-single-decky path.
synchronous — it validates against the worker's current state and raise HTTPException(
returns the would-be services without spawning a task or touching status_code=501,
docker, so the wizard's preview path stays cheap.""" detail="Per-decky mutate is performed via /deploy with updated services",
if req.dry_run:
from decnet.config import load_state
state = load_state()
if state is None:
raise HTTPException(
status_code=404,
detail="no active deployment on this worker",
)
cfg, _ = state
decky = next((d for d in cfg.deckies if d.name == req.decky_id), None)
if decky is None:
raise HTTPException(
status_code=404,
detail=f"decky {req.decky_id!r} not found in worker state",
)
return JSONResponse(
status_code=200,
content={
"status": "dry_run",
"decky_id": req.decky_id,
"services": list(req.services),
},
)
asyncio.create_task(
_exec.mutate_async(req.decky_id, list(req.services)),
name=f"mutate-{req.decky_id}",
) )
return {
"status": "accepted",
"decky_id": req.decky_id,
"services": list(req.services),
}

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Thin adapter between the agent's HTTP endpoints and the existing """Thin adapter between the agent's HTTP endpoints and the existing
``decnet.engine.deployer`` code path. ``decnet.engine.deployer`` code path.
@@ -81,99 +80,6 @@ async def deploy(config: DecnetConfig, dry_run: bool = False, no_cache: bool = F
await asyncio.to_thread(_deployer.deploy, config, dry_run, no_cache, False) await asyncio.to_thread(_deployer.deploy, config, dry_run, no_cache, False)
async def deploy_async(
config: DecnetConfig, *, dry_run: bool = False, no_cache: bool = False,
) -> None:
"""Background-task body for /deploy: run the deploy, then push a
lifecycle delta to the master so it observes terminal transitions
immediately rather than waiting for the next scheduled heartbeat.
Per-decky lifecycle deltas — master pivots them onto the matching
open DeckyLifecycle rows via the heartbeat handler. Errors are
captured and pushed as ``failed`` deltas; the task itself never
raises (a crashed task would just leave master rows wedged).
"""
from datetime import datetime, timezone
from decnet.agent.heartbeat import push_lifecycle_delta
decky_names = [d.name for d in config.deckies]
try:
await deploy(config, dry_run=dry_run, no_cache=no_cache)
except Exception as exc: # noqa: BLE001
log.exception("agent.deploy_async failed")
err = f"{type(exc).__name__}: {exc}"
deltas = [
{
"decky_name": name, "operation": "deploy",
"status": "failed", "error": err[:2000],
"completed_at": datetime.now(timezone.utc).isoformat(),
}
for name in decky_names
]
await push_lifecycle_delta(deltas)
return
deltas = [
{
"decky_name": name, "operation": "deploy",
"status": "succeeded",
"completed_at": datetime.now(timezone.utc).isoformat(),
}
for name in decky_names
]
await push_lifecycle_delta(deltas)
async def mutate_async(decky_id: str, services: list[str]) -> None:
"""Background-task body for /mutate. Same shape as deploy_async:
perform the work, then push a single lifecycle delta on
completion (success or failure)."""
import time
from datetime import datetime, timezone
from decnet.composer import write_compose
from decnet.config import load_state, save_state
from decnet.engine import _compose_with_retry
from decnet.agent.heartbeat import push_lifecycle_delta
def _delta(status: str, error: str | None = None) -> dict:
out = {
"decky_name": decky_id, "operation": "mutate",
"status": status,
"completed_at": datetime.now(timezone.utc).isoformat(),
}
if error is not None:
out["error"] = error[:2000]
return out
try:
state = load_state()
if state is None:
await push_lifecycle_delta(
[_delta("failed", "no active deployment on this worker")],
)
return
cfg, compose_path = state
decky = next((d for d in cfg.deckies if d.name == decky_id), None)
if decky is None:
await push_lifecycle_delta(
[_delta("failed", f"decky {decky_id!r} not found in worker state")],
)
return
decky.services = list(services)
decky.last_mutated = time.time()
save_state(cfg, compose_path)
write_compose(cfg, compose_path)
await asyncio.to_thread(
_compose_with_retry, "up", "-d", "--remove-orphans",
compose_file=compose_path,
)
except Exception as exc: # noqa: BLE001
log.exception("agent.mutate_async failed decky=%s", decky_id)
err = f"{type(exc).__name__}: {exc}"
await push_lifecycle_delta([_delta("failed", err)])
return
await push_lifecycle_delta([_delta("succeeded")])
async def teardown(decky_id: str | None = None) -> None: async def teardown(decky_id: str | None = None) -> None:
log.info("agent.teardown decky_id=%s", decky_id) log.info("agent.teardown decky_id=%s", decky_id)
await asyncio.to_thread(_deployer.teardown, decky_id) await asyncio.to_thread(_deployer.teardown, decky_id)
@@ -288,7 +194,7 @@ async def self_destruct() -> None:
argv = ["/bin/bash", path] argv = ["/bin/bash", path]
spawn_kwargs = {"start_new_session": True} spawn_kwargs = {"start_new_session": True}
subprocess.Popen( # type: ignore[call-overload] # nosec B603 subprocess.Popen( # nosec B603
argv, argv,
stdin=subprocess.DEVNULL, stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL, stdout=subprocess.DEVNULL,

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Agent → master liveness heartbeat loop. """Agent → master liveness heartbeat loop.
Every ``INTERVAL_S`` seconds the worker posts ``executor.status()`` to Every ``INTERVAL_S`` seconds the worker posts ``executor.status()`` to
@@ -51,11 +50,7 @@ def _resolve_agent_dir() -> pathlib.Path:
return pki.DEFAULT_AGENT_DIR return pki.DEFAULT_AGENT_DIR
async def _build_body( async def _tick(client: httpx.AsyncClient, url: str, host_uuid: str, agent_version: str) -> None:
host_uuid: str,
agent_version: str,
lifecycle: Optional[list[dict]] = None,
) -> dict:
snap = await _exec.status() snap = await _exec.status()
body: dict = { body: dict = {
"host_uuid": host_uuid, "host_uuid": host_uuid,
@@ -75,13 +70,7 @@ async def _build_body(
store.close() store.close()
except Exception: except Exception:
log.debug("heartbeat: topology state unavailable", exc_info=True) log.debug("heartbeat: topology state unavailable", exc_info=True)
if lifecycle:
body["lifecycle"] = lifecycle
return body
async def _tick(client: httpx.AsyncClient, url: str, host_uuid: str, agent_version: str) -> None:
body = await _build_body(host_uuid, agent_version)
resp = await client.post(url, json=body) resp = await client.post(url, json=body)
# 403 / 404 are terminal-ish — we still keep looping because an # 403 / 404 are terminal-ish — we still keep looping because an
# operator may re-enrol the host mid-session, but we log loudly so # operator may re-enrol the host mid-session, but we log loudly so
@@ -132,7 +121,7 @@ def start() -> Optional[asyncio.Task]:
return None return None
try: try:
from decnet import __version__ as _v # type: ignore[attr-defined] from decnet import __version__ as _v
agent_version = _v agent_version = _v
except Exception: except Exception:
agent_version = "unknown" agent_version = "unknown"
@@ -145,59 +134,6 @@ def start() -> Optional[asyncio.Task]:
return _task return _task
async def push_lifecycle_delta(deltas: list[dict]) -> None:
"""Fire a one-off heartbeat POST carrying *deltas* in the
``lifecycle`` field. Each delta: ``{decky_name, operation, status,
error?, completed_at?}``.
Called by the agent executor on /deploy and /mutate completion so
the master observes the terminal transition immediately rather than
waiting up to ``INTERVAL_S`` for the next scheduled tick. Failures
are logged and swallowed; the next scheduled heartbeat carries the
same deltas via DB-side reconciliation, since the worker has no
durable per-row state to lose.
"""
from decnet.env import (
DECNET_HOST_UUID,
DECNET_MASTER_HOST,
DECNET_SWARMCTL_PORT,
)
if not deltas:
return
if not DECNET_HOST_UUID or not DECNET_MASTER_HOST:
log.debug("push_lifecycle_delta: identity unconfigured — skipping")
return
agent_dir = _resolve_agent_dir()
try:
ssl_ctx = build_worker_ssl_context(agent_dir)
except Exception:
log.exception("push_lifecycle_delta: SSL context unavailable")
return
try:
from decnet import __version__ as _v # type: ignore[attr-defined]
agent_version = _v
except Exception:
agent_version = "unknown"
url = f"https://{DECNET_MASTER_HOST}:{DECNET_SWARMCTL_PORT}/swarm/heartbeat"
try:
async with httpx.AsyncClient(verify=ssl_ctx, timeout=_TIMEOUT) as client:
body = await _build_body(
DECNET_HOST_UUID, agent_version, lifecycle=deltas,
)
resp = await client.post(url, json=body)
if resp.status_code not in (200, 204):
log.warning(
"lifecycle delta push rejected status=%d body=%s",
resp.status_code, resp.text[:200],
)
except Exception:
log.exception("push_lifecycle_delta failed — next scheduled tick will retry")
async def stop() -> None: async def stop() -> None:
global _task global _task
if _task is None: if _task is None:

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Worker-agent uvicorn launcher. """Worker-agent uvicorn launcher.
Starts ``decnet.agent.app:app`` over HTTPS with mTLS enforcement. The Starts ``decnet.agent.app:app`` over HTTPS with mTLS enforcement. The

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Agent-side topology apply/teardown/state primitives. """Agent-side topology apply/teardown/state primitives.
Wraps the compose + bridge machinery from :mod:`decnet.engine.deployer` Wraps the compose + bridge machinery from :mod:`decnet.engine.deployer`
@@ -29,7 +28,6 @@ from decnet.engine.deployer import (
_compose_with_retry, _compose_with_retry,
_teardown_order, _teardown_order,
_topology_compose_path, _topology_compose_path,
_topology_compose_project,
) )
from decnet.logging import get_logger from decnet.logging import get_logger
from decnet.network import create_bridge_network, remove_bridge_network from decnet.network import create_bridge_network, remove_bridge_network
@@ -61,77 +59,6 @@ def _topology_id(hydrated: dict[str, Any]) -> str:
return str(tid) return str(tid)
def _check_hash_and_validate(hydrated: dict[str, Any], version_hash: str) -> str:
"""Verify hash integrity and structural validity; return topology_id."""
local_hash = canonical_hash(hydrated)
if local_hash != version_hash:
raise HashMismatch(
f"master hash {version_hash!r} does not match agent hash "
f"{local_hash!r} — refusing to apply"
)
issues = _validate_topology(hydrated)
if _validation_errors(issues):
raise ValidationError(issues)
return _topology_id(hydrated)
async def _teardown_superseded(topology_id: str, store: TopologyStore) -> None:
"""Tear down the current topology if it differs from topology_id.
Master is authoritative — a different pinned topology (fully applied,
partially applied, or drifted) is torn down before the new apply proceeds.
Refusing with 409 would leave the agent stuck in a state only a human
could resolve.
"""
existing = store.current()
if existing is None or existing.topology_id == topology_id:
return
log.info(
"superseding topology %s with %s on master authority",
existing.topology_id, topology_id,
)
try:
await teardown(existing.topology_id, store)
except Exception as exc: # noqa: BLE001 — we still want to try applying
log.warning(
"best-effort teardown of superseded topology %s failed: %s",
existing.topology_id, exc,
)
# Hard-clear the store row so the new apply isn't blocked by a
# half-torn-down predecessor. Leftover docker objects surface via
# the next heartbeat's observed block.
store.clear(existing.topology_id)
def _materialise(hydrated: dict[str, Any], topology_id: str) -> None:
"""Create bridge networks, write compose file, and bring up containers.
Sync/blocking — callers must dispatch via asyncio.to_thread.
``--always-recreate-deps`` keeps service containers' netns shares
fresh: every decky service joins its base's netns via
``network_mode: container:<base>``, and that share is bound at
service start time. If a base is recreated (e.g. when ``ports:``
changes after toggling ``forwards_l3``) but compose decides the
services are unchanged, the services keep a stale netns FD
pointing at the destroyed base — they end up in an empty
namespace with only ``lo``, and external traffic hits a closed
port on the live base. Forcing dependents to recreate alongside
the base is the cheapest way to make this race impossible.
"""
compose_path = _topology_compose_path(topology_id)
compose_project = _topology_compose_project(topology_id)
client = docker.from_env()
for lan in hydrated["lans"]:
net_name = _topology_network_name(topology_id, lan["name"])
create_bridge_network(client, net_name, lan["subnet"], internal=not lan["is_dmz"])
write_topology_compose(hydrated, compose_path)
_compose_with_retry(
"up", "--build", "-d", "--always-recreate-deps",
compose_file=compose_path, project=compose_project,
)
async def apply( async def apply(
hydrated: dict[str, Any], hydrated: dict[str, Any],
version_hash: str, version_hash: str,
@@ -146,11 +73,76 @@ async def apply(
Any docker / compose error propagates up; the endpoint maps it Any docker / compose error propagates up; the endpoint maps it
to 500 and records the message on the store row. to 500 and records the message on the store row.
""" """
topology_id = _check_hash_and_validate(hydrated, version_hash) local_hash = canonical_hash(hydrated)
await _teardown_superseded(topology_id, store) if local_hash != version_hash:
await asyncio.to_thread(_materialise, hydrated, topology_id) raise HashMismatch(
f"master hash {version_hash!r} does not match agent hash "
f"{local_hash!r} — refusing to apply"
)
issues = _validate_topology(hydrated)
if _validation_errors(issues):
raise ValidationError(issues)
topology_id = _topology_id(hydrated)
# Master is authoritative. If a different topology is pinned here
# — whether it fully applied, only partially applied (failure
# marker row + orphan containers), or drifted — teardown first,
# then accept the new one. Refusing with 409 would leave the
# agent stuck in a state only a human could resolve.
existing = store.current()
if existing is not None and existing.topology_id != topology_id:
log.info(
"superseding topology %s with %s on master authority",
existing.topology_id, topology_id,
)
try:
await teardown(existing.topology_id, store)
except Exception as exc: # noqa: BLE001 — we still want to try applying
log.warning(
"best-effort teardown of superseded topology %s failed: %s",
existing.topology_id, exc,
)
# Hard-clear the store row so the new apply isn't blocked
# by a half-torn-down predecessor. Leftover docker objects
# will surface via the next heartbeat's observed block.
store.clear(existing.topology_id)
lans = hydrated["lans"]
compose_path = _topology_compose_path(topology_id)
client = docker.from_env()
# Bridges + compose are sync/blocking; hop to a thread so we don't
# stall the event loop on a slow docker daemon.
def _materialise() -> None:
for lan in lans:
net_name = _topology_network_name(topology_id, lan["name"])
internal = not lan["is_dmz"]
create_bridge_network(
client, net_name, lan["subnet"], internal=internal
)
write_topology_compose(hydrated, compose_path)
# ``--always-recreate-deps`` keeps service containers' netns shares
# fresh: every decky service joins its base's netns via
# ``network_mode: container:<base>``, and that share is bound at
# service start time. If a base is recreated (e.g. when ``ports:``
# changes after toggling ``forwards_l3``) but compose decides the
# services are unchanged, the services keep a stale netns FD
# pointing at the destroyed base — they end up in an empty
# namespace with only ``lo``, and external traffic hits a closed
# port on the live base. Forcing dependents to recreate alongside
# the base is the cheapest way to make this race impossible.
_compose_with_retry(
"up", "--build", "-d", "--always-recreate-deps",
compose_file=compose_path,
)
await asyncio.to_thread(_materialise)
store.put(topology_id, version_hash, hydrated) store.put(topology_id, version_hash, hydrated)
log.info("topology %s applied on agent (%d LANs)", topology_id, len(hydrated["lans"])) log.info(
"topology %s applied on agent (%d LANs)", topology_id, len(lans)
)
async def teardown( async def teardown(
@@ -166,16 +158,12 @@ async def teardown(
# LAN membership list via the hydrated blob if available. # LAN membership list via the hydrated blob if available.
hydrated = row.hydrated if row and row.topology_id == topology_id else None hydrated = row.hydrated if row and row.topology_id == topology_id else None
compose_path = _topology_compose_path(topology_id) compose_path = _topology_compose_path(topology_id)
compose_project = _topology_compose_project(topology_id)
client = docker.from_env() client = docker.from_env()
def _dismantle() -> None: def _dismantle() -> None:
if compose_path.exists(): if compose_path.exists():
try: try:
_compose( _compose("down", "--remove-orphans", compose_file=compose_path)
"down", "--remove-orphans",
compose_file=compose_path, project=compose_project,
)
except subprocess.CalledProcessError as exc: except subprocess.CalledProcessError as exc:
log.warning( log.warning(
"topology %s compose down failed (continuing): %s", "topology %s compose down failed (continuing): %s",

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Agent-side sqlite cache of the currently-applied topology. """Agent-side sqlite cache of the currently-applied topology.
**This is a cache, not a source of truth.** The master is the only **This is a cache, not a source of truth.** The master is the only
@@ -64,7 +63,6 @@ class TopologyStore:
# The agent is single-process, so there's no real contention — # The agent is single-process, so there's no real contention —
# sqlite's own connection lock is enough. # sqlite's own connection lock is enough.
self._conn = sqlite3.connect(str(db_path), check_same_thread=False) self._conn = sqlite3.connect(str(db_path), check_same_thread=False)
self._conn.row_factory = sqlite3.Row
self._conn.execute( self._conn.execute(
"CREATE TABLE IF NOT EXISTS applied_topology (" "CREATE TABLE IF NOT EXISTS applied_topology ("
" topology_id TEXT PRIMARY KEY," " topology_id TEXT PRIMARY KEY,"
@@ -86,11 +84,11 @@ class TopologyStore:
if row is None: if row is None:
return None return None
return AppliedRow( return AppliedRow(
topology_id=row["topology_id"], topology_id=row[0],
applied_version_hash=row["applied_version_hash"], applied_version_hash=row[1],
hydrated=json.loads(row["hydrated_blob_json"]), hydrated=json.loads(row[2]),
applied_at=int(row["applied_at"]), applied_at=int(row[3]),
last_error=row["last_error"], last_error=row[4],
) )
# ---------------------------------------------------------------- writes # ---------------------------------------------------------------- writes

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
""" """
Machine archetype profiles for DECNET deckies. Machine archetype profiles for DECNET deckies.

View File

@@ -1 +0,0 @@
"""Artifact storage helpers shared between the web router and TTP workers."""

View File

@@ -1,86 +0,0 @@
"""
Shared on-disk artifact path resolution.
Honeypot decoys (SSH, SMTP) farm captured payloads into a host-mounted
quarantine tree:
/var/lib/decnet/artifacts/{decky}/{service}/{stored_as}
Two callers need to translate ``(decky, stored_as, service)`` into a
concrete ``Path`` rooted under that tree:
* The web router endpoint ``GET /api/v1/artifacts/{decky}/{stored_as}``
(``decnet.web.router.artifacts.api_get_artifact``) — admin-gated
download for the dashboard.
* The TTP ``EmailLifter`` (``decnet.ttp.impl.email_lifter``), which
reads the stored ``.eml`` at tag-time so body-aware predicates
(R0047 BEC, R0048 macro) don't need raw body text on the bus.
Both callers share the same validation rules and the same
defence-in-depth symlink-escape check; this module is the single
implementation. It is auth-agnostic — wrappers layer authentication
where appropriate (the router does ``require_admin``, the lifter does
not).
"""
from __future__ import annotations
import os
import re
from pathlib import Path
# decky names come from the deployer — lowercase alnum plus hyphens.
_DECKY_RE = re.compile(r"^[a-z0-9][a-z0-9-]{0,62}$")
# Services that own an artifacts subdir. Kept explicit so a caller
# can't pivot into arbitrary subpaths via a query string or bus payload.
_ALLOWED_SERVICES = frozenset({"ssh", "smtp"})
# stored_as is assembled by the capturing template as:
# ${ts}_${sha:0:12}_${base}
# where ts is ISO-8601 UTC (e.g. 2026-04-18T02:22:56Z), sha is 12 hex chars,
# and base is the original filename's basename. Keep the filename charset
# tight but allow common punctuation dropped files actually use.
_STORED_AS_RE = re.compile(
r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z_[a-f0-9]{12}_[A-Za-z0-9._-]{1,255}$"
)
# Module-level so tests can monkeypatch. Override via env in production
# (the systemd unit sets this) — the prod path matches the bind mount
# declared in decnet/services/{ssh,smtp}.py.
ARTIFACTS_ROOT = Path(
os.environ.get("DECNET_ARTIFACTS_ROOT", "/var/lib/decnet/artifacts")
)
class ArtifactPathError(ValueError):
"""Raised when (decky, stored_as, service) fails validation or escapes
the artifacts root.
The router catches this and re-raises HTTPException(400). The lifter
catches it and treats the event as having no body available (no-tag).
"""
def resolve_artifact_path(decky: str, stored_as: str, service: str) -> Path:
"""Validate inputs, resolve the on-disk path, and confirm it stays
inside the artifacts root.
Raises :class:`ArtifactPathError` on any violation. Does NOT check
that the file exists — callers handle that distinctly (404 for the
router, no-tag for the lifter).
"""
if service not in _ALLOWED_SERVICES:
raise ArtifactPathError("invalid service")
if not _DECKY_RE.fullmatch(decky):
raise ArtifactPathError("invalid decky name")
if not _STORED_AS_RE.fullmatch(stored_as):
raise ArtifactPathError("invalid stored_as")
root = ARTIFACTS_ROOT.resolve()
candidate = (root / decky / service / stored_as).resolve()
# defence-in-depth: even though the regexes reject `..`, make sure a
# symlink or weird filesystem state can't escape the root.
if root not in candidate.parents and candidate != root:
raise ArtifactPathError("path escapes artifacts root")
return candidate

View File

@@ -1,129 +0,0 @@
"""Shared asciinema shard helpers.
Extracted from ``decnet/web/router/transcripts/api_get_transcript.py``
so non-router callers (the BEHAVE-SHELL session-ended handler in
``decnet/profiler/worker.py``, the collector's session aggregator)
can resolve shard paths without crossing the layer boundary into the
FastAPI router.
Functions here speak in :class:`ValueError` — callers that want HTTP
semantics translate at the boundary. The router wrappers keep their
existing ``HTTPException`` behaviour for backwards compatibility.
PII boundary unchanged: shards live on disk; this module returns
:class:`pathlib.Path` pointers, never byte content. The ``_get_index``
cache stores byte offsets only.
"""
from __future__ import annotations
import os
import re
from collections import OrderedDict
from pathlib import Path
ARTIFACTS_ROOT = Path(
os.environ.get("DECNET_ARTIFACTS_ROOT", "/var/lib/decnet/artifacts"),
)
_DECKY_RE = re.compile(r"^[a-z0-9][a-z0-9-]{0,62}$")
_SERVICE_RE = re.compile(r"^(ssh|telnet)$")
_SHARD_BASENAME_RE = re.compile(r"^sessions-\d{4}-\d{2}-\d{2}\.jsonl$")
_SID_LINE_RE = re.compile(rb'"sid"\s*:\s*"([a-f0-9-]{36})"')
# (path, mtime_ns) → {sid: [(offset, length), ...]}
_INDEX_CACHE: "OrderedDict[tuple[str, int], dict[str, list[tuple[int, int]]]]" = (
OrderedDict()
)
_CACHE_MAX = 32
def validate_names(decky: str, service: str) -> None:
"""Raise :class:`ValueError` if ``decky`` / ``service`` look forged."""
if not _DECKY_RE.fullmatch(decky):
raise ValueError(f"invalid decky name: {decky!r}")
if not _SERVICE_RE.fullmatch(service):
raise ValueError(f"invalid service: {service!r}")
def resolve_shard(decky: str, service: str, shard_name: str) -> Path:
"""Resolve ``ARTIFACTS_ROOT/{decky}/{service}/transcripts/{shard_name}``
with escape-attempt detection. Raises :class:`ValueError` on
invalid inputs.
"""
validate_names(decky, service)
if not _SHARD_BASENAME_RE.fullmatch(shard_name):
raise ValueError(f"invalid shard name: {shard_name!r}")
root = ARTIFACTS_ROOT.resolve()
candidate = (root / decky / service / "transcripts" / shard_name).resolve()
if root not in candidate.parents and candidate != root:
raise ValueError(f"path escapes artifacts root: {candidate}")
return candidate
def _build_index(path: Path) -> dict[str, list[tuple[int, int]]]:
index: dict[str, list[tuple[int, int]]] = {}
with path.open("rb") as f:
offset = 0
for line in f:
length = len(line)
m = _SID_LINE_RE.search(line)
if m:
sid = m.group(1).decode("ascii")
index.setdefault(sid, []).append((offset, length))
offset += length
return index
def get_index(path: Path) -> tuple[dict[str, list[tuple[int, int]]], int]:
"""Return ``(sid → [(offset, length), …], file_size)``.
Cached by ``(path, mtime_ns)``; rebuilt when the shard changes.
"""
st = path.stat()
key = (str(path), st.st_mtime_ns)
if key in _INDEX_CACHE:
_INDEX_CACHE.move_to_end(key)
return _INDEX_CACHE[key], st.st_size
index = _build_index(path)
_INDEX_CACHE[key] = index
_INDEX_CACHE.move_to_end(key)
while len(_INDEX_CACHE) > _CACHE_MAX:
_INDEX_CACHE.popitem(last=False)
return index, st.st_size
def find_shard_with_sid(decky: str, service: str, sid: str) -> Path | None:
"""Scan every ``sessions-YYYY-MM-DD.jsonl`` under the decky's
transcripts dir until one claims this ``sid``.
Newest shards first — most lookups are for recent sessions. Caches
the per-shard sid index, so repeated calls are ~free until the
shard's mtime changes.
Returns ``None`` when nothing claims the sid OR when the
transcripts dir is missing / unreadable. Never raises on
filesystem-level errors — callers treat ``None`` as "skip".
"""
validate_names(decky, service)
root = ARTIFACTS_ROOT.resolve()
transcripts_dir = (root / decky / service / "transcripts").resolve()
if root not in transcripts_dir.parents:
return None
try:
if not transcripts_dir.is_dir():
return None
entries = list(transcripts_dir.iterdir())
except (OSError, PermissionError):
return None
shards = sorted(
(p for p in entries if _SHARD_BASENAME_RE.fullmatch(p.name)),
reverse=True,
)
for shard in shards:
try:
index, _size = get_index(shard)
except (OSError, PermissionError):
continue
if sid in index:
return shard
return None

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
""" """
IP-to-ASN enrichment — maps attacker IPs to BGP-announced AS numbers and IP-to-ASN enrichment — maps attacker IPs to BGP-announced AS numbers and
org names for attacker intelligence. org names for attacker intelligence.
@@ -7,7 +6,7 @@ Public surface mirrors :mod:`decnet.geoip` so callers can compose them:
* :func:`get_lookup` — returns the singleton :class:`AsnLookup`. * :func:`get_lookup` — returns the singleton :class:`AsnLookup`.
* :func:`enrich_ip` — takes an IP string, returns * :func:`enrich_ip` — takes an IP string, returns
``(asn_int, asn_name, bgp_prefix, provider_name)`` or ``(None, None, None, None)``. ``(asn_int, asn_name, provider_name)`` or ``(None, None, None)``.
Provider selection goes through :func:`~decnet.asn.factory.get_provider` Provider selection goes through :func:`~decnet.asn.factory.get_provider`
(env ``DECNET_ASN_PROVIDER``, default ``iptoasn``). Direct imports of (env ``DECNET_ASN_PROVIDER``, default ``iptoasn``). Direct imports of
@@ -52,8 +51,8 @@ def get_lookup(*, force_refresh: bool = False) -> AsnLookup:
return _lookup return _lookup
def enrich_ip(ip: str) -> Tuple[Optional[int], Optional[str], Optional[str], Optional[str]]: def enrich_ip(ip: str) -> Tuple[Optional[int], Optional[str], Optional[str]]:
"""Return ``(asn, as_name, bgp_prefix, provider_name)`` or ``(None, None, None, None)``. """Return ``(asn, as_name, provider_name)`` or ``(None, None, None)``.
Never raises — any lookup failure collapses to all-None so the Never raises — any lookup failure collapses to all-None so the
caller (profiler) can upsert the attacker row regardless. caller (profiler) can upsert the attacker row regardless.
@@ -63,15 +62,15 @@ def enrich_ip(ip: str) -> Tuple[Optional[int], Optional[str], Optional[str], Opt
touching provider config. touching provider config.
""" """
if os.environ.get("DECNET_ASN_ENABLED", "true").lower() == "false": if os.environ.get("DECNET_ASN_ENABLED", "true").lower() == "false":
return (None, None, None, None) return (None, None, None)
try: try:
lookup = get_lookup() lookup = get_lookup()
info = lookup.asn(ip) info = lookup.asn(ip)
if info is None: if info is None:
return (None, None, None, None) return (None, None, None)
return (info.asn, info.name or None, info.prefix, _provider_name or "unknown") return (info.asn, info.name or None, _provider_name or "unknown")
except Exception: except Exception:
return (None, None, None, None) return (None, None, None)
def _files_stale(provider) -> bool: def _files_stale(provider) -> bool:

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""ASN provider protocol — mirror of :mod:`decnet.geoip.base`. """ASN provider protocol — mirror of :mod:`decnet.geoip.base`.
Concrete providers (e.g. :mod:`decnet.asn.iptoasn`) implement this. Concrete providers (e.g. :mod:`decnet.asn.iptoasn`) implement this.

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""ASN provider factory — mirror of :mod:`decnet.geoip.factory`. """ASN provider factory — mirror of :mod:`decnet.geoip.factory`.
Dispatch key: ``DECNET_ASN_PROVIDER`` (default ``iptoasn``). Lazy Dispatch key: ``DECNET_ASN_PROVIDER`` (default ``iptoasn``). Lazy

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""iptoasn.com IP→ASN provider. """iptoasn.com IP→ASN provider.
Daily-refreshed gzipped TSV dump of the global BGP table, derived from Daily-refreshed gzipped TSV dump of the global BGP table, derived from

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""iptoasn.com bulk dump download. """iptoasn.com bulk dump download.
One file: ``ip2asn-v4.tsv.gz``, ~5 MB compressed, refreshed daily. One file: ``ip2asn-v4.tsv.gz``, ~5 MB compressed, refreshed daily.

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Parser for the iptoasn.com ``ip2asn-v4.tsv`` dump. """Parser for the iptoasn.com ``ip2asn-v4.tsv`` dump.
Line shape (gzipped, one row per BGP-announced prefix):: Line shape (gzipped, one row per BGP-announced prefix)::

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""iptoasn provider — orchestrates fetch + parse into an :class:`AsnLookup`. """iptoasn provider — orchestrates fetch + parse into an :class:`AsnLookup`.
Mirrors :class:`decnet.geoip.rir.provider.RirProvider` exactly: fetch, Mirrors :class:`decnet.geoip.rir.provider.RirProvider` exactly: fetch,
@@ -14,7 +13,7 @@ from typing import Sequence
from decnet.asn.base import Provider from decnet.asn.base import Provider
from decnet.asn.iptoasn.fetch import IPTOASN_SOURCES, fetch_all from decnet.asn.iptoasn.fetch import IPTOASN_SOURCES, fetch_all
from decnet.asn.iptoasn.parse import parse_file from decnet.asn.iptoasn.parse import parse_file
from decnet.asn.lookup import AsnLookup, Range from decnet.asn.lookup import AsnLookup
from decnet.asn.paths import ensure_root from decnet.asn.paths import ensure_root
logger = logging.getLogger("decnet.asn.iptoasn.provider") logger = logging.getLogger("decnet.asn.iptoasn.provider")
@@ -55,7 +54,7 @@ class IptoasnProvider(Provider):
"asn.iptoasn: cache load failed, rebuilding: %s", exc "asn.iptoasn: cache load failed, rebuilding: %s", exc
) )
ranges: list[Range] = [] ranges = []
for path in self.data_paths(): for path in self.data_paths():
if not path.exists(): if not path.exists():
continue continue

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Provider-agnostic IP→ASN lookup. """Provider-agnostic IP→ASN lookup.
A :class:`AsnLookup` is a frozen, sorted array of ``(start_ip, A :class:`AsnLookup` is a frozen, sorted array of ``(start_ip,
@@ -24,25 +23,11 @@ class AsnInfo:
asn: int asn: int
name: str # AS description / org name; "" if absent in the source data name: str # AS description / org name; "" if absent in the source data
prefix: Optional[str] = None # synthesized covering CIDR; set at lookup time, not at rest
Range = Tuple[int, int, AsnInfo] Range = Tuple[int, int, AsnInfo]
def _synthesize_prefix(start_int: int, end_int: int, queried_int: int) -> Optional[str]:
"""Return the most-specific CIDR from [start, end] that contains queried_int."""
try:
for net in ipaddress.summarize_address_range(
ipaddress.IPv4Address(start_int), ipaddress.IPv4Address(end_int)
):
if queried_int >= int(net.network_address) and queried_int <= int(net.broadcast_address):
return str(net)
except (ValueError, TypeError):
pass
return None
@dataclass @dataclass
class AsnLookup: class AsnLookup:
"""Indexed AS lookup over IPv4 ranges.""" """Indexed AS lookup over IPv4 ranges."""
@@ -103,9 +88,7 @@ class AsnLookup:
if idx < 0: if idx < 0:
return None return None
if n <= self._ends[idx]: if n <= self._ends[idx]:
info = self._infos[idx] return self._infos[idx]
prefix = _synthesize_prefix(self._starts[idx], self._ends[idx], n)
return AsnInfo(asn=info.asn, name=info.name, prefix=prefix)
return None return None
def __len__(self) -> int: def __len__(self) -> int:

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Filesystem layout for ASN data — mirror of :mod:`decnet.geoip.paths`. """Filesystem layout for ASN data — mirror of :mod:`decnet.geoip.paths`.
``ASN_ROOT`` is where providers drop their raw files and cache indexes. ``ASN_ROOT`` is where providers drop their raw files and cache indexes.

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""DECNET ServiceBus — pub/sub notification substrate. """DECNET ServiceBus — pub/sub notification substrate.
The bus is the notification layer for DECNET's worker constellation. The DB The bus is the notification layer for DECNET's worker constellation. The DB

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Process-wide bus singleton for request-serving workers (API, SSE routes). """Process-wide bus singleton for request-serving workers (API, SSE routes).
A single connected :class:`~decnet.bus.base.BaseBus` shared across request A single connected :class:`~decnet.bus.base.BaseBus` shared across request

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Bus abstractions: the :class:`Event` envelope and the :class:`BaseBus` ABC. """Bus abstractions: the :class:`Event` envelope and the :class:`BaseBus` ABC.
Every transport (NATS, in-process fake, null) speaks this contract. The Every transport (NATS, in-process fake, null) speaks this contract. The

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Bus factory — selects a :class:`~decnet.bus.base.BaseBus` implementation. """Bus factory — selects a :class:`~decnet.bus.base.BaseBus` implementation.
Dispatch key: the ``DECNET_BUS_TYPE`` environment variable. Dispatch key: the ``DECNET_BUS_TYPE`` environment variable.
@@ -77,7 +76,7 @@ def _maybe_wrap_telemetry(bus: BaseBus) -> BaseBus:
up at all we no-op. up at all we no-op.
""" """
try: try:
from decnet.telemetry import wrap_repository from decnet.telemetry import wrap_repository # type: ignore[attr-defined]
except ImportError: except ImportError:
return bus return bus
try: try:

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""In-process bus transports. """In-process bus transports.
* :class:`FakeBus` — real pub/sub semantics without touching a socket. Used * :class:`FakeBus` — real pub/sub semantics without touching a socket. Used

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Wire protocol for the DECNET bus UNIX-socket transport. """Wire protocol for the DECNET bus UNIX-socket transport.
Frame layout: Frame layout:

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Fire-and-forget publish helpers shared across every worker. """Fire-and-forget publish helpers shared across every worker.
Lifted out of ``decnet/mutator/engine.py`` once a second caller showed up Lifted out of ``decnet/mutator/engine.py`` once a second caller showed up
@@ -59,7 +58,7 @@ def make_thread_safe_publisher(
contract the rest of this module already upholds. contract the rest of this module already upholds.
""" """
if bus is None: if bus is None:
return lambda _topic, _payload, _event_type="": None # type: ignore[misc] return lambda _topic, _payload, _event_type="": None
def _publish(topic: str, payload: dict[str, Any], event_type: str = "") -> None: def _publish(topic: str, payload: dict[str, Any], event_type: str = "") -> None:
# Stream threads may keep draining after the bus owner closed it # Stream threads may keep draining after the bus owner closed it

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Canonical topic hierarchy for the DECNET ServiceBus. """Canonical topic hierarchy for the DECNET ServiceBus.
Locked early so consumers can subscribe with stable wildcard patterns. Locked early so consumers can subscribe with stable wildcard patterns.
@@ -18,7 +17,6 @@ Token structure (NATS-style, dot-separated):
attacker.scored attacker.scored
attacker.session.started attacker.session.started
attacker.session.ended attacker.session.ended
attacker.observation.{primitive}
identity.formed identity.formed
identity.observation.linked identity.observation.linked
identity.merged identity.merged
@@ -30,18 +28,12 @@ Token structure (NATS-style, dot-separated):
campaign.unmerged campaign.unmerged
credential.captured credential.captured
credential.reuse.detected credential.reuse.detected
attribution.profile.state_changed
attribution.profile.multi_actor_suspected
canary.{token_id}.triggered canary.{token_id}.triggered
canary.{token_id}.placed canary.{token_id}.placed
canary.{token_id}.revoked canary.{token_id}.revoked
system.log system.log
system.bus.health system.bus.health
system.{worker}.health system.{worker}.health
email.received
ttp.tagged
ttp.rule.fired.{technique_id}
ttp.rule.suppressed
Wildcards (per :func:`decnet.bus.base.matches`): Wildcards (per :func:`decnet.bus.base.matches`):
@@ -60,12 +52,8 @@ IDENTITY = "identity"
CAMPAIGN = "campaign" CAMPAIGN = "campaign"
SYSTEM = "system" SYSTEM = "system"
CREDENTIAL = "credential" CREDENTIAL = "credential"
ATTRIBUTION = "attribution"
ORCHESTRATOR = "orchestrator" ORCHESTRATOR = "orchestrator"
CANARY = "canary" CANARY = "canary"
SMTP = "smtp"
EMAIL = "email"
TTP = "ttp"
# ─── Leaf event-type constants (the last segment of each topic) ────────────── # ─── Leaf event-type constants (the last segment of each topic) ──────────────
@@ -95,24 +83,6 @@ DECKY_MUTATE_REQUEST = "mutate_request"
# syslog sidechannel too) to interleave substrate-change markers into # syslog sidechannel too) to interleave substrate-change markers into
# attacker traversals. # attacker traversals.
DECKY_MUTATION = "mutation" DECKY_MUTATION = "mutation"
# Per-service add/remove on a deployed decky (live; no full redeploy).
# Payload carries ``decky_name``, ``service_name``, optional
# ``topology_id``, and ``services`` (the post-mutation list). Consumers
# that watch substrate shape (correlator, dashboard, profiler) reconcile
# off these without waiting for the next decnet-state.json snapshot.
DECKY_SERVICE_ADDED = "service_added"
DECKY_SERVICE_REMOVED = "service_removed"
# Per-service config change (the schema-driven Inspector form). Payload
# carries ``decky_name``, ``service_name``, optional ``topology_id``,
# ``service_config`` (the new validated dict), and ``recreated`` — true
# when the operator hit Apply (container was force-recreated to pick up
# the new env), false when they only hit Save (DB-only).
DECKY_SERVICE_CONFIG_CHANGED = "service_config_changed"
# Async deploy/mutate operation transitions
# (pending/running/succeeded/failed). Payload: {lifecycle_id, operation,
# status, error?}. UI polling endpoint is the source of truth; this
# fires for live subscribers (dashboard, mutator-side audit, etc).
DECKY_LIFECYCLE = "lifecycle"
# Attacker event types (second token under the ``attacker`` root). First # Attacker event types (second token under the ``attacker`` root). First
# sighting, session boundary transitions, and score-threshold crossings # sighting, session boundary transitions, and score-threshold crossings
@@ -120,27 +90,10 @@ DECKY_LIFECYCLE = "lifecycle"
# the wildcard ``attacker.>``. # the wildcard ``attacker.>``.
ATTACKER_OBSERVED = "observed" ATTACKER_OBSERVED = "observed"
ATTACKER_SCORED = "scored" ATTACKER_SCORED = "scored"
# Published once per successful active probe result (JARM/HASSH/TCPfp/ipv6_leak). # Published once per successful active probe result (JARM/HASSH/TCPfp).
# Distinct from ``observed`` which is the correlator's first-sight signal — # Distinct from ``observed`` which is the correlator's first-sight signal —
# a fingerprint is additional evidence about an already-observed attacker. # a fingerprint is additional evidence about an already-observed attacker.
# Known payload ``kind`` discriminators carried in this topic:
# "jarm" — JARM TLS server hash (prober)
# "hassh" — HASSHServer SSH key-exchange hash (prober)
# "tcpfp" — TCP/IP stack fingerprint hash (prober)
# "tls_cert" — leaf TLS certificate SHA-256 (prober)
# "ipv6_leak" — fe80:: link-local address observed via passive sniffer
# or active ICMPv6 solicitation (prober + sniffer);
# payload: {attacker_ip, addr, iid_kind, mac_oui, vector,
# on_iface, observed_at}
ATTACKER_FINGERPRINTED = "fingerprinted" ATTACKER_FINGERPRINTED = "fingerprinted"
# Published when the prober observes a NEW hash for an
# (attacker_ip, port, probe_type) triple it has seen before — i.e. the
# attacker rotated their VPS, rebuilt their SSH server, swapped their
# TLS cert. Distinct from ``fingerprinted`` which fires on every probe
# result; ``fingerprint_rotated`` fires only on diff and carries both
# old_hash + new_hash. Producer: prober (via the rotation library);
# consumers: dashboard, forensics, attribution clustering.
ATTACKER_FINGERPRINT_ROTATED = "fingerprint_rotated"
ATTACKER_SESSION_STARTED = "session.started" ATTACKER_SESSION_STARTED = "session.started"
ATTACKER_SESSION_ENDED = "session.ended" ATTACKER_SESSION_ENDED = "session.ended"
# Published by the ``decnet enrich`` worker after an enrichment pass # Published by the ``decnet enrich`` worker after an enrichment pass
@@ -148,19 +101,6 @@ ATTACKER_SESSION_ENDED = "session.ended"
# returned a verdict). Payload carries the aggregate verdict + per- # returned a verdict). Payload carries the aggregate verdict + per-
# provider summary so SIEM-bound webhooks don't need to re-query the DB. # provider summary so SIEM-bound webhooks don't need to re-query the DB.
ATTACKER_INTEL_ENRICHED = "intel.enriched" ATTACKER_INTEL_ENRICHED = "intel.enriched"
# Per-primitive BEHAVE-SHELL observation. Full topic shape:
# attacker.observation.<primitive>
# e.g. ``attacker.observation.motor.input_modality``. Producer:
# ``decnet/profiler/behave_shell/`` (extractor library called from the
# profiler worker on ``attacker.session.ended``); consumers: dashboard
# SSE relay, attribution engine state machine, federation gossip
# (post-v0). See development/BEHAVE-INTEGRATION.md §"Bus topics" for
# the wire-format contract — the prefix is documentation + pattern
# match only; bus auth is socket file perms (DEBT-029 §2), not
# topic-level. The ``primitive`` segment MAY contain dots
# (``motor.shell_mastery.tab_completion``) — the same dotted-leaf
# rule that ``attacker.session.ended`` uses.
ATTACKER_OBSERVATION_PREFIX = "observation"
# Identity-resolution event types (second/third tokens under ``identity``). # Identity-resolution event types (second/third tokens under ``identity``).
# Published by the (future) clusterer worker — see # Published by the (future) clusterer worker — see
@@ -228,42 +168,6 @@ CAMPAIGN_UNMERGED = "unmerged"
CREDENTIAL_CAPTURED = "captured" CREDENTIAL_CAPTURED = "captured"
CREDENTIAL_REUSE_DETECTED = "reuse.detected" CREDENTIAL_REUSE_DETECTED = "reuse.detected"
# Attribution-engine event types (second/third tokens under
# ``attribution``). Published by the v0 attribution worker
# (``decnet.correlation.attribution_worker``) which subscribes to
# ``attacker.observation.>`` and runs the per-(identity, primitive)
# state machine. See ``development/ATTRIBUTION-ENGINE.md``.
#
# attribution.profile.state_changed — per-primitive state
# transition (e.g.
# stable → drifting).
# Payload: identity_uuid,
# primitive, old_state,
# new_state, current_value,
# confidence,
# observation_count, ts.
# attribution.profile.multi_actor_suspected — fires when ≥ 2
# primitives flag the same
# identity as multi_actor
# concurrently. Cross-
# primitive correlator;
# single-primitive
# multi_actor is too noisy
# on its own. Payload:
# identity_uuid, primitives,
# evidence_summary,
# confidence, ts.
#
# These are *derived* signals — distinct from
# ``identity.*`` (clusterer lifecycle, IDENTITY_RESOLUTION.md) and
# ``attacker.observation.*`` (raw extractor envelopes,
# BEHAVE-INTEGRATION.md). The three families compose: observations feed
# the attribution engine, the engine emits derived state, the clusterer
# reads observations + state to form / merge identities.
ATTRIBUTION_PROFILE_PREFIX = "profile"
ATTRIBUTION_PROFILE_STATE_CHANGED = "profile.state_changed"
ATTRIBUTION_PROFILE_MULTI_ACTOR_SUSPECTED = "profile.multi_actor_suspected"
# Canary-token event types (third token under ``canary``). # Canary-token event types (third token under ``canary``).
# #
# canary.{token_id}.placed — orchestrator/API successfully planted a # canary.{token_id}.placed — orchestrator/API successfully planted a
@@ -327,43 +231,6 @@ WORKER_CONTROL_START = "start"
# of patterns. Payload is currently empty; consumers only need the signal. # of patterns. Payload is currently empty; consumers only need the signal.
WEBHOOK_SUBSCRIPTIONS_CHANGED = "system.webhook.subscriptions_changed" WEBHOOK_SUBSCRIPTIONS_CHANGED = "system.webhook.subscriptions_changed"
# Email-receipt event — fired by smtp / smtp-relay services on full-message
# receipt (envelope + headers + body + attachments captured). Single-token
# leaf so the bus tokenizer accepts it directly under the ``email`` root.
# Consumed by the TTP ``email_lifter`` for header / body-pattern / attachment
# rules. PII rule (TTP_TAGGING.md "Hard parts §6"): payload carries hashes,
# counts, header names, and rcpt-domain sets — never rcpt addresses or body
# bytes.
EMAIL_RECEIVED = "received"
# TTP-tagging event types (second/third tokens under ``ttp``).
#
# ttp.tagged — one or more new tags written. Published
# only when ``INSERT OR IGNORE`` wrote at
# least one new row; idempotent
# re-evaluations publish nothing
# (loop-prevention invariant — see
# TTP_TAGGING.md).
# ttp.rule.fired.{technique_id} — per-technique fan-out for SIEM
# consumers that subscribe to a single
# technique. Topic key is the parent
# technique; sub_technique is in the
# payload. Built via :func:`ttp_rule_fired`.
# ttp.rule.suppressed — rule fired but the tag was dropped
# (confidence below floor, rate-limited,
# or the rule's RuleState was disabled).
# Observability signal for the dashboard.
#
# Per-rule reload + state-change topics. Built via
# :func:`ttp_rule_reloaded` / :func:`ttp_rule_state`; SIEM consumers
# subscribe to ``ttp.rule.reloaded.>`` (every rule) or
# ``ttp.rule.reloaded.R0001`` (one rule) at their preferred granularity.
TTP_TAGGED = "tagged"
TTP_RULE_FIRED = "rule.fired"
TTP_RULE_SUPPRESSED = "rule.suppressed"
TTP_RULE_RELOADED = "rule.reloaded"
TTP_RULE_STATE = "rule.state"
# ─── Builders ──────────────────────────────────────────────────────────────── # ─── Builders ────────────────────────────────────────────────────────────────
@@ -397,12 +264,6 @@ def decky_mutation(decky_id: str) -> str:
return f"{DECKY}.{decky_id}.{DECKY_MUTATION}" return f"{DECKY}.{decky_id}.{DECKY_MUTATION}"
def decky_lifecycle(decky_id: str) -> str:
"""Build ``decky.<id>.lifecycle``."""
_reject_tokens(decky_id)
return f"{DECKY}.{decky_id}.{DECKY_LIFECYCLE}"
def system(event_type: str) -> str: def system(event_type: str) -> str:
"""Build ``system.<event_type>``. """Build ``system.<event_type>``.
@@ -440,42 +301,6 @@ def attacker(event_type: str) -> str:
return f"{ATTACKER}.{event_type}" return f"{ATTACKER}.{event_type}"
def attacker_observation(primitive: str) -> str:
"""Build ``attacker.observation.<primitive>``.
*primitive* is the fully-qualified BEHAVE-SHELL primitive path
(e.g. ``motor.input_modality``,
``cognitive.feedback_loop_engagement``,
``motor.shell_mastery.tab_completion``). Dotted primitives are
permitted — this matches the format
``behave_shell.spec.event_adapter.event_topic_for`` produces
upstream, and DECNET's bus admits the dotted leaf the same way
:func:`attacker` does for ``session.started``.
Empty string is rejected so a downstream typo doesn't ship as
``attacker.observation.``.
"""
if not primitive:
raise ValueError(
"attacker_observation topic requires a non-empty primitive",
)
return f"{ATTACKER}.{ATTACKER_OBSERVATION_PREFIX}.{primitive}"
def attribution(event_type: str) -> str:
"""Build ``attribution.<event_type>``.
*event_type* is typically one of
:data:`ATTRIBUTION_PROFILE_STATE_CHANGED` or
:data:`ATTRIBUTION_PROFILE_MULTI_ACTOR_SUSPECTED` — both contain a
dot (``profile.state_changed``) which is permitted under the same
"trailing dotted leaf" rule that ``attacker.session.started`` uses.
"""
if not event_type:
raise ValueError("attribution topic requires a non-empty event_type")
return f"{ATTRIBUTION}.{event_type}"
def campaign(event_type: str) -> str: def campaign(event_type: str) -> str:
"""Build ``campaign.<event_type>``. """Build ``campaign.<event_type>``.
@@ -556,86 +381,6 @@ def system_control(worker: str) -> str:
return f"{SYSTEM}.{worker}.{SYSTEM_CONTROL}" return f"{SYSTEM}.{worker}.{SYSTEM_CONTROL}"
def smtp(event_type: str) -> str:
"""Build ``smtp.<event_type>``.
*event_type* may contain dots (e.g. ``probe.pending``).
"""
if not event_type:
raise ValueError("smtp topic requires a non-empty event_type")
return f"{SMTP}.{event_type}"
def email_topic(event_type: str) -> str:
"""Build ``email.<event_type>``.
Named ``email_topic`` rather than ``email`` to avoid shadowing the
Python ``email`` stdlib package at import sites that pull both.
*event_type* is typically :data:`EMAIL_RECEIVED`.
"""
if not event_type:
raise ValueError("email topic requires a non-empty event_type")
return f"{EMAIL}.{event_type}"
def ttp(event_type: str) -> str:
"""Build ``ttp.<event_type>``.
*event_type* is typically one of :data:`TTP_TAGGED`,
:data:`TTP_RULE_FIRED`, or :data:`TTP_RULE_SUPPRESSED`. Dotted
leaves (``rule.fired``) are permitted — same rationale as
:func:`system`. For per-technique fan-out use
:func:`ttp_rule_fired`.
"""
if not event_type:
raise ValueError("ttp topic requires a non-empty event_type")
return f"{TTP}.{event_type}"
def ttp_rule_fired(technique_id: str) -> str:
"""Build ``ttp.rule.fired.<technique_id>``.
Per-technique fan-out: SIEM subscribers can listen on
``ttp.rule.fired.>`` for everything, ``ttp.rule.fired.T1110`` for
one technique. *technique_id* is validated as a single segment —
sub-techniques like ``T1110.001`` are rejected because they would
split into two tokens. The topic key is the parent technique;
``sub_technique_id`` lives in the payload.
"""
_reject_tokens(technique_id)
return f"{TTP}.rule.fired.{technique_id}"
def ttp_rule_reloaded(rule_id: str) -> str:
"""Build ``ttp.rule.reloaded.<rule_id>``.
Per-rule fan-out fired by the :class:`~decnet.ttp.store.base.RuleStore`
when a rule's *definition* changes (YAML edit on the filesystem
backend, ``ttp_rule`` row update on the database backend). One event
per per-rule edit — never batched (the "incremental, never batched"
property in TTP_TAGGING.md §"Bus topics" inherits its granularity
from :meth:`RuleStore.subscribe_changes`).
Subscribers: ``ttp.rule.reloaded.>`` for every rule,
``ttp.rule.reloaded.R0001`` for one. *rule_id* is validated as a
single segment.
"""
_reject_tokens(rule_id)
return f"{TTP}.{TTP_RULE_RELOADED}.{rule_id}"
def ttp_rule_state(rule_id: str) -> str:
"""Build ``ttp.rule.state.<rule_id>``.
Per-rule fan-out fired by the :class:`~decnet.ttp.store.base.RuleStore`
when a rule's *operational state* changes (operator hits the disable
button, an ``expires_at`` TTL fires and auto-reverts the state).
*rule_id* is validated as a single segment.
"""
_reject_tokens(rule_id)
return f"{TTP}.{TTP_RULE_STATE}.{rule_id}"
def _reject_tokens(*parts: str) -> None: def _reject_tokens(*parts: str) -> None:
"""Reject topic segments that would break NATS-style tokenization. """Reject topic segments that would break NATS-style tokenization.

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""UNIX-socket client — :class:`UnixSocketBus` implementation of :class:`BaseBus`. """UNIX-socket client — :class:`UnixSocketBus` implementation of :class:`BaseBus`.
Holds one open socket to the local :class:`~decnet.bus.unix_server.BusServer`. Holds one open socket to the local :class:`~decnet.bus.unix_server.BusServer`.

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""UNIX-socket server for the DECNET bus. """UNIX-socket server for the DECNET bus.
One :class:`BusServer` per host. Accepts local connections on a UNIX-domain One :class:`BusServer` per host. Accepts local connections on a UNIX-domain

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""``decnet bus`` worker entrypoint. """``decnet bus`` worker entrypoint.
Starts a :class:`~decnet.bus.unix_server.BusServer` on the configured UNIX Starts a :class:`~decnet.bus.unix_server.BusServer` on the configured UNIX

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Canary tokens — decoy artifacts planted in decky filesystems. """Canary tokens — decoy artifacts planted in decky filesystems.
Public surface is exported here so callers can ``from decnet.canary Public surface is exported here so callers can ``from decnet.canary

View File

@@ -1,19 +0,0 @@
// SPDX-License-Identifier: AGPL-3.0-or-later
// Node helper invoked by decnet.canary.obfuscator.
// Reads {code, options} JSON from stdin, writes obfuscated JS to stdout.
// Kept dependency-light on purpose: only javascript-obfuscator.
const JsObf = require('javascript-obfuscator');
let raw = '';
process.stdin.setEncoding('utf8');
process.stdin.on('data', (chunk) => { raw += chunk; });
process.stdin.on('end', () => {
try {
const { code, options } = JSON.parse(raw);
const result = JsObf.obfuscate(code, options || {});
process.stdout.write(result.getObfuscatedCode());
} catch (e) {
process.stderr.write(String(e && e.stack || e));
process.exit(2);
}
});

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Canary generator / instrumenter ABCs and the artifact dataclass. """Canary generator / instrumenter ABCs and the artifact dataclass.
Two flavors of producer share the same return shape: Two flavors of producer share the same return shape:
@@ -101,12 +100,6 @@ class CanaryArtifact:
planting. Never leaked to the attacker-facing surface. planting. Never leaked to the attacker-facing surface.
""" """
fingerprint_nonce: Optional[str] = None
"""Per-mint HMAC nonce for fingerprint canaries; ``None`` for everything
else. Cultivator reads this and persists it on ``CanaryToken.fingerprint_nonce``
so the worker can validate incoming ``?k=`` params.
"""
class CanaryGenerator(ABC): class CanaryGenerator(ABC):
"""Produces a fake artifact from scratch.""" """Produces a fake artifact from scratch."""

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Realism contract adapter for canary generators. """Realism contract adapter for canary generators.
Stage 7 of the realism migration. The orchestrator's planner picks a Stage 7 of the realism migration. The orchestrator's planner picks a
@@ -47,8 +46,6 @@ _CLASS_TO_GENERATOR: dict[ContentClass, str] = {
ContentClass.CANARY_HONEYDOC_DOCX: "honeydoc_docx", ContentClass.CANARY_HONEYDOC_DOCX: "honeydoc_docx",
ContentClass.CANARY_HONEYDOC_PDF: "honeydoc_pdf", ContentClass.CANARY_HONEYDOC_PDF: "honeydoc_pdf",
ContentClass.CANARY_MYSQL_DUMP: "mysql_dump", ContentClass.CANARY_MYSQL_DUMP: "mysql_dump",
ContentClass.CANARY_FINGERPRINT_HTML: "fingerprint_html",
ContentClass.CANARY_FINGERPRINT_SVG: "fingerprint_svg",
} }
@@ -65,8 +62,6 @@ _GENERATOR_TO_KIND: dict[str, str] = {
"honeydoc_pdf": "http", "honeydoc_pdf": "http",
"ssh_key": "dns", # trip is DNS resolution of host comment "ssh_key": "dns", # trip is DNS resolution of host comment
"mysql_dump": "dns", # trip is DNS resolution of subdomain "mysql_dump": "dns", # trip is DNS resolution of subdomain
"fingerprint_html": "http", # obfuscated JS beacons GET /c/<slug>
"fingerprint_svg": "http", # same, embedded inside SVG <script>
} }
@@ -83,8 +78,6 @@ _DEFAULT_PATH: dict[ContentClass, str] = {
ContentClass.CANARY_HONEYDOC_DOCX: "/home/{persona}/Documents/Q3-Operations-Review.docx", ContentClass.CANARY_HONEYDOC_DOCX: "/home/{persona}/Documents/Q3-Operations-Review.docx",
ContentClass.CANARY_HONEYDOC_PDF: "/home/{persona}/Documents/Q3-Operations-Review.pdf", ContentClass.CANARY_HONEYDOC_PDF: "/home/{persona}/Documents/Q3-Operations-Review.pdf",
ContentClass.CANARY_MYSQL_DUMP: "/var/backups/db_backup.sql", ContentClass.CANARY_MYSQL_DUMP: "/var/backups/db_backup.sql",
ContentClass.CANARY_FINGERPRINT_HTML: "/home/{persona}/Documents/asset_directory.html",
ContentClass.CANARY_FINGERPRINT_SVG: "/home/{persona}/Documents/network_topology.svg",
} }
@@ -143,12 +136,10 @@ async def cultivate(
) )
callback_token = _new_callback_token() callback_token = _new_callback_token()
http_base_str: str = http_base or os.environ.get("DECNET_CANARY_HTTP_BASE") or ""
dns_zone_str: str = dns_zone or os.environ.get("DECNET_CANARY_DNS_ZONE") or ""
ctx = CanaryContext( ctx = CanaryContext(
callback_token=callback_token, callback_token=callback_token,
http_base=http_base_str, http_base=http_base or os.environ.get("DECNET_CANARY_HTTP_BASE", ""),
dns_zone=dns_zone_str, dns_zone=dns_zone or os.environ.get("DECNET_CANARY_DNS_ZONE", ""),
persona="linux", # all our deckies are POSIX in MVP persona="linux", # all our deckies are POSIX in MVP
) )
generator = get_generator(gen_name) generator = get_generator(gen_name)
@@ -163,7 +154,7 @@ async def cultivate(
# attribute a callback if the artifact trips during the plant # attribute a callback if the artifact trips during the plant
# itself (improbable but possible — DOCX viewers can preview # itself (improbable but possible — DOCX viewers can preview
# autoplay-style). # autoplay-style).
token_data: dict = { await repo.create_canary_token({
"kind": _GENERATOR_TO_KIND.get(gen_name, "http"), "kind": _GENERATOR_TO_KIND.get(gen_name, "http"),
"decky_name": plan.decky_name, "decky_name": plan.decky_name,
"instrumenter": None, "instrumenter": None,
@@ -174,10 +165,7 @@ async def cultivate(
"placed_at": datetime.now(timezone.utc), "placed_at": datetime.now(timezone.utc),
"created_by": created_by, "created_by": created_by,
"state": "planted", "state": "planted",
} })
if artifact.fingerprint_nonce is not None:
token_data["fingerprint_nonce"] = artifact.fingerprint_nonce
await repo.create_canary_token(token_data)
# Carry the placement_path on the artifact so the orchestrator's # Carry the placement_path on the artifact so the orchestrator's
# plant_file call uses it. We don't mutate the generator's # plant_file call uses it. We don't mutate the generator's

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Minimal authoritative DNS server for canary tokens (stdlib only). """Minimal authoritative DNS server for canary tokens (stdlib only).
We don't need a full resolver — only enough to: We don't need a full resolver — only enough to:
@@ -132,7 +131,7 @@ def _build_response(
question = qname_bytes + struct.pack("!HH", query.qtype, query.qclass) question = qname_bytes + struct.pack("!HH", query.qtype, query.qclass)
answer = b"" answer = b""
if an_count and answer_ip is not None: if an_count:
# Use a name pointer back to the question (offset 12). # Use a name pointer back to the question (offset 12).
ptr = struct.pack("!H", 0xC000 | 12) ptr = struct.pack("!H", 0xC000 | 12)
rdata = bytes(int(o) for o in answer_ip.split(".")) rdata = bytes(int(o) for o in answer_ip.split("."))
@@ -170,10 +169,10 @@ class CanaryDNSProtocol(asyncio.DatagramProtocol):
self._answer_ip = answer_ip self._answer_ip = answer_ip
self._transport: Optional[asyncio.DatagramTransport] = None self._transport: Optional[asyncio.DatagramTransport] = None
def connection_made(self, transport) -> None: def connection_made(self, transport) -> None: # type: ignore[override]
self._transport = transport self._transport = transport # type: ignore[assignment]
def datagram_received( def datagram_received( # type: ignore[override]
self, data: bytes, addr: Tuple[str, int], self, data: bytes, addr: Tuple[str, int],
) -> None: ) -> None:
try: try:
@@ -191,7 +190,7 @@ class CanaryDNSProtocol(asyncio.DatagramProtocol):
return return
# Known name — answer with our sinkhole IP, then fire the hook. # Known name — answer with our sinkhole IP, then fire the hook.
self._send(addr, _build_response(query, answer_ip=self._answer_ip)) self._send(addr, _build_response(query, answer_ip=self._answer_ip))
asyncio.ensure_future(self._hook(slug, query, addr[0])) asyncio.create_task(self._hook(slug, query, addr[0]))
def _slug_for(self, qname: str) -> Optional[str]: def _slug_for(self, qname: str) -> Optional[str]:
if not self._zone or not qname.endswith(self._suffix): if not self._zone or not qname.endswith(self._suffix):

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Generator and instrumenter factories. """Generator and instrumenter factories.
Same lazy-import pattern as :mod:`decnet.intel.factory` — concrete Same lazy-import pattern as :mod:`decnet.intel.factory` — concrete
@@ -22,8 +21,6 @@ KNOWN_GENERATORS: Tuple[str, ...] = (
"honeydoc_docx", "honeydoc_docx",
"honeydoc_pdf", "honeydoc_pdf",
"mysql_dump", "mysql_dump",
"fingerprint_html",
"fingerprint_svg",
) )
KNOWN_INSTRUMENTERS: Tuple[str, ...] = ( KNOWN_INSTRUMENTERS: Tuple[str, ...] = (
@@ -67,16 +64,6 @@ def get_generator(name: str) -> CanaryGenerator:
if name == "mysql_dump": if name == "mysql_dump":
from decnet.canary.generators.mysql_dump import MySQLDumpGenerator from decnet.canary.generators.mysql_dump import MySQLDumpGenerator
return MySQLDumpGenerator() return MySQLDumpGenerator()
if name == "fingerprint_html":
from decnet.canary.generators.fingerprint_html import (
FingerprintHtmlGenerator,
)
return FingerprintHtmlGenerator()
if name == "fingerprint_svg":
from decnet.canary.generators.fingerprint_svg import (
FingerprintSvgGenerator,
)
return FingerprintSvgGenerator()
raise ValueError( raise ValueError(
f"Unknown canary generator: {name!r}. Known: {KNOWN_GENERATORS}" f"Unknown canary generator: {name!r}. Known: {KNOWN_GENERATORS}"
) )

View File

@@ -1,292 +0,0 @@
// SPDX-License-Identifier: AGPL-3.0-or-later
// Canary fingerprint payload — the JS that runs inside an opened HTML/SVG
// canary, harvests browser primitives, and beacons the result back to the
// canary worker. Ported from canary-self-test.html with the rendering UI
// stripped out.
//
// Three placeholders are substituted by the Python builder BEFORE
// javascript-obfuscator runs:
//
// {{BEACON_URL}} → full URL to /c/<callback_token> (no trailing slash)
// {{MINT_UUID}} → per-mint UUID, baked into the string-array post-obf
// {{MINT_NONCE}} → 16-hex HMAC nonce; the worker rejects ?d=/?o= without it
//
// Beacon strategy (MVP): a bare GET pixel for "I was opened" reliability,
// then a fingerprint payload sent as a base64-URL query param on a second
// GET so the existing worker records the hit even before step-4 POST
// support lands. Both fail-open: any error short-circuits to next step.
(async function () {
var BEACON_URL = "{{BEACON_URL}}";
var MINT_UUID = "{{MINT_UUID}}";
var MINT_NONCE = "{{MINT_NONCE}}";
var fp = { mint: MINT_UUID };
function fire(url) {
try {
var img = new Image();
img.src = url;
} catch (e) { /* swallow */ }
}
// 1) bare-open beacon — fires regardless of whether the rest succeeds
fire(BEACON_URL + "?o=1&k=" + MINT_NONCE);
function sha256(str) {
var buf = new TextEncoder().encode(str);
return crypto.subtle.digest("SHA-256", buf).then(function (h) {
return Array.from(new Uint8Array(h))
.map(function (b) { return b.toString(16).padStart(2, "0"); })
.join("");
});
}
// navigator
try {
fp.nav = {
ua: navigator.userAgent,
pl: navigator.platform,
lg: navigator.language,
lgs: (navigator.languages || []).join(","),
ck: navigator.cookieEnabled,
dnt: navigator.doNotTrack,
hc: navigator.hardwareConcurrency,
dm: navigator.deviceMemory || null,
tp: navigator.maxTouchPoints,
wd: navigator.webdriver === true,
pdf: navigator.pdfViewerEnabled || null,
};
} catch (e) { fp.nav = { err: String(e) }; }
// screen
try {
fp.scr = {
w: screen.width, h: screen.height,
aw: screen.availWidth, ah: screen.availHeight,
cd: screen.colorDepth, pd: screen.pixelDepth,
dpr: window.devicePixelRatio,
iw: window.innerWidth, ih: window.innerHeight,
or: (screen.orientation && screen.orientation.type) || null,
};
} catch (e) { fp.scr = { err: String(e) }; }
// tz / locale
try {
var dtf = Intl.DateTimeFormat().resolvedOptions();
fp.tz = {
z: dtf.timeZone, lc: dtf.locale,
ca: dtf.calendar, ns: dtf.numberingSystem,
off: new Date().getTimezoneOffset(),
};
} catch (e) { fp.tz = { err: String(e) }; }
// connection
try {
var c = navigator.connection;
fp.cn = c ? {
t: c.effectiveType, dl: c.downlink, rtt: c.rtt, sd: c.saveData,
} : null;
} catch (e) { fp.cn = { err: String(e) }; }
// canvas
try {
var cv = document.createElement("canvas");
cv.width = 280; cv.height = 60;
var ctx = cv.getContext("2d");
ctx.textBaseline = "top";
ctx.font = "14px Arial";
ctx.fillStyle = "#f60";
ctx.fillRect(125, 1, 62, 20);
ctx.fillStyle = "#069";
ctx.fillText("c-" + String.fromCharCode(0x1f600), 2, 15);
ctx.fillStyle = "rgba(102,204,0,0.7)";
ctx.fillText("c-" + String.fromCharCode(0x1f600), 4, 17);
var dataURL = cv.toDataURL();
fp.cv = { h: await sha256(dataURL), n: dataURL.length };
} catch (e) { fp.cv = { err: String(e) }; }
// webgl
try {
var gc = document.createElement("canvas");
var gl = gc.getContext("webgl") || gc.getContext("experimental-webgl");
if (gl) {
var ext = gl.getExtension("WEBGL_debug_renderer_info");
fp.gl = {
v: gl.getParameter(gl.VENDOR),
r: gl.getParameter(gl.RENDERER),
ver: gl.getParameter(gl.VERSION),
sl: gl.getParameter(gl.SHADING_LANGUAGE_VERSION),
uv: ext ? gl.getParameter(ext.UNMASKED_VENDOR_WEBGL) : null,
ur: ext ? gl.getParameter(ext.UNMASKED_RENDERER_WEBGL) : null,
};
} else { fp.gl = { err: "unavailable" }; }
} catch (e) { fp.gl = { err: String(e) }; }
// audio
try {
var ACtx = window.OfflineAudioContext || window.webkitOfflineAudioContext;
if (ACtx) {
var actx = new ACtx(1, 44100, 44100);
var osc = actx.createOscillator();
var cmp = actx.createDynamicsCompressor();
osc.type = "triangle"; osc.frequency.value = 10000;
cmp.threshold.value = -50; cmp.knee.value = 40;
cmp.ratio.value = 12; cmp.attack.value = 0; cmp.release.value = 0.25;
osc.connect(cmp); cmp.connect(actx.destination);
osc.start(0);
var buf = await actx.startRendering();
var data = buf.getChannelData(0).slice(4500, 5000);
var sum = 0;
for (var i = 0; i < data.length; i++) sum += Math.abs(data[i]);
fp.au = { h: await sha256(sum.toString()), s: sum.toFixed(8) };
} else { fp.au = { err: "unavailable" }; }
} catch (e) { fp.au = { err: String(e) }; }
// fonts
try {
var bases = ["monospace", "sans-serif", "serif"];
var tests = [
"Arial", "Helvetica", "Times New Roman", "Courier New", "Verdana",
"Georgia", "Trebuchet MS", "Comic Sans MS", "Impact",
"Calibri", "Cambria", "Consolas", "Segoe UI", "Tahoma",
"JetBrains Mono", "Fira Code", "Cascadia Code", "SF Mono",
"Menlo", "Monaco", "Source Code Pro", "Inconsolata", "Hack",
"San Francisco", "Helvetica Neue", "Lucida Grande",
"DejaVu Sans", "DejaVu Sans Mono", "Liberation Sans",
"Liberation Mono", "Ubuntu", "Ubuntu Mono", "Roboto",
"Noto Sans", "Noto Mono",
"Microsoft YaHei", "SimSun", "PingFang SC", "Hiragino Sans",
"Hiragino Kaku Gothic Pro", "Yu Gothic", "Meiryo",
"Malgun Gothic", "Noto Sans CJK",
"Adobe Garamond Pro", "Myriad Pro", "Minion Pro",
"Bahnschrift", "Cyberpunk",
];
var sp = document.createElement("span");
sp.style.fontSize = "72px";
sp.style.position = "absolute";
sp.style.left = "-9999px";
sp.innerHTML = "mmmmmmmmmmlli";
document.body.appendChild(sp);
var bs = {};
for (var bi = 0; bi < bases.length; bi++) {
sp.style.fontFamily = bases[bi];
bs[bases[bi]] = { w: sp.offsetWidth, h: sp.offsetHeight };
}
var det = [];
for (var ti = 0; ti < tests.length; ti++) {
for (var bj = 0; bj < bases.length; bj++) {
sp.style.fontFamily = "'" + tests[ti] + "'," + bases[bj];
if (sp.offsetWidth !== bs[bases[bj]].w ||
sp.offsetHeight !== bs[bases[bj]].h) {
det.push(tests[ti]); break;
}
}
}
document.body.removeChild(sp);
fp.ft = {
h: await sha256(det.slice().sort().join(",")),
n: det.length, t: tests.length, d: det,
};
} catch (e) { fp.ft = { err: String(e) }; }
// webrtc local ip leak
try {
var ips = {}; var cands = [];
var RPC = window.RTCPeerConnection || window.webkitRTCPeerConnection ||
window.mozRTCPeerConnection;
if (RPC) {
var pc = new RPC({ iceServers: [{ urls: "stun:stun.l.google.com:19302" }] });
pc.createDataChannel("");
pc.onicecandidate = function (e) {
if (!e.candidate) return;
cands.push(e.candidate.candidate);
var m = e.candidate.candidate.match(
/(\d+\.\d+\.\d+\.\d+|[a-f0-9:]+::[a-f0-9:]+)/);
if (m) ips[m[1]] = 1;
};
var off = await pc.createOffer();
await pc.setLocalDescription(off);
await new Promise(function (r) { setTimeout(r, 1500); });
pc.close();
fp.rtc = { ip: Object.keys(ips), n: cands.length, c: cands.slice(0, 3) };
} else { fp.rtc = { err: "unavailable" }; }
} catch (e) { fp.rtc = { err: String(e) }; }
// battery
try {
if (navigator.getBattery) {
var bat = await navigator.getBattery();
fp.bt = {
c: bat.charging, l: bat.level,
ct: bat.chargingTime === Infinity ? "inf" : bat.chargingTime,
dt: bat.dischargingTime === Infinity ? "inf" : bat.dischargingTime,
};
} else { fp.bt = { err: "unavailable" }; }
} catch (e) { fp.bt = { err: String(e) }; }
// perf timing jitter
try {
var samples = [];
for (var pi = 0; pi < 1000; pi++) {
var pa = performance.now();
var x = 0;
for (var pj = 0; pj < 1000; pj++) x += Math.sqrt(pj);
samples.push(performance.now() - pa);
}
samples.sort(function (a, b) { return a - b; });
fp.pf = {
med: samples[500].toFixed(4),
p95: samples[950].toFixed(4),
mn: samples[0].toFixed(4),
mx: samples[999].toFixed(4),
};
} catch (e) { fp.pf = { err: String(e) }; }
// permissions
try {
if (navigator.permissions) {
var names = ["geolocation", "notifications", "camera", "microphone",
"persistent-storage", "clipboard-read", "clipboard-write"];
var st = {};
for (var ni = 0; ni < names.length; ni++) {
try {
var r = await navigator.permissions.query({ name: names[ni] });
st[names[ni]] = r.state;
} catch (e) { st[names[ni]] = "unsupported"; }
}
fp.pm = st;
} else { fp.pm = { err: "unavailable" }; }
} catch (e) { fp.pm = { err: String(e) }; }
// composite identity hash — stable inputs only
try {
var stable = [
fp.cv && fp.cv.h, fp.au && fp.au.h, fp.ft && fp.ft.h,
fp.gl && fp.gl.ur, fp.nav && fp.nav.pl,
fp.nav && fp.nav.hc, fp.tz && fp.tz.z,
fp.scr && (fp.scr.w + "x" + fp.scr.h),
].filter(Boolean).join("|");
fp.id = await sha256(stable);
} catch (e) { fp.id = { err: String(e) }; }
// 2) ship the payload as base64url JSON on a GET query param.
// The current worker records the hit on /c/<slug>; step-4 worker
// will decode ?d= and persist the fingerprint blob.
try {
var json = JSON.stringify(fp);
var b64 = btoa(unescape(encodeURIComponent(json)))
.replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/, "");
// chunk if URL would exceed safe limit (~6KB)
var MAX = 6000;
if (b64.length <= MAX) {
fire(BEACON_URL + "?d=" + b64 + "&k=" + MINT_NONCE);
} else {
var sid = (Math.random() * 1e9 | 0).toString(36);
var total = Math.ceil(b64.length / MAX);
for (var ci = 0; ci < total; ci++) {
var part = b64.substr(ci * MAX, MAX);
fire(BEACON_URL + "?s=" + sid + "&i=" + ci + "&n=" + total + "&d=" + part + "&k=" + MINT_NONCE);
}
}
} catch (e) { /* swallow */ }
})();

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Built-in canary generators (synthesised fake artifacts). """Built-in canary generators (synthesised fake artifacts).
Concrete classes live in sibling modules and are imported lazily by Concrete classes live in sibling modules and are imported lazily by

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Fake ``~/.aws/credentials`` block (passive bait). """Fake ``~/.aws/credentials`` block (passive bait).
This is the **passive** variant — no callback wiring. An attacker This is the **passive** variant — no callback wiring. An attacker

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Fake ``.env`` with embedded callback URLs. """Fake ``.env`` with embedded callback URLs.
Modern web stacks read environment variables for everything from Modern web stacks read environment variables for everything from

View File

@@ -1,141 +0,0 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""HTML fingerprint canary — plausible-looking page with an obfuscated
browser-fingerprinting payload inlined at the bottom of ``<body>``.
The visible content is a deliberately mundane "internal directory"
table — the kind of file a curious attacker pulls off a decky's
filesystem and opens locally to triage. When the file is opened in
*any* network-connected browser the obfuscated payload runs and beacons
to ``/c/<callback_token>``: first a bare-open pixel, then a chunked
fingerprint dump (canvas, audio, fonts, WebGL, WebRTC local IPs,
timing jitter, permissions, composite identity hash).
Determinism: the mint UUID is derived from the callback token via
:func:`uuid.uuid5` so the same ``ctx`` always produces byte-identical
output, satisfying the generator contract in :mod:`decnet.canary.base`.
The obfuscator's seed and polymorphic config bits are likewise
callback-token-derived (see :mod:`decnet.canary.obfuscator`).
"""
from __future__ import annotations
import hashlib
import uuid
from decnet.canary.base import CanaryArtifact, CanaryContext, CanaryGenerator
from decnet.canary.obfuscator import render_fingerprint_js, nonce_for
_MINT_NAMESPACE = uuid.UUID("a3f7c821-9d1e-4b6a-8c2d-1e4f9a7b3c5d")
def _mint_uuid_for(callback_token: str) -> str:
return str(uuid.uuid5(_MINT_NAMESPACE, callback_token))
def _stable_int(callback_token: str, salt: str = "") -> int:
"""Deterministic non-negative int derived from the callback token.
``builtins.hash`` is salted per-process — useless for a generator
that must be byte-identical across runs. SHA-256 prefix is
overkill but free.
"""
h = hashlib.sha256((callback_token + "|" + salt).encode("utf-8")).digest()
return int.from_bytes(h[:4], "big")
_PAGE_TEMPLATE = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Internal Asset Directory</title>
<style>
body{{font-family:Segoe UI,Arial,sans-serif;background:#fafafa;color:#222;
margin:24px;font-size:13px}}
h1{{font-size:18px;margin:0 0 4px 0}}
.sub{{color:#777;font-size:11px;margin-bottom:18px}}
table{{border-collapse:collapse;width:100%;background:#fff;
box-shadow:0 1px 2px rgba(0,0,0,.05)}}
th,td{{padding:6px 10px;border-bottom:1px solid #eee;text-align:left}}
th{{background:#f4f4f4;font-weight:600;font-size:11px;
text-transform:uppercase;letter-spacing:.5px;color:#555}}
tr:hover td{{background:#fafbff}}
.foot{{margin-top:16px;color:#999;font-size:11px}}
</style>
</head>
<body>
<h1>Internal Asset Directory</h1>
<div class="sub">last sync: {sync_label} · {row_count} entries · CONFIDENTIAL</div>
<table>
<tr><th>Hostname</th><th>Owner</th><th>Role</th><th>VLAN</th><th>Notes</th></tr>
{rows}
</table>
<div class="foot">page generated by directory-sync v2.4.1 — do not redistribute</div>
<script>{payload}</script>
</body>
</html>
"""
_ROW_POOL = (
("ny-app-01.corp.local", "k.tanaka", "app server", "vlan20", "primary"),
("ny-db-01.corp.local", "ops", "postgres primary", "vlan30", "backup nightly"),
("ny-build-02.corp.local", "ci-bot", "jenkins agent", "vlan40", ""),
("sf-vpn-01.corp.local", "netsec", "wireguard endpoint", "vlan10", "external"),
("ldn-mail-03.corp.local", "j.weber", "exchange edge", "vlan50", ""),
("hk-cache-01.corp.local", "ops", "redis replica", "vlan30", "lag <1s"),
("br-dev-04.corp.local", "m.silva", "dev sandbox", "vlan60", "ephemeral"),
("eu-bastion-02.corp.local", "secops", "ssh jump host", "vlan10", "mfa required"),
("us-archive-01.corp.local", "compliance", "log archive", "vlan70", "retain 7y"),
)
def _build_rows(callback_token: str) -> tuple[str, int]:
pick = _stable_int(callback_token, "pick") % len(_ROW_POOL)
take = 5 + (_stable_int(callback_token, "take") % 4)
selected = [_ROW_POOL[(pick + i) % len(_ROW_POOL)] for i in range(take)]
cells = "\n".join(
"<tr>" + "".join(f"<td>{c}</td>" for c in row) + "</tr>"
for row in selected
)
return cells, len(selected)
def _sync_label(callback_token: str) -> str:
day = _stable_int(callback_token, "day") % 28 + 1
hour = _stable_int(callback_token, "hour") % 24
return f"2026-04-{day:02d} {hour:02d}:14 UTC"
class FingerprintHtmlGenerator(CanaryGenerator):
"""Synthesise an HTML page that fingerprints the browser opening it."""
name = "fingerprint_html"
def generate(self, ctx: CanaryContext) -> CanaryArtifact:
mint_uuid = _mint_uuid_for(ctx.callback_token)
nonce = nonce_for(ctx.callback_token, mint_uuid)
payload = render_fingerprint_js(
callback_token=ctx.callback_token,
http_base=ctx.http_base,
mint_uuid=mint_uuid,
nonce=nonce,
)
rows, row_count = _build_rows(ctx.callback_token)
body = _PAGE_TEMPLATE.format(
sync_label=_sync_label(ctx.callback_token),
row_count=row_count,
rows=rows,
payload=payload,
)
beacon = f"{ctx.http_base.rstrip('/')}/c/{ctx.callback_token}"
return CanaryArtifact(
path="",
content=body.encode("utf-8"),
mode=0o644,
mtime_offset=-86400 * 14,
generator=self.name,
fingerprint_nonce=nonce,
notes=[
f"obfuscated fingerprinter beacons={beacon}",
f"mint_uuid={mint_uuid}",
],
)

View File

@@ -1,89 +0,0 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""SVG fingerprint canary — standalone SVG with an embedded ``<script>``
that runs the obfuscated fingerprinter when the file is opened directly
in a browser.
SVG ``<script>`` only fires when the SVG is loaded as a top-level
document (or via ``<object>``/``<iframe>``); it's *blocked* when the
SVG is referenced from another page's ``<img>``. That's the right
posture for canary use: an attacker browsing the decky filesystem and
double-clicking a stray ``network_diagram.svg`` triggers it; rendering
inside a sandboxed CMS preview does not.
Same determinism guarantees as :mod:`fingerprint_html`.
"""
from __future__ import annotations
from decnet.canary.base import CanaryArtifact, CanaryContext, CanaryGenerator
from decnet.canary.generators.fingerprint_html import _mint_uuid_for, _stable_int
from decnet.canary.obfuscator import render_fingerprint_js, nonce_for
_DIAGRAM_TEMPLATE = """<?xml version="1.0" encoding="UTF-8"?>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 600 360" width="600" height="360">
<style>
.box{{fill:#f7f9fb;stroke:#7a93ad;stroke-width:1.2}}
.lbl{{font:12px Segoe UI,Arial,sans-serif;fill:#2a3a4a}}
.edge{{stroke:#7a93ad;stroke-width:1.2;fill:none}}
.title{{font:bold 14px Segoe UI,Arial,sans-serif;fill:#1a2a3a}}
.cap{{font:10px Segoe UI,Arial,sans-serif;fill:#6a7a8a}}
</style>
<text class="title" x="20" y="28">Network Topology — {region} segment</text>
<text class="cap" x="20" y="44">draft v{ver} · last reviewed {review}</text>
<rect class="box" x="40" y="80" width="120" height="50" rx="4"/>
<text class="lbl" x="100" y="110" text-anchor="middle">edge gw</text>
<rect class="box" x="240" y="80" width="120" height="50" rx="4"/>
<text class="lbl" x="300" y="110" text-anchor="middle">core sw</text>
<rect class="box" x="440" y="80" width="120" height="50" rx="4"/>
<text class="lbl" x="500" y="110" text-anchor="middle">app cluster</text>
<rect class="box" x="240" y="220" width="120" height="50" rx="4"/>
<text class="lbl" x="300" y="250" text-anchor="middle">db tier</text>
<path class="edge" d="M160 105 L240 105"/>
<path class="edge" d="M360 105 L440 105"/>
<path class="edge" d="M300 130 L300 220"/>
<script type="application/ecmascript"><![CDATA[
{payload}
]]></script>
</svg>
"""
_REGIONS = ("us-east", "eu-central", "ap-south", "us-west", "sa-east")
class FingerprintSvgGenerator(CanaryGenerator):
"""Synthesise an SVG that fingerprints the browser opening it."""
name = "fingerprint_svg"
def generate(self, ctx: CanaryContext) -> CanaryArtifact:
mint_uuid = _mint_uuid_for(ctx.callback_token)
nonce = nonce_for(ctx.callback_token, mint_uuid)
payload = render_fingerprint_js(
callback_token=ctx.callback_token,
http_base=ctx.http_base,
mint_uuid=mint_uuid,
nonce=nonce,
)
region = _REGIONS[_stable_int(ctx.callback_token, "reg") % len(_REGIONS)]
ver = 1 + (_stable_int(ctx.callback_token, "ver") % 6)
day = _stable_int(ctx.callback_token, "day") % 28 + 1
body = _DIAGRAM_TEMPLATE.format(
region=region,
ver=ver,
review=f"2026-03-{day:02d}",
payload=payload,
)
beacon = f"{ctx.http_base.rstrip('/')}/c/{ctx.callback_token}"
return CanaryArtifact(
path="",
content=body.encode("utf-8"),
mode=0o644,
mtime_offset=-86400 * 30,
generator=self.name,
fingerprint_nonce=nonce,
notes=[
f"obfuscated fingerprinter beacons={beacon}",
f"mint_uuid={mint_uuid}",
],
)

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Fake ``.git/config`` with an attacker-bait remote URL. """Fake ``.git/config`` with an attacker-bait remote URL.
The ``[remote "origin"]`` ``url`` field is the natural place to embed The ``[remote "origin"]`` ``url`` field is the natural place to embed

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Built-in honeydoc — a minimal HTML "report" with a tracking pixel. """Built-in honeydoc — a minimal HTML "report" with a tracking pixel.
This is the *fallback* honeydoc used when the operator hasn't This is the *fallback* honeydoc used when the operator hasn't

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Real-DOCX honeydoc generator. """Real-DOCX honeydoc generator.
Synthesises a minimal but structurally valid DOCX from scratch via Synthesises a minimal but structurally valid DOCX from scratch via

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Real-PDF honeydoc generator (uses :mod:`pikepdf`). """Real-PDF honeydoc generator (uses :mod:`pikepdf`).
Builds a one-page PDF with the same Q3-review body as the HTML/DOCX Builds a one-page PDF with the same Q3-review body as the HTML/DOCX
@@ -44,7 +43,7 @@ class HoneydocPdfGenerator(CanaryGenerator):
def generate(self, ctx: CanaryContext) -> CanaryArtifact: def generate(self, ctx: CanaryContext) -> CanaryArtifact:
try: try:
from pikepdf import Pdf, Name, Dictionary, String from pikepdf import Pdf, Name, Dictionary, String # type: ignore[import-not-found]
except ImportError as e: except ImportError as e:
raise InstrumenterRejectedError( raise InstrumenterRejectedError(
"honeydoc_pdf requires pikepdf; install it (`pip install " "honeydoc_pdf requires pikepdf; install it (`pip install "

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Fake ``mysqldump`` output that phones home on import. """Fake ``mysqldump`` output that phones home on import.
Mirrors the Canarytokens.org MySQL-dump trick. When a victim runs Mirrors the Canarytokens.org MySQL-dump trick. When a victim runs

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Fake SSH private key with the callback host in the comment. """Fake SSH private key with the callback host in the comment.
OpenSSH private keys carry a free-form comment field — typically OpenSSH private keys carry a free-form comment field — typically

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Built-in canary instrumenters (operator-uploaded artifact mutation). """Built-in canary instrumenters (operator-uploaded artifact mutation).
Lazy-imported by :func:`decnet.canary.factory.get_instrumenter`. Lazy-imported by :func:`decnet.canary.factory.get_instrumenter`.

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""DOCX instrumenter — inject a remote image into the body. """DOCX instrumenter — inject a remote image into the body.
DOCX files are zip archives carrying ``word/document.xml`` (the body) DOCX files are zip archives carrying ``word/document.xml`` (the body)

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""HTML instrumenter — append a 1×1 tracking pixel. """HTML instrumenter — append a 1×1 tracking pixel.
Stdlib-only. We don't parse the HTML; we just inject the ``<img>`` Stdlib-only. We don't parse the HTML; we just inject the ``<img>``

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Image instrumenter — requires :mod:`PIL` (optional dependency). """Image instrumenter — requires :mod:`PIL` (optional dependency).
For PNG/JPEG/GIF we append a tEXt/EXIF chunk carrying the slug so For PNG/JPEG/GIF we append a tEXt/EXIF chunk carrying the slug so
@@ -33,7 +32,7 @@ class ImageInstrumenter(CanaryInstrumenter):
self, blob: bytes, ctx: CanaryContext, *, target_path: str, self, blob: bytes, ctx: CanaryContext, *, target_path: str,
) -> CanaryArtifact: ) -> CanaryArtifact:
try: try:
from PIL import Image, PngImagePlugin from PIL import Image, PngImagePlugin # type: ignore[import-not-found]
except ImportError as e: except ImportError as e:
raise InstrumenterRejectedError( raise InstrumenterRejectedError(
"image instrumenter requires Pillow; install it (`pip " "image instrumenter requires Pillow; install it (`pip "

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Passthrough instrumenter — bytes go to disk unchanged. """Passthrough instrumenter — bytes go to disk unchanged.
Used as the dispatch fallback for content types we can't safely Used as the dispatch fallback for content types we can't safely

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""PDF instrumenter — requires :mod:`pikepdf` (optional dependency). """PDF instrumenter — requires :mod:`pikepdf` (optional dependency).
PDF embedding is non-trivial: the cleanest place to put a callback PDF embedding is non-trivial: the cleanest place to put a callback
@@ -35,7 +34,7 @@ class PdfInstrumenter(CanaryInstrumenter):
self, blob: bytes, ctx: CanaryContext, *, target_path: str, self, blob: bytes, ctx: CanaryContext, *, target_path: str,
) -> CanaryArtifact: ) -> CanaryArtifact:
try: try:
import pikepdf import pikepdf # type: ignore[import-not-found]
except ImportError as e: except ImportError as e:
raise InstrumenterRejectedError( raise InstrumenterRejectedError(
"PDF instrumenter requires pikepdf; install it (`pip " "PDF instrumenter requires pikepdf; install it (`pip "

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Plain-text / config-file instrumenter. """Plain-text / config-file instrumenter.
Two embedding strategies, picked in order: Two embedding strategies, picked in order:

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""XLSX instrumenter — embed an external-image link. """XLSX instrumenter — embed an external-image link.
XLSX is structurally identical to DOCX (Office Open XML zip). The XLSX is structurally identical to DOCX (Office Open XML zip). The

View File

@@ -1,178 +0,0 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Per-mint JS obfuscator wrapper.
Thin Python wrapper around the ``javascript-obfuscator`` Node package.
Used by the fingerprint generators / instrumenters to produce a unique,
hard-to-statically-analyse JS blob per canary mint.
Two design choices flow from the canary contract in :mod:`base`:
* **Determinism.** Generators must return byte-identical artifacts for
the same ``(callback_token, http_base, dns_zone, persona)``. We
derive a numeric seed from the callback token and pass it to the
obfuscator's own ``seed`` option, and we derive the polymorphic
config bits from the same hash so a re-mint reproduces exactly.
* **Per-mint uniqueness.** Two different callback tokens produce
structurally different output: different identifier names, different
string-array rotation, optionally different transforms enabled.
The Node helper at ``_obfuscate_helper.js`` is invoked via subprocess.
We pass code+options as JSON on stdin and read the obfuscated result
from stdout. Stderr surfaces obfuscator failures.
"""
from __future__ import annotations
import hashlib
import hmac
import json
import os
import subprocess # nosec B404 — Node helper exec is the whole point
from pathlib import Path
from typing import Any
_HELPER = Path(__file__).parent / "_obfuscate_helper.js"
_PAYLOAD = Path(__file__).parent / "fingerprint_payload.js"
# Node binary path. Honor DECNET_NODE_BIN so deployments can pin a
# specific runtime; default to PATH lookup.
_NODE_BIN = os.environ.get("DECNET_NODE_BIN", "node")
# Hard timeout for the obfuscator subprocess. Real runs on the
# fingerprint payload sit well under 5s on a dev box.
_TIMEOUT_S = 30
class ObfuscatorError(RuntimeError):
"""Raised when the Node helper fails or returns empty output."""
class FingerprintSecretMissing(RuntimeError):
"""Raised when ``DECNET_CANARY_FINGERPRINT_SECRET`` is unset.
Fingerprint canaries embed a per-mint nonce derived from this
server-side secret; without it the worker cannot validate incoming
fingerprint beacons, so we fail loud at mint time rather than ship
a defeatable canary.
"""
_FINGERPRINT_SECRET_ENV = "DECNET_CANARY_FINGERPRINT_SECRET" # nosec B105 — this is an env var name, not a hardcoded password
def nonce_for(callback_token: str, mint_uuid: str) -> str:
"""Compute the per-mint fingerprint nonce.
HMAC-SHA256 keyed on the server-side master secret, message is
``callback_token + "|" + mint_uuid``. Truncated to 16 hex chars
(~64 bits of entropy) — enough to defeat slug-only forgery while
fitting comfortably into a query string.
"""
secret = os.environ.get(_FINGERPRINT_SECRET_ENV, "")
if not secret:
raise FingerprintSecretMissing(
f"{_FINGERPRINT_SECRET_ENV} is unset; fingerprint canaries cannot mint"
)
msg = f"{callback_token}|{mint_uuid}".encode("utf-8")
return hmac.new(secret.encode("utf-8"), msg, hashlib.sha256).hexdigest()[:16]
def _seed_from_token(callback_token: str) -> int:
"""Derive a 31-bit numeric seed from the callback token.
``javascript-obfuscator`` expects ``seed: number`` (int32-ish);
using a SHA-256-derived prefix gives us a uniform distribution
across the 31-bit positive range.
"""
h = hashlib.sha256(callback_token.encode("utf-8")).digest()
return int.from_bytes(h[:4], "big") & 0x7FFFFFFF
def _config_from_seed(seed: int) -> dict[str, Any]:
"""Build a deterministic, per-mint obfuscator config.
The hash bits drive *which* transforms apply — two mints get
structurally different outputs, not just different identifier names.
Defaults stay aggressive enough that reverse engineering is real
work; we never disable string-array or rename, only vary the dial.
"""
bits = seed
encodings = ("base64", "rc4")
string_array_encoding = [encodings[bits & 1]]
control_flow_threshold = 0.5 + ((bits >> 1) & 0xFF) / 512.0 # 0.5 .. ~1.0
dead_code_threshold = 0.2 + ((bits >> 9) & 0xFF) / 512.0 # 0.2 .. ~0.7
transform_object_keys = bool((bits >> 17) & 1)
numbers_to_expressions = bool((bits >> 18) & 1)
simplify = bool((bits >> 19) & 1)
return {
"compact": True,
"seed": seed,
"controlFlowFlattening": True,
"controlFlowFlatteningThreshold": round(control_flow_threshold, 3),
"deadCodeInjection": True,
"deadCodeInjectionThreshold": round(dead_code_threshold, 3),
"stringArray": True,
"stringArrayEncoding": string_array_encoding,
"stringArrayThreshold": 1,
"stringArrayRotate": True,
"stringArrayShuffle": True,
"splitStrings": True,
"splitStringsChunkLength": 4 + (bits & 7),
"transformObjectKeys": transform_object_keys,
"numbersToExpressions": numbers_to_expressions,
"simplify": simplify,
"selfDefending": False, # breaks SVG embed; not worth the cost
"renameGlobals": False,
"identifierNamesGenerator": "mangled-shuffled",
}
def obfuscate(code: str, *, callback_token: str) -> str:
"""Obfuscate *code* deterministically per *callback_token*.
Raises :class:`ObfuscatorError` if Node fails or returns empty.
"""
seed = _seed_from_token(callback_token)
options = _config_from_seed(seed)
payload = json.dumps({"code": code, "options": options})
try:
proc = subprocess.run( # nosec B603 — argv-form, no shell, fixed helper path; payload is JSON on stdin, not in argv
[_NODE_BIN, str(_HELPER)],
input=payload, capture_output=True, text=True,
timeout=_TIMEOUT_S, check=False,
)
except FileNotFoundError as e:
raise ObfuscatorError(f"node binary not found: {_NODE_BIN!r}") from e
except subprocess.TimeoutExpired as e:
raise ObfuscatorError("javascript-obfuscator timed out") from e
if proc.returncode != 0:
raise ObfuscatorError(
f"javascript-obfuscator failed rc={proc.returncode} "
f"stderr={proc.stderr.strip()[:400]}"
)
out = proc.stdout
if not out.strip():
raise ObfuscatorError("javascript-obfuscator returned empty output")
return out
def render_fingerprint_js(
*, callback_token: str, http_base: str, mint_uuid: str, nonce: str,
) -> str:
"""Build the obfuscated fingerprint JS for a single mint.
Substitutes ``{{BEACON_URL}}``, ``{{MINT_UUID}}``, and
``{{MINT_NONCE}}`` in the payload template, then runs it through
:func:`obfuscate` with a seed derived from the callback token.
The nonce is appended as ``&k=`` on every beacon URL the JS emits;
the worker rejects fingerprint payloads whose ``?k=`` doesn't match
the row's :attr:`CanaryToken.fingerprint_nonce`.
"""
template = _PAYLOAD.read_text(encoding="utf-8")
beacon = f"{http_base.rstrip('/')}/c/{callback_token}"
src = (
template
.replace("{{BEACON_URL}}", beacon)
.replace("{{MINT_UUID}}", mint_uuid)
.replace("{{MINT_NONCE}}", nonce)
)
return obfuscate(src, callback_token=callback_token)

View File

@@ -1,10 +0,0 @@
{
"name": "decnet-canary-obfuscator",
"version": "0.1.0",
"private": true,
"description": "Node helper for decnet.canary.obfuscator — javascript-obfuscator wrapper invoked via subprocess.",
"main": "_obfuscate_helper.js",
"dependencies": {
"javascript-obfuscator": "^5.4.2"
}
}

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Persona-aware path resolution for canary artifacts. """Persona-aware path resolution for canary artifacts.
Linux-persona deckies use POSIX-shaped paths under ``/home/<user>``. Linux-persona deckies use POSIX-shaped paths under ``/home/<user>``.
@@ -29,8 +28,6 @@ _LINUX_DEFAULTS: dict[str, str] = {
"honeydoc": "/home/{user}/Documents/quarterly_report.html", "honeydoc": "/home/{user}/Documents/quarterly_report.html",
"honeydoc_docx": "/home/{user}/Documents/quarterly_report.docx", "honeydoc_docx": "/home/{user}/Documents/quarterly_report.docx",
"honeydoc_pdf": "/home/{user}/Documents/quarterly_report.pdf", "honeydoc_pdf": "/home/{user}/Documents/quarterly_report.pdf",
"fingerprint_html": "/home/{user}/Documents/asset_directory.html",
"fingerprint_svg": "/home/{user}/Documents/network_topology.svg",
} }
_WINDOWS_DEFAULTS: dict[str, str] = { _WINDOWS_DEFAULTS: dict[str, str] = {
@@ -41,8 +38,6 @@ _WINDOWS_DEFAULTS: dict[str, str] = {
"honeydoc": "/home/{user}/Documents/quarterly_report.html", "honeydoc": "/home/{user}/Documents/quarterly_report.html",
"honeydoc_docx": "/home/{user}/Documents/quarterly_report.docx", "honeydoc_docx": "/home/{user}/Documents/quarterly_report.docx",
"honeydoc_pdf": "/home/{user}/Documents/quarterly_report.pdf", "honeydoc_pdf": "/home/{user}/Documents/quarterly_report.pdf",
"fingerprint_html": "/home/{user}/Documents/asset_directory.html",
"fingerprint_svg": "/home/{user}/Documents/network_topology.svg",
} }

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Plant / revoke canary artifacts inside running decky containers. """Plant / revoke canary artifacts inside running decky containers.
Single entry point per operation: Single entry point per operation:
@@ -21,8 +20,11 @@ shape but speaks bytes-via-base64 over the wire.
""" """
from __future__ import annotations from __future__ import annotations
import asyncio
import base64
import os import os
from datetime import datetime, timedelta, timezone import shlex
import time
from secrets import token_urlsafe from secrets import token_urlsafe
from typing import Any, Iterable, Optional from typing import Any, Iterable, Optional
@@ -32,16 +34,13 @@ from decnet.bus.factory import get_bus
from decnet.canary.base import CanaryArtifact, CanaryContext from decnet.canary.base import CanaryArtifact, CanaryContext
from decnet.canary.factory import get_generator from decnet.canary.factory import get_generator
from decnet.canary.paths import default_path_for from decnet.canary.paths import default_path_for
from decnet.decky_io import (
delete_file_from_container,
resolve_topology_container,
write_file_to_container,
)
from decnet.logging import get_logger from decnet.logging import get_logger
from decnet.web.db.repository import BaseRepository from decnet.web.db.repository import BaseRepository
log = get_logger("canary.planter") log = get_logger("canary.planter")
_DOCKER = "docker"
_TIMEOUT = 8.0
# Container suffix — matches the orchestrator SSH driver's convention # Container suffix — matches the orchestrator SSH driver's convention
# (``<decky_name>-ssh``). Canary placement always happens through the # (``<decky_name>-ssh``). Canary placement always happens through the
# ssh container because every decky has one and it carries the most # ssh container because every decky has one and it carries the most
@@ -53,16 +52,62 @@ def _container_for(decky_name: str) -> str:
return f"{decky_name}{_SSH_CONTAINER_SUFFIX}" return f"{decky_name}{_SSH_CONTAINER_SUFFIX}"
# resolve_topology_container is re-exported from decky_io for back-compat def _dirname(path: str) -> str:
# with callers (tests, deploy hook) that imported it from this module idx = path.rfind("/")
# before the decky_io extraction. if idx <= 0:
__all__ = [ return "/"
"plant", return path[:idx]
"revoke",
"resolve_topology_container",
"seed_baseline", async def _run(
"seed_baseline_topology", argv: list[str], *, stdin_bytes: Optional[bytes] = None,
] ) -> tuple[int, str, str]:
try:
proc = await asyncio.create_subprocess_exec(
*argv,
stdin=asyncio.subprocess.PIPE if stdin_bytes is not None else None,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
except FileNotFoundError as exc:
return 127, "", f"argv[0] not found: {exc}"
try:
stdout, stderr = await asyncio.wait_for(
proc.communicate(input=stdin_bytes), timeout=_TIMEOUT,
)
except asyncio.TimeoutError:
try:
proc.kill()
except ProcessLookupError:
pass
return 124, "", "timeout"
return (
proc.returncode if proc.returncode is not None else -1,
stdout.decode("utf-8", "replace"),
stderr.decode("utf-8", "replace"),
)
def _build_plant_command(artifact: CanaryArtifact) -> tuple[str, bytes]:
"""Compose the ``sh -c`` script + stdin payload for one artifact.
Binary safety: we base64-encode on the host and stream the result
over stdin to ``base64 -d`` inside the container, so the bytes
never touch the argv (kernel ARG_MAX would reject anything larger
than ~128KB-2MB depending on the host). Both ``base64`` (coreutils)
and ``touch -d @<unix_ts>`` are present on every Linux base image
we ship, so there's no per-distro branching.
"""
encoded = base64.b64encode(artifact.content)
mtime = int(time.time() + artifact.mtime_offset)
mode_str = oct(artifact.mode)[2:]
parts = [
f"mkdir -p {shlex.quote(_dirname(artifact.path))}",
f"base64 -d > {shlex.quote(artifact.path)}",
f"chmod {mode_str} {shlex.quote(artifact.path)}",
f"touch -d @{mtime} {shlex.quote(artifact.path)}",
]
return " && ".join(parts), encoded
async def _publish( async def _publish(
@@ -94,7 +139,6 @@ async def plant(
repo: Optional[BaseRepository] = None, repo: Optional[BaseRepository] = None,
publish: bool = True, publish: bool = True,
bus: Optional[BaseBus] = None, bus: Optional[BaseBus] = None,
container: Optional[str] = None,
) -> tuple[bool, Optional[str]]: ) -> tuple[bool, Optional[str]]:
"""Write *artifact* into the decky's ssh container. """Write *artifact* into the decky's ssh container.
@@ -113,12 +157,13 @@ async def plant(
await repo.update_canary_token_state(token_uuid, "failed", err) await repo.update_canary_token_state(token_uuid, "failed", err)
return False, err return False, err
target_container = container or _container_for(decky_name) sh_cmd, stdin_payload = _build_plant_command(artifact)
mtime = datetime.now(timezone.utc) + timedelta(seconds=artifact.mtime_offset) # ``-i`` keeps stdin attached so base64 -d inside the container can
success, error = await write_file_to_container( # consume the encoded payload streamed from the host.
target_container, artifact.path, artifact.content, argv = [_DOCKER, "exec", "-i", _container_for(decky_name), "sh", "-c", sh_cmd]
mode=artifact.mode, mtime=mtime, rc, _stdout, stderr = await _run(argv, stdin_bytes=stdin_payload)
) success = rc == 0
error = None if success else (stderr.strip()[:256] or f"rc={rc}")
if repo is not None: if repo is not None:
if success: if success:
@@ -137,8 +182,8 @@ async def plant(
if not success: if not success:
log.warning( log.warning(
"canary.plant failed decky=%s token=%s container=%s err=%r", "canary.plant failed decky=%s token=%s rc=%d stderr=%r",
decky_name, token_uuid, target_container, error, decky_name, token_uuid, rc, stderr[:120],
) )
return success, error return success, error
@@ -151,7 +196,6 @@ async def revoke(
repo: Optional[BaseRepository] = None, repo: Optional[BaseRepository] = None,
publish: bool = True, publish: bool = True,
bus: Optional[BaseBus] = None, bus: Optional[BaseBus] = None,
container: Optional[str] = None,
) -> tuple[bool, Optional[str]]: ) -> tuple[bool, Optional[str]]:
"""Best-effort unlink + state transition + bus publish. """Best-effort unlink + state transition + bus publish.
@@ -159,10 +203,11 @@ async def revoke(
the file is gone after the call (whether we deleted it or it was the file is gone after the call (whether we deleted it or it was
already missing); only docker / container-down errors return False. already missing); only docker / container-down errors return False.
""" """
target_container = container or _container_for(decky_name) sh_cmd = f"rm -f {shlex.quote(placement_path)}"
success, error = await delete_file_from_container( argv = [_DOCKER, "exec", _container_for(decky_name), "sh", "-c", sh_cmd]
target_container, placement_path, rc, _stdout, stderr = await _run(argv)
) success = rc == 0
error = None if success else (stderr.strip()[:256] or f"rc={rc}")
if repo is not None: if repo is not None:
await repo.update_canary_token_state(token_uuid, "revoked", error if not success else None) await repo.update_canary_token_state(token_uuid, "revoked", error if not success else None)
@@ -205,7 +250,6 @@ async def seed_baseline(
persona: str = "linux", persona: str = "linux",
created_by: str = "system", created_by: str = "system",
bus: Optional[BaseBus] = None, bus: Optional[BaseBus] = None,
container: Optional[str] = None,
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
"""Plant the configured baseline canary set on one decky. """Plant the configured baseline canary set on one decky.
@@ -249,59 +293,9 @@ async def seed_baseline(
await plant( await plant(
decky_name, artifact, decky_name, artifact,
token_uuid=token_uuid, repo=repo, publish=True, bus=bus, token_uuid=token_uuid, repo=repo, publish=True, bus=bus,
container=container,
) )
out.append({ out.append({
"token_uuid": token_uuid, "generator": gen_name, "kind": kind, "token_uuid": token_uuid, "generator": gen_name, "kind": kind,
"callback_token": slug, "placement_path": artifact.path, "callback_token": slug, "placement_path": artifact.path,
}) })
return out return out
async def seed_baseline_topology(
repo: BaseRepository,
topology_id: str,
*,
created_by: str = "system",
bus: Optional[BaseBus] = None,
) -> list[dict[str, Any]]:
"""Plant baseline canaries on every decky in a MazeNET topology.
Mirrors :func:`seed_baseline` for the topology path. Container name
resolution uses :func:`resolve_topology_container` since topology
deckies may not have an ssh service — in that case we target the
base container instead.
Best-effort: failures on any single decky are logged inside
:func:`plant`; the deploy hook treats the return value as
informational. Returns a flat list of per-token dicts (with an added
``decky_name`` key) across all deckies.
"""
from decnet.topology.persistence import hydrate
hydrated = await hydrate(repo, topology_id)
if hydrated is None:
log.warning(
"canary.seed_baseline_topology: topology %s not found", topology_id,
)
return []
out: list[dict[str, Any]] = []
for decky in hydrated["deckies"]:
cfg = decky.get("decky_config") or {}
decky_name = cfg.get("name") or decky.get("name")
if not decky_name:
continue
services = decky.get("services") or []
container = resolve_topology_container(topology_id, decky_name, services)
# MazeNET deckies don't carry an OS persona today; default to
# linux (every base image we ship is Linux).
rows = await seed_baseline(
decky_name, repo,
persona="linux", created_by=created_by, bus=bus,
container=container,
)
for r in rows:
r["decky_name"] = decky_name
out.append(r)
return out

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Filesystem store for operator-uploaded canary blobs. """Filesystem store for operator-uploaded canary blobs.
Blobs live under ``/var/lib/decnet/canary/blobs/<sha256>`` (override Blobs live under ``/var/lib/decnet/canary/blobs/<sha256>`` (override

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""``decnet canary`` worker — HTTP + DNS callback receivers. """``decnet canary`` worker — HTTP + DNS callback receivers.
Two surfaces, one process: Two surfaces, one process:
@@ -27,14 +26,9 @@ crashes loudly rather than masking failures.
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import base64
import binascii
import json
import os import os
import time
import uuid
from datetime import datetime, timezone from datetime import datetime, timezone
from typing import Any, Optional from typing import Optional
from fastapi import FastAPI, Request, Response from fastapi import FastAPI, Request, Response
@@ -56,41 +50,6 @@ _TRANSPARENT_GIF = bytes.fromhex(
) )
# Namespace used by fingerprint generators to derive mint UUID.
# Must stay in sync with fingerprint_html._MINT_NAMESPACE.
_MINT_NAMESPACE = uuid.UUID("a3f7c821-9d1e-4b6a-8c2d-1e4f9a7b3c5d")
# In-memory per-(token_uuid, src_ip) rate limiter for fingerprint persists.
# Maps (token_uuid, src_ip) -> list of monotonic timestamps.
# Not shared across worker restarts or processes — acceptable for MVP.
_FP_RATE_WINDOW_S = 60
_FP_RATE_LIMIT = 30
_fp_rate_buckets: dict[tuple[str, str], list[float]] = {}
def _fp_rate_allowed(token_uuid: str, src_ip: str) -> bool:
key = (token_uuid, src_ip)
now = time.monotonic()
cutoff = now - _FP_RATE_WINDOW_S
bucket = _fp_rate_buckets.get(key, [])
bucket = [t for t in bucket if t > cutoff]
if len(bucket) >= _FP_RATE_LIMIT:
_fp_rate_buckets[key] = bucket
return False
bucket.append(now)
_fp_rate_buckets[key] = bucket
return True
def _is_valid_fp_shape(fp: dict) -> bool:
"""Layer B — structural sanity check on a decoded fingerprint blob."""
if not isinstance(fp.get("mint"), str) or not fp["mint"]:
return False
known_keys = {"nav", "scr", "tz", "cv", "gl", "au", "ft", "rtc"}
present = sum(1 for k in known_keys if isinstance(fp.get(k), dict))
return present >= 3
def _http_base() -> str: def _http_base() -> str:
return os.environ.get("DECNET_CANARY_HTTP_BASE", "http://localhost:8088").rstrip("/") return os.environ.get("DECNET_CANARY_HTTP_BASE", "http://localhost:8088").rstrip("/")
@@ -145,11 +104,6 @@ def _build_app(repo: BaseRepository, bus: BaseBus) -> FastAPI:
@app.get("/c/{slug}") @app.get("/c/{slug}")
async def callback(slug: str, request: Request) -> Response: async def callback(slug: str, request: Request) -> Response:
raw_nonce = request.query_params.get("k")
fp_meta, parsed_fp = _extract_fingerprint(request.query_params)
merged_headers = dict(request.headers)
if fp_meta:
merged_headers.update(fp_meta)
await _record_hit( await _record_hit(
repo, bus, repo, bus,
slug=slug, slug=slug,
@@ -157,9 +111,7 @@ def _build_app(repo: BaseRepository, bus: BaseBus) -> FastAPI:
user_agent=request.headers.get("user-agent"), user_agent=request.headers.get("user-agent"),
request_path=str(request.url.path), request_path=str(request.url.path),
dns_qname=None, dns_qname=None,
raw_headers=merged_headers, raw_headers=dict(request.headers),
parsed_fp=parsed_fp,
raw_nonce=raw_nonce,
) )
# Always 200 with a tiny image so the attacker's client sees # Always 200 with a tiny image so the attacker's client sees
# a "success" — same return regardless of whether the slug is # a "success" — same return regardless of whether the slug is
@@ -177,67 +129,6 @@ def _build_app(repo: BaseRepository, bus: BaseBus) -> FastAPI:
return app return app
# Per-chunk size cap. Real fingerprints fit in one ~3KB GET; honest
# overflow is handled via chunking (s/i/n + d). Anything larger than
# this on a single request is junk, so we drop it instead of letting an
# attacker inflate a trigger row indefinitely.
_FP_CHUNK_MAX = 8 * 1024
def _extract_fingerprint(qp: Any) -> tuple[dict[str, Any], Optional[dict]]:
"""Decode fingerprint-payload query params into (meta_dict, parsed_fp).
The obfuscated browser payload may send three shapes on ``GET /c/<slug>``:
* ``?o=1`` — bare-open beacon, fired before fingerprinting starts.
* ``?d=<b64url-json>`` — single-shot fingerprint dump.
* ``?s=<sid>&i=<idx>&n=<total>&d=<b64url-chunk>`` — chunked dump.
Returns a tuple of:
- ``meta`` — flat dict with ``_fp_*`` keys to merge into raw_headers.
- ``parsed_fp`` — the decoded fingerprint dict for validation, or ``None``
when there's no ``?d=`` or decoding fails.
"""
out: dict[str, Any] = {}
parsed_fp: Optional[dict] = None
if not qp:
return out, parsed_fp
o = qp.get("o") if hasattr(qp, "get") else None
if o:
out["_fp_open"] = "1"
d = qp.get("d") if hasattr(qp, "get") else None
if not d:
return out, parsed_fp
if len(d) > _FP_CHUNK_MAX:
out["_fp_oversize"] = "1"
return out, parsed_fp
sid = qp.get("s")
idx = qp.get("i")
total = qp.get("n")
if sid and idx and total:
out["_fp_sid"] = sid
out["_fp_idx"] = idx
out["_fp_total"] = total
out["_fp_chunk"] = d
return out, parsed_fp
# Single-shot: decode and pass back as parsed_fp; validation runs in
# _record_hit after token lookup so we have the stored nonce at hand.
try:
padded = d + "=" * (-len(d) % 4)
raw = base64.urlsafe_b64decode(padded.encode("ascii"))
parsed = json.loads(raw.decode("utf-8"))
except (binascii.Error, ValueError, UnicodeDecodeError):
out["_fp_decode_error"] = "1"
return out, parsed_fp
if isinstance(parsed, dict):
parsed_fp = parsed
else:
out["_fp_decode_error"] = "1"
return out, parsed_fp
def _client_ip(request: Request) -> str: def _client_ip(request: Request) -> str:
# Honor X-Forwarded-For if the operator deployed behind a reverse # Honor X-Forwarded-For if the operator deployed behind a reverse
# proxy. Take the leftmost address in the chain; everything after # proxy. Take the leftmost address in the chain; everything after
@@ -263,58 +154,16 @@ async def _record_hit(
request_path: Optional[str], request_path: Optional[str],
dns_qname: Optional[str], dns_qname: Optional[str],
raw_headers: Optional[dict], raw_headers: Optional[dict],
parsed_fp: Optional[dict] = None,
raw_nonce: Optional[str] = None,
) -> None: ) -> None:
"""Resolve slug -> token, persist a trigger, publish on the bus. """Resolve slug -> token, persist a trigger, publish on the bus.
Unknown slugs are silently swallowed: returning the same response Unknown slugs are silently swallowed: returning the same response
for known and unknown slugs is the stealth posture, and persisting for known and unknown slugs is the stealth posture, and persisting
every random scan would clutter the DB. every random scan would clutter the DB.
When *parsed_fp* is present (single-shot fingerprint decode succeeded),
it is validated through four layers before being merged into raw_headers:
A) nonce match against CanaryToken.fingerprint_nonce,
B) structural shape check,
C) mint UUID consistency,
D) per-(token, IP) rate limit.
Each failure drops the structured ``_fp`` and sets a ``_fp_*_invalid`` flag.
The trigger row always lands regardless — the GET hit is itself forensic.
""" """
token = await repo.get_canary_token_by_slug(slug) token = await repo.get_canary_token_by_slug(slug)
if token is None: if token is None:
return return
final_headers: dict[str, Any] = dict(raw_headers or {})
if parsed_fp is not None:
stored_nonce: Optional[str] = token.get("fingerprint_nonce")
# Layer A — nonce
if stored_nonce is not None and raw_nonce != stored_nonce:
final_headers["_fp_invalid_nonce"] = "1"
parsed_fp = None
# Layer B — shape (only when nonce passed or no nonce enforced)
if parsed_fp is not None and not _is_valid_fp_shape(parsed_fp):
final_headers["_fp_invalid_shape"] = "1"
parsed_fp = None
# Layer C — mint UUID consistency
if parsed_fp is not None:
expected_mint = str(uuid.uuid5(_MINT_NAMESPACE, slug))
if parsed_fp.get("mint") != expected_mint:
final_headers["_fp_invalid_mint"] = "1"
parsed_fp = None
# Layer D — rate limit
if parsed_fp is not None and not _fp_rate_allowed(token["uuid"], src_ip):
final_headers["_fp_rate_limited"] = "1"
parsed_fp = None
if parsed_fp is not None:
final_headers["_fp"] = parsed_fp
trigger_id = await repo.record_canary_trigger({ trigger_id = await repo.record_canary_trigger({
"token_uuid": token["uuid"], "token_uuid": token["uuid"],
"occurred_at": datetime.now(timezone.utc), "occurred_at": datetime.now(timezone.utc),
@@ -322,7 +171,7 @@ async def _record_hit(
"user_agent": user_agent, "user_agent": user_agent,
"request_path": request_path, "request_path": request_path,
"dns_qname": dns_qname, "dns_qname": dns_qname,
"raw_headers": final_headers, "raw_headers": raw_headers or {},
}) })
try: try:
await bus.publish( await bus.publish(
@@ -340,22 +189,6 @@ async def _record_hit(
except Exception as e: # noqa: BLE001 — best effort except Exception as e: # noqa: BLE001 — best effort
log.warning("canary.triggered publish failed slug=%s err=%s", slug, e) log.warning("canary.triggered publish failed slug=%s err=%s", slug, e)
# Auto-deregister fingerprint canaries after the first valid fingerprint
# is collected. Slug goes dark; the stealth posture means the attacker
# sees the same 200 + GIF on the next hit — nothing reveals the revocation.
# Guard: only fingerprint tokens have a non-NULL fingerprint_nonce; plain
# http/dns canaries are NOT auto-revoked.
if parsed_fp is not None and token.get("fingerprint_nonce") is not None:
try:
await repo.update_canary_token_state(token["uuid"], "revoked")
await bus.publish(
topics.canary(token["uuid"], topics.CANARY_REVOKED),
{"token_id": token["uuid"], "trigger_id": trigger_id,
"reason": "fingerprint_collected"},
)
except Exception as e: # noqa: BLE001 — trigger row already landed; best effort
log.warning("canary.deregister failed token=%s err=%s", token["uuid"], e)
# ---------------------------- DNS surface -------------------------------- # ---------------------------- DNS surface --------------------------------
@@ -381,7 +214,7 @@ async def _start_dns_server(
local_addr=(_dns_bind(), _dns_port()), local_addr=(_dns_bind(), _dns_port()),
) )
log.info("canary.dns listening zone=%s port=%d", zone, _dns_port()) log.info("canary.dns listening zone=%s port=%d", zone, _dns_port())
return transport return transport # type: ignore[return-value]
# ---------------------------- entry point -------------------------------- # ---------------------------- entry point --------------------------------

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
""" """
DECNET CLI — entry point for all commands. DECNET CLI — entry point for all commands.
@@ -40,7 +39,6 @@ from . import (
swarm, swarm,
swarmctl, swarmctl,
topology, topology,
ttp,
updater, updater,
web, web,
webhook, webhook,
@@ -61,7 +59,7 @@ for _mod in (
swarm, swarm,
deploy, lifecycle, workers, inventory, deploy, lifecycle, workers, inventory,
web, profiler, orchestrator, realism, reconciler, sniffer, db, web, profiler, orchestrator, realism, reconciler, sniffer, db,
topology, bus, geoip, init, webhook, canary, ttp, topology, bus, geoip, init, webhook, canary,
): ):
_mod.register(app) _mod.register(app)

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import os import os

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import os import os

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import typer import typer

View File

@@ -1,14 +1,8 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""``decnet canary`` — HTTP + DNS callback receiver for canary tokens. """``decnet canary`` — HTTP + DNS callback receiver for canary tokens.
Two entry points share this module: Worker process. Mirrors the shape of :mod:`decnet.cli.webhook`: a
``@app.command(name="canary")`` Typer entry point that delegates to
* ``decnet canary`` — runs the worker process. Mirrors the shape of :func:`decnet.canary.worker.run`.
:mod:`decnet.cli.webhook`. Invoked by the ``decnet-canary.service``
systemd unit so its argv must stay stable.
* ``decnet canary-install-toolchain`` — provisions the Node side of
the fingerprint-canary obfuscator. Idempotent; safe to call from
the API service unit's ``ExecStartPre``.
Not master-only — any host that hosts deckies can run its own Not master-only — any host that hosts deckies can run its own
canary worker (the bus events stay local; the webhook worker on canary worker (the bus events stay local; the webhook worker on
@@ -17,17 +11,11 @@ in ``development/let-s-move-to-the-enumerated-pike.md``).
""" """
from __future__ import annotations from __future__ import annotations
import shutil
import subprocess # nosec B404 — npm exec is the whole point of the toolchain installer
from pathlib import Path
import typer import typer
from . import utils as _utils from . import utils as _utils
from .utils import console, log from .utils import console, log
_TOOLCHAIN_TIMEOUT_S = 180
def register(app: typer.Typer) -> None: def register(app: typer.Typer) -> None:
@app.command(name="canary") @app.command(name="canary")
@@ -52,53 +40,3 @@ def register(app: typer.Typer) -> None:
asyncio.run(run()) asyncio.run(run())
except KeyboardInterrupt: except KeyboardInterrupt:
console.print("\n[yellow]Canary worker stopped.[/]") console.print("\n[yellow]Canary worker stopped.[/]")
@app.command(name="canary-install-toolchain")
def canary_install_toolchain(
npm_bin: str = typer.Option(
"npm", "--npm-bin", help="Path to the npm executable. Defaults to PATH lookup.",
),
) -> None:
"""Install the Node-side toolchain used by fingerprint canaries.
Runs ``npm install --omit=dev`` under the installed ``decnet/canary/``
directory so the obfuscator's helper script can ``require()``
``javascript-obfuscator`` at mint time. Requires Node >= 18.
Idempotent: re-running on an already-installed tree is fast
(npm short-circuits when ``node_modules/`` is up-to-date).
"""
import decnet.canary as _canary_pkg
canary_dir = Path(_canary_pkg.__file__).resolve().parent
if not (canary_dir / "package.json").is_file():
console.print(
f"[red]canary package.json not found under {canary_dir}; "
"wheel may be missing the JS toolchain payload.[/]"
)
raise typer.Exit(code=2)
if shutil.which(npm_bin) is None:
console.print(
f"[red]npm executable {npm_bin!r} not found on PATH. "
"Install Node >= 18 and re-run.[/]"
)
raise typer.Exit(code=2)
console.print(
f"[cyan]installing canary toolchain[/] in {canary_dir}",
)
try:
proc = subprocess.run( # nosec B603 — argv-form, no shell, fixed cwd, npm_bin checked above
[npm_bin, "install", "--omit=dev", "--no-fund", "--no-audit"],
cwd=str(canary_dir),
capture_output=True, text=True,
timeout=_TOOLCHAIN_TIMEOUT_S, check=False,
)
except subprocess.TimeoutExpired:
console.print("[red]npm install timed out after 3 minutes[/]")
raise typer.Exit(code=3) from None
if proc.returncode != 0:
console.print(
f"[red]npm install failed rc={proc.returncode}[/]\n"
f"{proc.stderr.strip()}"
)
raise typer.Exit(code=proc.returncode)
console.print("[green]canary toolchain ready[/]")

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
from typing import Optional from typing import Optional

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
from typing import Optional from typing import Optional

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Role-based CLI gating. """Role-based CLI gating.
MAINTAINERS: when you add a new Typer command (or add_typer group) that is MAINTAINERS: when you add a new Typer command (or add_typer group) that is
@@ -31,10 +30,6 @@ MASTER_ONLY_COMMANDS: frozenset[str] = frozenset({
"mutate", "listener", "profiler", "mutate", "listener", "profiler",
"services", "distros", "correlate", "archetypes", "web", "services", "distros", "correlate", "archetypes", "web",
"db-reset", "init", "webhook", "clusterer", "campaign-clusterer", "db-reset", "init", "webhook", "clusterer", "campaign-clusterer",
# `ttp` runs on agents — local SMTP decoys persist .eml files into the
# agent's artifacts tree and the EmailLifter disk-reaches them in-process
# (DEBT-047). `ttp-backfill` stays master-only: it walks the master DB.
"ttp-backfill",
}) })
MASTER_ONLY_GROUPS: frozenset[str] = frozenset( MASTER_ONLY_GROUPS: frozenset[str] = frozenset(
{"swarm", "topology", "geoip", "realism"} {"swarm", "topology", "geoip", "realism"}
@@ -70,7 +65,7 @@ def _gate_commands_by_mode(_app: typer.Typer) -> None:
return return
_app.registered_commands = [ _app.registered_commands = [
c for c in _app.registered_commands c for c in _app.registered_commands
if (c.name or (c.callback.__name__ if c.callback else "")) not in MASTER_ONLY_COMMANDS if (c.name or c.callback.__name__) not in MASTER_ONLY_COMMANDS
] ]
_app.registered_groups = [ _app.registered_groups = [
g for g in _app.registered_groups g for g in _app.registered_groups

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""GeoIP CLI — refresh and lookup subcommands (master-only). """GeoIP CLI — refresh and lookup subcommands (master-only).
Usage:: Usage::

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
""" """
`decnet init` — one-shot master-host bootstrap. `decnet init` — one-shot master-host bootstrap.
@@ -45,12 +44,6 @@ _CONFIG_PLACEHOLDER = """\
# EnvironmentFile= — never in a group-readable INI. # EnvironmentFile= — never in a group-readable INI.
[decnet] [decnet]
# DECNET-service user/group as configured at `decnet init` time.
# Resolved to a uid/gid on each host at deploy time via pwd.getpwnam,
# so the same user name can have different numeric uids on master vs
# agents without breaking artifact ownership.
api-user = {api_user}
api-group = {api_group}
# mode = master # or "agent" # mode = master # or "agent"
# [api] # [api]
@@ -81,7 +74,6 @@ api-group = {api_group}
# master-host = 10.0.0.1 # master-host = 10.0.0.1
# syslog-port = 6514 # syslog-port = 6514
# swarmctl-port = 8770 # swarmctl-port = 8770
# swarmctl-host = 127.0.0.1
# [logging] # [logging]
# system-log = /var/log/decnet/decnet.system.log # system-log = /var/log/decnet/decnet.system.log
@@ -205,17 +197,14 @@ def _ensure_dir(
return f"skip: {path} already present" if existed else "ok" return f"skip: {path} already present" if existed else "ok"
def _ensure_config( def _ensure_config(path: Path, group: str, *, dry_run: bool) -> str:
path: Path, group: str, *, user: str, dry_run: bool,
) -> str:
if path.exists(): if path.exists():
return f"skip: {path} already present" return f"skip: {path} already present"
if dry_run: if dry_run:
console.print(f" [dim]would write:[/] {path}") console.print(f" [dim]would write:[/] {path}")
return "ok" return "ok"
path.parent.mkdir(parents=True, exist_ok=True) path.parent.mkdir(parents=True, exist_ok=True)
rendered = _CONFIG_PLACEHOLDER.format(api_user=user, api_group=group) path.write_text(_CONFIG_PLACEHOLDER)
path.write_text(rendered)
try: try:
os.chmod(path, 0o640) os.chmod(path, 0o640)
gid = grp.getgrnam(group).gr_gid gid = grp.getgrnam(group).gr_gid
@@ -612,7 +601,7 @@ def register(app: typer.Typer) -> None:
# (Path("/"). / "/opt/decnet" == Path("/opt/decnet"), dropping pfx). # (Path("/"). / "/opt/decnet" == Path("/opt/decnet"), dropping pfx).
_install_rel = install_dir.lstrip("/") _install_rel = install_dir.lstrip("/")
required_tools: tuple[str, ...] = ("systemctl",) if deinit else ( required_tools = ("systemctl",) if deinit else (
"systemctl", "useradd", "groupadd", "systemd-tmpfiles", "systemctl", "useradd", "groupadd", "systemd-tmpfiles",
) )
if deinit: if deinit:
@@ -669,7 +658,7 @@ def register(app: typer.Typer) -> None:
) )
_step( _step(
"systemctl daemon-reload", "systemctl daemon-reload",
lambda: (_run(["systemctl", "daemon-reload"], dry_run=dry_run), "ok")[1], # type: ignore[func-returns-value] lambda: (_run(["systemctl", "daemon-reload"], dry_run=dry_run), "ok")[1],
) )
_step( _step(
f"remove {etc_decnet / 'decnet.ini'}", f"remove {etc_decnet / 'decnet.ini'}",
@@ -765,13 +754,6 @@ def register(app: typer.Typer) -> None:
(pfx / _install_rel, 0o755, user, group), (pfx / _install_rel, 0o755, user, group),
(pfx / "var/lib/decnet", 0o750, user, group), (pfx / "var/lib/decnet", 0o750, user, group),
(pfx / "var/lib/decnet/geoip", 0o755, user, group), (pfx / "var/lib/decnet/geoip", 0o755, user, group),
# DEBT-035 / DEBT-047: artifact root carries setgid (the
# 0o2... bit) so every file written under it inherits the
# decnet group regardless of which container's uid created
# it. Group-write (0o2775) lets the API process and the
# local TTP worker read each other's outputs without a
# manual chown after every fresh deploy.
(pfx / "var/lib/decnet/artifacts", 0o2775, user, group),
(pfx / "var/log/decnet", 0o750, user, group), (pfx / "var/log/decnet", 0o750, user, group),
(etc_decnet, 0o755, "root", group), (etc_decnet, 0o755, "root", group),
(pfx / "run/decnet", 0o755, "root", group), (pfx / "run/decnet", 0o755, "root", group),
@@ -793,15 +775,12 @@ def register(app: typer.Typer) -> None:
for path, mode, d_owner, d_group in dirs: for path, mode, d_owner, d_group in dirs:
_step( _step(
f"ensure dir {path}", f"ensure dir {path}",
lambda p=path, m=mode, o=d_owner, g=d_group: # type: ignore[misc] lambda p=path, m=mode, o=d_owner, g=d_group:
_ensure_dir(p, mode=m, owner=o, group=g, dry_run=dry_run), _ensure_dir(p, mode=m, owner=o, group=g, dry_run=dry_run),
) )
_step( _step(
f"write {etc_decnet / 'decnet.ini'}", f"write {etc_decnet / 'decnet.ini'}",
lambda: _ensure_config( lambda: _ensure_config(etc_decnet / "decnet.ini", group, dry_run=dry_run),
etc_decnet / "decnet.ini", group,
user=user, dry_run=dry_run,
),
) )
_step( _step(
"install systemd units", "install systemd units",
@@ -833,7 +812,7 @@ def register(app: typer.Typer) -> None:
) )
_step( _step(
"systemctl daemon-reload", "systemctl daemon-reload",
lambda: (_run(["systemctl", "daemon-reload"], dry_run=dry_run), "ok")[1], # type: ignore[func-returns-value] lambda: (_run(["systemctl", "daemon-reload"], dry_run=dry_run), "ok")[1],
) )
if no_start: if no_start:
@@ -844,7 +823,7 @@ def register(app: typer.Typer) -> None:
_step( _step(
"systemctl enable --now decnet.target", "systemctl enable --now decnet.target",
lambda: ( lambda: (
_run( # type: ignore[func-returns-value] _run(
["systemctl", "enable", "--now", "decnet.target"], ["systemctl", "enable", "--now", "decnet.target"],
dry_run=dry_run, dry_run=dry_run,
), ),

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import typer import typer

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import subprocess # nosec B404 import subprocess # nosec B404

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
from typing import Optional from typing import Optional

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import typer import typer

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""``decnet realism ...`` — content-engine maintenance commands. """``decnet realism ...`` — content-engine maintenance commands.
After stage 5 of the realism migration, this is the only remaining After stage 5 of the realism migration, this is the only remaining

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import typer import typer

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import typer import typer

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""`decnet swarm ...` — master-side operator commands (HTTP to local swarmctl).""" """`decnet swarm ...` — master-side operator commands (HTTP to local swarmctl)."""
from __future__ import annotations from __future__ import annotations

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import os import os
@@ -17,16 +16,8 @@ from .utils import console, log
def register(app: typer.Typer) -> None: def register(app: typer.Typer) -> None:
@app.command() @app.command()
def swarmctl( def swarmctl(
port: int = typer.Option( port: int = typer.Option(8770, "--port", help="Port for the swarm controller"),
8770, "--port", host: str = typer.Option("127.0.0.1", "--host", help="Bind address for the swarm controller"),
envvar="DECNET_SWARMCTL_PORT",
help="Port for the swarm controller. Defaults to [swarm] swarmctl-port from /etc/decnet/decnet.ini, else 8770.",
),
host: str = typer.Option(
"127.0.0.1", "--host",
envvar="DECNET_SWARMCTL_HOST",
help="Bind address for the swarm controller. Defaults to [swarm] swarmctl-host from /etc/decnet/decnet.ini, else 127.0.0.1.",
),
daemon: bool = typer.Option(False, "--daemon", "-d", help="Detach to background as a daemon process"), daemon: bool = typer.Option(False, "--daemon", "-d", help="Detach to background as a daemon process"),
no_listener: bool = typer.Option(False, "--no-listener", help="Do not auto-spawn the syslog-TLS listener alongside swarmctl"), no_listener: bool = typer.Option(False, "--no-listener", help="Do not auto-spawn the syslog-TLS listener alongside swarmctl"),
tls: bool = typer.Option(False, "--tls", help="Serve over HTTPS with mTLS (required for cross-host worker heartbeats)"), tls: bool = typer.Option(False, "--tls", help="Serve over HTTPS with mTLS (required for cross-host worker heartbeats)"),

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""MazeNET topology CLI: generate / deploy / teardown / list / show.""" """MazeNET topology CLI: generate / deploy / teardown / list / show."""
from __future__ import annotations from __future__ import annotations
@@ -234,8 +233,8 @@ def _delete(
topo = await repo.get_topology(topology_id) topo = await repo.get_topology(topology_id)
if topo is None: if topo is None:
return False, "not-found" return False, "not-found"
if topo.status in _RUNNING: if topo["status"] in _RUNNING:
return False, str(topo.status) return False, str(topo["status"])
ok = await repo.delete_topology_cascade(topology_id) ok = await repo.delete_topology_cascade(topology_id)
return ok, None return ok, None

View File

@@ -1,310 +0,0 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""``decnet ttp`` — TTP-tagging worker and admin commands.
Two flat commands share this module:
* ``decnet ttp`` — runs the long-running tagger worker. Bus-woken on
``attacker.session.ended`` / ``attacker.observed`` /
``attacker.intel.enriched`` / ``identity.{formed,merged}`` /
``credential.reuse.detected`` / ``email.received`` / ``canary.>``;
dispatches each event through :class:`CompositeTagger` (RuleEngine +
Behavioral / Intel / CanaryFingerprint / Email / Identity / Credential
lifters), persists ``ttp_tag`` rows via the idempotent
``INSERT OR IGNORE`` write, and publishes ``ttp.tagged`` +
``ttp.rule.fired.<technique_id>`` only when the insert returned a
non-zero rowcount (loop-prevention invariant from TTP_TAGGING.md
§"Bus topics"). Invoked by the ``decnet-ttp.service`` systemd unit
so its argv must stay stable.
* ``decnet ttp-backfill`` — replays historical events (shell commands
recorded on :class:`Attacker.commands`, :class:`CanaryTrigger` rows)
through the live tagger. Writes ``ttp_tag`` rows using the same
idempotent insert path. **Does not publish** to the bus — replay must
not re-trigger SIEM/webhook fan-out on already-attributed events.
Both are master-only — gated via ``MASTER_ONLY_COMMANDS`` in
:mod:`decnet.cli.gating`.
"""
from __future__ import annotations
import asyncio
import time
from datetime import datetime, timedelta, timezone
from typing import Any
import typer
from decnet.ttp.factory import CompositeTagger, get_tagger
from . import utils as _utils
from .utils import console, log
_BACKFILL_SOURCES = ("command", "canary", "all")
def register(app: typer.Typer) -> None:
@app.command(name="ttp")
def ttp(
poll_interval_secs: float = typer.Option(
60.0, "--poll-interval", "-i",
help="Slow-tick fallback when the bus is idle or unavailable (seconds)",
),
daemon: bool = typer.Option(
False, "--daemon", "-d",
help="Detach to background as a daemon process",
),
) -> None:
"""TTP-tagging worker — MITRE ATT&CK technique tagging."""
from decnet.ttp.worker import run_ttp_worker_loop
from decnet.web.dependencies import repo
if daemon:
log.info("ttp daemonizing poll=%s", poll_interval_secs)
_utils._daemonize()
log.info("ttp command invoked poll=%s", poll_interval_secs)
console.print(
f"[bold cyan]TTP tagging worker starting[/] "
f"poll={poll_interval_secs}s"
)
console.print("[dim]Press Ctrl+C to stop[/]")
async def _run() -> None:
await repo.initialize()
await run_ttp_worker_loop(
repo, poll_interval_secs=poll_interval_secs,
)
try:
asyncio.run(_run())
except KeyboardInterrupt:
console.print("\n[yellow]TTP tagging worker stopped.[/]")
@app.command(name="ttp-backfill")
def ttp_backfill(
since_days: int = typer.Option(
7, "--since-days", "-s",
min=1, max=3650,
help="Replay events whose source row is newer than N days ago.",
),
source: str = typer.Option(
"all", "--source",
help=f"Source slice to replay. One of: {', '.join(_BACKFILL_SOURCES)}.",
),
dry_run: bool = typer.Option(
False, "--dry-run",
help="Run the tagger but skip insert_tags. Reports counts only.",
),
batch_size: int = typer.Option(
500, "--batch-size",
min=1, max=100_000,
help="Number of tags accumulated before each repo.insert_tags call.",
),
) -> None:
"""Replay historical attacker activity through the live tagger.
Walks ``Attacker.commands`` (per-IP shell-command history) and
``CanaryTrigger`` (canary callback log) since N days ago,
builds the same :class:`TaggerEvent` shape the live worker
emits, and persists tags via the idempotent INSERT OR IGNORE
write. Re-running is safe — a second pass over identical
source rows reports ``inserted=0``.
Bus publish is intentionally suppressed; SIEM / webhook fan-out
sees only live events, never replays.
"""
from decnet.cli.gating import _require_master_mode
from decnet.web.dependencies import repo
_require_master_mode("ttp-backfill")
if source not in _BACKFILL_SOURCES:
console.print(
f"[red]invalid --source {source!r}; expected one of "
f"{_BACKFILL_SOURCES}[/]"
)
raise typer.Exit(code=2)
cutoff = datetime.now(tz=timezone.utc) - timedelta(days=since_days)
console.print(
f"[bold cyan]TTP backfill[/] since={cutoff.isoformat()} "
f"source={source} dry_run={dry_run} batch_size={batch_size}"
)
async def _run() -> None:
await repo.initialize()
await _backfill(
repo,
cutoff=cutoff,
sources=_resolve_sources(source),
dry_run=dry_run,
batch_size=batch_size,
)
try:
asyncio.run(_run())
except KeyboardInterrupt:
console.print("\n[yellow]Backfill interrupted.[/]")
def _resolve_sources(name: str) -> tuple[str, ...]:
if name == "all":
return ("command", "canary")
return (name,)
async def _backfill(
repo: Any,
*,
cutoff: datetime,
sources: tuple[str, ...],
dry_run: bool,
batch_size: int,
) -> None:
"""Drive the per-source backfill loops and report structured counts.
One :class:`CompositeTagger` is built once and reused for every
source — the per-lifter watch fan-out the live worker performs is
inlined here as a `watch_store()` startup task per
:class:`WatchableTagger`, so the dispatch indexes hydrate before
we start feeding events.
"""
# Import-time bound so tests can monkeypatch ``decnet.cli.ttp.get_tagger``
# to inject a recording fake without touching the global factory.
tagger = get_tagger()
watch_tasks: list[asyncio.Task[None]] = []
if isinstance(tagger, CompositeTagger):
for watchable in tagger.iter_watchables():
watch_tasks.append(asyncio.create_task(watchable.watch_store()))
# Yield once so each watch_store gets a chance to run its
# initial `load_compiled` before we feed the first event.
await asyncio.sleep(0.05)
try:
if "command" in sources:
await _backfill_commands(
repo, tagger, cutoff=cutoff,
dry_run=dry_run, batch_size=batch_size,
)
if "canary" in sources:
await _backfill_canaries(
repo, tagger, cutoff=cutoff,
dry_run=dry_run, batch_size=batch_size,
)
finally:
for task in watch_tasks:
task.cancel()
for task in watch_tasks:
try:
await task
except (asyncio.CancelledError, Exception): # noqa: BLE001
pass
async def _backfill_commands(
repo: Any,
tagger: Any,
*,
cutoff: datetime,
dry_run: bool,
batch_size: int,
) -> None:
from decnet.ttp.base import TaggerEvent
started = time.monotonic()
rows_seen = 0
cmds_seen = 0
inserted = 0
pending: list[Any] = []
async for attacker, commands in repo.iter_attacker_commands_since(cutoff):
rows_seen += 1
for idx, cmd in enumerate(commands):
cmds_seen += 1
text = cmd.get("command_text") or cmd.get("text")
if not isinstance(text, str):
continue
cmd_id = (
cmd.get("id")
or cmd.get("uuid")
or cmd.get("command_id")
or f"{attacker.uuid}#cmd{idx}"
)
event = TaggerEvent(
source_kind="command",
source_id=str(cmd_id),
attacker_uuid=attacker.uuid,
identity_uuid=getattr(attacker, "identity_id", None),
session_id=cmd.get("session_id"),
decky_id=cmd.get("decky_id") or cmd.get("decky"),
payload={**cmd, "command_text": text},
)
tags = await tagger.tag(event)
if tags:
pending.extend(tags)
if len(pending) >= batch_size:
inserted += await _flush(repo, pending, dry_run)
pending = []
if pending:
inserted += await _flush(repo, pending, dry_run)
elapsed = time.monotonic() - started
console.print(
f"source=command rows={rows_seen} commands={cmds_seen} "
f"inserted={inserted} dry_run={dry_run} elapsed_s={elapsed:.2f}"
)
async def _backfill_canaries(
repo: Any,
tagger: Any,
*,
cutoff: datetime,
dry_run: bool,
batch_size: int,
) -> None:
from decnet.ttp.base import TaggerEvent
started = time.monotonic()
rows_seen = 0
inserted = 0
pending: list[Any] = []
async for trigger in repo.iter_canary_triggers_since(cutoff):
rows_seen += 1
event = TaggerEvent(
source_kind="canary_fingerprint",
source_id=trigger.uuid,
attacker_uuid=trigger.attacker_id,
identity_uuid=None,
session_id=None,
decky_id=None,
payload={
"token_uuid": trigger.token_uuid,
"src_ip": trigger.src_ip,
"ua_signature": trigger.user_agent or "",
"user_agent": trigger.user_agent,
"request_path": trigger.request_path,
"dns_qname": trigger.dns_qname,
"headers": trigger.headers(),
},
)
tags = await tagger.tag(event)
if tags:
pending.extend(tags)
if len(pending) >= batch_size:
inserted += await _flush(repo, pending, dry_run)
pending = []
if pending:
inserted += await _flush(repo, pending, dry_run)
elapsed = time.monotonic() - started
console.print(
f"source=canary rows={rows_seen} inserted={inserted} "
f"dry_run={dry_run} elapsed_s={elapsed:.2f}"
)
async def _flush(repo: Any, tags: list[Any], dry_run: bool) -> int:
if dry_run:
return 0
return int(await repo.insert_tags(tags))

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import pathlib as _pathlib import pathlib as _pathlib

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Shared CLI helpers: console, logger, process management, swarm HTTP client. """Shared CLI helpers: console, logger, process management, swarm HTTP client.
Submodules reference these as ``from . import utils`` then ``utils.foo(...)`` Submodules reference these as ``from . import utils`` then ``utils.foo(...)``
@@ -12,7 +11,7 @@ import signal
import subprocess # nosec B404 import subprocess # nosec B404
import sys import sys
from pathlib import Path from pathlib import Path
from typing import Any, Callable, Optional from typing import Optional
import typer import typer
from rich.console import Console from rich.console import Console
@@ -97,7 +96,7 @@ def _is_running(match_fn) -> int | None:
return None return None
def _service_registry(log_file: str) -> list[tuple[str, Callable[..., Any], list[str]]]: def _service_registry(log_file: str) -> list[tuple[str, callable, list[str]]]:
"""Return the microservice registry for health-check and relaunch. """Return the microservice registry for health-check and relaunch.
On agents these run as systemd units invoking /usr/local/bin/decnet, On agents these run as systemd units invoking /usr/local/bin/decnet,
@@ -196,7 +195,7 @@ _DEFAULT_SWARMCTL_URL = "http://127.0.0.1:8770"
def _swarmctl_base_url(url: Optional[str]) -> str: def _swarmctl_base_url(url: Optional[str]) -> str:
return url or os.environ.get("DECNET_SWARMCTL_URL") or _DEFAULT_SWARMCTL_URL return url or os.environ.get("DECNET_SWARMCTL_URL", _DEFAULT_SWARMCTL_URL)
def _http_request(method: str, url: str, *, json_body: Optional[dict] = None, timeout: float = 30.0): def _http_request(method: str, url: str, *, json_body: Optional[dict] = None, timeout: float = 30.0):

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import typer import typer

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
import typer import typer

View File

@@ -1,4 +1,3 @@
# SPDX-License-Identifier: AGPL-3.0-or-later
from __future__ import annotations from __future__ import annotations
from typing import Optional from typing import Optional
@@ -193,70 +192,6 @@ def register(app: typer.Typer) -> None:
except KeyboardInterrupt: except KeyboardInterrupt:
console.print("\n[yellow]Reuse correlator stopped.[/]") console.print("\n[yellow]Reuse correlator stopped.[/]")
@app.command(name="attribution")
def attribution(
multi_actor_tick_secs: float = typer.Option(
60.0, "--multi-actor-tick", "-t",
help=(
"Cross-primitive multi_actor correlator tick interval (seconds). "
"Walks attribution_state for identities flagged on >= 2 "
"primitives and emits attribution.profile.multi_actor_suspected."
),
),
daemon: bool = typer.Option(
False, "--daemon", "-d",
help="Detach to background as a daemon process",
),
) -> None:
"""Attribution engine v0 — per-(identity, primitive) state machine.
Subscribes to ``attacker.observation.>`` and, for each event,
ensures a stub identity row, runs the merger over the full
per-(identity, primitive) observation series, upserts the
derived state, and publishes
``attribution.profile.state_changed`` only on transition.
Periodic tick fires
``attribution.profile.multi_actor_suspected`` when >= 2
primitives flag the same identity.
Closes DEBT-051. Bright-line scope: behavioural coherence and
drift only — never persona attribution to natural persons.
"""
import asyncio
from decnet.correlation.attribution_worker import (
run_attribution_loop,
)
from decnet.web.dependencies import repo
if daemon:
log.info(
"attribution worker daemonizing tick=%s",
multi_actor_tick_secs,
)
_utils._daemonize()
log.info(
"attribution worker command invoked tick=%s",
multi_actor_tick_secs,
)
console.print(
f"[bold cyan]Attribution engine starting[/] "
f"multi_actor_tick={multi_actor_tick_secs}s"
)
console.print("[dim]Press Ctrl+C to stop[/]")
async def _run() -> None:
await repo.initialize()
await run_attribution_loop(
repo,
multi_actor_tick_secs=multi_actor_tick_secs,
)
try:
asyncio.run(_run())
except KeyboardInterrupt:
console.print("\n[yellow]Attribution engine stopped.[/]")
@app.command(name="clusterer") @app.command(name="clusterer")
def clusterer( def clusterer(
poll_interval_secs: float = typer.Option( poll_interval_secs: float = typer.Option(
@@ -360,10 +295,3 @@ def register(app: typer.Typer) -> None:
asyncio.run(_run()) asyncio.run(_run())
except KeyboardInterrupt: except KeyboardInterrupt:
console.print("\n[yellow]Campaign clusterer stopped.[/]") console.print("\n[yellow]Campaign clusterer stopped.[/]")
# ``decnet ttp`` and ``decnet ttp-backfill`` moved to
# :mod:`decnet.cli.ttp` — the TTP CLI surface (worker + admin verbs)
# is colocated there, mirroring the per-feature CLI split used by
# :mod:`decnet.cli.canary`, :mod:`decnet.cli.webhook`, etc. The
# ``decnet-ttp.service`` systemd unit's ExecStart still resolves to
# ``decnet ttp`` because the command name is unchanged.

Some files were not shown because too many files have changed in this diff Show More