23 Commits

Author SHA1 Message Date
0b9873982d refactor(realism): move emailgen LLM/personas/prompt into shared library
Lift the format-agnostic pieces from decnet/orchestrator/emailgen/
into the new decnet/realism/ library so file-class content generation
(stage 3 of the realism migration) can reuse them. Email-specific
delivery (RFC 2822 EML, IMAP/POP3 spool, thread chains) stays in
orchestrator/.

Renames (history-preserving git mv):
  emailgen/personas.py     -> realism/personas.py
  emailgen/prompt.py       -> realism/prompts/email.py
  emailgen/global_pool.py  -> realism/personas_pool.py
  emailgen/llm/            -> realism/llm/

Env-var clean break (pre-v1, no aliases):
  DECNET_EMAILGEN_LLM      -> DECNET_REALISM_LLM
  DECNET_EMAILGEN_MODEL    -> DECNET_REALISM_MODEL
  DECNET_EMAILGEN_TIMEOUT  -> DECNET_REALISM_TIMEOUT
  DECNET_EMAILGEN_PERSONAS -> DECNET_REALISM_PERSONAS
  DECNET_EMAILGEN_FAKE_OUTPUT -> DECNET_REALISM_FAKE_OUTPUT

Importers rewritten in: orchestrator/emailgen/scheduler.py,
orchestrator/drivers/email.py, web/router/{emailgen,topology}/
api_personas.py, cli/emailgen.py. Tests for moved modules relocated
to tests/realism/; tests for stay-put modules updated in place.

API URL `/api/v1/emailgen/personas` and CLI `decnet emailgen
import-personas` keep their public names until the service-collapse
commit (stage 5).
2026-04-27 16:05:43 -04:00
ee176a6f79 Revert "feat(mazenet): per-LAN swarm host pin"
This reverts commit 0d92170a57.
2026-04-25 03:26:19 -04:00
0d92170a57 feat(mazenet): per-LAN swarm host pin
Adds nullable LAN.host_uuid (FK swarm_hosts.uuid). Resolution order
when deploying a LAN: lan.host_uuid → topology.target_host_uuid →
master. A LAN is one Docker bridge so the bridge cannot span hosts;
this pin forces every decky in the LAN onto the named host.

LANCreateRequest / LANUpdateRequest accept host_uuid; both validate
that the host exists, returning 400 on unknown UUIDs. PATCH still
gated by the existing pending-only guard, so reassignment of a live
LAN is not yet possible (deferred to mutator support).

LANRow surfaces the field so the frontend can render per-host badges.
2026-04-25 03:04:23 -04:00
05d225ae38 fix(engine): surface CalledProcessError.stderr in deploy-failure log + status reason
str(CalledProcessError) is just 'Command ... returned non-zero exit
status N' — the stderr (where the buildx recovery hint lives) was
being silently dropped from both the deploy log line and the
persisted 'failed' status reason.

New _format_subprocess_error helper appends .stderr when the
exception is a CalledProcessError. Applied to transition_status
reason and the background-deploy log message so operators and the
UI see the real failure, not just the exit code.

This is what makes the buildx preflight hint from 86b9dec actually
reach the user.
2026-04-24 19:31:37 -04:00
c214cdd7bb fix(api/topology): map duplicate-name IntegrityError to 409
POST /topologies raised a 500 with a raw SQLAlchemy IntegrityError
traceback when the name collided with an existing topology. Catch
the error at the router, verify it's the ix_topologies_name
constraint (so unrelated integrity failures still surface as 500s
with their real traceback), and return 409 with a helpful detail.

Test covers the create-then-duplicate-create flow.
2026-04-24 19:06:37 -04:00
f3408d5e62 fix(topology/allocator): widen default subnet base to /12 for mass-scale
A 30-LAN generate request already fits in 172.20.0.0/16, but trees
with depth/branching that multiply past 256 (e.g. depth=6,
branching=4 ≈ 5k LANs) hit AllocatorExhausted before the first
write.

SubnetAllocator now accepts a full CIDR base ("172.16.0.0/12" →
4096 /24s) in addition to the legacy two-octet shorthand ("172.20",
auto-lifted to /16). The parent must be ≤/24; a /24 base yields
exactly one slot. Iteration order is preserved for /16 bases so
existing topologies keep their third-octet sweep; /12 adds a
second-octet dimension underneath.

Defaults bumped to 172.16.0.0/12: TopologyConfig.subnet_base_prefix,
/next-subnet query param, and the mutator's add-LAN fallback. The
field pattern widens to accept CIDR. create-blank and manual LAN
CRUD still use "10.0" (lifts to /16) — one DMZ LAN per topology,
256 is plenty.
2026-04-24 18:57:55 -04:00
162f7c1194 feat(api/sse): per-user connection cap + viewer-safe invariant
New decnet/web/sse_limits.py provides sse_connection_slot, an async
context manager that counts live SSE connections per user UUID and
raises 429 when a per-user cap is exceeded (default 5, override via
DECNET_SSE_MAX_PER_USER). Wired into both SSE generators as their
first async with, so the cap check fires before any stream data is
yielded.

The cap must sit inside the generator — StreamingResponse returns
before the generator body runs, so a handler-level wrapper would
release the slot immediately. Put prefetch + slot + loop all under
the one async with.

Also documents F6/I (role leakage) as mitigated-by-construction via
handler docstrings: every event type on both streams wraps data
already reachable via viewer-gated REST, so no per-event filter is
needed until a new event family is introduced. The invariant is
written into the handler docstrings so a future PR can't silently
add admin-only events.

Resolves THREAT_MODEL F6/I and F6/D.
2026-04-24 15:01:20 -04:00
df84981954 feat(api): pin response_model on dict-returning mutation routes
Every mutation route that returned an untyped dict now declares
response_model at the decorator. MessageResponse covers the eight
{"message": ...} envelopes (change-password, mutate-decky, mutate-
interval, update-deployment-limit, update-global-mutation-interval,
delete-user, update-user-role, reset-user-password). Purpose-built
models cover the richer shapes (DeployResponse for /deckies/deploy,
PurgeResponse for /config/reinit, ReapReportResponse for /reap-orphans,
UserResponse for /config/users). 204-No-Content and Response/
ORJSONResponse routes stay as-is.

The wire shape for clients is unchanged — the envelopes already only
shipped a message field. What changes is that a handler which
accidentally returns a richer dict (e.g. a full user row including
password_hash) would be silently stripped to the declared fields at
serialization time.

Also flips F4/D "expensive LIKE" to accepted (new DA-09) — the /logs
and /attackers search routes LIKE-scan unbounded columns, but both are
admin-gated, limit-capped, and operator rate-limit scope per DA-04.
FTS5 stays a performance TODO, not a security blocker.
2026-04-24 14:27:58 -04:00
a935bf2663 feat(api): cap offset on list-topologies and transcript endpoints
The other five query endpoints (/logs, /attackers, /attacker-commands,
/bounties, /topologies/{id}) already declared le=2147483647 on offset;
these two were inconsistently uncapped. Bring them in line to close
the F4/D deep-pagination row.

Also resolves F4/T (ORM sort injection — already mitigated by the
regex pattern on /attackers sort_by, no other route accepts a column
name) and F4/D (limit cap — already universal) with code pointers.
2026-04-24 14:14:25 -04:00
6f537f52c2 fix(topology): remove DMZ gateway auto-attach on LAN create
POST /topologies/{id}/lans previously called _auto_attach_gateway()
whenever a non-DMZ LAN was created, which wired the DMZ gateway decky
to every new subnet. That's why a deployed gateway ended up with
eth0..ethN on every LAN regardless of what the user drew in MazeNET.

Drop the auto-attach helper entirely. The DMZ_ORPHAN deploy-time
validator (decnet/topology/validate.py:65-110) stays strict — users
must explicitly wire the gateway to each subnet they want bridged,
which is the whole point of having a topology editor.

useMazeApi.ts: drop stale auto-bridge reference from comment.
2026-04-22 17:14:09 -04:00
8f25ff677f feat(engine,api): add orphan topology resource reaper
Topology rows deleted without a proper teardown leave Docker containers
and bridge networks behind, holding IPAM pools that cause 403 "Pool
overlaps" on the next deploy at the same subnet.

- engine/reaper.py walks the local Docker daemon, extracts the 8-char
  topology prefix from every decnet_t_* resource, and force-removes
  containers + networks whose prefix is not in the repo.
- POST /api/v1/topologies/reap-orphans (admin-only) returns a report
  of live/orphan prefixes and what was removed.
- Resources belonging to live topologies are never touched; per-resource
  errors are captured without aborting the sweep.
2026-04-21 22:13:44 -04:00
f611e7363b feat(mutator,web): live topology mutation pipeline backend (DEBT-030)
Wire the mutator and web API into the service bus so live-topology
edits flow sub-second from enqueue to UI:

- Mutator publishes every state transition on the bus (mutation.applying
  /applied/failed + topology.status). Fire-and-forget; DB stays source
  of truth.
- Mutator watch loop subscribes to topology.*.mutation.enqueued and
  wakes early via asyncio.Event — the 10s poll becomes a fallback
  heartbeat, not the primary dispatch trigger.
- POST /topologies/{id}/mutations publishes mutation.enqueued after
  the DB write succeeds.
- New GET /topologies/{id}/events SSE route: snapshot on connect
  (status + in-flight mutations), live forwards topology.{id}.>
  bus events, 15s keepalive. ?token= auth mirrors /stream.
- New decnet/bus/app.py — process-wide lazy bus singleton for the
  API, closed cleanly on lifespan shutdown.
2026-04-21 14:38:25 -04:00
071312fc0c feat(web/api): expose archetype catalog endpoint
/api/v1/topologies/archetypes returns the archetype registry (slug,
display name, description, preferred services/distros, nmap_os
fingerprint) so the frontend wizard can render a live catalog instead
of hardcoding a copy.
2026-04-21 10:24:01 -04:00
5a0cf5d7c8 feat(topology): add target_host_uuid to pin topologies to swarm agents
Adds the `target_host_uuid` FK on `Topology` plus wiring through the
two create endpoints (`POST /topologies`, `POST /topologies/blank`).
Validates the mode/host pair: `mode='agent'` now requires a known,
routable host; `mode='unihost'` must leave the field unset.
Surfaced on `TopologySummary` so list/detail responses expose it.
Purely additive at the schema level — existing unihost flows unchanged
(field defaults to `NULL`).

Step 1 of the agent <-> topology integration.
2026-04-21 01:19:45 -04:00
b261e8e5fa feat(topology): add teardown endpoint + UI button
Active/degraded/failed/deploying topologies cannot be deleted
without first transitioning to torn_down, but the UI had no way
to trigger that. Add POST /topologies/{id}/teardown mirroring the
deploy endpoint (background task, 202 Accepted), and a
click-to-arm TEARDOWN button on the topology list card that shows
whenever the row is in a teardown-eligible state.
2026-04-20 23:41:37 -04:00
be4e1b1891 feat(mazenet): auto-bridge new LANs to the DMZ gateway
When a non-DMZ LAN is created via POST /lans, look up the topology's
gateway (decky with forwards_l3=True attached to the DMZ) and insert
an edge binding it to the new LAN. The gateway becomes multi-homed
to every internal LAN automatically, so DMZ_ORPHAN cannot arise
from ordinary editor use.

Also fixes delete_lan: the home-decky guard used scalar_one_or_none,
which blew up when the gateway already had >1 'other' LAN edge.
Switch to scalars().first() — we only need to know *some* other
edge exists, not a unique one.
2026-04-20 23:07:19 -04:00
cc9765e54e fix(mazenet): drop fictional host-mode on DMZ gateway stub
POST /topologies/blank seeded the gateway decky with
archetype=host-gateway + network_mode=host, but neither was wired:
no compose writer reads network_mode and host-gateway is not a real
archetype. Replace with archetype=deaddeck + forwards_l3=true so the
gateway is a normal multi-homed bridge decky, consistent with how
compose.py interprets forwards_l3 (sysctl + NET_ADMIN).

Edge marked is_bridge=true, forwards_l3=true so downstream readers
(generator, compose, validator) see a real bridge attachment.
2026-04-20 23:06:54 -04:00
d06b04221f feat(api/topology): live mutation queue endpoints (POST/GET /mutations) 2026-04-20 19:38:55 -04:00
ff0b2efbb0 feat(api/topology): pending-only child CRUD for LANs, deckies, edges 2026-04-20 19:37:16 -04:00
999113e3c3 feat(api/topology): POST/DELETE/deploy endpoints for MazeNET topologies 2026-04-20 19:34:35 -04:00
38db76dd14 fix(api): document 400 on topology read endpoints for schemathesis contract
DECNET's app-level RequestValidationError handler remaps structural
422→400, including query/path constraint violations (limit bounds,
the next-subnet base pattern, etc.).  Schemathesis fuzzing will drive
those code paths and fail response_schema_conformance unless 400 is
declared in responses={}.  Adds the entry to every phase-3 read route.
2026-04-20 18:30:32 -04:00
f182c98ffa feat(api): phase 3 step 2 — topology read endpoints (list/get/status/catalog)
GET /api/v1/topologies — paginated list with status filter. Extends
repo.list_topologies() to accept limit/offset and adds count_topologies()
for the total envelope field.

GET /api/v1/topologies/{id} — hydrated TopologyDetail; 404 if missing.
GET /api/v1/topologies/{id}/status-events — audit trail, limit-capped.

Catalog helpers for the phase-4 canvas UI:
* GET /topologies/services — full service catalog.
* GET /topologies/next-subnet?base=172.20 — wraps SubnetAllocator against
  reserved_subnets across non-torn-down topologies.
* GET /topologies/{id}/lans/{lan_id}/next-ip — IPAllocator pre-seeded
  with existing decky IPs in that LAN.

All read routes are viewer-or-admin. Sub-routers are included in an
order that keeps literal catalog paths (/services, /next-subnet) from
being shadowed by the /{topology_id} trie branch.
2026-04-20 18:25:33 -04:00
2379b2aeda feat(api): phase 3 step 1 — topology request/response models + router skeleton
Add Pydantic DTOs in decnet/web/db/models.py covering every phase-3
endpoint shape: TopologyGenerateRequest, TopologySummary/Detail, child
create/update requests, MutationEnqueueRequest (Literal op guard),
MutationRow with JSON-payload decoder, validation/version/not-editable
error envelopes, and the three catalog responses.

Create decnet/web/router/topology/ as an import-safe package exporting
topology_router (prefix /topologies) — sub-routers land step-by-step in
subsequent commits. Mount under the main api router alongside swarm_mgmt.

tests/api/topology/test_models.py pins repo-dict ↔ DTO parity so future
repo-row drift breaks the contract test before the endpoints.
2026-04-20 18:16:30 -04:00