diff --git a/Deployment-Modes.md b/Deployment-Modes.md new file mode 100644 index 0000000..811fff1 --- /dev/null +++ b/Deployment-Modes.md @@ -0,0 +1,175 @@ +# Deployment Modes: UNIHOST and SWARM + +DECNET's `deploy` command takes a `--mode` flag that selects how the fleet of +deckies is laid out across real hardware. Two modes are defined: + +- `unihost` — a single real host runs all deckies as containers. +- `swarm` — multiple real hosts each run a subset of deckies (multi-host). + +The mode is validated in `decnet/cli.py` (`deploy --mode`) and stored on the +`DecnetConfig` object consumed by `decnet/engine/deployer.py`. The orchestrator +in `decnet/composer.py` emits the Docker Compose file used by either mode. + +See also: [CLI reference](CLI-Reference), [INI format](INI-Config-Format), +[Networking: MACVLAN and IPvlan](Networking-MACVLAN-IPVLAN), +[Teardown](Teardown-and-State). + +--- + +## When to use which + +| Criterion | UNIHOST | SWARM | +|---------------------------------|----------------------------|-------------------------------------| +| Hosts involved | 1 | N | +| Setup complexity | low | higher (needs fleet orchestration) | +| Deckies per host | all of them | split across hosts | +| Fits home lab / single server | yes | overkill | +| Fits multi-subnet / multi-site | no | yes | +| Failure blast radius | whole fleet on one box | isolated per host | +| Recommended for first runs | yes | no | + +UNIHOST is the default (`--mode unihost`) and what the README's quickstart +assumes. SWARM is the deployment posture for a real honeynet spanning multiple +boxes. + +--- + +## UNIHOST + +``` + attacker LAN 192.168.1.0/24 + +---------------------------------------------------+ + | | + | [host] eth0 decnet_lan (MACVLAN / IPvlan L2) | + | |- decky-01 .10 | + | |- decky-02 .11 | + | |- decky-03 .12 | + | |- decky-04 .13 | + | '- decky-05 .14 | + +---------------------------------------------------+ + | + (isolated mgmt path) + v + logging / SIEM network +``` + +One host, one MACVLAN or IPvlan L2 network, N deckies. The host also runs the +collector/API and forwards logs out-of-band to the isolated logging network. + +### CLI + +``` +# Dry-run: generate compose, no containers +decnet deploy --mode unihost --deckies 3 --randomize-services --dry-run + +# Full deploy (root required for MACVLAN) +sudo decnet deploy \ + --mode unihost \ + --deckies 5 \ + --interface eth0 \ + --randomize-services +``` + +Flags worth knowing (full table in [CLI reference](CLI-Reference)): + +- `--deckies / -n` — number of deckies on this host. +- `--interface / -i` — parent NIC. Auto-detected via `detect_interface()`. +- `--ipvlan` — switch to IPvlan L2. Required on WiFi. +- `--services`, `--randomize-services`, `--archetype` — service selection. +- `--distro`, `--randomize-distros` — OS heterogeneity. +- `--mutate-interval` — rotate services every N minutes. +- `--dry-run` — write the compose file under `deploy/` without starting. + +### INI + +The same deployment can be expressed declaratively via `--config`: + +``` +decnet deploy --mode unihost --config ./my-fleet.ini +``` + +See [INI format](INI-Config-Format) for the full schema; `cli.py`'s config +path calls `load_ini` and `build_deckies_from_ini`. + +--- + +## SWARM (multihost) + +``` + attacker network(s) + ------------------------------ + | | | + [host-A] [host-B] [host-C] + decky-01 decky-04 decky-07 + decky-02 decky-05 decky-08 + decky-03 decky-06 decky-09 + + \__________________|__________________/ + | + isolated mgmt / SIEM +``` + +Each real host runs a UNIHOST-shaped deployment over its own slice of the IP +space. An external orchestrator (Ansible, sshpass-driven scripts, etc.) +invokes `decnet deploy --mode swarm ...` on each host in turn. The CLI +currently accepts `swarm` as a valid mode — the fleet-wide orchestration layer +lives outside the DECNET binary and is the operator's responsibility. See the +README's architecture section for the intended shape. + +### CLI + +Run on each host, coordinating IP ranges so deckies do not collide: + +``` +# host-A +sudo decnet deploy \ + --mode swarm \ + --deckies 3 \ + --interface eth0 \ + --ip-start 192.168.1.10 \ + --randomize-services + +# host-B +sudo decnet deploy \ + --mode swarm \ + --deckies 3 \ + --interface eth0 \ + --ip-start 192.168.1.20 \ + --randomize-services +``` + +`--ip-start` is the operator's primary tool for partitioning the subnet across +hosts; `allocate_ips` in `decnet/network.py` starts sequentially from that +address and skips reserved / in-use IPs. + +### INI + +For reproducible swarm rollouts, give each host its own INI and drive the +rollout from Ansible (or similar): + +``` +decnet deploy --mode swarm --config ./host-A.ini +decnet deploy --mode swarm --config ./host-B.ini +``` + +--- + +## What is common to both modes + +- Compose generation in `decnet/composer.py` is mode-agnostic; the `mode` + field is recorded on `DecnetConfig` for state/telemetry but the compose + shape (base container + service containers sharing its network namespace + via `network_mode: service:`) is identical. +- Network driver selection (MACVLAN vs IPvlan L2) is per-host, controlled by + `--ipvlan`. See [Networking: MACVLAN and IPvlan](Networking-MACVLAN-IPVLAN). +- Teardown is per-host: run `sudo decnet teardown --all` on each host in the + swarm. See [Teardown](Teardown-and-State). + +--- + +## Quick checklist + +- Single box, one LAN -> UNIHOST. +- Multiple boxes across one or more LANs -> SWARM, driven by external + orchestration, with non-overlapping `--ip-start` per host. +- Not sure -> start with UNIHOST and a small `--deckies` count. diff --git a/Networking-MACVLAN-IPVLAN.md b/Networking-MACVLAN-IPVLAN.md new file mode 100644 index 0000000..322e703 --- /dev/null +++ b/Networking-MACVLAN-IPVLAN.md @@ -0,0 +1,163 @@ +# Networking: MACVLAN and IPvlan + +DECNET's deckies must look like real, independent machines on the LAN. Each +decky owns an IP (and ideally a MAC) drawn from the same subnet as the host's +real NIC, so that attackers scanning the network see a heterogeneous fleet of +hosts rather than a single container host. + +This page covers how that is wired up: the two Docker network drivers DECNET +supports, the host-side hairpin interface it creates, and the limitations you +will hit on WiFi or WSL. + +Source of truth: `decnet/network.py` (constants `MACVLAN_NETWORK_NAME`, +`HOST_MACVLAN_IFACE`, `HOST_IPVLAN_IFACE`; functions `setup_host_macvlan`, +`teardown_host_macvlan`, `setup_host_ipvlan`, `teardown_host_ipvlan`). + +See also: [CLI reference](CLI-Reference), [INI format](INI-Config-Format), +[Teardown](Teardown-and-State). + +--- + +## Topology + +``` + Internet / attacker + | + | (public IP, port forwards) + | + +-----------------+ + | DECNET host | eth0 -> decnet_lan (MACVLAN / IPvlan L2) + +--------+--------+ + | + ==============+=================== LAN 192.168.1.0/24 + | | | + decky-01 decky-02 decky-03 (each = own IP, + .10 .11 .12 MACVLAN = own MAC too) + + | + +-------------+-------------+ + | decnet_macvlan0 (host) | hairpin: host <-> deckies reachability + +---------------------------+ + + --- isolated mgmt path --- + host -> logging/SIEM network + (not reachable from decnet_lan) +``` + +The decoy LAN is attacker-facing. The logging/aggregation network is reachable +only from the host and must not be routable from `decnet_lan`. + +--- + +## Driver 1: MACVLAN (default) + +MACVLAN is the default driver. Each decky gets a kernel-assigned MAC address on +the parent interface, so it is a distinct L2 endpoint on the LAN. + +- Docker network name: `decnet_lan` (constant `MACVLAN_NETWORK_NAME`). +- Parent NIC: `--interface` (auto-detected via `detect_interface()` if omitted). +- Mode: `macvlan / bridge` (see `create_macvlan_network` and the + `ip link add ... type macvlan mode bridge` call in `setup_host_macvlan`). +- Requires the parent NIC to support **promiscuous mode**. Most wired NICs do. + +### Host hairpin (`decnet_macvlan0`) + +MACVLAN has a well-known limitation: the host cannot talk directly to its own +MACVLAN children on the parent NIC. Without a workaround, `decnet status` and +the log collector (which run on the host) would be unable to reach deckies. + +DECNET fixes this by creating a second host-side MACVLAN interface named +`decnet_macvlan0` (constant `HOST_MACVLAN_IFACE`) attached to the same parent +NIC, assigning it a /32 on the host, and routing the decky IP range out of it. +The relevant sequence in `setup_host_macvlan`: + +``` +ip link add decnet_macvlan0 link type macvlan mode bridge +ip addr add /32 dev decnet_macvlan0 +ip link set decnet_macvlan0 up +ip route add dev decnet_macvlan0 +``` + +--- + +## Driver 2: IPvlan L2 (`--ipvlan`) + +Pass `--ipvlan` to opt into IPvlan L2. Each decky gets a unique IP but all +share the host MAC. This is the fallback for environments where MACVLAN is not +workable: + +- **WiFi.** Most access points drop frames whose source MAC is not registered + with the radio, so random MACVLAN MACs never reach the LAN. IPvlan L2 works + because every frame leaves with the (registered) host MAC. +- NICs that refuse promiscuous mode. + +### Host hairpin (`decnet_ipvlan0`) + +Same trick as MACVLAN, different link type. `setup_host_ipvlan` adds a +host-side interface named `decnet_ipvlan0` (constant `HOST_IPVLAN_IFACE`) of +type `ipvlan mode l2`, gives it a /32, and routes the decky range through it. +`teardown_host_ipvlan` removes the route and the link on teardown. + +### Trade-offs + +| Aspect | MACVLAN | IPvlan L2 | +|--------------------|----------------------------------|---------------------------------| +| Per-decky MAC | yes (unique, kernel-assigned) | no (shares host MAC) | +| Works on WiFi | usually no | yes | +| Promisc NIC needed | yes | no | +| Decoy realism | higher (distinct L2 identities) | lower (all share one MAC) | +| Attacker fingerprinting via `arp -a` | sees many MACs | sees one MAC, many IPs | + +Prefer MACVLAN on wired, bare-metal or VM deployments. Use IPvlan L2 when the +environment forces it. + +--- + +## WiFi and WSL limitations + +- **WiFi:** MACVLAN typically does not work. Use `--ipvlan`. +- **WSL2:** MACVLAN has known limitations under WSL. Bare metal or a full Linux + VM is the supported testing environment (README "Requirements" section). +- **Promiscuous mode:** some virtualization platforms (including vSphere with + default security policies) block promiscuous mode on the vNIC; either relax + the policy on the virtual switch or use IPvlan. + +--- + +## Teardown + +Teardown of the Docker network and the host hairpin interface is automatic +when you run `decnet teardown --all`. The relevant steps, from +`decnet/engine/deployer.py`: + +``` +if config.ipvlan: + teardown_host_ipvlan(decky_range) +else: + teardown_host_macvlan(decky_range) +remove_macvlan_network(client) +``` + +`teardown_host_macvlan` / `teardown_host_ipvlan` each do: + +``` +ip route del dev # decnet_macvlan0 or decnet_ipvlan0 +ip link del +``` + +Both use `check=False`, so a partial or previously-cleaned state is safe. +Removing the Docker network `decnet_lan` happens separately via +`remove_macvlan_network`. + +See [Teardown](Teardown-and-State) for the full teardown flow including +container removal and state-file cleanup. + +--- + +## Quick checklist + +- Host NIC is wired and supports promiscuous mode -> MACVLAN, no flag needed. +- Host NIC is WiFi, or promisc is blocked -> pass `--ipvlan`. +- Running under WSL -> switch to a Linux VM or bare metal. +- `decnet status` says a decky is unreachable from the host -> confirm + `decnet_macvlan0` (or `decnet_ipvlan0`) exists with `ip link show`.