test(clustering): fixture 2 vpn_hopping + fingerprint/asn references

One campaign, one DSL actor, ip_pool: rotating + rotation_count: 5
across 5 synthetic private-use ASNs (RFC 6996 64512-64516). Stable
JA3, HASSH, and payload_hash across every rotation — these are the
"signals the attacker can't cheaply rotate" per IDENTITY_RESOLUTION.md
and the load-bearing reason all 5 observation rows must resolve to
one identity / one campaign.

Two new reference clusterers in fixture_harness.py:

* fingerprint_clusterer — groups by (ja3, hassh). Un-fingerprinted
  rows stay singleton so it doesn't trivially fuse all noise into one
  mega-cluster. Approximates the stable-signal arm of the planned
  similarity graph.

* asn_clusterer — deliberately-bad reference for fixture 2's
  adversarial test. Group-by-ASN shatters the campaign into 5
  singletons; completeness collapses to 0.

Four tests in test_vpn_hopping_fixture.py: corpus shape (5 rows, 1
identity, 1 campaign, 5 distinct ASNs/IPs, stable fingerprints),
pass at campaign level, pass at identity level (asserts ARI exactly
1.0), asn_clusterer breaches the completeness floor.
This commit is contained in:
2026-04-26 07:34:18 -04:00
parent 943bb3a39d
commit 0def6f7e37
4 changed files with 247 additions and 1 deletions

View File

@@ -0,0 +1,25 @@
# Bounds for fixture 2 (vpn_hopping).
#
# Ground truth at campaign-level: 1 campaign of 5 observation rows.
# Ground truth at identity-level: 1 identity of 5 observation rows.
# A correct algorithm scores 1.0 across every metric on this fixture.
#
# Completeness is the load-bearing metric: a clusterer that fragments
# the campaign by IP/ASN tanks completeness (the one true class is
# split across many predicted clusters). The adversarial asn_clusterer
# in the test file demonstrates this and the bound below rejects it.
#
# No true singletons in this fixture — singleton_recall is trivially
# 1.0 (the metric returns 1.0 when truth has no singletons).
#
# Bounds are loose at v1; tighten as the algorithm matures. Loosening
# any bound to make CI pass requires PR-comment justification (per
# CAMPAIGN_CLUSTERING.md §2).
adjusted_rand_index:
min: 0.85
homogeneity:
min: 0.90
completeness:
min: 0.80
singleton_recall:
min: 0.95

View File

@@ -0,0 +1,55 @@
# Fixture 2 (vpn_hopping) — see development/CAMPAIGN_CLUSTERING.md §2
# and development/IDENTITY_RESOLUTION.md.
#
# One campaign, one actor, rotating across 5 distinct ASNs. JA3, HASSH,
# and payload_hash are STABLE across every rotation — these are the
# signals "the attacker can't cheaply rotate" (per the identity
# resolution design doc) and they're the reason a clusterer should
# recover all 5 observation rows as ONE identity, ONE campaign.
#
# Ground truth (verified at every level):
# - 5 observations → 1 identity → 1 campaign (per truth_labels())
#
# Pass condition: a fingerprint-driven clusterer must fold all 5 rows
# into one cluster at both campaign-level and identity-level scoring.
#
# Adversarial condition: an asn_clusterer (group attackers by ASN —
# the textbook bad heuristic) must fragment the campaign into 5
# pieces and breach the completeness floor. This is what proves "ASN
# match" is correctly weighted "very low" in the planned similarity
# graph (per TODO clusterer feature list).
#
# ASN choice: synthetic private-use values (RFC 6996 6451264534) so
# the fixture never collides with real-world data and signals "not
# real" to readers at a glance.
campaign:
id: vpn-hopping-001
actors:
- id: hopper-a
asn: 64512 # primary; rotation_asns overrides per row
ip_pool: rotating
rotation_count: 5
rotation_asns: [64512, 64513, 64514, 64515, 64516]
ja3: "771,4865-4866-4867-49195-49199-49196-49200,0-23-65281-10-11-35-16-5-13-18-51-45-43-27,29-23-24,0"
hassh: "vpn-hopper-cccccccc-cccccccc-cccccccc"
hours_active_utc: [12, 13, 14, 15, 16]
jitter_seconds: 60
phases:
- name: delivery
actor: hopper-a
target_selector: { service: ssh, count: 5 }
dwell_seconds: 1
- name: exploitation
actor: hopper-a
tool_signature:
# Stable payload across every rotation — same dropper from
# whatever staging the operator uses, regardless of which VPN
# exit they emerge from.
payload_hash: "vpn-hopper-stage1-payload"
target_selector: { service: ssh, count: 5 }
dwell_seconds: 5
- name: discovery
actor: hopper-a
target_selector: { service: ssh, count: 5 }
dwell_seconds: 5
duration_days: 2