test(clustering): fixture 1 (shared_wordlist) + fixture-harness extraction
Two campaigns sharing a credential wordlist; everything else (ASN, IPs,
JA3, HASSH, active hours) divergent. Pass condition: clusterer must NOT
merge. Protects against the "credential overlap is identity" failure
mode that commodity wordlists invite.
* tests/clustering/fixture_harness.py — shared assert_fixture_bounds
helper + identity_clusterer (placeholder, trivially correct on
all-singleton fixtures) + credential_jaccard_clusterer (deliberately-
bad reference used to PROVE the fixture catches what it should).
* tests/clustering/test_shared_wordlist_fixture.py — bounds pass with
identity, bounds FAIL (homogeneity → 0) with the bad credential
clusterer. The latter is the proof the fixture earns its keep.
* tests/fixtures/campaigns/shared_wordlist.{yaml,expected.yaml}.
* tests/clustering/test_lone_wolf_fixture.py — refactored onto the
shared harness. No behavior change.
This commit is contained in:
21
tests/fixtures/campaigns/shared_wordlist.expected.yaml
vendored
Normal file
21
tests/fixtures/campaigns/shared_wordlist.expected.yaml
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
# Bounds for fixture 1 (shared_wordlist).
|
||||
#
|
||||
# Ground truth: two distinct campaigns, one actor each → 2 truth-labels
|
||||
# of size 1. The clusterer must keep them separate. A correct algorithm
|
||||
# scores 1.0 across every metric on this fixture.
|
||||
#
|
||||
# Homogeneity is the load-bearing metric here: a clusterer that merges
|
||||
# the two campaigns based on shared credentials will tank homogeneity
|
||||
# (one predicted cluster contains members of two true campaigns).
|
||||
#
|
||||
# Bounds are loose at v1; tighten as the algorithm matures. Loosening
|
||||
# any bound to make CI pass requires PR-comment justification (per
|
||||
# CAMPAIGN_CLUSTERING.md §2).
|
||||
adjusted_rand_index:
|
||||
min: 0.85
|
||||
homogeneity:
|
||||
min: 0.90
|
||||
completeness:
|
||||
min: 0.80
|
||||
singleton_recall:
|
||||
min: 0.95
|
||||
Reference in New Issue
Block a user