feat(ttp): extract intel_lifter provider mappings to YAML data + ATT&CK external_reference enrichment

The four provider→technique tables (AbuseIPDB cat→techniques,
GreyNoise tag→techniques, ThreatFox threat_type→techniques, plus
the Feodo binary-listed signal) used to live as Final[dict] constants
in intel_lifter.py. Two real problems with that:

1. Drift between rules/ttp/R0054.yaml..R0058.yaml (which declare
   the full slate per provider) and the Python dicts (which decide
   which slate-member fires per signal). The v2 audit comment in
   intel_lifter.py documented that they had silently drifted.
2. No ATT&CK provenance on emissions — the loaded STIX bundle has
   rich external_references (canonical attack.mitre.org URLs) that
   never surfaced because the lifter had no path back to them.

Mappings now live as YAML at decnet/ttp/data/intel/{provider}.yaml,
validated at load against the loaded ATT&CK bundle, with each entry
enriched by attack_stix._attack_pattern_by_id to attach the canonical
MITRE URL to every emission.

- decnet/ttp/data/intel_loader.py: pydantic-validated schema +
  ProviderMapping/Signal/TechniqueEmission frozen dataclasses +
  load_provider_mapping(provider) lru-cached.
- Per-technique high_score_threshold inlined into YAML
  (collapses the separate _ABUSEIPDB_HIGH_SCORE_GATED dict).
- external_reference field follows the STIX 2.1 external-reference
  shape (source_name + url + optional external_id) so the future
  STIX/MISP exporter is a direct translation.
- intel_lifter.py: dicts deleted, decision functions read from
  ProviderMapping accessors. Decision-flow constants (T1071/T1595
  bare-classification fallbacks in _greynoise_decisions) stay in
  code — they're not table rows.
- Each emit slot's evidence_extra now carries mitre_url for any
  technique resolved in the bundle (every one in practice).
- tests/ttp/test_intel_mappings.py: snapshot equivalence vs the
  legacy dicts, high-score gate behavior, every-signal-has-an-
  external-reference, every-emission-has-a-mitre-url, negative
  paths (unknown technique_id raises AttackBundleError, mismatched
  provider field rejected, dir listing matches expected providers).

The YAML schema + mitre_url enrichment lays groundwork for the
future STIX exporter; this commit does NOT build that exporter.
This commit is contained in:
2026-05-09 06:18:25 -04:00
parent a3f1cea2d6
commit d25f69ba1b
9 changed files with 853 additions and 95 deletions

View File

@@ -0,0 +1,74 @@
# GreyNoise tag → ATT&CK technique mapping.
#
# Mirrors what _GREYNOISE_TAG_TO_TECHNIQUES used to encode in
# decnet/ttp/impl/intel_lifter.py. Note: GreyNoise's Community
# endpoint does not return tags; these fire only when operators wire
# a non-Community provider (Visualizer / Enterprise / RIOT). Kept
# canonical here so the upgrade path is a column populate, not a
# code change. Decision-flow constants for bare ``classification ==
# "scanner"`` (T1595) and bare ``classification == "malicious"``
# (T1071 at 0.5×) stay in code — they're not table rows.
provider: greynoise
mapping_version: "1"
attack_release: ">=15.1"
signals:
- id: tor_exit_node
label: "Tor exit node"
external_reference:
source_name: greynoise
url: "https://docs.greynoise.io/docs/understanding-greynoise-tags"
external_id: tor_exit_node
techniques:
- technique_id: T1090
- id: ssh_bruteforcer
label: "SSH brute-forcer"
external_reference:
source_name: greynoise
url: "https://docs.greynoise.io/docs/understanding-greynoise-tags"
external_id: ssh_bruteforcer
techniques:
- technique_id: T1110
- id: web_crawler
label: "Web crawler"
external_reference:
source_name: greynoise
url: "https://docs.greynoise.io/docs/understanding-greynoise-tags"
external_id: web_crawler
techniques:
- technique_id: T1595
- id: cobalt_strike
label: "Cobalt Strike"
external_reference:
source_name: greynoise
url: "https://docs.greynoise.io/docs/understanding-greynoise-tags"
external_id: cobalt_strike
techniques:
- technique_id: T1071
- technique_id: T1588
- id: metasploit
label: "Metasploit"
external_reference:
source_name: greynoise
url: "https://docs.greynoise.io/docs/understanding-greynoise-tags"
external_id: metasploit
techniques:
- technique_id: T1071
- technique_id: T1588
- id: sliver
label: "Sliver"
external_reference:
source_name: greynoise
url: "https://docs.greynoise.io/docs/understanding-greynoise-tags"
external_id: sliver
techniques:
- technique_id: T1071
- technique_id: T1588
- id: havoc
label: "Havoc"
external_reference:
source_name: greynoise
url: "https://docs.greynoise.io/docs/understanding-greynoise-tags"
external_id: havoc
techniques:
- technique_id: T1071
- technique_id: T1588