perf(pytest): 194s → 4s collection — lazy heavy imports + norecursedirs

Four-part fix for the collection bottleneck that was blocking the dev loop:

1. Lazy mitreattack.stix20 import in attack_stix.py — deferred to first
   _load() call (TYPE_CHECKING guard at top level)

2. Lazy misp_stix_converter import in both MISP export routers — moved
   from module level into the route handler body

3. Lazy attack_catalog / attack_stix in ttp.py repo mixin — thin wrapper
   functions so the import chain never fires at module load time

4. tests/api/conftest.py — `from decnet.web.api import app` moved inside
   the `client()` fixture; `pytest_ignore_collect` broadened to skip all
   test_schemathesis*.py variants (not just test_schemathesis.py), which
   were launching a subprocess server at module-import time

5. pyproject.toml — `norecursedirs` for tests/live, tests/stress,
   tests/service_testing, tests/docker, tests/perf so these directories
   are never entered; `-m` filter removed from addopts (now redundant);
   `--dist loadscope` → `--dist load` to unblock workers immediately

6. behave_core / behave_shell rename — BEHAVE packages dropped the
   `decnet_` prefix; reinstalled editable installs and updated all 14
   import sites across profiler, ttp, bus, and correlation modules
This commit is contained in:
2026-05-10 06:41:25 -04:00
parent f63aca4186
commit e4626879f6
24 changed files with 59 additions and 36 deletions

View File

@@ -2,7 +2,7 @@
emitted Observation envelope.
Mirrors the BEHAVE-SHELL ``Observation`` Pydantic envelope
(``decnet_behave_core.spec.envelope.Observation``) field-for-field, plus
(``behave_core.spec.envelope.Observation``) field-for-field, plus
one DECNET-side denormalisation (``attacker_uuid``) for cheap joins.
The class is named ``ObservationRow`` to avoid colliding with the
BEHAVE Pydantic class when both are imported into the same module —

View File

@@ -13,7 +13,7 @@ Composed onto :class:`SQLModelRepository`. Three public methods:
drift charts.
PII discipline is the BEHAVE envelope's job
(``core/decnet_behave_core/spec/envelope.py:3-19``); this mixin does
(``core/behave_core/spec/envelope.py:3-19``); this mixin does
not validate values — that happens at construction time by the BEHAVE
``Observation`` subclass before the dict reaches us.
"""

View File

@@ -20,8 +20,6 @@ from typing import Any
from sqlalchemy import func, select
from sqlmodel import col
from decnet.ttp.attack_catalog import technique_name as _technique_name
from decnet.ttp.attack_stix import mitre_url_for as _mitre_url_for
from decnet.web.db.models import (
Attacker,
AttackerIdentity,
@@ -34,6 +32,16 @@ from decnet.web.db.models.canary import CanaryTrigger
from decnet.web.db.sqlmodel_repo._helpers import _MixinBase
def _technique_name(tid: str | None) -> str | None:
from decnet.ttp.attack_catalog import technique_name # heavy — lazy on first call
return technique_name(tid)
def _mitre_url_for(tid: str | None) -> str | None:
from decnet.ttp.attack_stix import mitre_url_for # heavy — lazy on first call
return mitre_url_for(tid)
# Confidence floor: tags computed below this value are silently dropped
# at insert time. Pinned by tests/ttp/test_confidence.py.
_CONFIDENCE_FLOOR: float = 0.3

View File

@@ -14,7 +14,6 @@ from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.responses import Response
from decnet.telemetry import traced as _traced
from decnet.ttp.misp_export import build_attacker_misp_event
from decnet.web.dependencies import require_viewer, repo
router = APIRouter()
@@ -79,6 +78,7 @@ async def api_export_attacker_misp(
observations = cast(list[dict[str, Any]], results[8])
fingerprint_bounties = cast(list[dict[str, Any]], results[9])
from decnet.ttp.misp_export import build_attacker_misp_event # heavy — lazy on first call
event = build_attacker_misp_event(
attacker=attacker,
behavior=behavior,

View File

@@ -19,7 +19,6 @@ from fastapi import APIRouter, Depends
from fastapi.responses import Response
from decnet.telemetry import traced as _traced
from decnet.ttp.misp_export import build_fleet_misp_collection
from decnet.web.dependencies import require_viewer, repo
router = APIRouter()
@@ -49,6 +48,7 @@ async def api_export_attackers_misp(
repo.get_all_observations_for_export(),
repo.get_all_fingerprint_bounties_for_export(),
)
from decnet.ttp.misp_export import build_fleet_misp_collection # heavy — lazy on first call
collection = build_fleet_misp_collection(
rows=rows,
ttp_by_attacker=ttp_by_attacker,