added: profiling toolchain (py-spy, pyinstrument, pytest-benchmark, memray, snakeviz)
New `profile` optional-deps group, opt-in Pyinstrument ASGI middleware gated by DECNET_PROFILE_REQUESTS, bench marker + tests/perf/ micro-benchmarks for repository hot paths, and scripts/profile/ helpers for py-spy/cProfile/memray.
This commit is contained in:
69
tests/perf/README.md
Normal file
69
tests/perf/README.md
Normal file
@@ -0,0 +1,69 @@
|
||||
# DECNET Profiling
|
||||
|
||||
Five complementary lenses. Pick whichever answers the question you have.
|
||||
|
||||
## 1. Whole-process sampling — py-spy
|
||||
|
||||
Attach to a running API and record a flamegraph for 30s. Requires `sudo`
|
||||
(Linux ptrace scope).
|
||||
|
||||
```bash
|
||||
./scripts/profile/pyspy-attach.sh # auto-finds uvicorn pid
|
||||
sudo py-spy record -o profile.svg -p <PID> -d 30 --subprocesses
|
||||
```
|
||||
|
||||
If py-spy "doesn't work", it is almost always one of:
|
||||
- Attached to the Typer CLI PID, not the uvicorn worker PID (use `pgrep -f 'uvicorn decnet.web.api'`).
|
||||
- `kernel.yama.ptrace_scope=1` — run with `sudo` or `sudo sysctl kernel.yama.ptrace_scope=0`.
|
||||
- The API isn't actually running (a `--dry-run` deploy starts nothing).
|
||||
|
||||
## 2. Per-request flamegraphs — Pyinstrument
|
||||
|
||||
Set the env flag, hit endpoints, find HTML flamegraphs under `./profiles/`.
|
||||
|
||||
```bash
|
||||
DECNET_PROFILE_REQUESTS=true decnet deploy --mode unihost --deckies 1
|
||||
# in another shell:
|
||||
curl http://127.0.0.1:8000/api/v1/health
|
||||
open profiles/*.html
|
||||
```
|
||||
|
||||
Off by default — zero overhead when the flag is unset.
|
||||
|
||||
## 3. Deterministic call graph — cProfile + snakeviz
|
||||
|
||||
For one-shot profiling of CLI commands or scripts.
|
||||
|
||||
```bash
|
||||
./scripts/profile/cprofile-cli.sh services # profiles `decnet services`
|
||||
snakeviz profiles/cprofile.prof
|
||||
```
|
||||
|
||||
## 4. Micro-benchmarks — pytest-benchmark
|
||||
|
||||
Regression-gate repository hot paths.
|
||||
|
||||
```bash
|
||||
pytest -m bench tests/perf/ -n0 # SQLite backend (default)
|
||||
DECNET_DB_TYPE=mysql pytest -m bench tests/perf/ -n0
|
||||
```
|
||||
|
||||
Note: `-n0` disables xdist. `pytest-benchmark` refuses to measure under
|
||||
parallel workers, which is the project default (`-n logical --dist loadscope`).
|
||||
|
||||
## 5. Memory allocation — memray
|
||||
|
||||
Hunt leaks and allocation hot spots in the API / workers.
|
||||
|
||||
```bash
|
||||
./scripts/profile/memray-api.sh # runs uvicorn under memray
|
||||
memray flamegraph profiles/memray.bin
|
||||
```
|
||||
|
||||
## Load generation
|
||||
|
||||
Pair any of the in-process lenses (2, 5) with Locust for realistic traffic:
|
||||
|
||||
```bash
|
||||
pytest -m stress tests/stress/
|
||||
```
|
||||
0
tests/perf/__init__.py
Normal file
0
tests/perf/__init__.py
Normal file
36
tests/perf/conftest.py
Normal file
36
tests/perf/conftest.py
Normal file
@@ -0,0 +1,36 @@
|
||||
import asyncio
|
||||
import pytest
|
||||
|
||||
from decnet.web.db.factory import get_repository
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def event_loop():
|
||||
loop = asyncio.new_event_loop()
|
||||
yield loop
|
||||
loop.close()
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def repo(tmp_path_factory, event_loop):
|
||||
path = tmp_path_factory.mktemp("perf") / "bench.db"
|
||||
r = get_repository(db_path=str(path))
|
||||
event_loop.run_until_complete(r.initialize())
|
||||
return r
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def seeded_repo(repo, event_loop):
|
||||
async def _seed():
|
||||
for i in range(1000):
|
||||
await repo.add_log({
|
||||
"decky": f"decky-{i % 10:02d}",
|
||||
"service": ["ssh", "ftp", "smb", "rdp"][i % 4],
|
||||
"event_type": "connect",
|
||||
"attacker_ip": f"10.0.{i // 256}.{i % 256}",
|
||||
"raw_line": f"event {i}",
|
||||
"fields": "{}",
|
||||
"msg": "",
|
||||
})
|
||||
event_loop.run_until_complete(_seed())
|
||||
return repo
|
||||
60
tests/perf/test_repo_bench.py
Normal file
60
tests/perf/test_repo_bench.py
Normal file
@@ -0,0 +1,60 @@
|
||||
"""
|
||||
Micro-benchmarks for the repository hot paths.
|
||||
|
||||
Run with:
|
||||
pytest -m bench tests/perf/
|
||||
|
||||
These do NOT run in the default suite (see `addopts` in pyproject.toml).
|
||||
"""
|
||||
import pytest
|
||||
|
||||
|
||||
pytestmark = pytest.mark.bench
|
||||
|
||||
|
||||
def test_add_log_bench(benchmark, repo, event_loop):
|
||||
payload = {
|
||||
"decky": "decky-bench",
|
||||
"service": "ssh",
|
||||
"event_type": "connect",
|
||||
"attacker_ip": "10.0.0.1",
|
||||
"raw_line": "bench event",
|
||||
"fields": "{}",
|
||||
"msg": "",
|
||||
}
|
||||
|
||||
def run():
|
||||
event_loop.run_until_complete(repo.add_log(payload))
|
||||
|
||||
benchmark(run)
|
||||
|
||||
|
||||
def test_get_logs_bench(benchmark, seeded_repo, event_loop):
|
||||
def run():
|
||||
return event_loop.run_until_complete(seeded_repo.get_logs(limit=50, offset=0))
|
||||
|
||||
result = benchmark(run)
|
||||
assert len(result) == 50
|
||||
|
||||
|
||||
def test_get_total_logs_bench(benchmark, seeded_repo, event_loop):
|
||||
def run():
|
||||
return event_loop.run_until_complete(seeded_repo.get_total_logs())
|
||||
|
||||
benchmark(run)
|
||||
|
||||
|
||||
def test_get_logs_search_bench(benchmark, seeded_repo, event_loop):
|
||||
def run():
|
||||
return event_loop.run_until_complete(
|
||||
seeded_repo.get_logs(limit=50, offset=0, search="service:ssh")
|
||||
)
|
||||
|
||||
benchmark(run)
|
||||
|
||||
|
||||
def test_get_user_by_username_bench(benchmark, seeded_repo, event_loop):
|
||||
def run():
|
||||
return event_loop.run_until_complete(seeded_repo.get_user_by_username("admin"))
|
||||
|
||||
benchmark(run)
|
||||
Reference in New Issue
Block a user