docs(wiki): add Database Drivers page (unit 9)

2026-04-18 06:04:24 -04:00
parent 9de6ea3dff
commit 3804641d4e

123
Database-Drivers.md Normal file

@@ -0,0 +1,123 @@
# Database Drivers
DECNET persists dashboard state, deployment metadata, event/capture rows, and auth
data through a single `BaseRepository` interface. Two concrete backends implement
that interface: SQLite (default) and MySQL. Both extend the same
`SQLModelRepository` base and only override the dialect-specific bits (engine
construction, URL resolution, schema bootstrap). This is a load-bearing design
choice — adding a new backend means subclassing the shared repo, not reimplementing
the query surface.
Source: `decnet/web/db/repository.py`, `decnet/web/db/sqlmodel_repo.py`,
`decnet/web/db/sqlite/repository.py`, `decnet/web/db/mysql/repository.py`.
## Why two backends
- **SQLite** fits single-host UNIHOST deploys. Zero setup, file-backed,
limited to one writer at a time.
- **MySQL** fits SWARM and any deployment with multi-process ingest or high
write volume (many sniffer/capture workers pushing concurrently). Scales
horizontally and survives multi-writer contention that SQLite cannot.
The wire format, ORM models, and repository API are identical — only the engine
and connection URL change.
## SQLite (default)
Driver: `aiosqlite >= 0.20.0` (see `pyproject.toml`). Pulled in automatically
by `pip install -e .` — nothing else to install.
Behavior:
- WAL journal mode and `synchronous=NORMAL` are enabled on every connect
(`decnet/web/db/sqlite/database.py`).
- Default `busy_timeout` is 30s to absorb short write contention.
- The DB file lives at the path passed to `SQLiteRepository(db_path=...)`
(defaults set by the caller; repo bootstrap will `create_all` on first use).
- WAL sidecar files (`*-wal`, `*-shm`) are expected and are ignored by
`.gitignore`.
Limits:
- Single-writer. Concurrent writes serialize; under heavy load you will see
`SQLITE_BUSY`. Raise `DECNET_DB_POOL_SIZE` / timeouts only as a stopgap —
switch to MySQL if contention is real.
- Not safe to share across hosts over NFS or similar.
## MySQL
Driver: `asyncmy >= 0.2.9` (see `pyproject.toml`). Installed by
`pip install -e .`. No separate package step.
### Setup
Create the database and a dedicated user:
```sql
CREATE DATABASE decnet CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'decnet'@'%' IDENTIFIED BY 'change-me';
GRANT ALL PRIVILEGES ON decnet.* TO 'decnet'@'%';
FLUSH PRIVILEGES;
```
Point DECNET at it with either a full URL or component vars (see
[Environment variables](Environment-Variables) for the complete list):
```bash
# Option A — full URL
export DECNET_DB_TYPE=mysql
export DECNET_DB_URL="mysql+asyncmy://decnet:change-me@db.internal:3306/decnet"
# Option B — components (password is percent-encoded automatically)
export DECNET_DB_TYPE=mysql
export DECNET_DB_HOST=db.internal
export DECNET_DB_PORT=3306
export DECNET_DB_NAME=decnet
export DECNET_DB_USER=decnet
export DECNET_DB_PASSWORD='change-me'
```
Precedence: explicit `url=` kwarg to `get_async_engine` > `DECNET_DB_URL` >
components. An empty password outside pytest raises — this is intentional.
Schema bootstrap: tables are created via `SQLModel.metadata.create_all` on the
first repository initialization. There is no migrator.
### Tuning
Pool sizing and recycle knobs (`DECNET_DB_POOL_SIZE`, `DECNET_DB_MAX_OVERFLOW`,
`DECNET_DB_POOL_RECYCLE`, `DECNET_DB_POOL_PRE_PING`) apply to both backends
and are documented on [Environment variables](Environment-Variables). MySQL
defaults `pool_pre_ping=true` to fail fast on dropped idle connections; SQLite
leaves it off since the "server" is a local file.
## Switching backends
Set `DECNET_DB_TYPE` to `sqlite` or `mysql` and restart the web/CLI process.
There is **no migration tool** — switching backends starts from an empty
schema. Export anything you care about beforehand. For MySQL teardown /
rebuild during development, `decnet db-reset` exists (MySQL-only; see
`decnet/cli.py`).
## Factory and DI
All code paths that need a repository must go through the factory:
```python
from decnet.web.db.factory import get_repository
repo = get_repository() # DECNET_DB_TYPE decides the subclass
```
In FastAPI handlers, depend on `get_repo` from `decnet/web/dependencies.py`
instead of constructing a repo inline. The factory also wraps the instance
with telemetry (`decnet.telemetry.wrap_repository`), so bypassing it loses
metrics.
**Rule (from `CLAUDE.md`):** never import `SQLiteRepository` (or
`MySQLRepository`) directly in feature code. Use `get_repository()` or the
`get_repo` FastAPI dependency. Direct imports break backend switching and
silently drop telemetry wrapping.
See the [Developer guide](Developer-Guide) for how repository-bound services
are wired into handlers and background workers.