1
Database Drivers
anti edited this page 2026-04-18 06:04:24 -04:00

Database Drivers

DECNET persists dashboard state, deployment metadata, event/capture rows, and auth data through a single BaseRepository interface. Two concrete backends implement that interface: SQLite (default) and MySQL. Both extend the same SQLModelRepository base and only override the dialect-specific bits (engine construction, URL resolution, schema bootstrap). This is a load-bearing design choice — adding a new backend means subclassing the shared repo, not reimplementing the query surface.

Source: decnet/web/db/repository.py, decnet/web/db/sqlmodel_repo.py, decnet/web/db/sqlite/repository.py, decnet/web/db/mysql/repository.py.

Why two backends

  • SQLite fits single-host UNIHOST deploys. Zero setup, file-backed, limited to one writer at a time.
  • MySQL fits SWARM and any deployment with multi-process ingest or high write volume (many sniffer/capture workers pushing concurrently). Scales horizontally and survives multi-writer contention that SQLite cannot.

The wire format, ORM models, and repository API are identical — only the engine and connection URL change.

SQLite (default)

Driver: aiosqlite >= 0.20.0 (see pyproject.toml). Pulled in automatically by pip install -e . — nothing else to install.

Behavior:

  • WAL journal mode and synchronous=NORMAL are enabled on every connect (decnet/web/db/sqlite/database.py).
  • Default busy_timeout is 30s to absorb short write contention.
  • The DB file lives at the path passed to SQLiteRepository(db_path=...) (defaults set by the caller; repo bootstrap will create_all on first use).
  • WAL sidecar files (*-wal, *-shm) are expected and are ignored by .gitignore.

Limits:

  • Single-writer. Concurrent writes serialize; under heavy load you will see SQLITE_BUSY. Raise DECNET_DB_POOL_SIZE / timeouts only as a stopgap — switch to MySQL if contention is real.
  • Not safe to share across hosts over NFS or similar.

MySQL

Driver: asyncmy >= 0.2.9 (see pyproject.toml). Installed by pip install -e .. No separate package step.

Setup

Create the database and a dedicated user:

CREATE DATABASE decnet CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'decnet'@'%' IDENTIFIED BY 'change-me';
GRANT ALL PRIVILEGES ON decnet.* TO 'decnet'@'%';
FLUSH PRIVILEGES;

Point DECNET at it with either a full URL or component vars (see Environment variables for the complete list):

# Option A — full URL
export DECNET_DB_TYPE=mysql
export DECNET_DB_URL="mysql+asyncmy://decnet:change-me@db.internal:3306/decnet"

# Option B — components (password is percent-encoded automatically)
export DECNET_DB_TYPE=mysql
export DECNET_DB_HOST=db.internal
export DECNET_DB_PORT=3306
export DECNET_DB_NAME=decnet
export DECNET_DB_USER=decnet
export DECNET_DB_PASSWORD='change-me'

Precedence: explicit url= kwarg to get_async_engine > DECNET_DB_URL > components. An empty password outside pytest raises — this is intentional.

Schema bootstrap: tables are created via SQLModel.metadata.create_all on the first repository initialization. There is no migrator.

Tuning

Pool sizing and recycle knobs (DECNET_DB_POOL_SIZE, DECNET_DB_MAX_OVERFLOW, DECNET_DB_POOL_RECYCLE, DECNET_DB_POOL_PRE_PING) apply to both backends and are documented on Environment variables. MySQL defaults pool_pre_ping=true to fail fast on dropped idle connections; SQLite leaves it off since the "server" is a local file.

Switching backends

Set DECNET_DB_TYPE to sqlite or mysql and restart the web/CLI process. There is no migration tool — switching backends starts from an empty schema. Export anything you care about beforehand. For MySQL teardown / rebuild during development, decnet db-reset exists (MySQL-only; see decnet/cli.py).

Factory and DI

All code paths that need a repository must go through the factory:

from decnet.web.db.factory import get_repository

repo = get_repository()  # DECNET_DB_TYPE decides the subclass

In FastAPI handlers, depend on get_repo from decnet/web/dependencies.py instead of constructing a repo inline. The factory also wraps the instance with telemetry (decnet.telemetry.wrap_repository), so bypassing it loses metrics.

Rule (from CLAUDE.md): never import SQLiteRepository (or MySQLRepository) directly in feature code. Use get_repository() or the get_repo FastAPI dependency. Direct imports break backend switching and silently drop telemetry wrapping.

See the Developer guide for how repository-bound services are wired into handlers and background workers.