Add web frontend with JWT auth, RBAC, SSE dashboard, and config editor

- FastAPI + htmx + Jinja2 web frontend, started with --web flag
- JWT HS256 auth (WEB_SECRET_KEY) with httpOnly cookies; access (15 min) +
  refresh (7 day) tokens; refresh rotation + JTI revocation in data/web.db
- RBAC: superadmin > admin > reader enforced per route
- Live SSE dashboard fed by tui/events broadcast queue
- Config editor: keyword groups and channel list saved to data/runtime_config.json
  and hot-reloaded in-process (scorer.reload_from_config, signal_channel_changed)
- config.py migrated to load groups/channels from runtime_config.json;
  falls back to hardcoded defaults when file absent
- tui/events.py: subscribe/unsubscribe broadcast, set_bot_context/signal_channel_changed
- utils/scorer.py: import config as _config (fixes local binding); reload_from_config()
- utils/database.py: count_by_severity, recent_for_domains, count_by_severity_for_domains
- 53 new tests (events bus, JWT lifecycle, web DB CRUD, RBAC enforcement,
  config round-trip); total 141 passing
This commit is contained in:
2026-04-02 11:41:46 -03:00
parent b28168c846
commit 4c104cddd2
32 changed files with 2093 additions and 47 deletions

View File

@@ -1,28 +1,43 @@
# tui/events.py
Thread-safe event bus between the bot backend thread and the Textual TUI.
The bot thread calls `post()`. The TUI drains the queue every 100ms via `_drain_bus()`.
The bot thread calls `post()`. The TUI drains the queue every 100ms via `_drain_bus()`.
Web SSE consumers call `subscribe()` / `unsubscribe()` to get their own broadcast queue.
## Public API
```python
from tui import events as bus # from core/ and tui/app.py
from tui.events import post, init_bus, get_bus, tui_active
from tui.events import subscribe, unsubscribe
from tui.events import set_bot_context, signal_channel_changed
```
### `init_bus() -> queue.Queue`
Creates the `queue.Queue`. Called inside `MonitorApp.on_mount()`**must run on Textual's event loop**, not before `App.run()`.
### `post(event: Any) -> None`
Fire-and-forget from any thread. Silently drops if bus not initialised.
Fire-and-forget from any thread. Delivers to the TUI queue **and** all subscriber queues.
Uses `queue.Queue.put_nowait()` — never blocks.
### `get_bus() -> queue.Queue | None`
Returns the queue for the TUI consumer to drain.
Returns the TUI queue for `_drain_bus()` to consume.
### `tui_active: bool`
Set to `True` by `init_bus()`. Checked by `core/tdl_downloader.py` to decide whether to pipe tdl output or inherit the terminal.
### `subscribe() -> queue.Queue`
Register a new subscriber. Returns a private `queue.Queue` that receives every future `post()`. Thread-safe. Call once per SSE connection.
### `unsubscribe(q: queue.Queue) -> None`
Remove a subscriber queue. Safe to call if already removed. Call on SSE disconnect.
### `set_bot_context(loop, event) -> None`
Called by `_bot_main()` once the bot asyncio loop and `_ch_changed` event exist. Enables `signal_channel_changed()`.
### `signal_channel_changed() -> None`
Wake the bot's `_watch_channels()` coroutine from any thread. Used by web config routes after channel list is updated.
---
## Event types
@@ -49,18 +64,24 @@ Set to `True` by `init_bus()`. Checked by `core/tdl_downloader.py` to decide whe
Bot thread (own asyncio loop)
└─ bus.post(event) ← queue.Queue.put_nowait() [thread-safe]
queue.Queue
Textual thread (Textual's loop)
└─ _drain_bus() [set_interval 100ms]
└─ q.get_nowait() loop
└─ dispatch to widgets [safe, same thread as Textual]
_queue (TUI) + _subscribers[0..n] (web SSE)
Textual thread Web thread (uvicorn)
_drain_bus() SSE generator: q.get_nowait()
```
Channel changes flow the other way:
Channel changes from TUI:
```
_drain_bus sees EvChannelAdded/Removed
→ _signal_channel_changed()
→ loop.call_soon_threadsafe(asyncio.Event.set)
→ bot thread's _watch_channels() wakes
```
Channel changes from web:
```
PUT /config/channels
→ config.save_runtime_config()
→ bus.signal_channel_changed() ← uses stored loop + event
→ bot thread's _watch_channels() wakes
```

View File

@@ -7,8 +7,12 @@ queue.Queue (thread-safe), and the TUI consumer polls it from Textual's loop
using asyncio.get_event_loop().run_in_executor() bridging.
post() is safe to call from any thread or any asyncio loop.
Web frontend: call subscribe() to get a private queue that receives every
event post() delivers. Call unsubscribe() on disconnect.
"""
import asyncio
import queue
import threading
from dataclasses import dataclass, field
@@ -22,6 +26,49 @@ _queue_lock = threading.Lock()
# writing directly to the terminal.
tui_active: bool = False
# ─── Web subscriber broadcast ─────────────────────────────────────────────────
_subscribers: list[queue.Queue] = []
_subscribers_lock = threading.Lock()
def subscribe() -> queue.Queue:
"""Return a new Queue that receives every future post(). Thread-safe."""
q: queue.Queue = queue.Queue()
with _subscribers_lock:
_subscribers.append(q)
return q
def unsubscribe(q: queue.Queue) -> None:
"""Remove a subscriber queue. Safe to call even if already removed."""
with _subscribers_lock:
try:
_subscribers.remove(q)
except ValueError:
pass
# ─── Bot-loop channel-change signal (for web routes) ─────────────────────────
# The TUI sets this via set_bot_context() after the bot asyncio loop starts.
# Web config routes call signal_channel_changed() to wake _watch_channels().
_bot_loop: asyncio.AbstractEventLoop | None = None
_bot_ch_ev: asyncio.Event | None = None
def set_bot_context(loop: asyncio.AbstractEventLoop, event: asyncio.Event) -> None:
"""Called by _bot_main() once the bot loop and channel event exist."""
global _bot_loop, _bot_ch_ev
_bot_loop = loop
_bot_ch_ev = event
def signal_channel_changed() -> None:
"""Wake the bot's _watch_channels() coroutine from any thread."""
if _bot_loop is not None and _bot_ch_ev is not None:
_bot_loop.call_soon_threadsafe(_bot_ch_ev.set)
def init_bus() -> queue.Queue:
"""Call once from MonitorApp.on_mount() to create the queue."""
@@ -36,12 +83,19 @@ def get_bus() -> queue.Queue | None:
def post(event: Any) -> None:
"""Fire-and-forget from any thread. Silently drops if bus not up."""
"""Fire-and-forget from any thread. Broadcasts to TUI queue + all subscribers."""
if _queue is not None:
try:
_queue.put_nowait(event)
except queue.Full:
pass
with _subscribers_lock:
subs = list(_subscribers)
for q in subs:
try:
q.put_nowait(event)
except queue.Full:
pass
# ─── Event types ──────────────────────────────────────────────────────────────