Add web frontend with JWT auth, RBAC, SSE dashboard, and config editor
- FastAPI + htmx + Jinja2 web frontend, started with --web flag - JWT HS256 auth (WEB_SECRET_KEY) with httpOnly cookies; access (15 min) + refresh (7 day) tokens; refresh rotation + JTI revocation in data/web.db - RBAC: superadmin > admin > reader enforced per route - Live SSE dashboard fed by tui/events broadcast queue - Config editor: keyword groups and channel list saved to data/runtime_config.json and hot-reloaded in-process (scorer.reload_from_config, signal_channel_changed) - config.py migrated to load groups/channels from runtime_config.json; falls back to hardcoded defaults when file absent - tui/events.py: subscribe/unsubscribe broadcast, set_bot_context/signal_channel_changed - utils/scorer.py: import config as _config (fixes local binding); reload_from_config() - utils/database.py: count_by_severity, recent_for_domains, count_by_severity_for_domains - 53 new tests (events bus, JWT lifecycle, web DB CRUD, RBAC enforcement, config round-trip); total 141 passing
This commit is contained in:
@@ -1,28 +1,43 @@
|
||||
# tui/events.py
|
||||
|
||||
Thread-safe event bus between the bot backend thread and the Textual TUI.
|
||||
The bot thread calls `post()`. The TUI drains the queue every 100ms via `_drain_bus()`.
|
||||
The bot thread calls `post()`. The TUI drains the queue every 100ms via `_drain_bus()`.
|
||||
Web SSE consumers call `subscribe()` / `unsubscribe()` to get their own broadcast queue.
|
||||
|
||||
## Public API
|
||||
|
||||
```python
|
||||
from tui import events as bus # from core/ and tui/app.py
|
||||
from tui.events import post, init_bus, get_bus, tui_active
|
||||
from tui.events import subscribe, unsubscribe
|
||||
from tui.events import set_bot_context, signal_channel_changed
|
||||
```
|
||||
|
||||
### `init_bus() -> queue.Queue`
|
||||
Creates the `queue.Queue`. Called inside `MonitorApp.on_mount()` — **must run on Textual's event loop**, not before `App.run()`.
|
||||
|
||||
### `post(event: Any) -> None`
|
||||
Fire-and-forget from any thread. Silently drops if bus not initialised.
|
||||
Fire-and-forget from any thread. Delivers to the TUI queue **and** all subscriber queues.
|
||||
Uses `queue.Queue.put_nowait()` — never blocks.
|
||||
|
||||
### `get_bus() -> queue.Queue | None`
|
||||
Returns the queue for the TUI consumer to drain.
|
||||
Returns the TUI queue for `_drain_bus()` to consume.
|
||||
|
||||
### `tui_active: bool`
|
||||
Set to `True` by `init_bus()`. Checked by `core/tdl_downloader.py` to decide whether to pipe tdl output or inherit the terminal.
|
||||
|
||||
### `subscribe() -> queue.Queue`
|
||||
Register a new subscriber. Returns a private `queue.Queue` that receives every future `post()`. Thread-safe. Call once per SSE connection.
|
||||
|
||||
### `unsubscribe(q: queue.Queue) -> None`
|
||||
Remove a subscriber queue. Safe to call if already removed. Call on SSE disconnect.
|
||||
|
||||
### `set_bot_context(loop, event) -> None`
|
||||
Called by `_bot_main()` once the bot asyncio loop and `_ch_changed` event exist. Enables `signal_channel_changed()`.
|
||||
|
||||
### `signal_channel_changed() -> None`
|
||||
Wake the bot's `_watch_channels()` coroutine from any thread. Used by web config routes after channel list is updated.
|
||||
|
||||
---
|
||||
|
||||
## Event types
|
||||
@@ -49,18 +64,24 @@ Set to `True` by `init_bus()`. Checked by `core/tdl_downloader.py` to decide whe
|
||||
Bot thread (own asyncio loop)
|
||||
└─ bus.post(event) ← queue.Queue.put_nowait() [thread-safe]
|
||||
↓
|
||||
queue.Queue
|
||||
↓
|
||||
Textual thread (Textual's loop)
|
||||
└─ _drain_bus() [set_interval 100ms]
|
||||
└─ q.get_nowait() loop
|
||||
└─ dispatch to widgets [safe, same thread as Textual]
|
||||
_queue (TUI) + _subscribers[0..n] (web SSE)
|
||||
↓ ↓
|
||||
Textual thread Web thread (uvicorn)
|
||||
_drain_bus() SSE generator: q.get_nowait()
|
||||
```
|
||||
|
||||
Channel changes flow the other way:
|
||||
Channel changes from TUI:
|
||||
```
|
||||
_drain_bus sees EvChannelAdded/Removed
|
||||
→ _signal_channel_changed()
|
||||
→ loop.call_soon_threadsafe(asyncio.Event.set)
|
||||
→ bot thread's _watch_channels() wakes
|
||||
```
|
||||
|
||||
Channel changes from web:
|
||||
```
|
||||
PUT /config/channels
|
||||
→ config.save_runtime_config()
|
||||
→ bus.signal_channel_changed() ← uses stored loop + event
|
||||
→ bot thread's _watch_channels() wakes
|
||||
```
|
||||
|
||||
@@ -7,8 +7,12 @@ queue.Queue (thread-safe), and the TUI consumer polls it from Textual's loop
|
||||
using asyncio.get_event_loop().run_in_executor() bridging.
|
||||
|
||||
post() is safe to call from any thread or any asyncio loop.
|
||||
|
||||
Web frontend: call subscribe() to get a private queue that receives every
|
||||
event post() delivers. Call unsubscribe() on disconnect.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import queue
|
||||
import threading
|
||||
from dataclasses import dataclass, field
|
||||
@@ -22,6 +26,49 @@ _queue_lock = threading.Lock()
|
||||
# writing directly to the terminal.
|
||||
tui_active: bool = False
|
||||
|
||||
# ─── Web subscriber broadcast ─────────────────────────────────────────────────
|
||||
|
||||
_subscribers: list[queue.Queue] = []
|
||||
_subscribers_lock = threading.Lock()
|
||||
|
||||
|
||||
def subscribe() -> queue.Queue:
|
||||
"""Return a new Queue that receives every future post(). Thread-safe."""
|
||||
q: queue.Queue = queue.Queue()
|
||||
with _subscribers_lock:
|
||||
_subscribers.append(q)
|
||||
return q
|
||||
|
||||
|
||||
def unsubscribe(q: queue.Queue) -> None:
|
||||
"""Remove a subscriber queue. Safe to call even if already removed."""
|
||||
with _subscribers_lock:
|
||||
try:
|
||||
_subscribers.remove(q)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
|
||||
# ─── Bot-loop channel-change signal (for web routes) ─────────────────────────
|
||||
# The TUI sets this via set_bot_context() after the bot asyncio loop starts.
|
||||
# Web config routes call signal_channel_changed() to wake _watch_channels().
|
||||
|
||||
_bot_loop: asyncio.AbstractEventLoop | None = None
|
||||
_bot_ch_ev: asyncio.Event | None = None
|
||||
|
||||
|
||||
def set_bot_context(loop: asyncio.AbstractEventLoop, event: asyncio.Event) -> None:
|
||||
"""Called by _bot_main() once the bot loop and channel event exist."""
|
||||
global _bot_loop, _bot_ch_ev
|
||||
_bot_loop = loop
|
||||
_bot_ch_ev = event
|
||||
|
||||
|
||||
def signal_channel_changed() -> None:
|
||||
"""Wake the bot's _watch_channels() coroutine from any thread."""
|
||||
if _bot_loop is not None and _bot_ch_ev is not None:
|
||||
_bot_loop.call_soon_threadsafe(_bot_ch_ev.set)
|
||||
|
||||
|
||||
def init_bus() -> queue.Queue:
|
||||
"""Call once from MonitorApp.on_mount() to create the queue."""
|
||||
@@ -36,12 +83,19 @@ def get_bus() -> queue.Queue | None:
|
||||
|
||||
|
||||
def post(event: Any) -> None:
|
||||
"""Fire-and-forget from any thread. Silently drops if bus not up."""
|
||||
"""Fire-and-forget from any thread. Broadcasts to TUI queue + all subscribers."""
|
||||
if _queue is not None:
|
||||
try:
|
||||
_queue.put_nowait(event)
|
||||
except queue.Full:
|
||||
pass
|
||||
with _subscribers_lock:
|
||||
subs = list(_subscribers)
|
||||
for q in subs:
|
||||
try:
|
||||
q.put_nowait(event)
|
||||
except queue.Full:
|
||||
pass
|
||||
|
||||
|
||||
# ─── Event types ──────────────────────────────────────────────────────────────
|
||||
|
||||
Reference in New Issue
Block a user