Initial commit: ULPgrammer

- Core Telegram monitoring pipeline (scraper, processor, notifier, downloaders)
- Textual TUI frontend with thread-safe event bus
- SQLite persistence, severity scoring, dedup cache
- Fixed ULP parser: handles https:// truncation, port+path URLs, semicolon separator
- Test suite: 88 tests across scorer, cache, database, processor

tui/__init__.py

@@ -0,0 +1 @@
+"""tui — Textual TUI frontend and event bus."""

tui/app.md

@@ -0,0 +1,130 @@
+# tui/app.py
+
+Textual TUI frontend. Entry point: `run_tui()`.
+
+## Entry point
+
+```python
+from tui.app import run_tui
+run_tui()   # called by main.py
+```
+
+---
+
+## Screen hierarchy
+
+```
+MonitorApp (App)
+├── [default screen]
+│   ├── Header
+│   ├── #top-row (Horizontal)
+│   │   ├── DownloadPanel  #dl-panel
+│   │   └── HitsPanel      #hits-panel
+│   ├── StatsPanel         #stats-panel
+│   ├── ChannelPanel       #ch-panel
+│   └── Footer
+├── SearchScreen     (push/pop via 's')
+├── HitsDBScreen     (push/pop via 'h')
+└── KeywordsScreen   (push/pop via 'k')
+```
+
+---
+
+## MonitorApp
+
+### Threading model
+- **Bot backend** → `threading.Thread(daemon=True)` with its own `asyncio.new_event_loop()`  
+  Runs `_bot_main()` — Telethon is completely isolated from Textual's loop.
+- **TUI drain** → `set_interval(0.1, _drain_bus)` — polls `queue.Queue` every 100ms on Textual's loop.
+
+### Key methods
+
+| Method | Description |
+|--------|-------------|
+| `on_mount()` | Calls `bus.init_bus()`, starts bot thread, sets drain interval |
+| `_drain_bus()` | Drains all pending events from `queue.Queue`, dispatches to widgets |
+| `_run_bot_thread()` | Thread entry: creates event loop, runs `_bot_main()` |
+| `_bot_main()` | Async bot backend: connect, auth, backfill, live handler loop |
+| `_signal_channel_changed()` | Thread-safely sets the bot loop's `asyncio.Event` via `call_soon_threadsafe` |
+
+### Keybindings
+
+| Key | Action |
+|-----|--------|
+| `s` | Push `SearchScreen` |
+| `h` | Push `HitsDBScreen` |
+| `k` | Push `KeywordsScreen` |
+| `c` | Clear download + hits logs |
+| `r` | Force-refresh stats bar |
+| `q` / `ctrl+c` | Quit |
+
+---
+
+## Widgets
+
+### DownloadPanel
+Left panel. Two `RichLog` widgets separated by a dashed line:
+- **top** (`#tdl-out`): raw tdl output lines (ANSI stripped)
+- **bottom** (`#dl-log`): structured download status entries
+
+Methods: `tdl_line(line)`, `queued(filename, size_mb, source, password)`, `status(filename, state, via)`, `clear_logs()`
+
+States for `status()`: `queued` · `downloading` · `done_tdl` · `done_tel` · `failed`
+
+### HitsPanel
+Right panel. Single `RichLog` with color-coded hit entries.  
+Reactive `hit_count` updates the panel title badge automatically.
+
+Methods: `add_hit(severity, raw, source, filename, reasons)`, `clear_log()`
+
+### StatsPanel
+Slim horizontal bar. Polls `utils.database.stats()` every 10s via `set_interval`.  
+Also refreshed immediately on each `EvHit` event.
+
+### ChannelPanel
+Bottom panel. `ListView` + `Input` + buttons.  
+Add/remove posts `EvChannelAdded` / `EvChannelRemoved` onto the bus.  
+Changes apply immediately (handler re-registered). Not persisted to `config.py` automatically.
+
+---
+
+## Screens
+
+### SearchScreen (`s`)
+- Text input → queries `utils.database.search(keyword)`
+- Results in a `DataTable` with columns: Sev, Time, URL, Username, Password, Source, File
+- Submit with `↵` or Search button; `Escape` to dismiss
+
+### HitsDBScreen (`h`)
+- Toolbar buttons + number keys filter by severity
+- `r` → recent 50, `1`→CRITICAL, `2`→HIGH, `3`→MEDIUM, `4`→LOW
+- Calls `utils.database.recent()` / `by_severity()`
+
+### KeywordsScreen (`k`)
+- Live-edit `config.TARGET_KEYWORDS`
+- Validates regex before adding
+- On change: rebuilds `utils.scorer.EMPLOYEE_DOMAINS` and `ORG_DOMAINS`
+- Bot handler recompiles patterns on the next incoming message automatically
+- **Changes are in-memory only** — copy to `config.py` to persist
+
+---
+
+## Bot auth flow (`_bot_main`)
+
+```
+await bot_client.connect()
+await bot_client.is_user_authorized()? → sign_in(bot_token=...)
+await user_client.connect()
+await user_client.is_user_authorized()? → log error + return (must run --no-tui first)
+warm_entity_cache()
+_make_handler(channels)       ← registers NewMessage handler
+backfill_all()
+run_until_disconnected()  ┐
+_watch_channels()         ┘  gathered
+```
+
+Channel-change signal path:
+```
+ChannelPanel button → EvChannel* on bus → _drain_bus → _signal_channel_changed()
+  → call_soon_threadsafe(asyncio.Event.set) → _watch_channels() wakes → _make_handler()
+```

tui/events.md

@@ -0,0 +1,66 @@
+# tui/events.py
+
+Thread-safe event bus between the bot backend thread and the Textual TUI.  
+The bot thread calls `post()`. The TUI drains the queue every 100ms via `_drain_bus()`.
+
+## Public API
+
+```python
+from tui import events as bus   # from core/ and tui/app.py
+from tui.events import post, init_bus, get_bus, tui_active
+```
+
+### `init_bus() -> queue.Queue`
+Creates the `queue.Queue`. Called inside `MonitorApp.on_mount()` — **must run on Textual's event loop**, not before `App.run()`.
+
+### `post(event: Any) -> None`
+Fire-and-forget from any thread. Silently drops if bus not initialised.  
+Uses `queue.Queue.put_nowait()` — never blocks.
+
+### `get_bus() -> queue.Queue | None`
+Returns the queue for the TUI consumer to drain.
+
+### `tui_active: bool`
+Set to `True` by `init_bus()`. Checked by `core/tdl_downloader.py` to decide whether to pipe tdl output or inherit the terminal.
+
+---
+
+## Event types
+
+| Class | Fields | Posted by | Consumed by |
+|-------|--------|-----------|-------------|
+| `EvDownloadQueued` | `batch_id, filename, size_mb, source, password` | `tdl_downloader`, `scraper` | `DownloadPanel.queued()` |
+| `EvDownloadStarted` | `batch_id, filename` | `tdl_downloader`, `scraper` | `DownloadPanel.status("downloading")` |
+| `EvDownloadDone` | `batch_id, filename, via` | `tdl_downloader`, `scraper` | `DownloadPanel.status("done_tdl"\|"done_tel")` |
+| `EvDownloadFailed` | `batch_id, filename, reason` | `tdl_downloader`, `scraper` | `DownloadPanel.status("failed")` |
+| `EvTdlOutput` | `line` | `tdl_downloader._relay()` | `DownloadPanel.tdl_line()` |
+| `EvHit` | `severity, raw, source, filename, reasons` | `notifier.notify()` | `HitsPanel.add_hit()` + `StatsPanel.refresh_stats()` |
+| `EvChannelAdded` | `channel` | `ChannelPanel.on_button_pressed()` | `_drain_bus` → `_signal_channel_changed()` |
+| `EvChannelRemoved` | `channel` | `ChannelPanel.on_button_pressed()` | `_drain_bus` → `_signal_channel_changed()` |
+| `EvStatus` | `text, level` | everywhere | `MonitorApp.notify()` toast |
+
+`level` on `EvStatus`: `"info"` (default) · `"warning"` · `"error"`
+
+---
+
+## Threading model
+
+```
+Bot thread (own asyncio loop)
+  └─ bus.post(event)          ← queue.Queue.put_nowait() [thread-safe]
+        ↓
+  queue.Queue
+        ↓
+Textual thread (Textual's loop)
+  └─ _drain_bus() [set_interval 100ms]
+       └─ q.get_nowait() loop
+            └─ dispatch to widgets [safe, same thread as Textual]
+```
+
+Channel changes flow the other way:
+```
+_drain_bus sees EvChannelAdded/Removed
+  → _signal_channel_changed()
+       → loop.call_soon_threadsafe(asyncio.Event.set)
+            → bot thread's _watch_channels() wakes
+```

tui/events.py

@@ -0,0 +1,114 @@
+"""
+tui_events.py — Thread-safe event bus between the bot backend and the TUI.
+
+The bot backend runs in a dedicated thread with its own asyncio event loop
+(completely isolated from Textual's loop).  Events are posted via a standard
+queue.Queue (thread-safe), and the TUI consumer polls it from Textual's loop
+using asyncio.get_event_loop().run_in_executor() bridging.
+
+post() is safe to call from any thread or any asyncio loop.
+"""
+
+import queue
+import threading
+from dataclasses import dataclass, field
+from typing import Any
+
+# Thread-safe queue — works across the bot thread and Textual's thread.
+_queue: queue.Queue | None = None
+_queue_lock = threading.Lock()
+
+# Set to True when the TUI is running so tdl pipes output instead of
+# writing directly to the terminal.
+tui_active: bool = False
+
+
+def init_bus() -> queue.Queue:
+    """Call once from MonitorApp.on_mount() to create the queue."""
+    global _queue, tui_active
+    _queue = queue.Queue()
+    tui_active = True
+    return _queue
+
+
+def get_bus() -> queue.Queue | None:
+    return _queue
+
+
+def post(event: Any) -> None:
+    """Fire-and-forget from any thread. Silently drops if bus not up."""
+    if _queue is not None:
+        try:
+            _queue.put_nowait(event)
+        except queue.Full:
+            pass
+
+
+# ─── Event types ──────────────────────────────────────────────────────────────
+
+@dataclass
+class EvDownloadQueued:
+    """A file has been accepted and is waiting for tdl."""
+    batch_id:  str
+    filename:  str
+    size_mb:   float
+    source:    str
+    password:  str | None
+
+
+@dataclass
+class EvDownloadStarted:
+    """tdl has begun transferring this file."""
+    batch_id:  str
+    filename:  str
+
+
+@dataclass
+class EvDownloadDone:
+    """File fully downloaded (tdl or Telethon fallback)."""
+    batch_id:  str
+    filename:  str
+    via:       str   # "tdl" | "telethon"
+
+
+@dataclass
+class EvDownloadFailed:
+    """All download attempts failed."""
+    batch_id:  str
+    filename:  str
+    reason:    str
+
+
+@dataclass
+class EvTdlOutput:
+    """A line of output from tdl's stdout/stderr (TUI mode only)."""
+    line: str
+
+
+@dataclass
+class EvHit:
+    """A scored credential hit to display in the hits panel."""
+    severity:  str
+    raw:       str
+    source:    str
+    filename:  str
+    reasons:   list[str] = field(default_factory=list)
+
+
+@dataclass
+class EvChannelAdded:
+    """A channel was added to the live watch list."""
+    channel: str | int
+
+
+@dataclass
+class EvChannelRemoved:
+    """A channel was removed from the live watch list."""
+    channel: str | int
+
+
+@dataclass
+class EvStatus:
+    """Generic one-line status message (startup, errors, etc.)."""
+    text: str
+    level: str = "info"   # "info" | "warning" | "error"