anti
48f486ac97
- Core Telegram monitoring pipeline (scraper, processor, notifier, downloaders) - Textual TUI frontend with thread-safe event bus - SQLite persistence, severity scoring, dedup cache - Fixed ULP parser: handles https:// truncation, port+path URLs, semicolon separator - Test suite: 88 tests across scorer, cache, database, processor
68 lines
2.1 KiB
Markdown
68 lines
2.1 KiB
Markdown
# core/notifier.py
|
|
|
|
Scores hits, deduplicates, persists to disk and DB, sends Telegram alerts.
|
|
|
|
## Public API
|
|
|
|
```python
|
|
from core.notifier import notify, send_status
|
|
```
|
|
|
|
### `notify(bot, hits: list[str], source: str, filename: str)`
|
|
**async.** Full notification pipeline:
|
|
1. `score_hits(hits)` → `list[ScoredHit]`
|
|
2. Deduplicate via SHA-256 hashes (`data/dedup.json`)
|
|
3. `insert_hits()` into SQLite for new + dupes (flagged accordingly)
|
|
4. `write_hits()` → append to `data/hits.txt`
|
|
5. `write_hits_csv()` → append to `data/hits.csv`
|
|
6. `send_alert()` → Telegram message for CRITICAL/HIGH/MEDIUM only
|
|
7. Post `EvHit` events onto the TUI bus for each new hit
|
|
|
|
### `send_status(bot, message: str)`
|
|
**async.** Sends a plain Markdown message to `config.NOTIFY_CHAT_ID`. Used for startup/status notifications.
|
|
|
|
---
|
|
|
|
## Internal functions
|
|
|
|
| Function | Description |
|
|
|----------|-------------|
|
|
| `deduplicate(hits)` | Returns `(new_hits, dupe_hits)`; updates `data/dedup.json` |
|
|
| `write_hits(scored_hits, source)` | Appends grouped human-readable block to `data/hits.txt` |
|
|
| `write_hits_csv(scored_hits, source, filename)` | Appends rows to `data/hits.csv`; writes header on first call |
|
|
| `send_alert(bot, scored_hits, source, filename)` | Sends Telegram message grouped by severity; skips if all LOW |
|
|
|
|
---
|
|
|
|
## Output files
|
|
|
|
| File | Format | Notes |
|
|
|------|--------|-------|
|
|
| `data/hits.txt` | Plain text, grouped by severity | Human-readable, append-only |
|
|
| `data/hits.csv` | CSV with header | Columns: `timestamp, severity, score, url, username, password, reasons, source, filename` |
|
|
| `data/dedup.json` | JSON array of SHA-256 hex strings | Hashes of `line.strip().lower()` |
|
|
|
|
---
|
|
|
|
## Alert behaviour
|
|
|
|
- CRITICAL / HIGH / MEDIUM → Telegram alert sent immediately
|
|
- LOW → stored in DB + files, **no** Telegram alert
|
|
- Duplicates → stored in DB with `seen_before=1`, no alert, no file write
|
|
|
|
## Telegram alert format
|
|
|
|
```
|
|
🚨 Credential hit(s) detected
|
|
📁 `filename`
|
|
📢 `source`
|
|
🕐 `timestamp`
|
|
|
|
Summary: 🔴 N 🟠 N 🟡 N 🟢 N
|
|
|
|
🔴 CRITICAL (N)
|
|
`url:user:pass`
|
|
↳ reason | reason
|
|
... (up to 10 per severity; remainder counted)
|
|
```
|