DISTILLATION

Sentinel sorts every headline it ingests into one of eight topic classes, and the thing doing the sorting is a keyword regex whose if-chain returns on first match. The order of the branches decides the answer, so "Amazon Prime Day" is tech, "SpaceX launch" is tech, and any headline containing `china` is geopolitics regardless of what it is about. Everything that matches no keyword falls through to `general` — 74.2% of the corpus. The obvious fix is to call a frontier API on every headline, which costs money forever, adds a network round trip to something that has to be fast, and breaks when somebody else's service does. This project measures the other option: train a small model to do the job.

Rebuild the corpus from the same 63 public RSS feeds the product already reads — 3,706 headlines across 54 outlets, zero credentials, zero user data. Split off 500 held-out examples and freeze them before generating a single label. Label the rest with a 35B open-weight teacher running locally through Ollama, hand-audit 50 of its answers to establish the ceiling before training anything, then LoRA fine-tune a 4B open model on those labels on the same laptop. Score three arms — the incumbent regex ported with its defects intact, the teacher, and the student — through one harness on the identical held-out 500, and report quality, cost and latency for all three.

Sentinel files every headline it ingests into one of eight topic classes, and it does that with a keyword regex — a first-match-wins if-chain in the production backend. It is broken in ways you can see just by reading it. `amazon` sits in the tech rule ahead of the consumer rule, so "Amazon Prime Day" is tech and the consumer rule's own `amazon prime` keyword can never be reached. `spacex` appears in both tech and science, and tech is checked first. Any headline containing `china` is geopolitics forever. Everything unmatched becomes `general`, which is most things: 74.2% of this corpus, and 375 of the 500 held-out headlines.

The obvious fix is to call a frontier model on every headline. That costs money on every request forever, puts a network round trip in front of something that has to be fast, and hands your uptime to somebody else. So the project measured the other option instead: have a big open model label the data, and train a small one to do the job by itself.

The corpus was rebuilt from the same 63 public RSS feeds the product already reads — 3,706 headlines across 54 outlets, no database access, no user data, no API keys. 500 were split off and frozen before a single label was generated. A 35B open-weight teacher (Apache-2.0, run locally through Ollama) labelled the rest at 0.00% unparseable output. Fifty of its answers were then graded by hand, before any training, and it agreed with a human 84% of the time — the ceiling on everything downstream, measured first rather than assumed.

A 4B open model was then LoRA fine-tuned on those labels, on the laptop, and all three arms were scored through one harness on the same held-out 500. The student reaches 0.840 macro-F1 and agrees with its teacher on 85.4% of headlines, at 2.4x lower latency and — on list prices applied to measured token counts, never measured spend — 41.3x lower cost. The regex it would replace scores 0.337 and cannot emit the `consumer` class even once in 500 headlines.

Getting there took two training runs. The first died at roughly iteration 1,170 of 1,200 in a macOS GPU-driver kernel panic that took its log with it; it was retrained from scratch rather than resumed, because resuming restarts the iteration counter and yields a curve that cannot be honestly plotted. The shipped weights are not the final ones either: iteration 800 beat iteration 1200 by 8.0 macro-F1 points, chosen on a validation split by a module that cannot read the test set. Both evaluations are committed.

One module per stage under src/, each runnable on its own, wired together by a single reproduce command that regenerates every result and chart the README cites — and deliberately does not train, because a 70-minute GPU job with its own failure modes should be launched on purpose rather than as a side effect of asking for a chart.

feeds.py holds 63 production feeds plus 83 same-outlet sections. harvest.py / rss.py / store.py fetch them asynchronously and dedup on normalised URL. split.py draws the held-out 500 once, freezes membership, and asserts disjointness rather than assuming it, so a harvester that keeps adding rows can never contaminate the evaluation set. teacher.py drives Qwen3.5-35B-A3B (Q4_K_M) through Ollama with schema-constrained decoding and parser guards that mark unparseable output rather than coercing it to the majority class. prepare_training.py owns student_messages(), the single shared prompt shape, so training and inference cannot drift apart. configs/lora.yaml pins the base model by revision SHA and feeds mlx_lm.lora at r=16, bf16, 16 of 32 layers, mask_prompt on — the adapter is 0.096% of the model's parameters. regex_baseline.py is the faithful port, compiled with re.ASCII to match JavaScript's word-boundary semantics. evaluate.py is one harness for all three arms, which is what makes the comparison a measurement rather than three separate stories.

Sequencing was dictated by 34 GB of free disk, not by RAM. The naive ordering needs about 37 GB — 19 GB of teacher, 8 GB of base model, 8 GB merged, plus the environment — so the plan deletes the teacher before the student's base model is pulled. That is why teacher latency had to be measured while its weights were still resident: sequentially, one request at a time, warm, over the same held-out 500, never inferred from batched labelling throughput.

The thing being replaced was audited before anything was trained. Sentinel's production topic classifier is a first-match-wins keyword if-chain, and it is wrong in ways you can read rather than measure: `amazon` sits in the tech rule ahead of the consumer rule, so "Amazon Prime Day" files as tech and the consumer rule's own `amazon prime` keyword is unreachable dead code; `spacex` appears in both the tech and science rules and only tech can ever win; any headline containing `china` is geopolitics forever; and anything matching no keyword falls through to `general`, which on this corpus is 74.2% of it. The port in src/regex_baseline.py keeps every defect intact and tests/test_regex_baseline.py pins each one with a passing test, so a later "fix" fails loudly instead of quietly turning the baseline into a different program. On the held-out 500 the incumbent sends 375 headlines to `general`; the teacher moves 310 of those somewhere else.

The audit turned up a sixth defect nobody had listed. Every keyword in the chain is a bare noun and the word boundary demands a non-word character after it, so `russia` never matches "Russian", `china` never matches "Chinese", `israel` never matches "Israeli" — and headlines use the adjectival form constantly, so every one of them lands in `general`. Porting it faithfully also needed re.ASCII: JavaScript's word boundary is ASCII-based and Python's is Unicode-aware, so without the flag the incumbent arm would have been quietly weaker on accented text than the code actually running in production, on a corpus that includes DW, France24, SCMP and Haaretz. Three more failures were silent rather than loud. An architecture probe run before training caught Qwen3.5-4B opening a thinking block at inference while the training data carried a closed one — the adapter answered "Thinking Process:" for all five test cases, and the same adapter scored 5/5 once enable_thinking=False reproduced the training prefix byte for byte. Ollama defaulted the 35B teacher to a 32,768-token context for ~250-token prompts and drove the machine into 7.7 GB of swap, taking free disk from 12 GB to 4 GB mid-run; pinning num_ctx to 2048 stopped it, and the 60 examples labelled before the change were re-labelled to prove it moved no label (60/60 identical). And the first training run died at roughly iteration 1,170 of 1,200 in a macOS GPU-driver kernel panic that took its stdout log with it — rather than resume from a checkpoint, which restarts the iteration counter and produces a curve that cannot be honestly plotted, it was retrained from scratch with the log written inside the repo. One design choice removed a whole class of this: held-out membership is frozen on the first split rather than re-drawn, so a background harvester that keeps adding rows cannot leak into the evaluation set — impossible by construction rather than by remembering the rule.

On the same 500 held-out headlines, the student reaches macro-F1 0.8400 / accuracy 0.8540 against the incumbent's 0.3372 / 0.3420, with zero unparseable outputs from either. It wins all eight classes on F1 and loses exactly one on recall: `general`, where the regex scores 0.985 only because it answers `general` three times in four, at 0.173 precision. Head to head, the student is right where the regex is wrong 285 times; the regex is right where the student is wrong 29 times. The regex never emits `consumer` once in 500 headlines — precision, recall and F1 all exactly 0.0. Two measurements changed what shipped. Selecting the best-validation checkpoint (iteration 800) over mlx-lm's default final weights (iteration 1200) is worth +8.0 macro-F1 points, 0.8400 against 0.7599, and both evaluations are committed rather than only the flattering one. And the last 200 iterations were an optimisation excursion rather than overfitting: validation loss went 0.075 to 0.280 while mean training loss rose with it, 0.071 to 0.256 — 1,200 iterations was simply too many. The cost model was also wrong before it was right: all four token constants were wrong when re-tokenised over the real 500 held-out prompts, moving the published headline from 45.4x to 41.3x cheaper. Teacher reproducibility is evidenced three ways — 100/100 unanimous at temperature 0, 60/60 across a mid-run config change, and 500/500 on an independent re-prediction of the whole held-out set.