Trail: A Log Parser That Shows Its Work

⬅️ Back to Projects

🔧 Trail

Most log parsers hand you templates and a score. I wanted the receipt. Trail parses in one deterministic pass and writes one JSON record per input line: cluster, similarity, decision, resulting template. When grouping goes wrong you can point at the exact line where it happened.

What I learned

1. One accuracy number hides two different failures. On OpenSSH_2k Trail scores GA 0.542 against Drain at 0.789, and nearly the whole gap sits in two clusters holding 908 of 2000 lines. One is a confident merge at similarity 0.90, a pam_unix failure line with and without a trailing user field. I side with the miner there. Same event, optional field. The other is a borderline swallow at line 964, similarity 0.625, where one rare disconnected by user line lands in 413 Bye Bye lines some 950 lines after the cluster was born. GA counts both the same. The records keep them apart.
2. Anchors carry the score, and they cut both ways. Trail only merges clusters whose token counts differ by at most one and whose first two tokens match exactly. Without anchors, Linux GA falls from 0.690 to 0.186 while Apache collapses from 37 templates to 6 and scores a perfect 1.000. The same rule that saves Linux causes the Apache over-fragmentation. One anchor keeps almost all of the Linux gain. Meanwhile the similarity threshold is a dead knob. I ran st at 0.4, 0.5, and 0.6 across all three sets and got bit-identical output every time. Structure decides the merges here. The threshold never binds.
3. One global config tying tuned Drain on Linux beats three tuned wins. Drain runs per-dataset settings in my harness, st 0.5, 0.39, 0.6 with depth 4, 6, 5. Trail runs st 0.5 with two anchors and slack 1 everywhere and still ties Drain GA on Linux at 0.690. Slack 0 alone would take OpenSSH to 0.790, but that is tuning after seeing the labels, so the report states it as future work instead of shipping a tuned aggregate.
4. Keep the model out of the parse. A local Qwen3 2B distill reviews only low-confidence joins under 0.7, and its proposals land in a separate assisted CSV. The deterministic CSV and audit never change. Anything touching more than 10 lines is held for a human and never auto-applied. The gate exists because the first prompt version took SecOps tight GA from 0.967 to 0.682 through two false accepts. The stricter prompt makes zero auto-applies and holds three cases. Cost on my M2: 463 seconds for 16 candidates, about 40 seconds of model load. Slow, local, private. That is the point.
5. My synthetic win is a sanity check, not a victory lap. On SecOps-2k, 2000 sshd, sudo, firewall, and auditd lines over 25 tight templates, Trail reaches GA 0.967 against Drain at 0.772. But I built that set alongside the miner and its templates exercise my own identity keys and masks. No organic noise, no clock skew, no interleaving. It shows the machinery works, nothing about production logs.

How it works

Strip the syslog header with one regex, split on whitespace, mask a small pinned list of whole tokens. For each line, consider only clusters within length slack 1 whose first two tokens match, reject merges that disagree on an identity key like PROTO, and join the best candidate above similarity 0.5. Templates only ever generalize, never split.

{
  "line": 964,
  "cluster": "T8",
  "decision": "matched",
  "similarity": 0.625,
  "template": "Received disconnect from <*> 11: <*> <*> <*>"
}

That record is both the explanation and the review queue. Anything under 0.7 becomes a candidate for the local model loop.

Steal this if: you parse logs where a wrong merge has consequences and you need to defend the grouping to someone who was not in the room.

Related TMFNK Content

• autoresearch: An AI That Runs LLM Experiments While You Sleep Overnight agent-run LLM experiments with pinned configs and local models, the same reproducibility instinct as Trail’s golden files.
• Learn Harness Engineering: Make AI Coding Agents Actually Reliable What a good eval harness looks like, and Trail’s reproduce script with verified goldens is a small working example.
• Fine-Tune a Local SLM to Clean Master Data on Your Mac Small-local-model work on a Mac, and why Trail’s 2B reviewer runs offline through llama.cpp instead of an API.

Crepi il lupo! 🐺