Anomaly detection
Isolation Forest
Builds 200 random trees; windows that can be isolated with very few splits are unusual. Great at catching behaviour unlike anything else in the log.
Under the Hood
How It Works
From raw text to a verdict in seconds: a streaming parser, columnar statistics, twelve models, a security scanner and two AI systems, all behind encryption that expires in an hour.
The Pipeline
Eight Steps From Upload to Insight
- 01
Stream and Encrypt
The file streams straight into AES-256-GCM encrypted chunks. A per-upload key is derived with HKDF from a server key and a secret only your browser holds.
- 02
Detect the Format
The first 800 lines are scored against 17 format signatures and 12 timestamp styles. The best match drives parsing; unknown formats fall back to a universal parser.
- 03
Learn Message Patterns
The Drain algorithm masks variables such as numbers, IPs and IDs, then groups lines into templates. Millions of lines become a few hundred patterns.
- 04
Build Columnar Data
Timestamps, severity, components, HTTP and network fields go into compact NumPy columns, so statistics over millions of lines take milliseconds.
- 05
Run the Models
Windowed features feed five anomaly detectors, clustering, sequence models, forecasting and rule mining in an isolated worker process.
- 06
Scan for Threats
Rules look for brute force, logins after guessing, privilege escalation, port scans, floods, injection attempts, scanners and leaked secrets.
- 07
Add AI Insight
A redacted summary goes to JEV for structured decisions and to Groq for plain-English summaries, root causes and answers to your questions.
- 08
Shred After One Hour
Access ends one hour after upload and a sweeper overwrites and deletes every file. If the server was asleep, it shreds expired files as soon as it wakes. You can delete sooner at any time.
Formats
Supported Log Formats
If it is text with one event per line, it works. These formats get specialised parsing.
| Family | Formats |
|---|---|
| Web servers | Nginx and Apache access logs (combined, common, with response times), Apache error logs |
| System | Syslog RFC 3164 and 5424, Linux, macOS, Windows servicing (CBS) logs, supercomputer RAS logs |
| Security | SSH and auth.log, sudo and su, PAM, Windows logon failures |
| Network | UFW and iptables kernel logs, Cisco ASA, proxy connection logs |
| Applications | JSON lines, logfmt, Java, Spring Boot, Log4j, Hadoop, HDFS, Spark, Zookeeper, OpenStack, Android logcat |
| Anything else | Any line-based text log: timestamps, severity and components are inferred from the text |
Machine Learning
The Models, Explained
Each model looks for a different kind of problem. Together they catch what any one would miss.
Anomaly detection
Local Outlier Factor
Compares each window's density with its nearest neighbours, catching periods that are odd relative to similar periods rather than globally.
Anomaly detection
PCA Residual Detector
Learns the normal correlations between message types with Principal Component Analysis. Windows that can't be reconstructed (high residual) broke the usual mix.
Anomaly detection
Robust Z-Score
Uses the median and median absolute deviation instead of mean and standard deviation, so a single spike can't hide itself by inflating the baseline.
Sequence modelling
Markov Chain
Learns the probability that one message follows another within each component. Windows full of improbable transitions are flagged, and rare transitions are listed.
Decision
Ensemble Vote
A window is only reported as unusual when at least two of the models agree. Each report explains which features moved and by how much.
Clustering
K-Means
Groups message patterns into themes using TF-IDF vectors, and groups clients or sources into behavioural segments. The number of groups is chosen by silhouette score.
Clustering
DBSCAN
Density clustering on reduced message vectors. Messages that don't belong to any dense group are surfaced as rare events worth reading.
Time series
Change-Point Detection
Binary segmentation with a BIC-style penalty finds the exact windows where volume, errors or latency shifted to a new level.
Time series
Holt-Winters Forecasting
Damped trend with optional daily or hourly seasonality, tuned by grid search and back-tested on held-out data so you can see how reliable it is.
Pattern mining
Apriori Association Rules
Treats each window as a basket of warnings and errors, and finds combinations that occur together far more often than chance (lift).
Pattern mining
Periodicity Detection
Measures the regularity of gaps between repeated messages to find cron jobs, health checks and heartbeats, including missed runs.
AI Engineering
Two AI Systems, Each Doing What It Does Best
JEV by TypeSafe
A decision model that answers structured questions with calibrated probabilities: the overall state (healthy, degraded, incident or critical), whether action is needed today, the primary type of problem, the stability outlook for investors, the urgency of every recommendation and the likely user impact of every incident.
Groq
Open-weight GPT-OSS models on Groq's fast inference write the executive and developer summaries, root-cause hypotheses, key moments and risks, and answer your questions with retrieval over message patterns and real log lines. A token budget per model and automatic fallback keep it within the free tier.
Redaction First
Passwords, tokens, keys, card numbers and emails are removed and IPs partially masked before any AI call.
Prompt-Injection Safe
Log text is treated as untrusted data. Instructions hidden inside a log are ignored.
Graceful Fallback
If an AI provider is slow or out of quota, a summary is written from the statistics and you can retry later.
System Design
Built to Fail Safely
No Database
Each analysis is a self-contained encrypted folder on a mounted volume. Nothing to migrate, nothing to leak in bulk.
Isolated Workers
Analysis runs in a separate process with a timeout. A crash or runaway file can't take the API down, and the pool restarts itself.
Bounded Queue
A small concurrency limit and queue cap protect the server. When it's busy you get a clear message to retry, not a hang.
Streaming Everywhere
Uploads, decryption and the log explorer all stream in 1 MB chunks, so a 100 MB file never needs to fit in memory.
Idempotent Uploads
Each upload carries an idempotency key, so a double-click or network retry never creates a duplicate analysis.
Crash Recovery
On restart, interrupted jobs are marked clearly so you know to re-upload, and expired data is swept immediately.