
A machine learning toolkit for log parsing [ICSE'19, DSN'16]
Logparser provides a machine learning toolkit and benchmarks for automated log parsing, which is a crucial step for structured log analytics. By applying logparser, users can automatically extract event templates from unstructured logs and convert raw log messages into a sequence of structured events. The process of log parsing is also known as message template extraction, log key extraction, or log message clustering in the literature.

An example of log parsing
pip install logparser3.| Publication | Parser | Paper Title | Benchmark |
|---|---|---|---|
| IPOM'03 | SLCT | A Data Clustering Algorithm for Mining Patterns from Event Logs, by Risto Vaarandi. | ↗️ |
| QSIC'08 | AEL | Abstracting Execution Logs to Execution Events for Enterprise Applications, by Zhen Ming Jiang, Ahmed E. Hassan, Parminder Flora, Gilbert Hamann. | ↗️ |
| KDD'09 | IPLoM | Clustering Event Logs Using Iterative Partitioning, by Adetokunbo Makanju, A. Nur Zincir-Heywood, Evangelos E. Milios. | ↗️ |
| ICDM'09 | LKE | Execution Anomaly Detection in Distributed Systems through Unstructured Log Analysis, by Qiang Fu, Jian-Guang Lou, Yi Wang, Jiang Li. [Microsoft] | ↗️ |
| MSR'10 | LFA | Abstracting Log Lines to Log Event Types for Mining Software System Logs, by Meiyappan Nagappan, Mladen A. Vouk. | ↗️ |
| CIKM'11 | LogSig | LogSig: Generating System Events from Raw Textual Logs, by Liang Tang, Tao Li, Chang-Shing Perng. | ↗️ |
| SCC'13 | SHISO | Incremental Mining of System Log Format, by Masayoshi Mizutani. | ↗️ |