Merlion: A Machine Learning Library for Time Series

Table of Contents

1. Introduction
2. Comparison with Related Libraries
3. Installation
4. Documentation
5. Getting Started
   1. Anomaly Detection
   2. Forecasting
6. Evaluation and Benchmarking
7. Technical Report and Citing Merlion

Introduction

Merlion is a Python library for time series intelligence. It provides an end-to-end machine learning framework that includes loading and transforming data, building and training models, post-processing model outputs, and evaluating model performance. It supports various time series learning tasks, including forecasting, anomaly detection, and change point detection for both univariate and multivariate time series. This library aims to provide engineers and researchers a one-stop solution to rapidly develop models for their specific time series needs, and benchmark them across multiple time series datasets.

Merlion's key features are

• Standardized and easily extensible data loading & benchmarking for a wide range of forecasting and anomaly detection datasets. This includes transparent support for custom datasets.
• A library of diverse models for anomaly detection, forecasting, and change point detection, all unified under a shared interface. Models include classic statistical methods, tree ensembles, and deep learning approaches. Advanced users may fully configure each model as desired.
• Abstract DefaultDetector and DefaultForecaster models that are efficient, robustly achieve good performance, and provide a starting point for new users.
• AutoML for automated hyperaparameter tuning and model selection.
• Unified API for using a wide range of models to forecast with exogenous regressors.
• Practical, industry-inspired post-processing rules for anomaly detectors that make anomaly scores more interpretable, while also reducing the number of false positives.
• Easy-to-use ensembles that combine the outputs of multiple models to achieve more robust performance.
• Flexible evaluation pipelines that simulate the live deployment & re-training of a model in production, and evaluate performance on both forecasting and anomaly detection.
• Native support for visualizing model predictions, including with a clickable visual UI.
• Distributed computation backend using PySpark, which can be used to serve time series applications at industrial scale.

Comparison with Related Libraries

The table below provides a visual overview of how Merlion's key features compare to other libraries for time series anomaly detection and/or forecasting.

	Merlion	Prophet	Alibi Detect	Kats	darts	statsmodels	nixtla	GluonTS	RRCF	STUMPY	Greykite	pmdarima
Univariate Forecasting	✅	✅		✅	✅	✅	✅	✅			✅	✅
Multivariate Forecasting	✅			✅	✅	✅	✅	✅
Univariate Anomaly Detection	✅	✅	✅	✅	✅				✅	✅	✅	✅
Multivariate Anomaly Detection	✅		✅	✅	✅				✅	✅
Pre Processing	✅		✅	✅	✅		✅	✅			✅	✅
Post Processing	✅		✅
AutoML	✅			✅								✅
Ensembles	✅			✅	✅					✅
Benchmarking	✅				✅	✅	✅				✅
Visualization	✅	✅		✅	✅						✅

The following features are new in Merlion 2.0:

	Merlion	Prophet	Alibi Detect	Kats	darts	statsmodels	nixtla	GluonTS	RRCF	STUMPY	Greykite	pmdarima
Exogenous Regressors	✅	✅			✅	✅					✅	✅
Change Point Detection	✅	✅	✅	✅							✅
Clickable Visual UI	✅
Distributed Backend	✅						✅

Installation

Merlion consists of two sub-repos: merlion implements the library's core time series intelligence features, and ts_datasets provides standardized data loaders for multiple time series datasets. These loaders load time series as pandas.DataFrame s with accompanying metadata.

You can install merlion from PyPI by calling pip install salesforce-merlion. You may install from source by cloning this repoand calling pip install Merlion/, or pip install -e Merlion/ to install in editable mode. You may install additional dependencies via pip install salesforce-merlion[all], or by calling pip install "Merlion/[all]" if installing from source. Individually, the optional dependencies include dashboard for a GUI dashboard, spark for a distributed computation backend with PySpark, and deep-learning for all deep learning models.

To install the data loading package ts_datasets, clone this repo and call pip install -e Merlion/ts_datasets/. This package must be installed in editable mode (i.e. with the -e flag) if you don't want to manually specify the root directory of every dataset when initializing its data loader.

Note the following external dependencies:

1. Some of our forecasting models depend on OpenMP. If using conda, please conda install -c conda-forge lightgbm before installing our package. This will ensure that OpenMP is configured to work with the lightgbm package (one of our dependencies) in your conda environment. If using Mac, please install Homebrew and call brew install libomp so that the OpenMP libary is available for the model.

2. Some of our anomaly detection models depend on the Java Development Kit (JDK). For Ubuntu, call sudo apt-get install openjdk-11-jdk. For Mac OS, install Homebrew and call brew tap adoptopenjdk/openjdk && brew install --cask adoptopenjdk11. Also ensure that java can be found on your PATH, and that the JAVA_HOME environment variable is set.

Documentation

For example code and an introduction to Merlion, see the Jupyter notebooks in examples, and the guided walkthrough here. You may find detailed API documentation (including the example code) here. The technical report outlines Merlion's overall architecture and presents experimental results on time series anomaly detection & forecasting for both univariate and multivariate time series.