SQLancer (Synthesized Query Lancer) is a tool to automatically test Database Management Systems (DBMS) in order to find logic bugs in their implementation. We refer to logic bugs as those bugs that cause the DBMS to fetch an incorrect result set (e.g., by omitting a record).

SQLancer operates in the following two phases:

1. Database generation: The goal of this phase is to create a populated database, and stress the DBMS to increase the probability of causing an inconsistent database state that could be detected subsequently. First, random tables are created. Then, randomly SQL statements are chosen to generate, modify, and delete data. Also other statements, such as those to create indexes as well as views and to set DBMS-specific options are sent to the DBMS.
2. Testing: The goal of this phase is to detect the logic bugs based on the generated database. See Testing Approaches below.

Requirements:

• Java 8 or above
• Maven (sudo apt install maven on Ubuntu)
• The DBMS that you want to test (SQLite is an embedded DBMS and is included)

The following commands clone SQLancer, create a JAR, and start SQLancer to test SQLite using Non-optimizing Reference Engine Construction (NoREC):

git clone https://github.com/sqlancer/sqlancer
cd sqlancer
mvn package -DskipTests
cd target
java -jar sqlancer-*.jar --num-threads 4 sqlite3 --oracle NoREC

If the execution prints progress information every five seconds, then the tool works as expected. Note that SQLancer might find bugs in SQLite. Before reporting these, be sure to check that they can still be reproduced when using the latest development version. The shortcut CTRL+C can be used to terminate SQLancer manually. If SQLancer does not find any bugs, it executes infinitely. The option --num-tries can be used to control after how many bugs SQLancer terminates. Alternatively, the option --timeout-seconds can be used to specify the maximum duration that SQLancer is allowed to run.

If you launch SQLancer without parameters, available options and commands are displayed. Note that general options that are supported by all DBMS-testing implementations (e.g., --num-threads) need to precede the name of DBMS to be tested (e.g., sqlite3). Options that are supported only for specific DBMS (e.g., --test-rtree for SQLite3), or options for which each testing implementation provides different values (e.g. --oracle NoREC) need to go after the DBMS name.

This project should at this stage still be seen as a research prototype. We believe that the tool is not ready to be used. However, we have received many requests by companies, organizations, and individual developers, which is why we decided to prematurely release the tool. Expect errors, incompatibilities, lack of documentation, and insufficient code quality. That being said, we are working hard to address these issues and enhance SQLancer to become a production-quality piece of software. We welcome any issue reports, extension requests, and code contributions.

Please find the .bib entries here.

Since SQL dialects differ widely, each DBMS to be tested requires a separate implementation.

SQLancer stores logs in the target/logs subdirectory. By default, the option --log-each-select is enabled, which results in every SQL statement that is sent to the DBMS being logged. The corresponding file names are postfixed with -cur.log. In addition, if SQLancer detects a logic bug, it creates a file with the extension .log, in which the statements to reproduce the bug are logged.

After finding a bug, it is useful to produce a minimal test case before reporting the bug, to save the DBMS developers' time and effort. For many test cases, C-Reduce does a great job. In addition, we have been working on a SQL-specific reducer, which we plan to release soon.

We would appreciate it if you mention SQLancer when you report bugs found by it. We would also be excited to know if you are using SQLancer to find bugs, or if you have extended it to test another DBMS (also if you do not plan to contribute it to this project). SQLancer has found over 400 bugs in widely-used DBMS, which are listed here.

We have created a Slack workspace to discuss SQLancer, and DBMS testing in general. SQLancer's official Twitter handle is @sqlancer_dbms.

• Contributing to SQLancer
• Papers and .bib entries

Official release are available on:

• GitHub
• Maven Central
• DockerHub

• A talk on Ternary Logic Partitioning (TLP) and SQLancer is available on YouTube.
• An (older) Pivoted Query Synthesis (PQS) talk is available on YouTube.
• PingCAP has implemented PQS, NoREC, and TLP in a tool called go-sqlancer.
• More information on our DBMS testing efforts and the bugs we found is available here.