google/silifuzzFuzzes CPU implementations by generating test inputs from software proxies, then executes them on real hardware to detect microarchitecture defects and errata.
SiliFuzz is a system that finds CPU defects by fuzzing software proxies, like CPU simulators or disassemblers, and then executing the accumulated test inputs (known as the corpus) on actual CPUs on a large scale. SiliFuzz is a work in progress, please refer to the paper for details.
Fuzzing is a technique of testing a target (an application, or an API) with a large number of test inputs generated on the fly. The goal is to make these inputs as interesting and as diverse as possible in order to trigger corner cases. In other words, fuzzing aims to maximize the combined code coverage. Code coverage may have different meanings, e.g. which basic blocks are executed or which paths are taken in the program.
For the purposes of SiliFuzz, a proxy is any software or hardware system that behaves similar to some aspects of the target CPU. For example, a CPU emulator or a disassembler. A proxy is needed when we cannot directly collect coverage information from the target.
By applying fuzzing techniques to a proxy, we can generate a series of test inputs (corpus) that produce interesting behavior in the proxy. Our underlying assumption is that this translates into similarly interesting behavior in the target. Read more in the docs.
A collection of inputs used for testing the target is known as corpus.
A reasonably large corpus contains millions of inputs and is usually split into multiple non-overlapping chunks called shards.
A SiliFuzz snapshot describes a short sequence of CPU instructions plus an
initial state of CPU registers and memory to deterministically execute that
sequence. A typical snapshot contains less than 100 bytes of code and runs in
microseconds, but it can be arbitrarily large. Snapshots are stored as
silifuzz.proto.Snapshot protocol buffers.
Snapshots are typically created from the inputs generated by a fuzzing engine. For CPU testing purposes these inputs are filtered to eliminate all non-deterministic snapshots. Read more in the docs.
An end state describes the contents of registers and memory expected to exist at the end of a Snapshot execution. If the snapshot executes differently on different CPU microarchitectures it will have multiple expected end states.
A Snap is an in-memory representation of a Snapshot that can be easily loaded and executed by the Runner. Snaps are typically loaded from disk by a reading runner. The Snap on-disk format is essentially the same as the in-memory one except that native pointers are replaced with offsets. See this header for details. This format is often referred to as relocatable. Each Snap contains exactly one expected end state i.e. Snaps are microarchitecture-specific. Read more in the docs.
Runner is a binary for testing a single CPU core. A runner consumes a corpus shard, repeatedly executes random Snaps in it and checks that the expected end state is reached. Runner is a single-threaded process.
The orchestrator is a process that drives multiple runners. In a typical setting the orchestrator will continuously execute one runner per logical CPU core, and accumulate and report any failures produced by the individual runner processes.
See this file for the list of supported microarchitectures.
SiliFuzz runs on x86_64 and aarch64 Linux systems. It has been tested with Linux kernel versions 5.x and 6.x. There is no guarantee that it is compatible with older kernel versions. Legacy vsyscall ABI must be turned off to avoid false positives.
A non-exhaustive list of bugs and defects SiliFuzz has found.
A logic bug is an invalid CPU behavior inherent to a particular CPU microarchitecture or stepping. SiliFuzz has identified the following bugs:
An (electrical) defect is an invalid CPU behavior that happens only on one or several chips. SiliFuzz has found the following defects we described in the paper
git clone https://github.com/google/silifuzz.git && cd silifuzz
SILIFUZZ_SRC_DIR=`pwd`
./install_build_dependencies.sh # Currently, works for the latest Ubuntu only.
bazel build -c opt @silifuzz//tools:{snap_corpus_tool,fuzz_filter_tool,snap_tool,silifuzz_platform_id,simple_fix_tool_main} \
@silifuzz//runner:reading_runner_main_nolibc \
@silifuzz//orchestrator:silifuzz_orchestrator_main
SILIFUZZ_BIN_DIR=`pwd`/bazel-bin
cd "${SILIFUZZ_BIN_DIR}"
NOTE: You can use a Docker container to avoid polluting the host system: docker run -it --tty --security-opt seccomp=unconfined --mount type=bind,source=${SILIFUZZ_SRC_DIR},target=/app ubuntu:noble /bin/bash -c "cd /app && ./install_build_dependencies.sh && bazel build ... && bazel test ..."
For Bazel, use the following commands.
cd "${SILIFUZZ_SRC_DIR}"
COV_FLAGS_FILE="$(bazel info output_base)/external/fuzztest+/centipede/clang-flags.txt"
bazel build -c opt --copt=-UNDEBUG --dynamic_mode=off \
--per_file_copt=unicorn/.*@$(xargs < "${COV_FLAGS_FILE}" |sed -e 's/,/\\,/g' -e 's/ /,/g') @//proxies:unicorn_x86_64
bazel build -c opt @fuzztest//centipede:centipede
mkdir -p /tmp/wd
# Fuzz the Unicorn proxy under Centipede 1000 times with parallelism of 30.
"${SILIFUZZ_BIN_DIR}/external/fuzztest+/centipede/centipede" \
--binary="${SILIFUZZ_BIN_DIR}/proxies/unicorn_x86_64" \
--workdir=/tmp/wd \
-j=30 --num_runs=1000
NOTE: Please refer to Centipede documentation on how to efficiently run the fuzzing engine.
This helper tool is to check that the machine you're running on is supported.
$ ${SILIFUZZ_BIN_DIR}/tools/silifuzz_platform_id --short
intel-skylake
NOTE: SiliFuzz CPU detection logic does not account for certain desktop variants of otherwise supported CPUs. The tool will report "Unsupported platform" in such cases.