test-fuzz

test-fuzz is a Cargo subcommand and a collection of Rust macros to automate certain tasks related to fuzzing with [afl.rs], including:

• generating a fuzzing corpus
• implementing a fuzzing harness

test-fuzz accomplishes these (in part) using Rust's testing facilities. For example, to generate a fuzzing corpus, test-fuzz records a target's arguments each time it is called during an invocation of cargo test. Similarly, test-fuzz implements a fuzzing harness as an additional test in a cargo-test-generated binary. This tight integration with Rust's testing facilities is what motivates the name test-fuzz.

Contents

1. [Installation]
2. [Overview]
3. [Components]
   • [test_fuzz macro]
   • [test_fuzz_impl macro]
   • [cargo test-fuzz command]
   • [Convenience functions and macros]
4. [test-fuzz package features]
5. [Auto-generated corpus files]
6. [Environment variables]
7. [Limitations]
8. [Tips and tricks]
9. [Semantic versioning policy]
10. [License]

Installation

Install cargo-test-fuzz and [afl.rs] with the following command:

cargo install cargo-test-fuzz cargo-afl

Overview

Fuzzing with test-fuzz is essentially three steps:*

1. Identify a fuzz target:
   • Add the following dependencies to the target crate's Cargo.toml file:

     serde = "*"
     test-fuzz = "*"
     

   • Precede the target function with the [test_fuzz] macro:

     #[test_fuzz::test_fuzz]
     fn foo(...) {
         ...
     }
     

2. Generate a corpus by running cargo test:

   cargo test
   

3. Fuzz your target by running [cargo test-fuzz]:

   cargo test-fuzz foo
   

* An additional, preliminary step may be necessary following a reboot:

cargo afl system-config

Note that the above command runs sudo internally. Hence, you may be prompted to enter your password.

Components

test_fuzz macro

Preceding a function with the test_fuzz macro indicates that the function is a fuzz target.

The primary effects of the test_fuzz macro are:

• Add instrumentation to the target to serialize its arguments and write them to a corpus file each time the target is called. The instrumentation is guarded by #[cfg(test)] so that corpus files are generated only when running tests (however, see [enable_in_production] below).
• Add a test to read and deserialize arguments from standard input and apply the target to them. The test checks an environment variable, set by [cargo test-fuzz], so that the test does not block trying to read from standard input during a normal invocation of cargo test. The test is enclosed in a module to reduce the likelihood of a name collision. Currently, the name of the module is target_fuzz, where target is the name of the target (however, see [rename] below).

Arguments

bounds = "where_predicates"

Impose where_predicates (e.g., trait bounds) on the struct used to serialize/deserialize arguments. This may be necessary, e.g., if a target's argument type is an associated type. For an example, see [associated_type.rs] in this repository.

generic_args = "parameters"

Use parameters as the target's type parameters when fuzzing. Example:

#[test_fuzz(generic_args = "String")]
fn foo<T: Clone + Debug + Serialize>(x: &T) {
    ...
}

Note: The target's arguments must be serializable for every instantiation of its type parameters. But the target's arguments are required to be deserializable only when the target is instantiated with parameters.

impl_generic_args = "parameters"

Use parameters as the target's Self type parameters when fuzzing. Example:

#[test_fuzz_impl]
impl<T: Clone + Debug + Serialize> for Foo {
    #[test_fuzz(impl_generic_args = "String")]
    fn bar(&self, x: &T) {
        ...
    }
}

Note: The target's arguments must be serializable for every instantiation of its Self type parameters. But the target's arguments are required to be deserializable only when the target's Self is instantiated with parameters.

convert = "X, Y"

When serializing the target's arguments, convert values of type X to type Y using Y's implementation of From<X>, or of type &X to type Y using Y's implementation of the non-standard trait test_fuzz::FromRef<X>. When deserializing, convert those values back to type X using Y's implementation of the non-standard trait test_fuzz::Into<X>.

That is, use of convert = "X, Y" must be accompanied by certain implementations. If X implements [Clone], then Y may implement the following:

impl From<X> for Y {
    fn from(x: X) -> Self {
        ...
    }
}

If X does not implement [Clone], then Y must implement the following:

impl test_fuzz::FromRef<X> for Y {
    fn from_ref(x: &X) -> Self {
        ...
    }
}

Additionally, Y must implement the following (regardless of whether X implements [Clone]):

impl test_fuzz::Into<X> for Y {
    fn into(self) -> X {
        ...
    }
}

The definition of test_fuzz::Into is identical to that of [std::convert::Into]. The reason for using a non-standard trait is to avoid conflicts that could arise from blanket implementations of standard traits.

enable_in_production

Generate corpus files when not running tests, provided the environment variable [TEST_FUZZ_WRITE] is set. The default is to generate corpus files only when running tests, regardless of whether [TEST_FUZZ_WRITE] is set. When running a target from outside its package directory, set [TEST_FUZZ_MANIFEST_PATH] to the path of the package's Cargo.toml file.

WARNING: Setting enable_in_production could introduce a denial-of-service vector. For example, setting this option for a function that is called many times with different arguments could fill up the disk. The check of [TEST_FUZZ_WRITE] is meant to provide some defense against this possibility. Nonetheless, consider this option carefully before using it.

execute_with = "function"

Rather than call the target directly:

• construct a closure of type FnOnce() -> R, where R is the target's return type, so that calling the closure calls the target;
• call function with the closure.

Calling the target in this way allows function to set up the call's environment. This can be useful, e.g., for fuzzing [Substrate externalities].

no_auto_generate

Do not try to [auto-generate corpus files] for the target.

only_generic_args

Record the target's generic args when running tests, but do not generate corpus files and do not implement a fuzzing harness. This can be useful when the target is a generic function, but it is unclear what type parameters should be used for fuzzing.

The intended workflow is: enable only_generic_args, then run cargo test followed by cargo test-fuzz --display generic-args. One of the resulting generic args might be usable as generic_args's parameters. Similarly, generic args resulting from cargo test-fuzz --display impl-generic-args might be usable as impl_generic_args's parameters.

Note, however, that just because a target was called with certain parameters during tests, it does not imply the target's arguments are serializable/deserializable when those parameters are used. The results of --display generic-args/--display impl-generic-args are merely suggestive.

rename = "name"

Treat the target as though its name is name when adding a module to the enclosing scope. Expansion of the test_fuzz macro adds a module definition to the enclosing scope. By default, the module is named as follows: