Skip to content

Instantly share code, notes, and snippets.

@Som1Lse
Created April 19, 2024 15:50
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Som1Lse/95c2bf99385138451b614d8a94066ed7 to your computer and use it in GitHub Desktop.
Save Som1Lse/95c2bf99385138451b614d8a94066ed7 to your computer and use it in GitHub Desktop.

Fuzzing CDT: Finding, reproducing, and reporting bugs

Introduction

This is a tutorial on how to write a fuzzer for a non-trivial real-world library, namely Artem Amirkhanov's CDT. It is a library for computing Constrained Delaunay Triangulations (CDTs, hence the name of the library). We will be working from the 9d99b32ae56b26cd2781678dc4405c98b8679a9f commit, since that is what I originally wrote the fuzzer for, and that way, we will be able to rediscover the same bugs I found back then.

If you want to follow along, clone the library using

$ git clone https://github.com/artem-ogre/CDT
$ cd CDT
$ git checkout 9d99b32ae56b26cd2781678dc4405c98b8679a9f

This article will cover

  • what makes a library good for fuzzing,
  • what a fuzzer is,
  • how to use a fuzzer,
  • how to understand and simplify its output,
  • and how using a fuzzer can lead to better library design.

Background

This article is not about CDTs, so I will not explain what they are, but having a basic understanding will probably still help. I would recommend skimming the Wikipedia page for Delaunay triangulations, and these articles for DTs and CDTs until you have a rough idea of what it is.

Or, if you like abstraction: Ultimately, all that matters is this: A CDT can be computed from

  • a set of 2D points, called vertices, and
  • a set of line segments between them, called edges, and the library knows how to do it.

The code to do this using the library is

CDT::Triangulation<double> Triangulation = {};
Triangulation.insertVertices({
    {0.0, 0.0},
    {1.0, 0.0},
    {1.0, 1.0},
    {0.5, 0.5},
    {0.0, 1.0},
    {0.2, 0.7},
    {0.8, 0.7},
});
Triangulation.insertEdges({
    {0, 1},
    {1, 2},
    {2, 3},
    {3, 4},
    {4, 0},
});

An important thing to note is despite using floating point numbers, the library uses exact predicates to make decisions. This means errors cannot simply be chalked up to floating-point imprecision. It also detects edge intersections and handles them accordingly. Without this clear line between what is and what isn't allowed fuzzing becomes impossible, as well as just relying on the library in general.

What in the heck is a fuzzer?

A fuzzer is a monkey typing on a typewriter, that gets a banana whenever it happens to type something interesting. It then remembers what it wrote and tries to write something similar, using the previous success as a starting point, in hopes of getting another banana. It is more complicated than that but this is enough for now.

The idea is, that if it happens to type something that makes our program crash, we've found a bug and a test case that triggers it. We can then begin to debug and fix it. If, on the other hand, the fuzzer runs for a long time without finding a bug, we can be more confident that our code works.

Our job is to turn the monkey's typings into input the library can use. You might have thought "Ahh, it's reinforcement learning, a fuzzer is an AI!" when reading the above paragraph, but that is completely wrong. While it sounds like reinforcement learning, the fuzzer never actually learns anything, it is just using different starting points. This means the data we get will be very random, so to make good use of the fuzzer we should strive to make valid input as likely as possible.

CDT has a test suite (in the tests-folder) with a fairly simple text-based format:

  1. Two integers, n and m, representing the number of vertices and the number of edges.
  2. n vertices, represented by two floating point values.
  3. m edges, represented by indices into the array of vertices.

We could simply copy the parsing code from the test suite directly into the fuzzer, add some input validation, and be done with it. Unfortunately, what makes a format good for testing does not necessarily make it good for fuzzing: The above format is easy to read for humans who are used to text, but the monkey is a digital monkey. It just spits out random bytes. Sometimes it would happen to write text, and the above would work, but it would be slow, and less likely to fully explore the program and uncover bugs.

We should use a binary format instead. Binary formats also tend to be easier to validate, since we don't have to worry about numbers being too large to fit in our types, or containing invalid characters, so it is a win all around.

A natural way to fix this would simply be to translate the above into a binary format:

  1. Treat the first eight bytes as two integers n and m.1
  2. Read n vertices (float pairs).
  3. Validate each vertex. (Not infinite or NaN.)
  4. Read m edges (index pairs).
  5. Validate each edge. (Both indices are < n, and are different.)

This is fairly intuitive to us, but remember that the fuzzer is incredibly stupid, and we should do everything we can to make things easier for it.

First of all, m is completely unnecessary: We can just treat the rest of the input as edges. Second, the fuzzer doesn't know that the number n should be followed by n points. It is not beyond a fuzzer to figure out, but it is generally better to use a sentinel value if we can, and we have an easy solution: Simply stop at the first invalid point, and treat the rest as edges.

This is implemented in the following parser:

bool parse_input(
    simple_stream Stream,
    std::vector<CDT::V2d<double>>* Vertices,
    std::vector<CDT::Edge>* Edges
){
    for(float Vert[2]; Stream.read(&Vert) && std::isfinite(Vert[0]);){
        if(!std::isfinite(Vert[1])){
            return false;
        }

        Vertices->push_back({Vert[0], Vert[1]});
    }

    for(std::uint32_t Edge[2]; Stream.read(&Edge);){
        if(Edge[0] == Edge[1] || Edge[0] >= Vertices->size() || Edge[1] >= Vertices->size()){
            return false;
        }

        Edges->push_back(Edge[0], Edge[1]);
    }

    return true;
}

I've used a utility class simple_stream to simplify parsing the input. Its implementation can be found in Appendix A. It returns false whenever it encounters invalid input. This will become relevant later.

Now that we can parse input, we just need a way to get it. Today, most fuzz engines have standardised on a simple interface, namely the LLVMFuzzerTestOneInput function:

extern "C" int LLVMFuzzerTestOneInput(const void* Data, std::size_t Size);

It is a function we are supposed to provide, which the fuzzer will then call with whatever data it generates. We can use the return value to indicate whether to discard the input by returning -1. Since we already have a way to parse the buffer, the rest is fairly trivial: We simply call the parse_input function, return -1 if it fails, and then pass along the result to the library in the same way as the example code:

extern "C" int LLVMFuzzerTestOneInput(const void* Data, std::size_t Size) {
    std::vector<CDT::V2d<double>> Vertices = {};
    std::vector<CDT::Edge> Edges = {};

    if(!parse_input(simple_stream(Data, Size), &Vertices, &Edges)){
        return -1;
    }

    CDT::Triangulation<double> Triangulation = {};
    Triangulation.insertVertices(Vertices);
    Triangulation.insertEdges(Edges);

    return 0;
}

It is actually that simple. The above is a fully working fuzzer when compiled with the right flags. To start with, we will be using libFuzzer, which is supported in both Clang and MSVC, so if have either, simply compile the program with the -fsanitize=fuzzer flag and you should be set:2

clang++ fuzz1.cpp -g -fsanitize=fuzzer,address -ICDT/CDT/include -o fuzz1

We are also compiling with -g to improve the reports we get, and -fsanitize=address to catch bugs sooner.3

Running the fuzzer

If you run it, you will get some output on the console, (probably) followed by the program freezing up. The reason this happens is the fuzzer found an input that caused the program to freeze. Success! We can provide a timeout (in seconds) to the fuzzer using the -timeout parameter, which will cause it to give up after that time has elapsed and treat it as a crash. Another thing I will do is provide a -seed value. It is entirely optional, but this way, if you are following along you will get the same result as me:4

$ ./fuzz1 -timeout=2 -seed=167
[...]
artifact_prefix='./'; Test unit written to ./timeout-5e87565de85cf3abd59c018e57b81d266ad7cac0
[...]
SUMMARY: libFuzzer: timeout

We can open timeout-5e87565de85cf3abd59c018e57b81d266ad7cac0 in a hex editor and we see the following bytes, here grouped into 32-bit words to increase readability:5

55555555 ADADADAD 55555555 ADAD5555

This corresponds to the points (1.4660155E+13, -1.974495E-11) and (1.4660155E+13, -1.9705718E-11). Notice the x-coordinates are the same.

This is certainly interesting and worth reporting, but it is not particularly juicy: It is a small case that is easy to detect. Unfortunately, as long as it exists the fuzzer will keep reporting it, and never find anything else. We have a few options here:

  1. Report the bug and wait for it to be fixed, then run the fuzzer again and hope to find something else.
  2. Fix the bug.
  3. Reject any input that might trigger it.

I find the first option to be somewhat disrespectful to the author, as it would result in a bunch of small bug reports one after the other, so I will not do that. The second option is nice, and if you are the maintainer, you should definitely go for that one, however, since I am not familiar with the code, I will choose option 3.

I mentioned earlier that the x-coordinates were the same. Let's update parse_input so it verifies that the bounding box is actually a box:

float x0, x1, y0, y1;
x0 = y0 = std::numeric_limits<float>::max();
x1 = y1 = -x0;

for(float Vert[2]; Stream.read(&Vert) && std::isfinite(Vert[0]);){
    if(!std::isfinite(Vert[1])){
        return false;
    }

    x0 = std::min(x0, Vert[0]);
    y0 = std::min(y0, Vert[1]);
    x1 = std::max(x1, Vert[0]);
    y1 = std::max(y1, Vert[1]);

    Vertices->push_back({Vert[0], Vert[1]});
}

if(x0 >= x1 || y0 >= y1){
    return false;
}
// [...]

Running that we get the following timeout:

0A130A0A 0A3CE6E6 0A130A0A 0A3CE6E6
B4E6E6E6 69193B19

Notice that 0A130A0A 0A3CE6E6 appears twice? That's two points at (7.0796807E-33, 9.0952979E-33). Again, might be worth reporting, but still not very juicy, since it contains a duplicate point. Again, we can augment parse_input to detect it:

// [...]
for(float Vert[2]; Stream.read(&Vert) && std::isfinite(Vert[0]);){
    /// [...]

    for(auto& p:*Vertices){
        if(p.x == Vert[0] && p.y == Vert[1]){
            return false;
        }
    }

    Vertices->push_back({Vert[0], Vert[1]});
}
// [...]

Running it again, we get the following report:

$ ./fuzz3 -timeout=2 -seed=167
[...]
fuzz3: CDT/CDT/include/CDTUtils.hpp:188: Index CDT::edgeNeighborInd(const VerticesArr3 &, const VertInd, const VertInd): Assertion `vv[0] == iVedge1 || vv[1] == iVedge1 || vv[2] == iVedge1' failed.
[...]
artifact_prefix='./'; Test unit written to ./crash-47de6e7388835f9f18717a7b20711b8a8a81c69d
[...]
SUMMARY: libFuzzer: deadly signal

This is more interesting. We've hit an assertion failure; some invariant inside the library was broken, so it is definitely a bug.

This incidentally, is one of the reasons asserts are incredibly useful, and why any code you write should be chock full of them. Not only do they help document assumptions to readers of your code. They also help catch bugs when testing, especially when tested with randomised inputs like in a fuzzer, and make the code crash early.

Analysing a crash

The crash

EAEA3BC4 3BEAEAEA 19FF3B3B 00000005
19000319 0A191919 00000019 1E0A1900
1996D419 3B969696 BCBCBCBC BCBCBCBC
BCBCBCBC 00000000 13000000 81000000
00000000 19190000 01010101 01010101
01010101 01070001 B9B90101 011919B9
00000000 190A0000 00C10000 10000000
FFFFC330 010101F9 00000001 00000000

is a lot longer than the previous ones, and simply looking at the bytes or using a hex editor is no longer feasible. We have to be smarter.

The first thing to note is this is simply whatever the fuzzer happened to stumble upon. It might be significantly longer than necessary. Going back to the monkey metaphor, it started with nothing and got a banana whenever it typed out something interesting, specifically whenever it triggered a new code path. Instead, we could start the monkey off with a crash, and give it a banana whenever it manages to type something smaller, that still crashes. This is called a minimizer and is built into most fuzz engines, including libFuzzer. We can do this by passing the -minimize_crash=1 flag, along with the name of the crash:

$ ./fuzz3 -timeout=2 -seed=167 -minimize_crash=1 ./crash-47de6e7388835f9f18717a7b20711b8a8a81c69d
[...]
CRASH_MIN: './crash-47de6e7388835f9f18717a7b20711b8a8a81c69d' (134 bytes) caused a crash. Will try to minimize it further
[...]
CRASH_MIN: './minimized-from-47de6e7388835f9f18717a7b20711b8a8a81c69d' (126 bytes) caused a crash. Will try to minimize it further
[...]
CRASH_MIN: './minimized-from-cf8190c2b24580da4a8747293ff9427fcd43c618' (121 bytes) caused a crash. Will try to minimize it further
[...]
CRASH_MIN: './minimized-from-97b3cf12f3768543513c3893ccd85adbf6b4838c' (97 bytes) caused a crash. Will try to minimize it further
[...]
CRASH_MIN: './minimized-from-2a6d7b3ec56986a915e9b34294ca05bce9b67dfd' (89 bytes) caused a crash. Will try to minimize it further
[...]
CRASH_MIN: './minimized-from-8c6e4080e3b69d4a2a7c72bc30fffd4c2f98ff1e' (81 bytes) caused a crash. Will try to minimize it further
[...]
CRASH_MIN: './minimized-from-9cb09e7232de109043653baf69259a49d2783122' (80 bytes) caused a crash. Will try to minimize it further
[...]
CRASH_MIN: failed to minimize beyond ./minimized-from-9cb09e7232de109043653baf69259a49d2783122 (80 bytes), exiting

Opening minimized-from-3751e2c20ef8c6522377e93bab87aa85e929e692 we see it is a good deal smaller:

EAEA3BC4 3BEAEAEA 19FF3B3B 00000003
13000319 81000000 01010100 01010101
01010101 01070001 B9B90101 011919B9
00000000 190A0000 00C10000 10000000
FFFFC330 010101F9 00000001 00000000

It is still not easy to see what is going on. This is where using a text-based format would have been useful, as it would be much easier for us to read. Fortunately, we are programmers: We can write a tool that converts the binary format used by the fuzzer to the text-based format used by the tests. We already have the parsing code, so we just need to put it in a header file and write a tool that translates it to text. This is fairly easy to do:

int main(int Argc, char** Argv){
    std::vector<char> Buffer = {};

    using stream_iterator = std::istreambuf_iterator<char>;

    for(int i = 1; i < Argc; ++i){
        std::ifstream File(Argv[i], std::ios::binary);
        if(File){
            if(i != 1){
                std::printf("----------------------------------------\n");
            }

            Buffer.assign(stream_iterator(File), stream_iterator());

            std::vector<CDT::V2d<double>> Vertices;
            std::vector<CDT::Edge> Edges;
            if(parse_input(simple_stream(Buffer.data(), Buffer.size()), &Vertices, &Edges)){
                std::printf("%zu %zu\n", Vertices.size(), Edges.size());
                for(auto Vert:Vertices){
                    std::printf("%.8G %.8G\n", Vert.x, Vert.y);
                }

                for(auto Edge:Edges){
                    std::printf("%u %u\n", Edge.v1(), Edge.v2());
                }
            }
        }
    }
}

We simply loop through all files passed on the command line, read them in the most naive way possible using a std::istreambuf_iterator, parse them using parse_input,6 then print them using std::printf. I insert a separator between outputs if there are multiple files given. I also wrote a similar tool that transforms the crashes into code we can compile and execute. It differs only in the printing code:

if(parse_input(simple_stream(Buffer.data(), Buffer.size()), &Vertices, &Edges)){
    std::printf("CDT::Triangulation<double> Triangulation = {};\n");

    if(!Vertices.empty()){
        std::printf("Triangulation.insertVertices({\n");

        for(auto Vert:Vertices){
            std::printf("    {%.8G, %.8G},\n", Vert.x, Vert.y);
        }

        std::printf("});\n");
    }

    if(!Edges.empty()){
        std::printf("Triangulation.insertEdges({\n");

        for(auto Edge:Edges){
            std::printf("    {%u, %u},\n", Edge.v1(), Edge.v2());
        }

        std::printf("});\n");
    }
}

Converting the minimized crash to code we get the following repro (code that it reproduces a bug).

CDT::Triangulation<double> Triangulation = {};
Triangulation.insertVertices({
    {-1.4158544E+26, 0.0071691172},
    {2.6390305E-23, 4.2038954E-45},
    {1.6157399E-27, -2.3509887E-38},
    {2.3694275E-38, 2.3694278E-38},
    {2.3694278E-38, 2.4795587E-38},
    {-0.00035286698, 2.8120117E-38},
    {0, 7.1344328E-24},
    {1.7724251E-38, 2.5243549E-29},
});
Triangulation.insertEdges({
    {0, 1},
});

We can wrap that in a main function and run it, and we do indeed get a crash. We could simplify the code further by hand; trying to remove points, make the coordinates smaller, etc. Sometimes that is necessary, but there is an alternative: We control the fuzzer. We can force it to only accept points that lie within a certain range until we have a simpler crash, and then simplify that further manually.

We define a function that checks whether a value is "simple" enough to accept. We accept zero, and any number with a magnitude between 0.01 and 100:

bool issimple(float x){
    return x == 0.0f || (1E-2f <= std::abs(x) && std::abs(x) < 1E2f);
}

Then we check it inside the parser:

// [...]
if(!std::isfinite(Vert[1]) || !issimple(Vert[0]) || !issimple(Vert[1])){
    return false;
}
// [...]

It is worth noting this is only a temporary hack. It will reject a lot of inputs that could potentially result in a crash, especially ones with very large or very small numbers, so this code should not remain in the fuzzer. It is only helpful because we know there's a bug, and we're trying to force the fuzzer to generate a simpler crash.

When running the fuzzer it takes significantly longer than before, but it does eventually find a crash. After minimising and translating we get the following short repro:

CDT::Triangulation<double> Triangulation = {};
Triangulation.insertVertices({
    {0, 0},
    {0.12890625, 0.18572709},
    {0.37938875, 0.37938112},
    {0.18578431, 0.18578431},
    {0.37547487, 0.37536043},
    {0.18578812, 0.18649384},
    {0.16703048, 0.18578485},
    {0.18624496, 0},
});
Triangulation.insertEdges({
    {0, 2},
});

That is pretty good: None of the numbers are particularly large or small. We could simplify it further by hand, but, honestly, it's perfectly fine as it is. This is good enough to report, which is exactly what I did. The author was interested in fuzzing, which is part of why I am writing this article, though I sure have taken my sweet time.

Following up

After the bugs have been fixed, we can run the fuzzer again, trying to find new bugs. Let's check out the commit after the bugs were fixed

$ git checkout 6ec70ed46a4f1392b41cd6e9ff73ddcbbef638d3
Previous HEAD position was 9d99b32 Fix typo
HEAD is now at 6ec70ed Thank you, Som1Lse!🙇

and — oh look, that's me. Anyway, running the fuzzer we get another crash. Simplifying it gets us down to the following repro:

CDT::Triangulation<double> Triangulation = {};
Triangulation.insertVertices({
    {-97.378876, -47.561287},
    {0.125, 0.18578431},
    {0.18676087, 0.18726638},
    {0.125, 0.18554746},
    {0.13480471, 0.18554746},
    {0.12514207, 0.18578431},
    {0.12500092, 0.18578394},
    {0.18554989, 0.18578431},
    {0.26414675, 0.18285516},
    {0, -10.702696},
    {0.71374136, 0.1857691},
    {0.18578431, 0.1862793},
    {0.18969236, 0.18554688},
});
Triangulation.insertEdges({
    {1, 10},
    {0, 2},
});

Notice there are two edges. If we graphed the points we'd see they're crossing. We could report this too, were it not for the fact that it has already been fixed in the very next commit with the following commit message:

Improve error handling in CDT

  • When triangulating detect duplicate vertices and throw an exception
  • Default mode: don't allow intersecting constraints. Check for intersecting constraints and throw if detected.
    • Possible to attempt to resolve intersections with IntersectingConstraintEdges::TryResolve
    • Possible to skip the checks with IntersectingConstraintEdges::DontCheck
  • Introduce custom ergonomic exceptions for errors in triangulation

Even fixed the duplicate vertex bug. We can simplify our fuzzer by removing the extra validation checks, and just add a simple try-catch-block around the calls to insertVertices and insertEdges. This is the final version of the code:

bool parse_input(
    simple_stream Stream,
    std::vector<CDT::V2d<double>>* Vertices,
    std::vector<CDT::Edge>* Edges
){
    for(float Vert[2]; Stream.read(&Vert) && std::isfinite(Vert[0]);){
        if(!std::isfinite(Vert[1])){
            return false;
        }

        Vertices->push_back({Vert[0], Vert[1]});
    }

    for(std::uint32_t Edge[2]; Stream.read(&Edge);){
        if(Edge[0] >= Vertices->size() || Edge[1] >= Vertices->size()){
            return false;
        }

        Edges->push_back({Edge[0], Edge[1]});
    }

    return true;
}

extern "C" int LLVMFuzzerTestOneInput(const void* Data, std::size_t Size) {
    std::vector<CDT::V2d<double>> Vertices = {};
    std::vector<CDT::Edge> Edges = {};

    if(!parse_input(simple_stream(Data, Size), &Vertices, &Edges)){
        return -1;
    }

    CDT::Triangulation<double> Triangulation = {};

    try {
        Triangulation.insertVertices(Vertices);
        Triangulation.insertEdges(Edges);
    }catch(...){}

    return 0;
}

That's a lot simpler. Running it we get a timeout with a single, vertex. This was actually introduced by the fix to the very first bug we found. Anyway, it is fixed in the commit immediately after again. I don't know if the author used fuzzing to find the bugs, but it seems plausible. Either way, it highlights a virtuous cycle with fuzzing: You run the fuzzer, it finds a bug. You fix the bug, run the fuzzer again, it finds a new bug. You fix that, rinse and repeat in a virtuous cycle of bug squashing.

I've run the fuzzer above against the current master commit, and I am happy to report that it has not yet found anything.

Comparison with unit testing

Let's take a moment to compare fuzzing with unit testing. Does fuzzing supplant unit testing? What are their strengths and weaknesses?

An obvious difference is unit tests should generally be readable, whereas we specifically designed the format to be fast for a machine to parse, at the expense of human readability. This means we had to write extra tools to make it easier for us to understand it. Fuzzers also need to turn bytes into usable data for the library, and, unless you are testing a parser, this is likely to involve a certain irreducible complexity. Unit tests can just embed the data directly in the code, which can make them significantly simpler.

An important part of unit tests, is they don't just serve to test your code, they also document how to use it. The same is true for fuzzers: They serve as examples of how to pass arbitrary data to the library, as well as how to validate its preconditions are satisfied.

Another point of unit tests is they serve as experience using the library: If writing tests is tedious, it is usually a sign that the API is bad. This is also true, in fact even more so, for fuzzers: Not only do they have to call the library. They also have to validate the input, which gives clear feedback to the library author on whether satisfying the library's preconditions is feasible. We directly saw this, as the fuzzer could be significantly simplified after the library was changed to throw exceptions.

An upside of unit tests is they can check the output matches an expected value. This is not possible for fuzzers, since the input is random. At best, we can validate certain postconditions are satisfied, but we cannot know the exact expected value.

So no, fuzzing does not supplant unit tests, at least not fully. They do however excel in many of the same areas, and I would personally recommend starting to fuzz before you start writing tests, and then focusing on writing particularly valuable tests, instead of testing every nook and cranny. A good tip is crashes found when fuzzing make good unit (regression) tests. I personally refer to this approach as fuzz-driven development.

Conclusion

Fuzzing is a useful tool for code testing and can complement unit tests nicely. It is important to have a clearly documented line between what is and what isn't acceptable behaviour of your code, and ideally enforce it in your code via assertions.

We've seen how to write a fast fuzzer, how to run it, how to interpret and simplify its results, and finally how that can improve the library both by fixing bugs, as well as improving its API, to make it easier to use.

My hope is this will encourage people to start using fuzzers if they aren't already.


1: Note we are using fixed-size integers and not std::size_t. This is deliberate, as it makes the format more portable: We can run the input on a 32-bit system and a 64-bit system and get the same result. If you want you could also handle endianness though this article won't.

2: Note that libFuzzer does not work with incremental linking in MSVC, which is on by default for debug builds in CMake. This can be fixed by passing -DCMAKE_EXE_LINKER_FLAGS_DEBUG="/debug /INCREMENTAL:NO" to CMake on the command line. Similarly, I recommend building with the release runtime, since the debug runtime shows a dialogue when std::abort is called. On CMake 3.15+ this can be achieved with -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded.

3: Generally I would also recommend specifying -fsanitize=undefined to enable UBSan and -fno-sanitize-recover=all to make UBSan actually crash the program (which is how the fuzzer knows there's a bug), as well as -O3 to make the monkey type faster (okay, it turns on optimisations, but it has the same effect). The reason I haven't done so here is I got inconsistent results between runs, which is inconvenient when writing an article. Normally that's not an issue, so I would recommend turning them on. Generally -g -O3 -fsanitize=fuzzer,address,undefined -fno-sanitize-recover=all should be your go-to options for fuzzing.

4: Related to the previous note, using different compilers, compiler options, and probably different compiler versions, can change the results you get. For reference, I used "Ubuntu clang version 16.0.6 (15)" running inside a Docker container. If you want to follow along exactly, try to replicate that setup as closely as possible. That said, even if you don't you will probably still get similar results, even if they're not bit for bit identical.

5: The actual result contains extra bytes at the end, which are simply ignored by our parser. I will not include them in this article since they are irrelevant.

6: It would make sense to split parsing and validation into two separate functions, so the translation tools can still parse old test cases that would be rejected by the fuzzer.

Appendix A: Simple stream

class simple_stream {
    const unsigned char* Buffer;
    std::size_t Size;

public:
    explicit simple_stream(const void* Buffer, std::size_t Size)
        :Buffer(static_cast<const unsigned char*>(Buffer)), Size(Size) {}

    template <typename T>
    bool read(T* t){
        static_assert(std::is_trivial<T>::value);

        if(Size < sizeof(T)){
            return false;
        }else{
            std::memcpy(t, Buffer, sizeof(T));
            Buffer += sizeof(T);
            Size -= sizeof(T);

            return true;
        }
    }
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment