diff options
| author | Abhishek Arya <inferno@chromium.org> | 2019-08-07 07:37:16 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2019-08-07 07:37:16 -0700 |
| commit | cf4af869728ac4fc3b136695241882333cf01055 (patch) | |
| tree | 81abbd70fae7a3eb10e113a3992e539db8d5215a /docs | |
| parent | c070f7fc7dbaede2e103e5d2cd79d7a2b7eb6255 (diff) | |
| download | oss-fuzz-cf4af869728ac4fc3b136695241882333cf01055.tar.gz | |
Switch docs to new structure (#2663)
Diffstat (limited to 'docs')
26 files changed, 808 insertions, 408 deletions
diff --git a/docs/advanced-topics/advanced_topics.md b/docs/advanced-topics/advanced_topics.md new file mode 100644 index 000000000..f6ea3d3a3 --- /dev/null +++ b/docs/advanced-topics/advanced_topics.md @@ -0,0 +1,9 @@ +--- +layout: default +title: Advanced topics +has_children: true +nav_order: 3 +permalink: /advanced-topics/ +--- + +# Advanced topics diff --git a/docs/code_coverage.md b/docs/advanced-topics/code_coverage.md index db02de6ef..e67f6f05e 100644 --- a/docs/code_coverage.md +++ b/docs/advanced-topics/code_coverage.md @@ -1,8 +1,20 @@ +--- +layout: default +title: Code coverage +parent: Advanced topics +nav_order: 2 +permalink: /advanced-topics/code-coverage +--- + # Code Coverage You can generate code coverage report for your project using [Clang Source-based Code Coverage]. +- TOC +{:toc} +--- + ## Pull the latest Docker images @@ -107,7 +119,6 @@ reports for your project, add the arguments into `project.yaml` file as follows: coverage_extra_args: -ignore-filename-regex=.*crc.* -ignore-filename-regex=.*adler.* <other_extra_args> ``` - [Clang Source-based Code Coverage]: https://clang.llvm.org/docs/SourceBasedCodeCoverage.html [gsutil tool]: https://cloud.google.com/storage/docs/gsutil_install [llvm-cov tool]: https://llvm.org/docs/CommandGuide/llvm-cov.html diff --git a/docs/advanced-topics/corpora.md b/docs/advanced-topics/corpora.md new file mode 100644 index 000000000..8bb48f218 --- /dev/null +++ b/docs/advanced-topics/corpora.md @@ -0,0 +1,67 @@ +--- +layout: default +title: Corpora +parent: Advanced topics +nav_order: 3 +permalink: /advanced-topics/corpora +--- + +# Accessing Corpora + +If you would like to access the corpora that we are using for your fuzz targets +(synthesized by the fuzzing engines), please follow these steps. + +- TOC +{:toc} +--- + +## Obtain access + +In order to get access to a project's corpora, you must first be listed as the +primary contact or an auto cc in the project's `project.yaml` file as described +in the [New Project Guide]({{ site.baseurl }}/getting-started/new-project-guide/#projectyaml). +If you don't do this, most of the links below won't work. + +## Install Google Cloud SDK + +The corpora for fuzz targets are stored on +[Google Cloud Storage](https://cloud.google.com/storage/). To access them, you +will need to [install](https://cloud.google.com/storage/docs/gsutil_install) the +gsutil tool, which is part of the Google Cloud SDK. +Follow the instructions on the installation page to login with a Google account +listed in your project's `project.yaml` file. + +## Viewing the corpus for a fuzz target + +The fuzzer statistics page for your project on +[ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) +will contain a link to the Google Cloud console for your corpus under the +"corpus_size" column. You can browse and download individual test inputs in the +corpus here. + + + +## Downloading the corpus + +If you would like to download the entire corpus, from the cloud console link, +copy the bucket path highlighted here: + + + +And then run the following command to copy the corpus to a directory on your +machine. + +```bash +gsutil -m rsync gs://<bucket_path> <local_directory> +``` +Following the expat example above, this would be: + +```bash +gsutil -m rsync gs://expat-corpus.clusterfuzz-external.appspot.com/libFuzzer/expat_parse_fuzzer <local_directory> +``` + +## Corpus backups + +We also keep daily zipped backups of your corpora. These can be accessed from +the `corpus_backup` column of the fuzzer statistics page. Downloading these can +also be significantly faster than `gsutil -m rsync` on the corpus bucket. diff --git a/docs/debugging.md b/docs/advanced-topics/debugging.md index 25fdb516d..7ceba5eef 100644 --- a/docs/debugging.md +++ b/docs/advanced-topics/debugging.md @@ -1,4 +1,18 @@ -## Debugging Build Scripts +--- +layout: default +title: Debugging +parent: Advanced topics +nav_order: 4 +permalink: /advanced-topics/debugging +--- + +# Debugging issues + +- TOC +{:toc} +--- + +## Debugging build scripts While developing your build script, it may be useful to run bash within the container: @@ -8,7 +22,7 @@ $ python infra/helper.py shell $PROJECT_NAME # runs /bin/bash within container $ compile # run compilation manually ``` -## Debugging Fuzzers with GDB +## Debugging fuzzers with GDB If you wish to debug a fuzz target with gdb, you may use the base-runner-debug image: @@ -25,5 +39,6 @@ $ gdb --args /out/$PROJECT_NAME/$FUZZ_TARGET_NAME /out/$PROJECT_NAME/testcase Note that the base-runner-debug image does not have access to your sources, so you will not be able to do source code level debugging. We recommend integrating -your fuzz target upstream as part of [ideal integration](ideal_integration.md) +your fuzz target upstream as part of +[ideal integration]({{ site.baseurl }}/advanced-topics/ideal-integration/) for debugging purposes. diff --git a/docs/ideal_integration.md b/docs/advanced-topics/ideal_integration.md index 912906996..6174077a8 100644 --- a/docs/ideal_integration.md +++ b/docs/advanced-topics/ideal_integration.md @@ -1,25 +1,43 @@ +--- +layout: default +title: Ideal integration +parent: Advanced topics +nav_order: 1 +permalink: /advanced-topics/ideal-integration +--- + # Ideal integration with OSS-Fuzz OSS projects have different build and test systems. So, we can not expect them -to have a unified way of implementing and maintaining fuzz targets and integrating -them with OSS-Fuzz. However, we will still try to give recommendations on the preferred ways. +to have a unified way of implementing and maintaining fuzz targets and +integrating them with OSS-Fuzz. However, we will still try to give +recommendations on the preferred ways. + +Here are several features (starting from the easiest) that will make automated +fuzzing simple and efficient, and will allow to catch regressions early on in +the development cycle. -Here are several features (starting from the easiest) that will make automated fuzzing -simple and efficient, and will allow to catch regressions early on in the development cycle. +- TOC +{:toc} +--- ## TL;DR Every [fuzz target](http://libfuzzer.info/#fuzz-target): * Is [maintained by code owners](#fuzz-target) in their RCS (Git, SVN, etc). * Is [built with the rest of the tests](#build-support) - no bit rot! * Has a [seed corpus](#seed-corpus) with good [code coverage](#coverage). -* Is [continuously tested on the seed corpus](#regression-testing) with [ASan/UBSan/MSan](https://github.com/google/sanitizers) -* Is [fast and has no OOMs](#performance) +* Is [continuously tested on the seed corpus](#regression-testing) with + [ASan/UBSan/MSan](https://github.com/google/sanitizers). +* Is [fast and has no OOMs](#performance). * Has a [fuzzing dictionary](#fuzzing-dictionary), if applicable ## Fuzz Target -The code of the [fuzz target(s)](http://libfuzzer.info/#fuzz-target) should be part of the project's source code repository. -All fuzz targets should be easily discoverable (e.g. reside in the same directory, or follow the same naming pattern, etc). +The code of the [fuzz target(s)](http://libfuzzer.info/#fuzz-target) should be +part of the project's source code repository. +All fuzz targets should be easily discoverable (e.g. reside in the same +directory, or follow the same naming pattern, etc). -This makes it easy to maintain the fuzzers and minimizes breakages that can arise as source code changes over time. +This makes it easy to maintain the fuzzers and minimizes breakages that can +arise as source code changes over time. Make sure to fuzz the target locally for a small period of time to ensure that it does not crash, hang, or run out of memory instantly. @@ -44,10 +62,11 @@ A plethora of different build systems exist in the open-source world. And the less OSS-Fuzz knows about them, the better it can scale. An ideal build integration for OSS-Fuzz would look like this: -* For every fuzz target `foo` in the project, there is a build rule that builds `foo_fuzzer`, -a binary that contains the fuzzing entry point (`LLVMFuzzerTestOneInput`) -and all the code it depends on, and that uses the `main()` function from `$LIB_FUZZING_ENGINE` -(env var [provided](new_project_guide.md) by OSS-Fuzz environment). +* For every fuzz target `foo` in the project, there is a build rule that +builds `foo_fuzzer`, a binary that contains the fuzzing entry point +(`LLVMFuzzerTestOneInput`) and all the code it depends on, and that uses the +`main()` function from `$LIB_FUZZING_ENGINE` +(env var [provided]({{ site.baseurl }}/getting-started/new-project-guide/) by OSS-Fuzz environment). * The build system supports changing the compiler and passing extra compiler flags so that the build command for a `foo_fuzzer` looks similar to this: @@ -57,23 +76,26 @@ flags so that the build command for a `foo_fuzzer` looks similar to this: $ make_or_whatever_other_command foo_fuzzer ``` -This will allow to have minimal OSS-Fuzz-specific configuration and thus be more robust. +This will allow to have minimal OSS-Fuzz-specific configuration and thus be +more robust. -There is no point in hardcoding the exact compiler flags in the build system because they -a) may change and b) are different depending on the fuzzing engine and the sanitizer being used. +There is no point in hardcoding the exact compiler flags in the build system +because they a) may change and b) are different depending on the fuzzing engine +and the sanitizer being used. ## Seed Corpus The *corpus* is a set of inputs for the fuzz target (stored as individual files). When starting the fuzzing process, one should have a "seed corpus", i.e. a set of inputs to "seed" the mutations. -The quality of the seed corpus has a huge impact on fuzzing efficiency as it allows the fuzzer -to discover new code paths more easily. +The quality of the seed corpus has a huge impact on fuzzing efficiency as it +allows the fuzzer to discover new code paths more easily. The ideal corpus is a minimal set of inputs that provides maximal code coverage. For better OSS-Fuzz integration, -the seed corpus should be available in revision control (can be same or different as the source code). -It should be regularly extended with the inputs that (used to) trigger bugs and/or touch new parts of the code. +the seed corpus should be available in revision control (can be same or +different as the source code). It should be regularly extended with the inputs +that (used to) trigger bugs and/or touch new parts of the code. Examples: [boringssl](https://github.com/google/boringssl/tree/master/fuzz), @@ -81,11 +103,13 @@ Examples: [nss](https://github.com/mozilla/nss-fuzzing-corpus) (corpus in a separate repo). ## Regression Testing -The fuzz targets should be regularly tested (not necessarily fuzzed!) as a part of the project's regression testing process. +The fuzz targets should be regularly tested (not necessarily fuzzed!) as a part +of the project's regression testing process. One way to do so is to link the fuzz target with a simple standalone driver (e.g. [this one](https://github.com/llvm-mirror/compiler-rt/tree/master/lib/fuzzer/standalone)) -that runs the provided inputs and use this driver with the seed corpus created in previous step. -It is recommended to use [sanitizers](https://github.com/google/sanitizers) during regression testing. +that runs the provided inputs and use this driver with the seed corpus created +in previous step. It is recommended to use +[sanitizers](https://github.com/google/sanitizers) during regression testing. Examples: [SQLite](https://www.sqlite.org/src/artifact/d9f1a6f43e7bab45), [openssl](https://github.com/openssl/openssl/blob/master/fuzz/test-corpus.c) @@ -100,25 +124,32 @@ Ideally, a dictionary should be maintained alongside the fuzz target. The syntax is described [here](http://libfuzzer.info/#dictionaries). ## Coverage -For a fuzz target to be useful, it must have good coverage in the code that it is testing. You can view the coverage -for your fuzz targets by looking at the [fuzzer stats](clusterfuzz.md#fuzzer-stats) dashboard on ClusterFuzz, as well as -[coverage reports](clusterfuzz.md#coverage-reports). +For a fuzz target to be useful, it must have good coverage in the code that it +is testing. You can view the coverage for your fuzz targets by looking at the +[fuzzer stats]({{ site.baseurl }}/furthur-reading/clusterfuzz#fuzzer-stats) +dashboard on ClusterFuzz, as well as +[coverage reports]({{ site.baseurl }}/furthur-reading/clusterfuzz#coverage-reports). To generate an aggregated code coverage report for your project, please see -[code coverage](code_coverage.md) +[code coverage]({{ site.baseurl }}/advanced-topics/code-coverage) documentation page. -Coverage can often be improved by adding dictionaries, more inputs for seed corpora, and fixing -timeouts/out-of-memory bugs in your targets. +Coverage can often be improved by adding dictionaries, more inputs for seed +corpora, and fixing timeouts/out-of-memory bugs in your targets. ## Performance -Fuzz targets should also be performant, as high memory usage and/or slow execution speed can slow the down -the growth of coverage and finding of new bugs. ClusterFuzz provides a -[performance analyzer](clusterfuzz.md) -for each fuzz target that shows problems that are impacting the performance of the fuzz target. +Fuzz targets should also be performant, as high memory usage and/or slow +execution speed can slow the down the growth of coverage and finding of new +bugs. ClusterFuzz provides a +[performance analyzer]({{ site.baseurl }}/furthur-reading/clusterfuzz) +for each fuzz target that shows problems that are impacting the performance of +the fuzz target. ## Example -You may look at a simple [example](../projects/example/my-api-repo) that covers most of the items above. + +You may look at a simple +[example](https://github.com/google/oss-fuzz/tree/master/projects/example/my-api-repo) +that covers most of the items above. ## Not a project member? @@ -128,8 +159,10 @@ and the project maintainers are not interested in helping. In such cases, we can host the fuzz targets, dictionaries, etc in OSS-Fuzz's repository and mention them in the Dockerfile. -Examples: [libxml2](../projects/libxml2), [c-ares](../projects/c-ares), [expat](../projects/expat). +Examples: [libxml2](https://github.com/google/oss-fuzz/tree/master/projects/libxml2), +[c-ares](https://github.com/google/oss-fuzz/tree/master/projects/c-ares), [expat](https://github.com/google/oss-fuzz/tree/master/projects/expat). This is far from ideal because the fuzz targets will not be continuously tested and hence may quickly bitrot. -If you are not a project maintainer, we may not be able to CC you to security bugs found by OSS-Fuzz. +If you are not a project maintainer, we may not be able to CC you to security +bugs found by OSS-Fuzz. diff --git a/docs/advanced-topics/reproducing.md b/docs/advanced-topics/reproducing.md new file mode 100644 index 000000000..57384b9a8 --- /dev/null +++ b/docs/advanced-topics/reproducing.md @@ -0,0 +1,127 @@ +--- +layout: default +title: Reproducing +parent: Advanced topics +nav_order: 5 +permalink: /advanced-topics/reproducing +--- + +# Reproducing OSS-Fuzz issues + +You've been CC'ed on an OSS-Fuzz issue +([examples](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=Type%3ABug%2CBug-Security)), +now what? Before attempting to fix the bug, you should be able to reliably +reproduce it. + +- TOC +{:toc} +--- + +## Fuzz target bugs +Every issue has a [reproducer]({{ site.baseurl }}/reference/glossary/#reproducer) +(aka "testcase") file attached. +Download it. If the issue is not public, you will need to login using your +[Google account](https://support.google.com/accounts/answer/176347?hl=en) +([why?]({{ site.baseurl }}/faq/#why-do-you-require-a-google-account-for-authentication)) +that the bug report CCs. +This file contains the bytes that were fed to the [Fuzz Target](http://libfuzzer.info/#fuzz-target). + +If you have already +[integrated]({{ site.baseurl }}/advanced-topics/ideal-integration/) +the fuzz target with your build and test system, all you do is run: +```bash +$ ./fuzz_target_binary <testcase_path> +``` + +If this is a timeout bug, add the **-timeout=25** argument. +If this is an OOM bug, add the **-rss_limit_mb=2048** argument. +Read more on how timeouts and OOMs are handed +[here]({{ site.baseurl }}/faq/#how-do-you-handle-timeouts-and-ooms). + +Depending on the nature of the bug, the fuzz target binary needs to be built +with the appropriate [sanitizer](https://github.com/google/sanitizers) +(e.g. if this is a buffer overflow, with +[AddressSanitizer](http://clang.llvm.org/docs/AddressSanitizer.html)). + +If you are not sure how to build the fuzzer using the project's build system, +you may also use Docker +([how?]({{ site.baseurl }}/getting-started/new-project-guide/#prerequisites), +[why?]({{ site.baseurl }}/faq/#why-do-you-use-docker)) commands +to replicate the exact build steps used by OSS-Fuzz and then feed the reproducer +input to the fuzz target. + +## Building using Docker + +### Pull the latest Docker images + +```bash +$ python infra/helper.py pull_images +``` + + Docker images get regularly updated with a newer version of build tools, build + configurations, scripts, and other changes. In some cases, a particular issue + can be reproduced only with a fresh image being used. + +### Build the image and the fuzzers + +```bash +$ python infra/helper.py build_image $PROJECT_NAME +$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME +``` + +## Reproducing bugs +```bash +$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path> +``` + +Find the type of sanitizer used in the report using the value in the +**Sanitizer** column. It is one of the following: + * **address** for AddressSanitizer + * **memory** for MemorySanitizer + * **undefined** for UndefinedBehaviorSanitizer + +E.g. for building [libxml2](https://github.com/google/oss-fuzz/tree/master/projects/libxml2) +project with UndefinedBehaviorSanitizer (undefined) instrumentation and +reproduce a crash testcase for a fuzzer named `libxml2_xml_read_memory_fuzzer`, +it will be: + +```bash +$ python infra/helper.py build_image libxml2 +$ python infra/helper.py build_fuzzers --sanitizer undefined libxml2 +$ python infra/helper.py reproduce libxml2 libxml2_xml_read_memory_fuzzer ~/Downloads/testcase +``` + +## Reproduce using local source checkout + +```bash +$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME <source_path> +$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path> +``` + +This is essentially the previous command that additionally mounts local sources +into the running container. + +- *Fix issue*. Write a patch to fix the issue in your local checkout and then + use the previous command to verify the fix (i.e. no crash occurred). + [Use gdb]({{ site.baseurl }}/advanced-topics/debugging/#debugging-fuzzers-with-gdb) + if needed. +- *Submit fix*. Submit the fix in the project's repository. ClusterFuzz will + automatically pick up the changes, recheck the testcase and will close the + issue (in < 1 day). +- *Improve fuzzing support*. Consider + [improving fuzzing support]({{ site.baseurl }}/advanced-topics/ideal-integration/) + in your project's build and test system. + +## Reproducing build failures +Our infrastructure runs some sanity tests to make sure that your build was +correctly configured, even if it succeeded. To reproduce these locally, run: + +```bash +$ python infra/helper.py build_image $PROJECT_NAME +$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME +$ python infra/helper.py check_build --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME <fuzz_target_name> +``` + +For reproducing a `coverage` build failure, follow +[Code Coverage page]({{ site.baseurl }}/advanced-topics/code-coverage) to build +your project and generate a code coverage report. diff --git a/docs/clusterfuzz.md b/docs/clusterfuzz.md deleted file mode 100644 index dd2a03421..000000000 --- a/docs/clusterfuzz.md +++ /dev/null @@ -1,52 +0,0 @@ -# ClusterFuzz - -[ClusterFuzz](https://github.com/google/clusterfuzz) is the distributed fuzzing infrastructure behind OSS-Fuzz. It was initially built -for fuzzing Chrome at scale. - -## Web interface - -ClusterFuzz provides a [web interface](https://oss-fuzz.com) -to view statistics about your fuzz targets, as well as current crashes. - -*Note*: Access is restricted to project developers who we auto CC on new bug reports. - -## Testcase reports - -ClusterFuzz will automatically de-duplicate and file reproducible crashes into -our [bug tracker](https://bugs.chromium.org/p/monorail). We provide a crash -report page that gives you the stack trace, a link to the crashing testcase, and -regression ranges where the bug was most likely introduced. - - - -## Fuzzer stats - -You can view statistics about your fuzz targets (e.g. speed, coverage information, -memory usage) on our fuzzer statistics dashboard. - - - - - -## Coverage reports - -We provide coverage reports, where we highlight the parts of source code that are being -reached by your fuzz target. Make sure to look at the uncovered code marked in red and -add appropriate fuzz targets to cover those usecases. - - - - -## Performance analyzer - -You can view performance issues that your fuzz target is running into (e.g. leaks, timeouts, -etc) by clicking on `Performance` link on our fuzzer statistics dashboard. Make sure to fix -all cited issues, so as to keep your fuzz target running efficiently and finding new bugs. - - - -## Crash stats - -You can view statistics of crashes over time on our crash statistics dashboard. - - diff --git a/docs/corpora.md b/docs/corpora.md deleted file mode 100644 index 09abb28fa..000000000 --- a/docs/corpora.md +++ /dev/null @@ -1,39 +0,0 @@ -# Accessing Corpora - -If you would like to access the corpora that we are using for your fuzz targets (synthesized by the fuzzing engines), please follow these steps. - -## Obtain access - -In order to get access to a project's corpora, you must first be listed as the primary contact or an auto cc in the project's `project.yaml` file as described in the [New Project Guide](new_project_guide.md#projectyaml). If you don't do this, most of the links below won't work. - -## Install Google Cloud SDK - -The corpora for fuzz targets are stored on [Google Cloud Storage](https://cloud.google.com/storage/). To access them, you will need to [install](https://cloud.google.com/storage/docs/gsutil_install) the gsutil tool, which is part of the Google Cloud SDK. -Follow the instructions on the installation page to login with a Google account listed in your project's `project.yaml` file. - -## Viewing the corpus for a fuzz target - -The fuzzer statistics page for your project on [ClusterFuzz](clusterfuzz.md) will contain a link to the Google Cloud console for your corpus under the "corpus_size" column. You can browse and download individual test inputs in the corpus here. - - - -## Downloading the corpus - -If you would like to download the entire corpus, from the cloud console link, copy the bucket path highlighted here: - - - -And then run the following command to copy the corpus to a directory on your machine. - -```bash -gsutil -m rsync gs://<bucket_path> <local_directory> -``` -Following the expat example above, this would be: - -```bash -gsutil -m rsync gs://expat-corpus.clusterfuzz-external.appspot.com/libFuzzer/expat_parse_fuzzer <local_directory> -``` - -## Corpus backups - -We also keep daily zipped backups of your corpora. These can be accessed from the `corpus_backup` column of the fuzzer statistics page. Downloading these can also be significantly faster than `gsutil -m rsync` on the corpus bucket. diff --git a/docs/faq.md b/docs/faq.md index 92acfb80b..b585f3fc4 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -1,15 +1,29 @@ +--- +layout: default +title: FAQ +has_children: true +nav_order: 7 +permalink: /faq/ +--- + # Frequently Asked Questions +- TOC +{:toc} +--- + ## What kind of projects are you accepting? -We accept established projects that have a critical impact on infrastructure and user security. -We will consider each request on a case-by-case basis, but some things we keep in mind are: +We accept established projects that have a critical impact on infrastructure and +user security. We will consider each request on a case-by-case basis, but some +things we keep in mind are: - - Exposure to remote attacks (e.g. libraries that are used to process untrusted input). + - Exposure to remote attacks (e.g. libraries that are used to process + untrusted input). - Number of users/other projects depending on this project. -We hope to relax this requirement in the future though, so keep an eye out even if we are not able -to accept your project at this time! +We hope to relax this requirement in the future though, so keep an eye out even +if we are not able to accept your project at this time! ## How can I find potential fuzz targets in my open source project? @@ -17,8 +31,10 @@ You should look for places in your code that: - consume un-trusted data from users or from the network. - consume complex data input or event if it's 'trusted'. - - use an algorithm that has two or more implementations (to verify their equivalence). - - look for existing fuzz target [examples](https://github.com/google/oss-fuzz/tree/master/projects) and find similarities. + - use an algorithm that has two or more implementations + (to verify their equivalence). + - look for existing fuzz target [examples](https://github.com/google/oss-fuzz/tree/master/projects) + and find similarities. ## My project is not open source. Can I use OSS-Fuzz? @@ -40,18 +56,23 @@ We will reconsider the GitHub issue tracker once the ## Why do you require a Google account for authentication? -Our [ClusterFuzz](clusterfuzz.md) fuzzing infrastructure and [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list) require a Google account for authentication. Note that an alternate email address associated with a Google account does not work due to appengine api limitations. +Our [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) fuzzing +infrastructure and [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list) +require a Google account for authentication. Note that an alternate email +address associated with a Google account does not work due to appengine api +limitations. ## Why do you use Docker? -Building fuzzers requires building your project with a fresh Clang compiler and special compiler flags. -An easy-to-use Docker image is provided to simplify toolchain distribution. This also simplifies our -support for a variety of Linux distributions and provides a reproducible and secure environment for fuzzer +Building fuzzers requires building your project with a fresh Clang compiler and +special compiler flags. An easy-to-use Docker image is provided to simplify +toolchain distribution. This also simplifies our support for a variety of Linux +distributions and provides a reproducible and secure environment for fuzzer building and execution. ## How do you handle timeouts and OOMs? -If a single input to a [fuzz target](glossary.md#fuzz-target) +If a single input to a [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target) requires more than **~25 seconds** or more than **2GB RAM** to process, we report this as a timeout or an OOM (out-of-memory) bug (examples: [timeouts](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=%22Crash+Type%3A+Timeout%22), @@ -72,15 +93,15 @@ Currently we do not offer ways to change the memory and time limits. ## Can I launch an additional process (e.g. a daemon) from my fuzz target? No. In order to get all the benefits of in-process, coverage-guided fuzz testing, -it is required to run everything inside a single process. Any child processes created -outside the main process introduces heavy launch overhead and is not monitored for -code coverage. +it is required to run everything inside a single process. Any child processes +created outside the main process introduces heavy launch overhead and is not +monitored for code coverage. Another rule of thumb is: "the smaller fuzz target is, the better it is". It is -expected that your project will have many fuzz targets to test different components, -instead of a single fuzz target trying to cover everything. Think of fuzz target -as a unit test, though it is much more powerful since it helps to test millions -of data permutations rather than just one. +expected that your project will have many fuzz targets to test different +components, instead of a single fuzz target trying to cover everything. +Think of fuzz target as a unit test, though it is much more powerful since it +helps to test millions of data permutations rather than just one. ## What if my fuzz target finds a bug in another project (dependency) ? @@ -89,20 +110,32 @@ Using that, you can debug the root cause and see which category the bug falls in - If this is a bug is due to an incorrect usage of the dependent project's API in your project, then you need to fix your usage to call the API correctly. -- If this is a real bug in the dependent project, then you should CC the maintainers -of that project on the bug. Once CCed, they will get automatic access to all the -information necessary to reproduce the issue. If this project is maintained in OSS-Fuzz, -you can search for contacts in the respective project.yaml file. +- If this is a real bug in the dependent project, then you should CC the +maintainers of that project on the bug. Once CCed, they will get automatic +access to all the information necessary to reproduce the issue. If this project +is maintained in OSS-Fuzz, you can search for contacts in the respective +project.yaml file. ## What if my fuzzer does not find anything? -If your fuzz target is running for many days and does not find bugs or new coverage, it may mean several things: -- We've covered all reachable code. In order to cover more code we need more fuzz targets. -- The [seed corpus](new_project_guide.md#seed-corpus) is not good enough and the fuzzing engine(s) are not able to go deeper based on the existing seeds. Need to add more seeds. -- There is some crypto/crc stuff in the code that will prevent any fuzzing engine from going deeper, in which case the crypto should be disabled in [fuzzing mode](http://libfuzzer.info#fuzzer-friendly-build-mode). Examples: [openssl](https://github.com/openssl/openssl/tree/master/fuzz#reproducing-issues), [boringssl](https://boringssl.googlesource.com/boringssl/+/HEAD/FUZZING.md#Fuzzer-mode) -- It is also possible that the fuzzer is running too slow (you may check the speed of your targets at https://oss-fuzz.com/) - -In either case, look at the [coverage reports](clusterfuzz.md#coverage-reports) for your target(s) and figure out why some parts of the code are not covered. +If your fuzz target is running for many days and does not find bugs or new +coverage, it may mean several things: +- We've covered all reachable code. In order to cover more code we need more + fuzz targets. +- The [seed corpus]({{ site.baseurl }}/getting-started/new-project-guide#seed-corpus) is not good enough and the + fuzzing engine(s) are not able to go deeper based on the existing seeds. + Need to add more seeds. +- There is some crypto/crc stuff in the code that will prevent any fuzzing + engine from going deeper, in which case the crypto should be disabled in + [fuzzing mode](http://libfuzzer.info#fuzzer-friendly-build-mode). + Examples: [openssl](https://github.com/openssl/openssl/tree/master/fuzz#reproducing-issues), + [boringssl](https://boringssl.googlesource.com/boringssl/+/HEAD/FUZZING.md#Fuzzer-mode) +- It is also possible that the fuzzer is running too slow + (you may check the speed of your targets at https://oss-fuzz.com/) + +In either case, look at the +[coverage reports]({{ site.baseurl }}/furthur-reading/clusterfuzz#coverage-reports) +for your target(s) and figure out why some parts of the code are not covered. ## Why are code coverage reports public? @@ -111,19 +144,33 @@ possible. We believe that public code coverage reports do not put users at risk, as they do not indicate the presence of bugs or lack thereof. ## What happens when I rename a fuzz target ? -If you rename your fuzz targets, the existing bugs for those targets will get closed and fuzzing will start from scratch from a fresh corpora (seed corpus only). Similar corpora will get accumulated over time depending on the number of cpu cycles that original fuzz target has run. If this is not desirable, make sure to copy the accumulated corpora from the original fuzz target (instructions to download [here](corpora.md#downloading-the-corpus)) and restore it to the new GCS location later (instruction to find the new location [here](corpora.md#viewing-the-corpus-for-a-fuzz-target)). +If you rename your fuzz targets, the existing bugs for those targets will get +closed and fuzzing will start from scratch from a fresh corpora +(seed corpus only). Similar corpora will get accumulated over time depending on +the number of cpu cycles that original fuzz target has run. If this is not +desirable, make sure to copy the accumulated corpora from the original fuzz +target (instructions to download +[here]({{ site.baseurl }}/advanced-topics/corpora/#downloading-the-corpus)) and +restore it to the new GCS location later (instruction to find the +new location [here]({{ site.baseurl }}/advanced-topics/corpora/#viewing-the-corpus-for-a-fuzz-target)). ## Does OSS-Fuzz support AFL? -OSS-Fuzz *uses* [AFL](http://lcamtuf.coredump.cx/afl/) as one of its [fuzzing engines](glossary.md#fuzzing-engine) but this is an implementation detail. Just follow the [ideal integration guide](ideal_integration.md) and OSS-Fuzz will use all its fuzzing engines on your code. +OSS-Fuzz *uses* [AFL](http://lcamtuf.coredump.cx/afl/) as one of its +[fuzzing engines]({{ site.baseurl }}/reference/glossary/#fuzzing-engine) but this is an implementation +detail. Just follow the +[ideal integration guide]({{ site.baseurl }}/advanced-topics/ideal-integration/) +and OSS-Fuzz will use all its fuzzing engines on your code. ## Does OSS-Fuzz support Honggfuzz? OSS-Fuzz (the service) does not use honggfuzz to find bugs. -That is because [ClusterFuzz](https://github.com/google/clusterfuzz) (the infrastructure OSS-Fuzz runs on) does not support honggfuzz. -However, OSS-Fuzz supports **building** fuzz targets for use with honggfuzz, similar to the way it -supports [AFL](#does-oss-fuzz-support-afl). +That is because [ClusterFuzz](https://github.com/google/clusterfuzz) +(the infrastructure OSS-Fuzz runs on) does not support honggfuzz. +However, OSS-Fuzz supports **building** fuzz targets for use with honggfuzz, +similar to the way it supports [AFL](#does-oss-fuzz-support-afl). Currently OSS-Fuzz builders do builds for libFuzzer, AFL, and honggfuzz. ## What are the specs on your machines? OSS-Fuzz builders have 32CPU/28.8GB RAM. -Fuzzing machines only have a single core and fuzz targets should not use more than 2GB of RAM. +Fuzzing machines only have a single core and fuzz targets should not use more +than 2GB of RAM. diff --git a/docs/furthur-reading/clusterfuzz.md b/docs/furthur-reading/clusterfuzz.md new file mode 100644 index 000000000..d37e256d2 --- /dev/null +++ b/docs/furthur-reading/clusterfuzz.md @@ -0,0 +1,67 @@ +--- +layout: default +title: ClusterFuzz +parent: Furthur reading +nav_order: 1 +permalink: /furthur-reading/clusterfuzz/ +--- + +# ClusterFuzz + +[ClusterFuzz](https://github.com/google/clusterfuzz) is the distributed fuzzing +infrastructure behind OSS-Fuzz. It was initially built for fuzzing Chrome at +scale. + +- TOC +{:toc} +--- + +## Web interface + +ClusterFuzz provides a [web interface](https://oss-fuzz.com) +to view statistics about your fuzz targets, as well as current crashes. + +*Note*: Access is restricted to project developers who we auto CC on new bug +reports. + +## Testcase reports + +ClusterFuzz will automatically de-duplicate and file reproducible crashes into +our [bug tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list). We provide +a crash report page that gives you the stack trace, a link to the crashing +testcase, and regression ranges where the bug was most likely introduced. + + + +## Fuzzer stats + +You can view statistics about your fuzz targets (e.g. speed, coverage +information, memory usage) on our fuzzer statistics dashboard. + + + + + +## Coverage reports + +We provide coverage reports, where we highlight the parts of source code that +are being reached by your fuzz target. Make sure to look at the uncovered code +marked in red and add appropriate fuzz targets to cover those usecases. + + + + +## Performance analyzer + +You can view performance issues that your fuzz target is running into (e.g. +leaks, timeouts, etc) by clicking on `Performance` link on our fuzzer statistics +dashboard. Make sure to fix all cited issues, so as to keep your fuzz target +running efficiently and finding new bugs. + + + +## Crash stats + +You can view statistics of crashes over time on our crash statistics dashboard. + + diff --git a/docs/furthur-reading/furthur_reading.md b/docs/furthur-reading/furthur_reading.md new file mode 100644 index 000000000..471e5e7c9 --- /dev/null +++ b/docs/furthur-reading/furthur_reading.md @@ -0,0 +1,9 @@ +--- +layout: default +title: Furthur reading +has_children: true +nav_order: 4 +permalink: /furthur-reading/ +--- + +# Furthur reading diff --git a/docs/fuzzer_environment.md b/docs/furthur-reading/fuzzer_environment.md index db7e2e321..6c51e4366 100644 --- a/docs/fuzzer_environment.md +++ b/docs/furthur-reading/fuzzer_environment.md @@ -1,14 +1,28 @@ +--- +layout: default +title: Fuzzer environment +parent: Furthur reading +nav_order: 2 +permalink: /furthur-reading/fuzzer-environment/ +--- + # Fuzzer environment on ClusterFuzz -Your fuzz targets will be run on a [Google Compute Engine](https://cloud.google.com/compute/) VM (Linux) with some [security restrictions](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-runner/run_minijail). +Your fuzz targets will be run on a +[Google Compute Engine](https://cloud.google.com/compute/) VM (Linux) with some +[security restrictions](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-runner/run_minijail). + +- TOC +{:toc} +--- ## Runtime Dependencies You should not make any assumptions on the availability of dependent packages in the execution environment. Packages that are installed via -[Dockerfile](new_project_guide.md#dockerfile) +[Dockerfile]({{ site.baseurl }}/getting-started/new-project-guide/#dockerfile) or built as part of -[build.sh](new_project_guide.md#buildsh) +[build.sh]({{ site.baseurl }}/getting-started/new-project-guide/#buildsh) are not available on the bot runtime environment (where the fuzz targets run). If you need these dependencies in the runtime environment, you can either @@ -17,22 +31,24 @@ If you need these dependencies in the runtime environment, you can either and then link statically against them ([example](https://github.com/google/oss-fuzz/blob/master/projects/tor/build.sh#L40)) - Or build the dependencies statically in -[build.sh](new_project_guide.md#buildsh) +[build.sh]({{ site.baseurl }}/getting-started/new-project-guide/#buildsh) ([example](https://github.com/google/oss-fuzz/blob/master/projects/ffmpeg/build.sh#L26)). -All build artifacts needed during fuzz target execution should be inside the `$OUT` -directory. Only those artifacts are archived and used on the bots. Everything else -is ignored (e.g. artifacts in `$WORK`, `$SRC`, etc) and hence is not available -in the execution environment. +All build artifacts needed during fuzz target execution should be inside the +`$OUT` directory. Only those artifacts are archived and used on the bots. +Everything else is ignored (e.g. artifacts in `$WORK`, `$SRC`, etc) and hence +is not available in the execution environment. -You should ensure that the fuzz target works correctly by using `run_fuzzer` command -(see instructions [here](new_project_guide.md#testing-locally)). This command uses -a clean base-runner docker container and not the base-builder docker container -created during build-time. +You should ensure that the fuzz target works correctly by using `run_fuzzer` +command (see instructions +[here]({{ site.baseurl }}/getting-started/new-project-guide/#testing-locally)). +This command uses a clean base-runner docker container and not the base-builder +docker container created during build-time. ## argv[0] -You must not modify `argv[0]`. It is required for certain things to work correctly. +You must not modify `argv[0]`. It is required for certain things to work +correctly. ## Current working directory @@ -42,8 +58,8 @@ directory where your fuzz target executable is located. ## File system -Everything except `/tmp` is read-only, including the directory that your fuzz target -executable lives in. +Everything except `/tmp` is read-only, including the directory that your fuzz +target executable lives in. `/dev` is also unavailable. diff --git a/docs/getting-started/accepting_new_projects.md b/docs/getting-started/accepting_new_projects.md new file mode 100644 index 000000000..da8f3a8af --- /dev/null +++ b/docs/getting-started/accepting_new_projects.md @@ -0,0 +1,38 @@ +--- +layout: default +title: Accepting new projects +parent: Getting started +nav_order: 1 +permalink: /getting-started/accepting-new-projects/ +--- + +## Accepting New Projects + +To be accepted to OSS-Fuzz, an open-source project must +have a significant user base and/or be critical to the global IT infrastructure. + +To submit a new project: + +* [Create a pull request](https://help.github.com/articles/creating-a-pull-request/) +with new `projects/<project_name>/project.yaml` file +([example](https://github.com/google/oss-fuzz/tree/master/projects/libarchive/project.yaml)) +giving at least the following information: + * project homepage. + * e-mail of the engineering contact person to be CCed on new issues. It should: + * belong to an established project committer (according to VCS logs). + If this is not you or the email address differs from VCS, an informal + e-mail verification will be required. + * be associated with a Google account + ([why?]({{ site.baseurl }}/faq/#why-do-you-require-a-google-account-for-authentication)). + If you use an alternate email address + [linked to a Google Account](https://support.google.com/accounts/answer/176347?hl=en), + it will ONLY give you access to filed bugs in + [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list) and + NOT to [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) + dashboard (due to appengine api limitations). + * Note that `project_name` can only contain alphanumeric characters, + underscores(_) or dashes(-). + +* Once accepted by an OSS-Fuzz project member, follow the + [New Project Guide]({{ site.baseurl }}/getting-started/new-project-guide/) + to configure your project. diff --git a/docs/getting-started/bug_disclosure_guidelines.md b/docs/getting-started/bug_disclosure_guidelines.md new file mode 100644 index 000000000..f82a75100 --- /dev/null +++ b/docs/getting-started/bug_disclosure_guidelines.md @@ -0,0 +1,23 @@ +--- +layout: default +title: Bug disclosure guidelines +parent: Getting started +nav_order: 4 +permalink: /getting-started/bug-disclosure-guidelines/ +--- + +## Bug Disclosure Guidelines + +Following [Google's standard disclosure policy](https://googleprojectzero.blogspot.com/2015/02/feedback-and-data-driven-updates-to.html), +OSS-Fuzz will adhere to following disclosure principles: + + - **Deadline**. After notifying project authors, we will open reported + issues to the public in 90 days, or 30 days after the fix is released + (whichever comes earlier). + - **Weekends and holidays**. If a deadline is due to expire on a weekend, + the deadline will be moved to the next normal work day. + - **Grace period**. We have a 14-day grace period. If a 90-day deadline + expires but the upstream engineers let us know before the deadline that a + patch is scheduled for release on a specific day within 14 days following + the deadline, the public disclosure will be delayed until the availability + of the patch.
\ No newline at end of file diff --git a/docs/getting-started/getting_started.md b/docs/getting-started/getting_started.md new file mode 100644 index 000000000..15481c002 --- /dev/null +++ b/docs/getting-started/getting_started.md @@ -0,0 +1,11 @@ +--- +layout: default +title: Getting started +has_children: true +nav_order: 2 +permalink: /getting-started/ +--- + +# Getting started +These pages walk you through the process of integrating your open source project +with OSS-Fuzz. diff --git a/docs/getting-started/integration_rewards.md b/docs/getting-started/integration_rewards.md new file mode 100644 index 000000000..51b2a8ffa --- /dev/null +++ b/docs/getting-started/integration_rewards.md @@ -0,0 +1,13 @@ +--- +layout: default +title: Integration rewards +parent: Getting started +nav_order: 3 +permalink: /getting-started/integration-rewards/ +--- + +# Integration rewards + +We encourage you to apply for integration rewards (upto **$20,000**) once your project is +successfully integrated with OSS-Fuzz. Please see the details in our blog post +[here](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html). diff --git a/docs/new_project_guide.md b/docs/getting-started/new_project_guide.md index b2f0858a5..d3ca301e5 100644 --- a/docs/new_project_guide.md +++ b/docs/getting-started/new_project_guide.md @@ -1,8 +1,16 @@ +--- +layout: default +title: New project guide +parent: Getting started +nav_order: 2 +permalink: /getting-started/new-project-guide/ +--- + # Setting up a New Project ## Prerequisites -- [Integrate](ideal_integration.md) one or more [Fuzz Targets](glossary.md#fuzz-target) - with the project you want to fuzz.<BR> +- [Integrate]({{ site.baseurl }}/advanced-topics/ideal-integration/) one or more [Fuzz Targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) + with the project you want to fuzz. Examples: [boringssl](https://github.com/google/boringssl/tree/master/fuzz), [SQLite](https://www.sqlite.org/src/artifact/ad79e867fb504338), @@ -14,21 +22,29 @@ [pcre2](http://vcs.pcre.org/pcre2/code/trunk/src/pcre2_fuzzsupport.c?view=markup), [ffmpeg](https://github.com/FFmpeg/FFmpeg/blob/master/tools/target_dec_fuzzer.c). -- [Install Docker](installing_docker.md). ([Why Docker?](faq.md#why-do-you-use-docker)) +- Install Docker using the instructions + [here](https://docs.docker.com/engine/installation). + Googlers: [go/installdocker](https://goto.google.com/installdocker). + [Why Docker?]({{ site.baseurl }}/faq/#why-do-you-use-docker) + *NOTE: (Optional) If you want to run `docker` without `sudo`, follow the + [Create a docker group](https://docs.docker.com/engine/installation/linux/ubuntulinux/#/create-a-docker-group) section.* + *NOTE: Docker images can consume significant disk space. Run* + *[docker-cleanup](https://gist.github.com/mikea/d23a839cba68778d94e0302e8a2c200f)* + *periodically to garbage collect unused images.* ## Overview To add a new OSS project to OSS-Fuzz, you need a project subdirectory -inside the [`projects/`](../projects) directory in [OSS-Fuzz repository](https://github.com/google/oss-fuzz). +inside the [`projects/`](https://github.com/google/oss-fuzz/tree/master/projects) directory in [OSS-Fuzz repository](https://github.com/google/oss-fuzz). Example: [boringssl](https://github.com/google/boringssl) project is located in -[`projects/boringssl`](../projects/boringssl). +[`projects/boringssl`](https://github.com/google/oss-fuzz/tree/master/projects/boringssl). The project directory needs to contain the following three configuration files: * `projects/<project_name>/project.yaml` - provides metadata about the project. * `projects/<project_name>/Dockerfile` - defines the container environment with information -on dependencies needed to build the project and its [fuzz targets](glossary.md#fuzz-target). +on dependencies needed to build the project and its [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target). * `projects/<project_name>/build.sh` - build script that executes inside the container and generates project build. @@ -42,7 +58,7 @@ $ export PROJECT_NAME=<project_name> $ python infra/helper.py generate $PROJECT_NAME ``` -It is preferred to keep and maintain [fuzz targets](glossary.md#fuzz-target) in your own source code repository. If this is not possible due to various reasons, you can store them inside the OSS-Fuzz's project directory created above. +It is preferred to keep and maintain [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) in your own source code repository. If this is not possible due to various reasons, you can store them inside the OSS-Fuzz's project directory created above. ## project.yaml @@ -55,7 +71,7 @@ Project's homepage. Primary contact and CCs list. These people get access to ClusterFuzz which includes crash reports, fuzzer statistics, etc and are auto-cced on newly filed bugs in OSS-Fuzz tracker. To get full access to these artifacts, you should use a [Google account](https://support.google.com/accounts/answer/176347?hl=en) -here ([why?](faq.md#why-do-you-require-a-google-account-for-authentication)). +here ([why?]({{ site.baseurl }}/faq/#why-do-you-require-a-google-account-for-authentication)). ### sanitizers (optional) List of sanitizers to use. By default, it will use the default list of supported @@ -74,7 +90,7 @@ You may opt-in by adding "memory" to this list. If you want to test a particular sanitizer (e.g. memory) and see what crashes it generates without filing them in the issue tracker, you can set the experimental flag. The crashes can be accessed on [ClusterFuzz -homepage](clusterfuzz.md#web-interface). Example: +homepage]({{ site.baseurl }}/furthur-reading/clusterfuzz#web-interface). Example: ``` sanitizers: @@ -88,7 +104,7 @@ Example: [boringssl](https://github.com/google/oss-fuzz/blob/master/projects/bor ### help_url Link to a custom help URL in bug reports instead of the -[default OSS-Fuzz guide to reproducing crashes](reproducing.md). This can be useful if you assign +[default OSS-Fuzz guide to reproducing crashes]({{ site.baseurl }}/advanced-topics/reproducing/). This can be useful if you assign bugs to members of your project unfamiliar with OSS-Fuzz or if they should follow a different workflow for reproducing and fixing bugs than standard one outlined in the reproducing guide. @@ -96,7 +112,7 @@ Example: [skia](https://github.com/google/oss-fuzz/blob/master/projects/skia/pro ### experimental A boolean (either True or False) that indicates whether this project is in evaluation mode. This allows a project to be -fuzzed and generate crash findings, but not file them in the issue tracker. The crashes can be accessed on [ClusterFuzz homepage](clusterfuzz.md#web-interface). This should be only used if you are not a maintainer of the project and have +fuzzed and generate crash findings, but not file them in the issue tracker. The crashes can be accessed on [ClusterFuzz homepage]({{ site.baseurl }}/furthur-reading/clusterfuzz#web-interface). This should be only used if you are not a maintainer of the project and have less confidence in the efficacy of your fuzz targets. Example: ``` @@ -124,20 +140,20 @@ RUN git clone <git_url> <checkout_dir> # checkout all sources needed to build WORKDIR <checkout_dir> # current directory for build script COPY build.sh fuzzer.cc $SRC/ # copy build script and other fuzzer files in src dir ``` -Expat example: [expat/Dockerfile](../projects/expat/Dockerfile) +Expat example: [expat/Dockerfile](https://github.com/google/oss-fuzz/tree/master/projects/expat/Dockerfile) In the above example, the git clone will check out the source to `$SRC/<checkout_dir>`. ## build.sh -This file describes how to build binaries for [fuzz targets](glossary.md#fuzz-target) in your project. +This file describes how to build binaries for [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) in your project. The script will be executed within the image built from `Dockerfile`. In general, this script will need to: 1. Build the project using your build system *with* correct compiler and its flags provided as *environment variables* (see below). -2. Build the [fuzz targets](glossary.md#fuzz-target), linking your project's build and libFuzzer. +2. Build the [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target), linking your project's build and libFuzzer. Resulting binaries should be placed in `$OUT`. *Note*: @@ -145,7 +161,7 @@ In general, this script will need to: 1. Please don't assume that the fuzzing engine is libFuzzer and hardcode in your build scripts. We generate builds for both libFuzzer and AFL fuzzing engine configurations. So, link the fuzzing engine using $LIB_FUZZING_ENGINE, see example below. -2. Please make sure that the binary names for your [fuzz targets](glossary.md#fuzz-target) contain only +2. Please make sure that the binary names for your [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) contain only alphanumeric characters, underscore(_) or dash(-). Otherwise, they won't run on our infrastructure. 3. Please don't remove source code files. They are needed for code coverage. @@ -199,7 +215,7 @@ You *must* use `$CXX` as a linker, even if your project is written in pure C. Most well-crafted build scripts will automatically use these variables. If not, pass them manually to the build tool. -See [Provided Environment Variables](../infra/base-images/base-builder/README.md#provided-environment-variables) section in +See [Provided Environment Variables](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/README.md#provided-environment-variables) section in `base-builder` image documentation for more details. ## Disk space restrictions @@ -210,12 +226,12 @@ In addition to this, please keep the size of the build (everything copied to `$O ## Fuzzer execution environment -[This page](fuzzer_environment.md) gives information about the environment that -your [fuzz targets](glossary.md#fuzz-target) will run on ClusterFuzz, and the assumptions that you can make. +[This page]({{ site.baseurl }}/furthur-reading/fuzzer-environment/) gives information about the environment that +your [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) will run on ClusterFuzz, and the assumptions that you can make. ## Testing locally -Use the helper script to build docker image and [fuzz targets](glossary.md#fuzz-target). +Use the helper script to build docker image and [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target). ```bash $ cd /path/to/oss-fuzz @@ -241,10 +257,10 @@ $ python infra/helper.py run_fuzzer $PROJECT_NAME <fuzz_target> ``` If everything works locally, then it should also work on our automated builders and ClusterFuzz. -If it fails, check out [this](fuzzer_environment.md#dependencies) entry. +If it fails, check out [this]({{ site.baseurl }}/furthur-reading/fuzzer-environment/#dependencies) entry. It's recommended to look at code coverage as a sanity check to make sure that -[fuzz target](glossary.md#fuzz-target) gets to the code you expect. +[fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target) gets to the code you expect. ```bash $ python infra/helper.py build_fuzzers --sanitizer coverage $PROJECT_NAME @@ -257,8 +273,8 @@ of the supported build configurations with the above commands (build_fuzzers -> ## Debugging Problems -[Debugging](debugging.md) document lists ways to debug your build scripts or -[fuzz targets](glossary.md#fuzz-target) +[Debugging]({{ site.baseurl }}/advanced-topics/debugging/) document lists ways to debug your build scripts or +[fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) in case you run into problems. @@ -282,7 +298,7 @@ if (size < kMinInputLength || size > kMaxInputLength) return 0; ``` -For out of tree [fuzz targets](glossary.md#fuzz-target), you will likely add options file using docker's +For out of tree [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target), you will likely add options file using docker's `COPY` directive and will copy it into output in build script. (example: [woff2](https://github.com/google/oss-fuzz/blob/master/projects/woff2/convert_woff2ttf_fuzzer.options)). @@ -290,10 +306,10 @@ For out of tree [fuzz targets](glossary.md#fuzz-target), you will likely add opt ### Seed Corpus OSS-Fuzz uses evolutionary fuzzing algorithms. Supplying seed corpus consisting -of good sample inputs is one of the best ways to improve [fuzz target](glossary.md#fuzz-target)'s coverage. +of good sample inputs is one of the best ways to improve [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target)'s coverage. To provide a corpus for `my_fuzzer`, put `my_fuzzer_seed_corpus.zip` file next -to the [fuzz target](glossary.md#fuzz-target)'s binary in `$OUT` during the build. Individual files in this +to the [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target)'s binary in `$OUT` during the build. Individual files in this archive will be used as starting inputs for mutations. The name of each file in the corpus is the sha1 checksum (which you can get using the `sha1sum` or `shasum` comand) of its contents. You can store the corpus next to source files, generate during build or fetch it using curl or any other tool of your choice. @@ -303,7 +319,7 @@ Seed corpus files will be used for cross-mutations and portions of them might ap in bug reports or be used for further security research. It is important that corpus has an appropriate and consistent license. -See also [Accessing Corpora](corpora.md) for information about getting access to the corpus we are currently using for your fuzz targets. +See also [Accessing Corpora]({{ site.baseurl }}/advanced-topics/corpora/) for information about getting access to the corpus we are currently using for your fuzz targets. ### Dictionaries @@ -321,7 +337,7 @@ in .options file: dict = dictionary_name.dict ``` -It is common for several [fuzz targets](glossary.md#fuzz-target) +It is common for several [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) to reuse the same dictionary if they are fuzzing very similar inputs. (example: [expat](https://github.com/google/oss-fuzz/blob/master/projects/expat/parse_fuzzer.options)). @@ -358,10 +374,11 @@ If you are porting a fuzz target from Chromium, keep the original Chromium licen ## The end Once your change is merged, your project and fuzz targets should be automatically built and run on -ClusterFuzz after a short while (< 1 day)!<BR><BR> -Check your project's build status [here](https://oss-fuzz-build-logs.storage.googleapis.com/index.html).<BR> +ClusterFuzz after a short while (< 1 day)! + +Check your project's build status [here](https://oss-fuzz-build-logs.storage.googleapis.com/index.html). -Use [ClusterFuzz](clusterfuzz.md) web interface [here](https://oss-fuzz.com/) to checkout the following items: +Use [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) web interface [here](https://oss-fuzz.com/) to checkout the following items: * Crashes generated * Code coverage statistics * Fuzzer statistics diff --git a/docs/glossary.md b/docs/glossary.md deleted file mode 100644 index 764058923..000000000 --- a/docs/glossary.md +++ /dev/null @@ -1,55 +0,0 @@ -# Glossary - -For general fuzzing terms, see the [glossary] from [google/fuzzing] project, - -[glossary]: https://github.com/google/fuzzing/blob/master/docs/glossary.md -[google/fuzzing]: https://github.com/google/fuzzing - -## OSS-Fuzz specific terms - -### ClusterFuzz - -A scalable fuzzing infrastructure that is used for OSS-Fuzz backend. -[ClusterFuzz] is also used to fuzz Chrome and many other projects. A quick -overview of ClusterFuzz user interface is available on this [page]. - -[page]: clusterfuzz.md -[ClusterFuzz]: https://github.com/google/clusterfuzz - -### Fuzz Target - -In addition to its general definition, in OSS-Fuzz a fuzz target can be used to -[reproduce bug reports](reproducing.md). It is recommended to use it for -regression testing as well (see [ideal integration](ideal_integration.md)). - -### Job type - -Or **Fuzzer Build**. - -This refers to a build that contains all the [fuzz targets] for a given -[project](#project), is run with a specific [fuzzing engine], in a specific -build mode (e.g. with enabled/disabled assertions), and optionally combined -with a [sanitizer]. - -For example, we have a "libfuzzer_asan_sqlite" job type, indicating a build of -all sqlite3 [fuzz targets] using [libFuzzer](http://libfuzzer.info) and -[ASan](http://clang.llvm.org/docs/AddressSanitizer.html). - -### Project - -A project is an open source software project that is integrated with OSS-Fuzz. -Each project has a single set of configuration files -(example: [expat](https://github.com/google/oss-fuzz/tree/master/projects/expat)) -and may have one or more [fuzz targets] -(example: [openssl](https://github.com/openssl/openssl/blob/master/fuzz/)). - -### Reproducer - -Or a **testcase**. - -A [test input] that causes a specific bug to reproduce. - -[fuzz targets]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzz-target -[fuzzing engine]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzzing-engine -[sanitizer]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#sanitizer -[test input]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#test-input diff --git a/docs/index.md b/docs/index.md index 1e8291ca3..f71850b1c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -29,7 +29,7 @@ distributed execution. We support [libFuzzer](http://llvm.org/docs/LibFuzzer.html) and [AFL](http://lcamtuf.coredump.cx/afl/) as fuzzing engines in combination with [Sanitizers](https://github.com/google/sanitizers). -[ClusterFuzz](docs/clusterfuzz.md) +[ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) provides a distributed fuzzer execution environment and reporting. You can checkout ClusterFuzz [here](https://github.com/google/clusterfuzz). diff --git a/docs/installing_docker.md b/docs/installing_docker.md deleted file mode 100644 index dd25be5a2..000000000 --- a/docs/installing_docker.md +++ /dev/null @@ -1,14 +0,0 @@ -# Installing Docker - -Follow the [Official Guide](https://docs.docker.com/engine/installation). - -Googlers: [go/installdocker](https://goto.google.com/installdocker). - -[Why Docker?](faq.md#why-do-you-use-docker) - -*NOTE: (Optional) If you want to run `docker` without `sudo`, follow the [Create a docker group](https://docs.docker.com/engine/installation/linux/ubuntulinux/#/create-a-docker-group) section.* - -*NOTE: Docker images can consume significant disk space. Run* -*[docker-cleanup](https://gist.github.com/mikea/d23a839cba68778d94e0302e8a2c200f)* -*periodically to garbage collect unused images.* - diff --git a/docs/oss-fuzz/architecture.md b/docs/oss-fuzz/architecture.md new file mode 100644 index 000000000..bf7175c13 --- /dev/null +++ b/docs/oss-fuzz/architecture.md @@ -0,0 +1,33 @@ +--- +layout: default +title: Architecture +permalink: /architecture/ +nav_order: 1 +parent: OSS-Fuzz +--- + +# Architecture + + +The following process is used for projects in OSS-Fuzz: + +- A maintainer of an opensource project or an outside volunteer creates +one or more [fuzz targets](http://libfuzzer.info/#fuzz-target) +and [integrates]({{ site.baseurl }}/advanced-topics/ideal-integration/) them +with the project's build and test system. +- The project is [accepted to OSS-Fuzz]({{ site.baseurl }}/getting-started/accepting-new-projects/). +- When [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) finds a + bug, an issue is automatically reported in the OSS-Fuzz + [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list) + ([example](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9)). + ([Why use a different tracker?]({{ site.baseurl }}/faq/#why-do-you-use-a-different-issue-tracker-for-reporting-bugs-in-oss-projects)). + Project owners are CC-ed to the bug report. +- The project developer fixes the bug upstream and credits OSS-Fuzz for the + discovery (commit message should contain the string **'Credit to OSS-Fuzz'**). +- [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) automatically + verifies the fix, adds a comment and closes the issue + ([example](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53#c3)). +- 30 days after the fix is verified or 90 days after reporting (whichever is + earlier), the issue becomes *public* + ([guidelines]({{ site.baseurl }}/getting-started/bug-disclosure-guidelines/)). + diff --git a/docs/reference.md b/docs/reference.md deleted file mode 100644 index 9cac9d4dd..000000000 --- a/docs/reference.md +++ /dev/null @@ -1,30 +0,0 @@ -# Reference - -## Sanitizers - -Fuzzers are usually built with one or more [sanitizer](https://github.com/google/sanitizers) enabled. -You can select sanitizer configuration by specifying `$SANITIZER` build environment variable using `-e` option: - -```bash -python infra/helper.py build_fuzzers --sanitizer undefined json -``` - -Supported sanitizers: - -| `$SANITIZER` | Description -| ------------ | ---------- -| `address` *(default)* | [Address Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer) with [Leak Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer). -| `undefined` | [Undefined Behavior Sanitizer](http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html). -| `memory` | [Memory Sanitizer](https://github.com/google/sanitizers/wiki/MemorySanitizer).<br/>*NOTE: It is critical that you build __all__ the code in your program (including libraries it uses) with Memory Sanitizer. Otherwise, you will see false positive crashes due to an inability to see initializations in uninstrumented code.* -| `profile` | Used for generating code coverage reports. See [Code Coverage doc](code_coverage.md). - -Compiler flag values for predefined configurations are specified in the [Dockerfile](../infra/base-images/base-builder/Dockerfile). -These flags can be overridden by specifying `$SANITIZER_FLAGS` directly. - -You can choose which configurations to automatically run your fuzzers with in `project.yaml` file (e.g. [sqlite3](../projects/sqlite3/project.yaml)): - -```yaml -sanitizers: - - address - - undefined -``` diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md new file mode 100644 index 000000000..d5bf18bc6 --- /dev/null +++ b/docs/reference/glossary.md @@ -0,0 +1,96 @@ +--- +layout: default +title: Glossary +nav_order: 1 +permalink: /reference/glossary/ +parent: Reference +--- + +# Glossary + +For general fuzzing terms, see the [glossary] from [google/fuzzing] project, + +[glossary]: https://github.com/google/fuzzing/blob/master/docs/glossary.md +[google/fuzzing]: https://github.com/google/fuzzing + +- TOC +{:toc} +--- + +## OSS-Fuzz specific terms + +### ClusterFuzz + +A scalable fuzzing infrastructure that is used for OSS-Fuzz backend. +[ClusterFuzz] is also used to fuzz Chrome and many other projects. A quick +overview of ClusterFuzz user interface is available on this [page]. + +[page]: {{ site.baseurl }}/furthur-reading/clusterfuzz +[ClusterFuzz]: https://github.com/google/clusterfuzz + +### Fuzz Target + +In addition to its general definition, in OSS-Fuzz a fuzz target can be used to +[reproduce bug reports]({{ site.baseurl }}/advanced-topics/reproducing/). It is recommended to use it for +regression testing as well (see [ideal integration]({{ site.baseurl }}/advanced-topics/ideal-integration/)). + +### Job type + +Or **Fuzzer Build**. + +This refers to a build that contains all the [fuzz targets] for a given +[project](#project), is run with a specific [fuzzing engine], in a specific +build mode (e.g. with enabled/disabled assertions), and optionally combined +with a [sanitizer]. + +For example, we have a "libfuzzer_asan_sqlite" job type, indicating a build of +all sqlite3 [fuzz targets] using [libFuzzer](http://libfuzzer.info) and +[ASan](http://clang.llvm.org/docs/AddressSanitizer.html). + +### Project + +A project is an open source software project that is integrated with OSS-Fuzz. +Each project has a single set of configuration files +(example: [expat](https://github.com/google/oss-fuzz/tree/master/projects/expat)) +and may have one or more [fuzz targets] +(example: [openssl](https://github.com/openssl/openssl/blob/master/fuzz/)). + +### Reproducer + +Or a **testcase**. + +A [test input] that causes a specific bug to reproduce. + +[fuzz targets]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzz-target +[fuzzing engine]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzzing-engine +[sanitizer]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#sanitizer +[test input]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#test-input + +### Sanitizers + +Fuzzers are usually built with one or more [sanitizer](https://github.com/google/sanitizers) enabled. +You can select sanitizer configuration by specifying `$SANITIZER` build environment variable using `-e` option: + +```bash +python infra/helper.py build_fuzzers --sanitizer undefined json +``` + +Supported sanitizers: + +| `$SANITIZER` | Description +| ------------ | ---------- +| `address` *(default)* | [Address Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer) with [Leak Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer). +| `undefined` | [Undefined Behavior Sanitizer](http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html). +| `memory` | [Memory Sanitizer](https://github.com/google/sanitizers/wiki/MemorySanitizer).<br/>*NOTE: It is critical that you build __all__ the code in your program (including libraries it uses) with Memory Sanitizer. Otherwise, you will see false positive crashes due to an inability to see initializations in uninstrumented code.* +| `profile` | Used for generating code coverage reports. See [Code Coverage doc]({{ site.baseurl }}/advanced-topics/code-coverage/). + +Compiler flag values for predefined configurations are specified in the [Dockerfile](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/Dockerfile). +These flags can be overridden by specifying `$SANITIZER_FLAGS` directly. + +You can choose which configurations to automatically run your fuzzers with in `project.yaml` file (e.g. [sqlite3](https://github.com/google/oss-fuzz/tree/master/projects/sqlite3/project.yaml)): + +```yaml +sanitizers: + - address + - undefined +```
\ No newline at end of file diff --git a/docs/reference/reference.md b/docs/reference/reference.md new file mode 100644 index 000000000..0d41025cb --- /dev/null +++ b/docs/reference/reference.md @@ -0,0 +1,9 @@ +--- +layout: default +title: Reference +has_children: true +nav_order: 6 +permalink: /reference/ +--- + +# Reference diff --git a/docs/reference/useful_links.md b/docs/reference/useful_links.md new file mode 100644 index 000000000..50e57cf77 --- /dev/null +++ b/docs/reference/useful_links.md @@ -0,0 +1,44 @@ +--- +layout: default +title: Useful links +nav_order: 2 +permalink: /reference/useful-links +parent: Reference +--- + +# Useful links + +- TOC +{:toc} +--- + +## Web Interface + +* The main page: [oss-fuzz.com](https://oss-fuzz.com) + +## Build Status + +* [This page](https://oss-fuzz-build-logs.storage.googleapis.com/index.html) + gives the latest build logs for each project. + +* (Internal only) [Builds dashboard](https://builder.oss-fuzz.com). + +## Blog posts + +* 2016-12-01 ([1](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html), +[2](https://testing.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html), +[3](https://security.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html)) +* 2017-05-08 ([1](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html), +[2](https://testing.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html), +[3](https://security.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html)) +* 2018-11-06 ([1](https://security.googleblog.com/2018/11/a-new-chapter-for-oss-fuzz.html)) + +## Tutorials + +* [libFuzzer documentation](http://libfuzzer.info) +* [libFuzzer tutorial](http://tutorial.libfuzzer.info) +* [libFuzzer workshop](https://github.com/Dor1s/libfuzzer-workshop) +* [Structure-Aware Fuzzing with libFuzzer](https://github.com/google/fuzzer-test-suite/blob/master/tutorial/structure-aware-fuzzing.md) +* [Chromium Fuzzing Page](https://chromium.googlesource.com/chromium/src/testing/libfuzzer/) +* [Chromium Efficient Fuzzing Guide](https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/HEAD/efficient_fuzzing.md) +* [ClusterFuzz documentation](https://google.github.io/clusterfuzz/) diff --git a/docs/reproducing.md b/docs/reproducing.md deleted file mode 100644 index 5611d9a4b..000000000 --- a/docs/reproducing.md +++ /dev/null @@ -1,95 +0,0 @@ -# Reproducing OSS-Fuzz issues - -You've been CC'ed on an OSS-Fuzz issue -([examples](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=Type%3ABug%2CBug-Security)), now what? -Before attempting to fix the bug, you should be able to reliably reproduce it. - -## Fuzz target bugs -Every issue has a [reproducer](glossary.md#reproducer) (aka "testcase") file attached. -Download it. If the issue is not public, you will need to login using your -[Google account](https://support.google.com/accounts/answer/176347?hl=en) -([why?](faq.md#why-do-you-require-a-google-account-for-authentication)) -that the bug report CCs. -This file contains the bytes that were fed to the [Fuzz Target](http://libfuzzer.info/#fuzz-target). - -If you have already [integrated](ideal_integration.md) the fuzz target with your build and test system, -all you do is run: -```bash -$ ./fuzz_target_binary <testcase_path> -``` - -If this is a timeout bug, add the <b><i>-timeout=25</i></b> argument.<br /> -If this is an OOM bug, add the <b><i>-rss_limit_mb=2048</i></b> argument.<br /> -Read more on how timeouts and OOMs are handed [here](faq.md#how-do-you-handle-timeouts-and-ooms). - -Depending on the nature of the bug, the fuzz target binary needs to be built with the appropriate [sanitizer](https://github.com/google/sanitizers) -(e.g. if this is a buffer overflow, with [AddressSanitizer](http://clang.llvm.org/docs/AddressSanitizer.html)). - -If you are not sure how to build the fuzzer using the project's build system, -you may also use Docker ([how?](installing_docker.md), [why?](faq.md#why-do-you-use-docker)) commands -to replicate the exact build steps used by OSS-Fuzz and then feed the reproducer input to the fuzz target. - -## Building using Docker - -### Pull the latest Docker images - -```bash -$ python infra/helper.py pull_images -``` - - Docker images get regularly updated with a newer version of build tools, build - configurations, scripts, and other changes. In some cases, a particular issue - can be reproduced only with a fresh image being used. - -### Build the image and the fuzzers - -```bash -$ python infra/helper.py build_image $PROJECT_NAME -$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME -``` - -## Reproducing bugs -```bash -$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path> -``` - - Find the type of sanitizer used in the report using the value in the **Sanitizer** column. It is one of the - following: - * **address** for AddressSanitizer - * **memory** for MemorySanitizer - * **undefined** for UndefinedBehaviorSanitizer - - E.g. for building [libxml2](../projects/libxml2) project with UndefinedBehaviorSanitizer (undefined) - instrumentation and reproduce a crash testcase for a fuzzer named `libxml2_xml_read_memory_fuzzer`, - it will be: - -```bash -$ python infra/helper.py build_image libxml2 -$ python infra/helper.py build_fuzzers --sanitizer undefined libxml2 -$ python infra/helper.py reproduce libxml2 libxml2_xml_read_memory_fuzzer ~/Downloads/testcase -``` - -## Reproduce using local source checkout - -```bash -$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME <source_path> -$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path> -``` - - This is essentially the previous command that additionally mounts local sources into the running container. - -- *Fix issue*. Write a patch to fix the issue in your local checkout and then use the previous command to verify the fix (i.e. no crash occurred). - [Use gdb](debugging.md#debugging-fuzzers-with-gdb) if needed. -- *Submit fix*. Submit the fix in the project's repository. ClusterFuzz will automatically pick up the changes, recheck the testcase and will close the issue (in < 1 day). -- *Improve fuzzing support*. Consider [improving fuzzing support](ideal_integration.md) in your project's build and test system. - -## Reproducing build failures -Our infrastructure runs some sanity tests to make sure that your build was correctly configured, even if it succeeded. To reproduce these locally, run: - -```bash -$ python infra/helper.py build_image $PROJECT_NAME -$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME -$ python infra/helper.py check_build --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME <fuzz_target_name> -``` - -For reproducing a `coverage` build failure, follow [Code Coverage page](code_coverage.md) to build your project and generate a code coverage report. |