| Age | Commit message (Collapse) | Author |
|---|
|
By using LLVMFuzzerRunDriver, we can remove the LibfuzzerDriver class
and replace it by:
* a real main function that preprocesses arguments for libFuzzer and
starts a JVM;
* everything else turned into a proper library with no dependency on our
custom JVM class.
This will allow us to turn everything except the main function into a
JNI shared library, opening up the possibility to launch Jazzer from an
already running JVM.
|
|
The native library path for this example got lost during the migration
to rules_jni and is required for the fuzz target to be functional.
Also updates rules_foreign_cc and cleans up the attributes passed to
the cmake rule. This includes adding -fPIC as a speculative fix for
relocation issues reported by the linker.
|
|
Speculative fix for the following externally reported issue:
```
INFO: Instrumented com....
java.lang.StackOverflowError
at org.jacoco.core.internal.instr.ProbeInserter.visitLocalVariableAnnotation(ProbeInserter.java:126)
at org.jacoco.core.internal.instr.ProbeInserter.visitLocalVariableAnnotation(ProbeInserter.java:126)
...
...
...
at org.jacoco.core.internal.instr.ProbeInserter.visitLocalVariableAnnotation(ProbeInserter.java:126)
INFO: Instrumented com....
```
|
|
This release contains a number of fixes to coverage collection for
multi-language targets.
A new ErrorProne check has to be disabled for the JaCoCo build to pass.
|
|
|
|
Switch from the internal fork to the official JaCoCo version. This
looses the call optimizations but removes the burden of maintaining a
dedicated fork.
Tests using the example fuzzers and JMH don't show huge performance
differences. Some are more in favor of the fork, some of the official
version.
|
|
If we do not shade JaCoCo internally but only when preparing the agent
deploy jar, Jazzer tests themselves can't be instrumented with JaCoCo as
our patched version is incompatible with the one used by coverage tools.
|
|
|
|
|
|
The OW2 GitLab hasn't been very reliable in the past and just
encountered another outage. Getting the ASM jars from Maven should be
more reliable.
|
|
We were using the flags from a standalone build script for libFuzzer,
but should instead try to use the same set of flags as an LLVM release
would use.
|
|
|
|
|
|
|
|
|
|
The fork of JaCoCo is compatible with the exec files generated by
Jazzer's dumpCoverage.
This also reduces the size of the patches maintained in this repo.
|
|
|
|
|
|
|
|
This simplifies the libjvm location logic as well as native library
packaging. Incidentally, this fixes the libjpeg_turbo build.
In anticipation of Windows support and because it simplifies further
improvements to the fuzz target test setup, the wrapper is rewritten in
Java.
|
|
These repositories are no longer how starlarkified rules will be
published in the near future.
|
|
Using the (very fast) classpath traverser ClassPath, we can generate
coverage data for *all* classes on the classpath rather than just those
that were loaded during the fuzzing run.
|
|
|
|
|
|
|
|
The toolchain is only enabled in the CI by default as users should use
the same compiler toolchain for compiling the Jazzer driver as they use
to compile their JNI libraries. However, if they are only interested in
fuzzing pure Java libraries, they can pass --config=ci on the CLI to use
the toolchain, which greatly simplifies the build on macOS.
A significant complication arises because the ASan runtime library can't
be linked statically on macOS. To make the tests pass, it needs to be
exported from the toolchain and the driver has to conditionally depend
on it explicitly.
A further patch to the toolchain is required to ensure compatibility
with Ubuntu 21.04.
|
|
libjvm lives in different subpaths of JAVA_HOME, depending both on the
OS and the Java version. Since it is currently not possible to select a
dependency based on the Java version, supporting Java 8 required a
custom build setting. This also broke bazel query (but not cquery).
By loading libjvm from a simple repository rule, we can cover all
OSes and Java versions with a single dependency, even if libjvm.so is
installed in a non-standard location.
|
|
Also moves the quote command args patch upstream.
|
|
libFuzzer does not quote the arguments it passes to child processes during merge
and fork, which leads to arguments being lost if passing multiple jvm_args with
delimiter ';'.
This commit adds a libFuzzer patch that properly quotes all arguments as well as
a test that fails if quoting is not appropriate.
|
|
Our libFuzzer fork has been updated as some of our patches have been
upstreamed. It now also includes the get-covered-pcs patch.
|
|
|
|
Use a Bazel provided target for the JNI headers instead of a custom one.
|
|
The new --coverage_report option triggers a coverage report to be
written on fuzzer exit.
The report is generated with the JaCoCo analyzer. The information about
observed coverage IDs is obtained from libFuzzer and combined with the
coverage obtained during fuzzerInitialize as well as the current run.
|
|
In order to make our coverage information usable by the analyzer, probe
events need to be emitted by the MethodProbesAdapter rather than the
ProbeInserter. This commit moves the events to that class and simplifies
the code in EdgeCoverageInstrumentor.
|
|
|
|
|
|
These have been integrated into the fork.
|
|
|
|
|
|
|
|
The JVM frequently calls strcmp/memcmp/..., which fills up the table of
recent compares with entries that are either duplicates of values
already reported by the bytecode instrumentation or JDK-internal strings
that are not relevant for fuzzing.
This commit adds an ignorelist to the C stdlib interceptors that filters
out calls from known JVM libraries. If the fuzz target has not yet
loaded a native library, all such callbacks are ignored, which greatly
improves fuzzer performance for string-heavy targets. E.g.,
JsonSanitizerDenylistFuzzer takes < 1 million runs now when it used to
take over 3 million.
|
|
|
|
Building libFuzzer from source is easy and has multiple advantages:
* The clang distributed with XCode on macOS does not include libFuzzer.
* Applying a small patch to libFuzzer will allow us to replace the
--wrap linker feature, which is not supported on platforms other than
Linux.
|
|
|
|
This reverts commit 71ac55c6fc9d808bcc8a8e8d895f7f20141bec86.
|
|
* Replace uses of quick_exit and at_quick_exit
quick_exit is not supported on macOS, but can easily replaced by a call
to _Exit after running our cleanup manually.
* Run buildifier --lint=fix -r .
* Build libFuzzer from source
Building libFuzzer from source is easy and has multiple advantages:
* The clang distributed with XCode on macOS does not include libFuzzer.
* Applying a small patch to libFuzzer will allow us to replace the
--wrap linker feature, which is not supported on platforms other than
Linux.
* Replace -Wl,--wrap with a source code patch
* Pin non-native rules_python
* Print exit code on test failure
* Do not intercept JVM-internal C stdlib calls
The JVM frequently calls strcmp/memcmp/..., which fills up the table of
recent compares with entries that are either duplicates of values
already reported by the bytecode instrumentation or JDK-internal strings
that are not relevant for fuzzing.
This commit adds an ignorelist to the C stdlib interceptors that filters
out calls from known JVM libraries. If the fuzz target has not yet
loaded a native library, all such callbacks are ignored, which greatly
improves fuzzer performance for string-heavy targets. E.g.,
JsonSanitizerDenylistFuzzer takes < 1 million runs now when it used to
take over 3 million.
|
|
We are currently deriving edge coverage instrumentation from basic block
instrumentation via the AFL XOR-technique. This has several downsides:
* Different edges can be assigned the same position in the coverage map,
which leads to underreported coverage.
* The coverage map needs to be large enough for collisions to be
unlikely (on the order of num_edges^2). In addition to being wasteful,
it is also hard to determine the correct size given that we don't know
the number of edges.
In addition to the design limitations, the current implementation
additionally does not take into account that most Java method
invocations can throw exceptions and thus need to be instrumented.
These issues are resolved by switching to true LLVM-style edge coverage
instrumentation. The new coverage instrumentation is based on a lightly
patched version of the JaCoCo internals.
Note:
//agent/src/test/java/com/code_intelligence/jazzer/instrumentor:coverage_instrumentation_test
is not passing for this commit. It will be fixed with the next commit.
|
|
|
