# Debugging If you are using FRIDA mode and have hit some problems, then this guide may help you to diagnose any problems you are encountering. This assumes you have followed the [osx-lib](test/osx-lib) example to start fuzzing your target. It should be noted that attempting to debug code using gdb which has been instrumented in FRIDA is unlikely to be successful since the debugger will be inserting breakpoints by patching the code in memory. FRIDA works by reading this code and generating an instrumented copy to execute. In any case, unless you are very familiar with the implementation of Stalker, the instrumented code generated by FRIDA is likely to be very difficult to follow. For this reason, the following debugging strategies are outlined below. By convention, all files below should be provided with their path (they are omitted for readability) and all items in `` are placeholders and should be replaced accordingly. ## Select your version Test with both the `dev` and `stable` branches of AFL++. The `dev` branch should have the very latest version containing any fixes for identified issues. The `stable` branch is updated less frequently, but equally might avoid a problem if a regression has been introduced into the `dev` branch. ## Enable diagnostic information Run your target specifying the `AFL_DEBUG_CHILD=1` environment variable. This will print a lot more diagnostic information to the screen when the target starts up. If you have a simple configuration issue, then you will likely see a warning or error message in the output. ## Check your test harness If any of the following steps fail, then there is a problem with your test harness or your target library. Since this is running without FRIDA mode or `afl-fuzz` that greatly reduces the search area for your defect. This is why it is *VERY* important to carry out these basic steps first before taking on the additional complexity of debugging with FRIDA mode or `afl-fuzz`. - Run your harness outside of the fuzzer, passing it a representative seed as it's input `./harness `. - Pass your harness multiple seeds to check that it is stable when running multiple tests as it will when running in fork server mode `./harness `. - Build your test harness with `CFLAGS=-fsanitize=address` and `LDFLAGS=-fsanitize=address`. Then run it again with multiple inputs to check for errors (note that when fuzzing, your harness should not be built with any sanitizer options). ## Check the samples FRIDA mode contains a number of different sample targets in the `test` folder. Have a look through these and find one which is similar to your real target. Check whether you have any issues running the sample target and make sure you compare the command line used to launch the sample with the one you are using to launch your real target very carefully to check for any differences. If possible, start with one of these samples and gradually make changes one at a time re-testing as you go until you have migrated it to run your own target. ## FRIDA mode ### Basic First, just try running your target with `LD_PRELOAD=afl-frida-trace.so ./harness `. An error here means that your defect occurs when running with just FRIDA mode and isn't related to `afl-fuzz`. Now you can try commenting out the implementation of `LLVMFuzzerTestOneInput` so that the harness doesn't actually run your target library. This may also aid in narrowing down the problem. ```c int LLVMFuzzerTestOneInput(const unsigned char* data, size_t size){ // fpn_crashme(data, size); return 0; } ``` ### Persistent mode If your target is ok running in basic mode, you can try running it in persistent mode (if that is the configuration you are having issues with) as follows (again outside of `afl-fuzz`). This time, you will want to run it inside a debugger so that you can use the debugger to send the `SIGCONT` signals (by continuing) usually sent by `afl-fuzz` on each iteration. ```bash gdb \ --ex 'set environment __AFL_PERSISTENT=1' \ --ex 'set environment AFL_FRIDA_PERSISTENT_CNT=3' \ --ex 'set environment LD_PRELOAD=afl-frida-trace.so' \ --ex 'set environment AFL_FRIDA_PERSISTENT_ADDR=' \ --args ./harness ``` Note: - You have to manually set the `__AFL_PERSISTENT` environment variable which is usually passed by `afl-fuzz`. - Setting breakpoints etc. is likely to interfere with FRIDA and cause spurious errors. If this is successful, you can try additionally loading the hook library: ```bash gdb \ --ex 'set environment __AFL_PERSISTENT=1' \ --ex 'set environment AFL_FRIDA_PERSISTENT_CNT=3' \ --ex 'set environment LD_PRELOAD=afl-frida-trace.so' \ --ex 'set environment AFL_FRIDA_PERSISTENT_ADDR=' \ --ex 'set environment AFL_FRIDA_PERSISTENT_HOOK=frida_hook.so' --args ./harness ``` Note that the format of the hook used for FRIDA mode is subtly different to that used when running in QEMU mode as shown below. Thus the DSO used for the hook is not interchangeable. ```c void afl_persistent_hook(GumCpuContext *regs, uint8_t *input_buf, uint32_t input_buf_len); void afl_persistent_hook(struct x86_64_regs *regs, uint64_t guest_base, uint8_t *input_buf, uint32_t input_buf_len); ``` ### ASAN It is also possible to enable ASAN (if that is the configuration you are having issues with) without having to use `afl-fuzz`. This can be done as follows: Note: - The name of the asan DSO may need to be changed depending on your platform. - The asan DSO must appear first in the `LD_PRELOAD` environment variable. ```bash LD_PRELOAD=libclang_rt.asan-x86_64.so:afl-frida-trace.so \ ASAN_OPTIONS=detect_leaks=false,halt_on_error=0 \ AFL_USE_FASAN=1 \ ./harness ``` Note that care should be taken to ensure that if you set `AFL_INST_LIBS=1`, you use `AFL_FRIDA_INST_RANGES` or `AFL_FRIDA_EXCLUDE_RANGES` to exclude the ASAN DSO from coverage. Failure to do so will result in ASAN attempting to sanitize itself and as a result detecting failures when it attempts to update the shadow maps. ## Printf If you have an idea of where things are going wrong for you, then don't be scared to add `printf` statements to either AFL++ or FRIDA mode itself to show more diagnostic information. Just be sure to set `AFL_DEBUG=1` and `AFL_DEBUG_CHILD=1` when you are testing it. ## Core dumps Lastly, if your defect only occurs when using `afl-fuzz` (e.g., when using `CMPLOG` which cannot be tested outside of `afl-fuzz` due to its need for a shared memory mapping being created for it to record its data), it is possible to enable the creation of a core dump for post-mortem analysis. Firstly, check if your `/proc/sys/kernel/core_pattern` configuration is set to a filename (AFL++ encourages you to set it to the value `core` in any case since it doesn't want any handler applications getting in the way). Next, set `ulimit -c unlimited` to remove any size limitations for core files. Lastly, when you `afl-fuzz`, set the environment variable `AFL_DEBUG=1` to enable the creation of the `core` file. The file should be created in the working directory of the target application. If there is an existing `core` file already there, then it may not be overwritten. ## Reach out Get in touch on discord and ask for help. The groups are pretty active so someone may well be able to offer some advice. Better still, if you are able to create a minimal reproducer for your problem, it will make it easier to diagnose the issue.