feat(fuzz): Adding cargo-fuzz targets for load_file and parse_command.

[estr3llas]

The load_file target fuzzes:

let mut app = App::new();
let _ = app.load_file(path, 0, flags & 1 == 0); //path = path to temporary file containing the fuzz data

In which the readonly parameter is randomly chosen.

The parse_command target fuzzes:

let mut app = App::new();
let _ = parse_command(&mut app, s); //s = fuzz data

Additionally- A dictionary of valid dz6 commands can be found in fuzz/dict/commands.dict

The dz6 crate is not reachable from outside the crate itself, so I had to make a shim in fuzz/src/lib.rs, future work would be to add a lib.rs under the dz6 crate so it can be accessed from other consumer crates.

Both targets builds cleanly:

parse_command:

xiexie@ubuntu:~/dz6/fuzz$ cargo fuzz run -s none commands_parse_command -- -dict=dict/commands.dict -runs=1000
    Finished `release` profile [optimized + debuginfo] target(s) in 0.08s
    Finished `release` profile [optimized + debuginfo] target(s) in 0.07s
     Running `target/x86_64-unknown-linux-gnu/release/commands_parse_command -artifact_prefix=/home/xiexie/dz6/fuzz/artifacts/commands_parse_command/ -dict=dict/commands.dict -runs=1000 /home/xiexie/dz6/fuzz/corpus/commands_parse_command`
WARNING: Failed to find function "__sanitizer_acquire_crash_state".
WARNING: Failed to find function "__sanitizer_print_stack_trace".
WARNING: Failed to find function "__sanitizer_set_death_callback".
Dictionary: 48 entries
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 608063619
INFO: Loaded 1 modules   (25861 inline 8-bit counters): 25861 [0x56b74be62a08, 0x56b74be68f0d),
INFO: Loaded 1 PC tables (25861 PCs): 25861 [0x56b74be68f10,0x56b74becdf60),
INFO:      593 files found in /home/xiexie/dz6/fuzz/corpus/commands_parse_command
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 593 min: 1b max: 140b total: 11111b rss: 52Mb
#594    INITED cov: 3131 ft: 5434 corp: 396/8139b exec/s: 0 rss: 52Mb
#597    NEW    cov: 3131 ft: 5435 corp: 397/8252b lim: 140 exec/s: 0 rss: 52Mb L: 113/140 MS: 3 InsertByte-ChangeByte-InsertRepeatedBytes-
        NEW_FUNC[1/1]: 0x56b74bca52c1
#688    NEW    cov: 3134 ft: 5438 corp: 398/8269b lim: 140 exec/s: 0 rss: 52Mb L: 17/140 MS: 1 ManualDict- DE: "set ctrlchar \" \""-
#894    REDUCE cov: 3134 ft: 5438 corp: 398/8266b lim: 140 exec/s: 0 rss: 52Mb L: 7/140 MS: 1 EraseBytes-
#939    NEW    cov: 3135 ft: 5439 corp: 399/8288b lim: 140 exec/s: 0 rss: 52Mb L: 22/140 MS: 5 ManualDict-ManualDict-EraseBytes-ChangeBit-PersAutoDict- DE: "-h"-"theme"-"set ctrlchar \" \""-
#970    NEW    cov: 3136 ft: 5440 corp: 400/8305b lim: 140 exec/s: 0 rss: 52Mb L: 17/140 MS: 1 ManualDict- DE: "\\"-
#972    NEW    cov: 3136 ft: 5441 corp: 401/8314b lim: 140 exec/s: 0 rss: 52Mb L: 9/140 MS: 2 ManualDict-CMP- DE: "ctrlchar"-"r\000\000\000"-
#1000   DONE   cov: 3136 ft: 5441 corp: 401/8314b lim: 140 exec/s: 0 rss: 52Mb
###### Recommended dictionary. ######
"r\000\000\000" # Uses: 0
###### End of recommended dictionary. ######
Done 1000 runs in 0 second(s)

load_file:

xiexie@ubuntu:~/dz6/fuzz$ cargo fuzz run -s none app_load_file -- -runs=1000
   Compiling dz6-fuzz v0.0.0 (/home/xiexie/dz6/fuzz)
    Finished `release` profile [optimized + debuginfo] target(s) in 1.50s
    Finished `release` profile [optimized + debuginfo] target(s) in 0.08s
     Running `target/x86_64-unknown-linux-gnu/release/app_load_file -artifact_prefix=/home/xiexie/dz6/fuzz/artifacts/app_load_file/ -runs=1000 /home/xiexie/dz6/fuzz/corpus/app_load_file`
WARNING: Failed to find function "__sanitizer_acquire_crash_state".
WARNING: Failed to find function "__sanitizer_print_stack_trace".
WARNING: Failed to find function "__sanitizer_set_death_callback".
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 909058306
INFO: Loaded 1 modules   (21431 inline 8-bit counters): 21431 [0x562015bf9b08, 0x562015bfeebf),
INFO: Loaded 1 PC tables (21431 PCs): 21431 [0x562015bfeec0,0x562015c52a30),
INFO:       37 files found in /home/xiexie/dz6/fuzz/corpus/app_load_file
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 37 min: 1b max: 27b total: 217b rss: 52Mb
#38     INITED cov: 750 ft: 808 corp: 8/29b exec/s: 0 rss: 52Mb
#40     NEW    cov: 750 ft: 810 corp: 9/31b lim: 17 exec/s: 0 rss: 52Mb L: 2/17 MS: 2 InsertByte-CopyPart-
#56     REDUCE cov: 750 ft: 810 corp: 9/30b lim: 17 exec/s: 0 rss: 52Mb L: 1/17 MS: 1 EraseBytes-
#59     NEW    cov: 750 ft: 828 corp: 10/31b lim: 17 exec/s: 0 rss: 52Mb L: 1/17 MS: 3 ShuffleBytes-ShuffleBytes-ChangeBit-
#99     NEW    cov: 750 ft: 829 corp: 11/39b lim: 17 exec/s: 0 rss: 52Mb L: 8/17 MS: 5 CopyPart-CopyPart-InsertByte-InsertByte-ChangeBinInt-
#148    NEW    cov: 750 ft: 830 corp: 12/44b lim: 17 exec/s: 0 rss: 52Mb L: 5/17 MS: 4 ChangeByte-ChangeByte-InsertByte-ChangeBinInt-
#167    NEW    cov: 750 ft: 839 corp: 13/46b lim: 17 exec/s: 0 rss: 52Mb L: 2/17 MS: 4 ChangeByte-CopyPart-InsertByte-ChangeByte-
#247    REDUCE cov: 750 ft: 839 corp: 13/41b lim: 17 exec/s: 0 rss: 52Mb L: 3/17 MS: 5 EraseBytes-CopyPart-EraseBytes-ChangeByte-ShuffleBytes-
#269    NEW    cov: 752 ft: 842 corp: 14/43b lim: 17 exec/s: 0 rss: 52Mb L: 2/17 MS: 2 CopyPart-ChangeByte-
#332    NEW    cov: 753 ft: 843 corp: 15/44b lim: 17 exec/s: 0 rss: 52Mb L: 1/17 MS: 3 ShuffleBytes-CopyPart-ChangeBit-
#534    NEW    cov: 756 ft: 846 corp: 16/47b lim: 17 exec/s: 0 rss: 52Mb L: 3/17 MS: 2 CopyPart-ChangeBinInt-
#593    NEW    cov: 756 ft: 849 corp: 17/62b lim: 17 exec/s: 0 rss: 52Mb L: 15/17 MS: 4 ShuffleBytes-InsertByte-EraseBytes-InsertRepeatedBytes-
#602    REDUCE cov: 756 ft: 853 corp: 18/64b lim: 17 exec/s: 0 rss: 52Mb L: 2/17 MS: 4 ChangeBit-ChangeBit-ShuffleBytes-CopyPart-
#678    NEW    cov: 757 ft: 855 corp: 19/66b lim: 17 exec/s: 0 rss: 52Mb L: 2/17 MS: 1 CrossOver-
#1000   DONE   cov: 757 ft: 855 corp: 19/66b lim: 17 exec/s: 0 rss: 52Mb
Done 1000 runs in 0 second(s)

Cargo.lock is just cp'ed from dz6/Cargo.lock. Once (or if) dz6 becomes a library, it can be deleted.

[estr3llas]

Fuzzing documentation mostly follows pest's fuzzing documentation structure.

[merces]

Looks good! I didn't forget about this. I'll arrange some time to read it more carefully and to test it either tomorrow or next weekend. Thank you!

[merces]

I had to run cargo +nightly fuzz run app_load_file since nightly is not my default toolchain, but I'm currently traveling and it drained my laptop battery lol. Will run more tests later. Good stuff. :)

Out of curiosity @estr3llas , why does this need the nightly toolchain?

[estr3llas]

Hey @merces, I am sorry for the (huge) delay. Cargo fuzz relies on libFuzzer and its sanitizers to instrument and fuzz the SUT. As you may be aware,libFuzzer is not in active development since late 2022 (~October), also, Rust support to sanitizers such as AddressSanitizer is not in a stable state yet (I'd guess due to Rust's own unstable ABI), this combination of factors stops cargo fuzz from being incorporated into stable Rust.