To install headerpwn
, run the following command:
go install github.com/devanshbatham/headerpwn@v0.0.3
headerpwn allows you to test various headers on a target URL and analyze the responses. Here's how to use the tool:
-url
flag.-headers
flag to specify the path to this file.Example usage:
headerpwn -url https://example.com -headers my_headers.txt
my_headers.txt
should be like below:Proxy-Authenticate: foobar
Proxy-Authentication-Required: foobar
Proxy-Authorization: foobar
Proxy-Connection: foobar
Proxy-Host: foobar
Proxy-Http: foobar
Follow following steps to proxy requests through Burp Suite:
Export Burp's Certificate:
127.0.0.1:8080
Install Burp's Certificate:
You should be all set:
headerpwn -url https://example.com -headers my_headers.txt -proxy 127.0.0.1:8080
The headers.txt
file is compiled from various sources, including the SecLists">Seclists project. These headers are used for testing purposes and provide a variety of scenarios for analyzing how servers respond to different headers.
The original 403fuzzer.py :)
Fuzz 401/403ing endpoints for bypasses
This tool performs various checks via headers, path normalization, verbs, etc. to attempt to bypass ACL's or URL validation.
It will output the response codes and length for each request, in a nicely organized, color coded way so things are reaable.
I implemented a "Smart Filter" that lets you mute responses that look the same after a certain number of times.
You can now feed it raw HTTP requests that you save to a file from Burp.
usage: bypassfuzzer.py -h
Simply paste the request into a file and run the script!
- It will parse and use cookies
& headers
from the request. - Easiest way to authenticate for your requests
python3 bypassfuzzer.py -r request.txt
Specify a URL
python3 bypassfuzzer.py -u http://example.com/test1/test2/test3/forbidden.html
Specify cookies to use in requests:
some examples:
--cookies "cookie1=blah"
-c "cookie1=blah; cookie2=blah"
Specify a method/verb and body data to send
bypassfuzzer.py -u https://example.com/forbidden -m POST -d "param1=blah¶m2=blah2"
bypassfuzzer.py -u https://example.com/forbidden -m PUT -d "param1=blah¶m2=blah2"
Specify custom headers to use with every request Maybe you need to add some kind of auth header like Authorization: bearer <token>
Specify -H "header: value"
for each additional header you'd like to add:
bypassfuzzer.py -u https://example.com/forbidden -H "Some-Header: blah" -H "Authorization: Bearer 1234567"
Based on response code and length. If it sees a response 8 times or more it will automatically mute it.
Repeats are changeable in the code until I add an option to specify it in flag
NOTE: Can't be used simultaneously with -hc
or -hl
(yet)
# toggle smart filter on
bypassfuzzer.py -u https://example.com/forbidden --smart
Useful if you wanna proxy through Burp
bypassfuzzer.py -u https://example.com/forbidden --proxy http://127.0.0.1:8080
# skip sending headers payloads
bypassfuzzer.py -u https://example.com/forbidden -sh
bypassfuzzer.py -u https://example.com/forbidden --skip-headers
# Skip sending path normailization payloads
bypassfuzzer.py -u https://example.com/forbidden -su
bypassfuzzer.py -u https://example.com/forbidden --skip-urls
Provide comma delimited lists without spaces. Examples:
# Hide response codes
bypassfuzzer.py -u https://example.com/forbidden -hc 403,404,400
# Hide response lengths of 638
bypassfuzzer.py -u https://example.com/forbidden -hl 638
This is the companion code for the paper: 'Fuzzing Embedded Systems using Debugger Interfaces'. A preprint of the paper can be found here https://publications.cispa.saarland/3950/. The code allows the users to reproduce and extend the results reported in the paper. Please cite the above paper when reporting, reproducing or extending the results.
.
โโโ benchmark # Scripts to build Google's fuzzer test suite and run experiments
โโโ dependencies # Contains a Makefile to install dependencies for GDBFuzz
โโโ evaluation # Raw exeriment data, presented in the paper
โโโ example_firmware # Embedded example applications, used for the evaluation
โโโ example_programs # Contains a compiled example program and configs to test GDBFuzz
โโโ src # Contains the implementation of GDBFuzz
โโโ Dockerfile # For creating a Docker image with all GDBFuzz dependencies installed
โโโ LICENSE # License
โโโ Makefile # Makefile for creating the docker image or install GDBFuzz locally
โโโ README.md # This README file
The idea of GDBFuzz is to leverage hardware breakpoints from microcontrollers as feedback for coverage-guided fuzzing. Therefore, GDB is used as a generic interface to enable broad applicability. For binary analysis of the firmware, Ghidra is used. The code contains a benchmark setup for evaluating the method. Additionally, example firmware files are included.
GDBFuzz enables coverage-guided fuzzing for embedded systems, but - for evaluation purposes - can also fuzz arbitrary user applications. For fuzzing on microcontrollers we recommend a local installation of GDBFuzz to be able to send fuzz data to the device under test flawlessly.
GDBFuzz has been tested on Ubuntu 20.04 LTS and Raspberry Pie OS 32-bit. Prerequisites are java and python3. First, create a new virtual environment and install all dependencies.
virtualenv .venv
source .venv/bin/activate
make
chmod a+x ./src/GDBFuzz/main.py
GDBFuzz reads settings from a config file with the following keys.
[SUT]
# Path to the binary file of the SUT.
# This can, for example, be an .elf file or a .bin file.
binary_file_path = <path>
# Address of the root node of the CFG.
# Breakpoints are placed at nodes of this CFG.
# e.g. 'LLVMFuzzerTestOneInput' or 'main'
entrypoint = <entrypoint>
# Number of inputs that must be executed without a breakpoint hit until
# breakpoints are rotated.
until_rotate_breakpoints = <number>
# Maximum number of breakpoints that can be placed at any given time.
max_breakpoints = <number>
# Blacklist functions that shall be ignored.
# ignore_functions is a space separated list of function names e.g. 'malloc free'.
ignore_functions = <space separated list>
# One of {Hardware, QEMU, SUTRunsOnHost}
# Hardware: An external component starts a gdb server and GDBFuzz can connect to this gdb server.
# QEMU: GDBFuzz starts QEMU. QEMU emulates binary_file_path and starts gdbserver.
# SUTRunsOnHost: GDBFuzz start the target program within GDB.
target_mode = <mode>
# Set this to False if you want to start ghidra, analyze the SUT,
# and start the ghidra bridge server manually.
start_ghidra = True
# Space separated list of addresses where software breakpoints (for error
# handling code) are set. Execution of those is considered a crash.
# Example: software_breakpoint_addresses = 0x123 0x432
software_breakpoint_addresses =
# Whether all triggered software breakpoints are considered as crash
consider_sw_breakpoint_as_error = False
[SUTConnection]
# The class 'SUT_connection_class' in file 'SUT_connection_path' implements
# how inputs are sent to the SUT.
# Inputs can, for example, be sent over Wi-Fi, Serial, Bluetooth, ...
# This class must inherit from ./connections/SUTConnection.py.
# See ./connections/SUTConnection.py for more information.
SUT_connection_file = FIFOConnection.py
[GDB]
path_to_gdb = gdb-multiarch
#Written in address:port
gdb_server_address = localhost:4242
[Fuzzer]
# In Bytes
maximum_input_length = 100000
# In seconds
single_run_timeout = 20
# In seconds
total_runtime = 3600
# Optional
# Path to a directory where each file contains one seed. If you don't want to
# use seeds, leave the value empty.
seeds_directory =
[BreakpointStrategy]
# Strategies to choose basic blocks are located in
# 'src/GDBFuzz/breakpoint_strategies/'
# For the paper we use the following strategies
# 'RandomBasicBlockStrategy.py' - Randomly choosing unreached basic blocks
# 'RandomBasicBlockNoDomStrategy.py' - Like previous, but doesn't use dominance relations to derive transitively reached nodes.
# 'RandomBasicBlockNoCorpusStrategy.py' - Like first, but prevents growing the input corpus and therefore behaves like blackbox fuzzing with coverage measurement.
# 'BlackboxStrategy.py', - Doesn't set any breakpoints
breakpoint_strategy_file = RandomBasicBlockStrategy.py
[Dependencies]
path_to_qemu = dependencies/qemu/build/x86_64-linux-user/qemu-x86_64
path_to_ghidra = dependencies/ghidra
[LogsAndVisualizations]
# One of {DEBUG, INFO, WARNING, ERROR, CRITICAL}
loglevel = INFO
# Path to a directory where output files (e.g. graphs, logfiles) are stored.
output_directory = ./output
# If set to True, an MQTT client sends UI elements (e.g. graphs)
enable_UI = False
An example config file is located in ./example_programs/
together with an example program that was compiled using our fuzzing harness in benchmark/benchSUTs/GDBFuzz_wrapper/common/
. Start fuzzing for one hour with the following command.
chmod a+x ./example_programs/json-2017-02-12
./src/GDBFuzz/main.py --config ./example_programs/fuzz_json.cfg
We first see output from Ghidra analyzing the binary executable and susequently messages when breakpoints are relocated or hit.
Depending on the specified output_directory
in the config file, there should now be a folder trial-0
with the following structure
.
โโโ corpus # A folder that contains the input corpus.
โโโ crashes # A folder that contains crashing inputs - if any.
โโโ cfg # The control flow graph as adjacency list.
โโโ fuzzer_stats # Statistics of the fuzzing campaign.
โโโ plot_data # Table showing at which relative time in the fuzzing campaign which basic block was reached.
โโโ reverse_cfg # The reverse control flow graph.
By setting start_ghidra = False
in the config file, GDBFuzz connects to a Ghidra instance running in GUI mode. Therefore, the ghidra_bridge plugin needs to be started manually from the script manager. During fuzzing, reached program blocks are highlighted in green.
For fuzzing on Linux user applications, GDBFuzz leverages the standard LLVMFuzzOneInput
entrypoint that is used by almost all fuzzers like AFL, AFL++, libFuzzer,.... In benchmark/benchSUTs/GDBFuzz_wrapper/common
There is a wrapper that can be used to compile any compliant fuzz harness into a standalone program that fetches input via a named pipe at /tmp/fromGDBFuzz
. This allows to simulate an embedded device that consumes data via a well defined input interface and therefore run GDBFuzz on any application. For convenience we created a script in benchmark/benchSUTs
that compiles all programs from our evaluation with our wrapper as explained later.
NOTE: GDBFuzz is not intended to fuzz Linux user applications. Use AFL++ or other fuzzers therefore. The wrapper just exists for evaluation purposes to enable running benchmarks and comparisons on a scale!
The general effectiveness of our approach is shown in a large scale benchmark deployed as docker containers.
make dockerimage
To run the above experiment in the docker container (for one hour as specified in the config file), map the example_programs
and output
folder as volumes and start GDBFuzz as follows.
chmod a+x ./example_programs/json-2017-02-12
docker run -it --env CONFIG_FILE=/example_programs/fuzz_json_docker_qemu.cfg -v $(pwd)/example_programs:/example_programs -v $(pwd)/output:/output gdbfuzz:1.0
An output folder should appear in the current working directory with the structure explained above.
Our evaluation is split in two parts. 1. GDBFuzz on its intended setup, directly on the hardware. 2. GDBFuzz in an emulated environment to allow independend analysis and comparisons of the results.
GDBFuzz can work with any GDB server and therefore most debug probes for microcontrollers.
Regarding RQ1 from the paper, we execute GDBFuzz on different microcontrollers with different firmwares located in example_firmware
. For each experiment we run GDBFuzz with the RandomBasicBlock
and with the RandomBasicBlockNoCorpus
strategy. The latter behaves like fuzzing without feedback, but we can still measure the achieved coverage. For answering RQ1, we compare the achieved coverage of the RandomBasicBlock
and the RandomBasicBlockNoCorpus
strategy. Respective config files are in the corresponding subfolders and we now explain how to setup fuzzing on the four development boards.
GDBFuzz requires access to a GDB Server. In this case the B-L4S5I-IOT01A and its on-board debugger are used. This on-board debugger sets up a GDB server via the 'st-util' program, and enables access to this GDB server via localhost:4242.
sudo apt-get install stlink-tools gdb-multiarch
Build and flash a firmware for the STM32 B-L4S5I-IOT01A, for example the arduinojson project.
Prerequisite: Install platformio (pio)
cd ./example_firmware/stm32_disco_arduinojson/
pio run --target upload
For your info: platformio stored an .elf file of the SUT here: ./example_firmware/stm32_disco_arduinojson/.pio/build/disco_l4s5i_iot01a/firmware.elf
This .elf file is also later used in the user configuration for Ghidra.
Start a new terminal, and run the following to start the a GDB Server:
st-util
Run GDBFuzz with a user configuration for arduinojson. We can send data over the usb port to the microcontroller. The microcontroller forwards this data via serial to the SUT'. In our case /dev/ttyACM0
is the USB device to the microcontroller board. If your system assigned another device to the microcontroller board, change /dev/ttyACM0
in the config file to your device.
./src/GDBFuzz/main.py --config ./example_firmware/stm32_disco_arduinojson/fuzz_serial_json.cfg
Fuzzer statistics and logs are in the ./output/... directory.
Install pyocd
:
pip install --upgrade pip 'mbed-ls>=1.7.1' 'pyocd>=0.16'
Make sure that 'KitProg v3' is on the device and put Board into 'Arm DAPLink' Mode by pressing the appropriate button. Start the GDB server:
pyocd gdbserver --persist
Flash a firmware and start fuzzing e.g. with
gdb-multiarch
target remote :3333
load ./example_firmware/CY8CKIT_json/mtb-example-psoc6-uart-transmit-receive.elf
monitor reset
./src/GDBFuzz/main.py --config ./example_firmware/CY8CKIT_json/fuzz_serial_json.cfg
Build and flash a firmware for the ESP32, for instance the arduinojson example with platformio.
cd ./example_firmware/esp32_arduinojson/
pio run --target upload
Add following line to the openocd config file for the J-Link debugger: jlink.cfg
adapter speed 10000
Start a new terminal, and run the following to start the GDB Server:
get_idf
openocd -f interface/jlink.cfg -f target/esp32.cfg -c "telnet_port 7777" -c "gdb_port 8888"
Run GDBFuzz with a user configuration for arduinojson. We can send data over the usb port to the microcontroller. The microcontroller forwards this data via serial to the SUT'. In our case /dev/ttyUSB0
is the USB device to the microcontroller board. If your system assigned another device to the microcontroller board, change /dev/ttyUSB0
in the config file to your device.
./src/GDBFuzz/main.py --config ./example_firmware/esp32_arduinojson/fuzz_serial.cfg
Fuzzer statistics and logs are in the ./output/... directory.
Install TI MSP430 GCC from https://www.ti.com/tool/MSP430-GCC-OPENSOURCE
Start GDB Server
./gdb_agent_console libmsp430.so
or (more stable). Build mspdebug from https://github.com/dlbeer/mspdebug/ and use:
until mspdebug --fet-skip-close --force-reset tilib "opt gdb_loop True" gdb ; do sleep 1 ; done
Ghidra fails to analyze binaries for the TI MSP430 controller out of the box. To fix that, we import the file in the Ghidra GUI, choose MSP430X as architecture and skip the auto analysis. Next, we open the 'Symbol Table', sort them by name and delete all symbols with names like $C$L*
. Now the auto analysis can be executed. After analysis, start the ghidra bridge from the Ghidra GUI manually and then start GDBFuzz.
./src/GDBFuzz/main.py --config ./example_firmware/msp430_arduinojson/fuzz_serial.cfg
To access USB devices as non-root user with pyusb
we add appropriate rules to udev. Paste following lines to /etc/udev/rules.d/50-myusb.rules
:
SUBSYSTEM=="usb", ATTRS{idVendor}=="1234", ATTRS{idProduct}=="5678" GROUP="usbusers", MODE="666"
Reload udev:
sudo udevadm control --reload
sudo udevadm trigger
In RQ2 from the paper, we compare GDBFuzz against the emulation based approach Fuzzware. First we execute GDBFuzz and Fuzzware as described previously on the shipped firmware files. For each GDBFuzz experiment, we create a file with valid basic blocks from the control flow graph files as follows:
cut -d " " -f1 ./cfg > valid_bbs.txt
Now we can replay coverage against fuzzware result fuzzware genstats --valid-bb-file valid_bbs.txt
When crashing or hanging inputs are found, the are stored in the crashes
folder. During evaluation, we found the following three bugs:
GDBFuzz can also run on a Raspberry Pi host with slight modifications:
In file ./dependencies/ghidra/support/launch.sh:125
The JAVA_HOME
variable must be hardcoded therefore e.g. to JAVA_HOME="/usr/lib/jvm/default-java"
To fuzz software on other boards, GDBFuzz requires
src/GDBFuzz/connections
) that triggers execution of the code at the entry point e.g. serial connectionAll these properties need to be specified in the config file.
For RQ's 4 - 8 we run a large scale benchmark. First, build the Docker image as described previously and compile applications from Google's Fuzzer Test Suite with our fuzzing harness in benchmark/benchSUTs/GDBFuzz_wrapper/common
.
cd ./benchmark/benchSUTs
chmod a+x setup_benchmark_SUTs.py
make dockerbenchmarkimage
Next adopt the benchmark settings in benchmark/scripts/benchmark.py
and benchmark/scripts/benchmark_aflpp.py
to your demands (especially number_of_cores
, trials
, and seconds_per_trial
) and start the benchmark with:
cd ./benchmark/scripts
./benchmark.py $(pwd)/../benchSUTs/SUTs/ SUTs.json
./benchmark_aflpp.py $(pwd)/../benchSUTs/SUTs/ SUTs.json
A folder appears in ./benchmark/scripts
that contains plot files (coverage over time), fuzzer statistic files, and control flow graph files for each experiment as in evaluation/fuzzer_test_suite_qemu_runs
.
GDBFuzz has an optional feature where it plots the control flow graph of covered nodes. This is disabled by default. You can enable it by following the instructions of this section and setting 'enable_UI' to 'True' in the user configuration.
On the host:
Install
sudo apt-get install graphviz
Install a recent version of node, for example Option 2 from here. Use Option 2 and not option 1. This should install both node and npm. For reference, our version numbers are (but newer versions should work too):
โ node --version
v16.9.1
โ npm --version
7.21.1
Install web UI dependencies:
cd ./src/webui
npm install
Install mosquitto MQTT broker, e.g. see here
Update the mosquitto broker config: Replace the file /etc/mosquitto/conf.d/mosquitto.conf with the following content:
listener 1883
allow_anonymous true
listener 9001
protocol websockets
Restart the mosquitto broker:
sudo service mosquitto restart
Check that the mosquitto broker is running:
sudo service mosquitto status
The output should include the text 'Active: active (running)'
Start the web UI:
cd ./src/webui
npm start
Your web browser should open automatically on 'http://localhost:3000/'.
Start GDBFuzz and use a user config file where enable_UI is set to True. You can use the Docker container and arduinojson SUT from above. But make sure to set 'enable_UI' to 'True'.
The nodes covered in 'blue' are covered. White nodes are not covered. We only show uncovered nodes if their parent is covered (drawing the complete control flow graph takes too much time if the control flow graph is large).
Multi-cloud OSINT tool. Enumerate public resources in AWS, Azure, and Google Cloud.
Currently enumerates the following:
Amazon Web Services: - Open / Protected S3 Buckets - awsapps (WorkMail, WorkDocs, Connect, etc.)
Microsoft Azure: - Storage Accounts - Open Blob Storage Containers - Hosted Databases - Virtual Machines - Web Apps
Google Cloud Platform - Open / Protected GCP Buckets - Open / Protected Firebase Realtime Databases - Google App Engine sites - Cloud Functions (enumerates project/regions with existing functions, then brute forces actual function names) - Open Firebase Apps
See it in action in Codingo's video demo here.
Several non-standard libaries are required to support threaded HTTP requests and dns lookups. You'll need to install the requirements as follows:
pip3 install -r ./requirements.txt
The only required argument is at least one keyword. You can use the built-in fuzzing strings, but you will get better results if you supply your own with -m
and/or -b
.
You can provide multiple keywords by specifying the -k
argument multiple times.
Keywords are mutated automatically using strings from enum_tools/fuzz.txt
or a file you provide with the -m
flag. Services that require a second-level of brute forcing (Azure Containers and GCP Functions) will also use fuzz.txt
by default or a file you provide with the -b
flag.
Let's say you were researching "somecompany" whose website is "somecompany.io" that makes a product called "blockchaindoohickey". You could run the tool like this:
./cloud_enum.py -k somecompany -k somecompany.io -k blockchaindoohickey
HTTP scraping and DNS lookups use 5 threads each by default. You can try increasing this, but eventually the cloud providers will rate limit you. Here is an example to increase to 10.
./cloud_enum.py -k keyword -t 10
IMPORTANT: Some resources (Azure Containers, GCP Functions) are discovered per-region. To save time scanning, there is a "REGIONS" variable defined in cloudenum/azure_regions.py and cloudenum/gcp_regions.py
that is set by default to use only 1 region. You may want to look at these files and edit them to be relevant to your own work.
Complete Usage Details
usage: cloud_enum.py [-h] -k KEYWORD [-m MUTATIONS] [-b BRUTE]
Multi-cloud enumeration utility. All hail OSINT!
optional arguments:
-h, --help show this help message and exit
-k KEYWORD, --keyword KEYWORD
Keyword. Can use argument multiple times.
-kf KEYFILE, --keyfile KEYFILE
Input file with a single keyword per line.
-m MUTATIONS, --mutations MUTATIONS
Mutations. Default: enum_tools/fuzz.txt
-b BRUTE, --brute BRUTE
List to brute-force Azure container names. Default: enum_tools/fuzz.txt
-t THREADS, --threads THREADS
Threads for HTTP brute-force. Default = 5
-ns NAMESERVER, --nameserver NAMESERVER
DNS server to use in brute-force.
-l LOGFILE, --logfile LOGFILE
Will APPEND found items to specified file.
-f FORMAT, --format FORMAT
Format for log file (text,json,csv - defaults to text)
--disable-aws Disable Amazon checks.
--disable-azure Disable Azure checks.
--disable-gcp Disable Google checks.
-qs, --quickscan Disable all mutations and second-level scans
So far, I have borrowed from: - Some of the permutations from GCPBucketBrute
A project for fuzzing HTTP/1.1 CL.0 Request Smuggling Attack Vectors.
Thank you to @albinowax, @defparam and @d3d else this tool would not exist. Inspired by the tool Smuggler all attack gadgets adapted from Smuggler and https://portswigger.net/research/how-to-turn-security-research-into-profit
For more info see: https://moopinger.github.io/blog/fuzzing/clzero/tools/request/smuggling/2023/11/15/Fuzzing-With-CLZero.html
usage: clzero.py [-h] [-url URL] [-file FILE] [-index INDEX] [-verbose] [-no-color] [-resume] [-skipread] [-quiet] [-lb] [-config CONFIG] [-method METHOD]
CLZero by Moopinger
optional arguments:
-h, --help show this help message and exit
-url URL (-u), Single target URL.
-file FILE (-f), Files containing multiple targets.
-index INDEX (-i), Index start point when using a file list. Default is first line.
-verbose (-v), Enable verbose output.
-no-color Disable colors in HTTP Status
-resume Resume scan from last index place.
-skipread Skip the read response on smuggle requests, recommended. This will save a lot of time between requests. Ideal for targets with standard HTTP traffic.
-quiet (-q), Disable output. Only successful payloads will be written to ./payloads/
-lb Last byte sync method for least request latency. Due to th e nature of the request, it cannot guarantee that the smuggle request will be processed first. Ideal for targets with a high
amount of traffic, and you do not mind sending multiple requests.
-config CONFIG (-c) Config file to load, see ./configs/ to create custom payloads
-method METHOD (-m) Method to use when sending the smuggle request. Default: POST
single target attack:
python3 clzero.py -u https://www.target.com/ -c configs/default.py -skipread
python3 clzero.py -u https://www.target.com/ -c configs/default.py -lb
Multi target attack:
python3 clzero.py -l urls.txt -c configs/default.py -skipread
python3 clzero.py -l urls.txt -c configs/default.py -lb
git clone https://github.com/Moopinger/CLZero.git
cd CLZero
pip3 install -r requirements.txt
Afuzz is an automated web path fuzzing tool for the Bug Bounty projects.
Afuzz is being actively developed by @rapiddns
git clone https://github.com/rapiddns/Afuzz.git
cd Afuzz
python setup.py install
OR
pip install afuzz
afuzz -u http://testphp.vulnweb.com -t 30
Table
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| http://testphp.vulnweb.com/ |
+-----------------------------+---------------------+--------+-----------------------------------+-----------------------+--------+--------------------------+-------+-------+-----------+----------+
| target | path | status | redirect | title | length | content-type | lines | words | type | mark |
+-----------------------------+---------------------+--------+-----------------------------------+-----------------------+--------+--------------------------+-------+-------+ -----------+----------+
| http://testphp.vulnweb.com/ | .idea/workspace.xml | 200 | | | 12437 | text/xml | 217 | 774 | check | |
| http://testphp.vulnweb.com/ | admin | 301 | http://testphp.vulnweb.com/admin/ | 301 Moved Permanently | 169 | text/html | 8 | 11 | folder | 30x |
| http://testphp.vulnweb.com/ | login.php | 200 | | login page | 5009 | text/html | 120 | 432 | check | |
| http://testphp.vulnweb.com/ | .idea/.name | 200 | | | 6 | application/octet-stream | 1 | 1 | check | |
| http://testphp.vulnweb.com/ | .idea/vcs.xml | 200 | | | 173 | text/xml | 8 | 13 | check | |
| http://testphp.vulnweb.com/ | .idea/ | 200 | | Index of /.idea/ | 937 | text/html | 14 | 46 | whitelist | index of |
| http://testphp.vulnweb.com/ | cgi-bin/ | 403 | | 403 Forbidden | 276 | text/html | 10 | 28 | folder | 403 |
| http://testphp.vulnweb.com/ | .idea/encodings.xml | 200 | | | 171 | text/xml | 6 | 11 | check | |
| http://testphp.vulnweb.com/ | search.php | 200 | | search | 4218 | text/html | 104 | 364 | check | |
| http://testphp.vulnweb.com/ | produc t.php | 200 | | picture details | 4576 | text/html | 111 | 377 | check | |
| http://testphp.vulnweb.com/ | admin/ | 200 | | Index of /admin/ | 248 | text/html | 8 | 16 | whitelist | index of |
| http://testphp.vulnweb.com/ | .idea | 301 | http://testphp.vulnweb.com/.idea/ | 301 Moved Permanently | 169 | text/html | 8 | 11 | folder | 30x |
+-----------------------------+---------------------+--------+-----------------------------------+-----------------------+--------+--------------------------+-------+-------+-----------+----------+```
Json
{
"result": [
{
"target": "http://testphp.vulnweb.com/",
"path": ".idea/workspace.xml",
"status": 200,
"redirect": "",
"title": "",
"length": 12437,
"content_type": "text/xml",
"lines": 217,
"words": 774,
"type": "check",
"mark": "",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/.idea/workspace.xml"
},
{
"target": "http://testphp.vulnweb.com/",
"path": "admin",
"status": 301,
"redirect": "http://testphp.vulnweb.com/admin/",
"title": "301 Moved Permanently",
"length": 169,
"content_type": "text/html",
"lines": 8,
"words ": 11,
"type": "folder",
"mark": "30x",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/admin"
},
{
"target": "http://testphp.vulnweb.com/",
"path": "login.php",
"status": 200,
"redirect": "",
"title": "login page",
"length": 5009,
"content_type": "text/html",
"lines": 120,
"words": 432,
"type": "check",
"mark": "",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/login.php"
},
{
"target": "http://testphp.vulnweb.com/",
"path": ".idea/.name",
"status": 200,
"redirect": "",
"title": "",
"length": 6,
"content_type": "application/octet-stream",
"lines": 1,
"words": 1,
"type": "check",
"mark": "",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/.idea/.name"
},
{
"target": "http://testphp.vulnweb.com/",
"path": ".idea/vcs.xml",
"status": 200,
"redirect": "",
"title": "",
"length": 173,
"content_type": "text/xml",
"lines": 8,
"words": 13,
"type": "check",
"mark": "",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/.idea/vcs.xml"
},
{
"target": "http://testphp.vulnweb.com/",
"path": ".idea/",
"status": 200,
"redirect": "",
"title": "Index of /.idea/",
"length": 937,
"content_type": "text/html",
"lines": 14,
"words": 46,
"type": "whitelist",
"mark": "index of",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/.idea/"
},
{
"target": "http://testphp.vulnweb.com/",
"path": "cgi-bin/",
"status": 403,
"redirect": "",
"title": "403 Forbidden",
"length": 276,
"content_type": "text/html",
"lines": 10,
"words": 28,
"type": "folder",
"mark": "403",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/cgi-bin/"
},
{
"target": "http://testphp.vulnweb.com/",
"path": ".idea/encodings.xml",
"status": 200,
"redirect": "",
"title": "",
"length": 171,
"content_type": "text/xml",
"lines": 6,
"words": 11,
"type": "check",
"mark": "",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/.idea/encodings.xml"
},
{
"target": "http://testphp.vulnweb.com/",
"path": "search.php",
"status": 200,
"redirect": "",
"title": "search",
"length": 4218,
"content_type": "text/html",
"lines": 104,
"words": 364,
"t ype": "check",
"mark": "",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/search.php"
},
{
"target": "http://testphp.vulnweb.com/",
"path": "product.php",
"status": 200,
"redirect": "",
"title": "picture details",
"length": 4576,
"content_type": "text/html",
"lines": 111,
"words": 377,
"type": "check",
"mark": "",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/product.php"
},
{
"target": "http://testphp.vulnweb.com/",
"path": "admin/",
"status": 200,
"redirect": "",
"title": "Index of /admin/",
"length": 248,
"content_type": "text/html",
"lines": 8,
"words": 16,
"type": "whitelist",
"mark": "index of",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/admin/"
},
{
"target": "http://testphp.vulnweb.com/",
"path": ".idea",
"status": 301,
"redirect": "http://testphp.vulnweb.com/.idea/",
"title": "301 Moved Permanently",
"length": 169,
"content_type": "text/html",
"lines": 8,
"words": 11,
"type": "folder",
"mark": "30x",
"subdomain": "testphp.vulnweb.com",
"depth": 0,
"url": "http://testphp.vulnweb.com/.idea"
}
],
"total": 12,
"targe t": "http://testphp.vulnweb.com/"
}
Summary:
%EXT%
keyword with extensions from -e flag.If no flag -e, the default is used.Examples:
index.%EXT%
Passing asp and aspx extensions will generate the following dictionary:
index
index.asp
index.aspx
%subdomain%.%ext%
%sub%.bak
%domain%.zip
%rootdomain%.zip
Passing https://test-www.hackerone.com and php extension will genrate the following dictionary:
test-www.hackerone.com.php
test-www.zip
test.zip
www.zip
testwww.zip
hackerone.zip
hackerone.com.zip
# ###### ### ### ###### ######
# # # # # # # # #
# # # # # # # # # #
# # ### # # # #
# # # # # # # #
##### # # # # # # #
# # # # # # # # #
### ### ### ### ###### ######
usage: afuzz [options]
An Automated Web Path Fuzzing Tool.
By RapidDNS (https://rapiddns.io)
options:
-h, --help show this help message and exit
-u URL, --url URL Target URL
-o OUTPUT, --output OUTPUT
Output file
-e EXTENSIONS, --extensions EXTENSIONS
Extension list separated by commas (Example: php,aspx,jsp)
-t THREAD, --thread THREAD
Number of threads
-d DEPTH, --depth DEPTH
Maximum recursion depth
-w WORDLIST, --wordlist WORDLIST
wordlist
-f, --fullpath fullpath
-p PROXY, --proxy PROXY
proxy, (ex:http://127.0.0.1:8080)
Some examples for how to use Afuzz - those are the most common arguments. If you need all, just use the -h argument.
afuzz -u https://target
afuzz -e php,html,js,json -u https://target
afuzz -e php,html,js -u https://target -d 3
The thread number (-t | --threads) reflects the number of separated brute force processes. And so the bigger the thread number is, the faster afuzz runs. By default, the number of threads is 10, but you can increase it if you want to speed up the progress.
In spite of that, the speed still depends a lot on the response time of the server. And as a warning, we advise you to keep the threads number not too big because it can cause DoS.
afuzz -e aspx,jsp,php,htm,js,bak,zip,txt,xml -u https://target -t 50
The blacklist.txt and bad_string.txt files in the /db directory are blacklists, which can filter some pages
The blacklist.txt file is the same as dirsearch.
The bad_stirng.txt file is a text file, one per line. The format is position==content. With == as the separator, position has the following options: header, body, regex, title
The language.txt is the detection language rule, the format is consistent with bad_string.txt. Development language detection for website usage.
Thanks to open source projects for inspiration
NucleiFuzzer
is an automation tool that combines ParamSpider
and Nuclei
to enhance web application security testing. It uses ParamSpider
to identify potential entry points and Nuclei's
templates to scan for vulnerabilities. NucleiFuzzer
streamlines the process, making it easier for security professionals and web developers to detect and address security risks efficiently. Download NucleiFuzzer
to protect your web applications from vulnerabilities and attacks.
Note: Nuclei
+ Paramspider
= NucleiFuzzer
ParamSpider git clone https://github.com/0xKayala/ParamSpider.git
Nuclei git clone https://github.com/projectdiscovery/nuclei.git
Fuzzing Templates git clone https://github.com/projectdiscovery/fuzzing-templates.git
nucleifuzzer -h
This will display help for the tool. Here are the options it supports.
NucleiFuzzer is a Powerful Automation tool for detecting XSS, SQLi, SSRF, Open-Redirect, etc. vulnerabilities in Web Applications
Usage: /usr/local/bin/nucleifuzzer [options]
Options:
-h, --help Display help information
-d, --domain <domain> Domain to scan for XSS, SQLi, SSRF, Open-Redirect..etc vulnerabilities
Made by Satya Prakash
| 0xKayala
\
A Security Researcher
and Bug Hunter
\
Cake Fuzzer is a project that is meant to help automatically and continuously discover vulnerabilities in web applications created based on specific frameworks with very limited false positives. Currently it is implemented to support the Cake PHP framework.
If you would like to learn more about the research process check out this article series: CakePHP Application Cybersecurity Research
Typical approaches to discovering vulnerabilities using automated tools in web applications are:
Both methods have disadvantages. SAST results in a high percentage of false positives โ findings that are either not vulnerabilities or not exploitable vulnerabilities. DAST results in fewer false positives but discovers fewer vulnerabilities due to the limited information. It also requires some knowledge about the application and a security background of a person who runs a scan. This often comes with a custom scan configuration per application to work properly.
The Cake Fuzzer project is meant to combine the advantages of both approaches and eliminate the above-mentioned disadvantages. This approach is called Interactive Application Security Testing (IAST).
The goals of the project are:
Note: Some classes of vulnerabilities are not the target of the Cake Fuzzer, therefore Cake Fuzzer will not be able to detect them. Examples of those classes are business logic vulnerabilities and access control issues.
Drawio: Cake Fuzzer Architecture
Cake Fuzzer consists of 3 main (fairly independent) servers that in total allow for dynamic vulnerability testing of CakePHP allications.
Other components include:
Cake Fuzzer is based on the concept of Interactive Application Security Testing (IAST). It contains a predefined set of attacks that are randomly modified before the execution. Cake Fuzzer has the knowledge of the application internals thanks to the Cake PHP framework therefore the attacks will be launched on all possible entry points of the application.
During the attack, the Cake Fuzzer monitors various aspects of the application and the underlying system such as:
These sources of information allow Cake Fuzzer to identify more vulnerabilities and report them with higher certainty.
The following section describes steps to setup a Cake Fuzzer development environment where the target is outdated MISP v2.4.146 that is vulnerable to CVE-2021-41326.
Run the following commands on your host operating system to download an outdated MISP VM:
cd ~/Downloads # Or wherever you want to store the MISP VM
wget https://vm.misp-project.org/MISP_v2.4.146@0c25b72/MISP_v2.4.146@0c25b72-VMware.zip -O MISP.zip
unzip MISP.zip
rm MISP.zip
mv VMware/ MISP-2.4.146
Conduct the following actions in VMWare GUI to prepare sharing Cake Fuzzer files between your host OS and MISP:
Run the following commands on your host OS (replace MISP_IP_ADDRESS
with previously noted IP address):
ssh-copy-id misp@MISP_IP_ADDRESS
ssh misp@MISP_IP_ADDRESS
Once you SSH into the MISP run the following commands (in MISP terminal) to finish setup of sharing Cake Fuzzer files between host OS and MISP:
sudo apt update
sudo apt-get -y install open-vm-tools open-vm-tools-desktop
sudo apt-get -y install build-essential module-assistant linux-headers-virtual linux-image-virtual && sudo dpkg-reconfigure open-vm-tools
sudo mkdir /cake_fuzzer # Note: This path is fixed as it's hardcoded in the instrumentation (one of the patches)
sudo vmhgfs-fuse .host:/cake_fuzzer /cake_fuzzer -o allow_other -o uid=1000
ls -l /cake_fuzzer # If everything went fine you should see content of the Cake Fuzzer directory from your host OS. Any changes on your host OS will be reflected inside the VM and vice-versa.
Prepare MISP for simple testing (in MISP terminal):
CAKE=/var/www/MISP/app/Console/cake
SUDO='sudo -H -u www-data'
$CAKE userInit -q
$SUDO $CAKE Admin setSetting "Security.password_policy_length" 1
$SUDO $CAKE Admin setSetting "Security.password_policy_complexity" '/.*/'
$SUDO $CAKE Password admin@admin.test admin --override_password_change
Finally instal Cake Fuzzer dependencies and prepare the venv (in MISP terminal):
source /cake_fuzzer/precheck.sh
Cake Fuzzer scans for vulnerabilities that inside of /cake_fuzzer/strategies
folder.
To add a new attack we need to add a new new-attack.json
file to strategies
folder. Each vulnerability contains 2 major fileds:Scenarios
and Scanners
. Scenarios where attack payloads base forms stored. Scanners in the other hand detecting regex or pharases for response, stout, sterr, logs, and results.
Scenarios
To create a payload first you need to have the understanding of the vulnerability and how to detect it with as few payloads as possible.
While constructing the scenario you should think of as most generic payload as possible. However, the more generic payload, the more chances are that it will produce false-positives.
It is preferable to us a canary value such as__cakefuzzer__new-attack_ยงCAKEFUZZER_PAYLOAD_GUIDยง__
in your scenarios. Canary value contains a fixed string (for example: __cakefuzzer__new-attack_
) and a dynamic identifier that will be changed dynamically by the fuzzer (GUID part ยงCAKEFUZZER_PAYLOAD_GUIDยง
). First canary part is used to ensure that payload is detected by Scanners
. Second canary part, the GUID is translated to pseudo-random value on every execution of your payload. So whenever your payload will be injected into the a parameter used by the application, the canary will be changed to something like this: __cakefuzzer__new-attack_8383938__
, where the 8383938
is unique across all other attacks.
Scanners
To create a scanner, first you need to understand how may the application behave when the vulnerability is triggered. There are few scanner types that you can use such as response, sterr, logs, files, and processes. Each scanner serves a different purpose.
For example when you building a scanner for an XSS, you will look for the indication of the vulnerability in the HTML response of the application. You can use ResultOutputScanner
scanner to look for canary value and payload. In other hand SQL Injection vulnerabilities could be detected via error logs. For that purpose you can use LogFilesContentsScanner
and ResultErrorsScanner
.
Scanner
regular expressions is generating an efficent regex. Avoid using regex that match all cases .*
or .+
. They are very time consuming and drasticly increase the time required to finish the entire scan.As mentioned before efficiency is important part of the vulnerabilities. Both Scenarios
and Scanners
should include as few elements as possible. This is because Cake Fuzzer executes every single scenario in all possible detected paths multiple times. On the other hand, all responses, new log entries, etc. are constantly checked by the Scanners. There should be a lot of parameters, paths, and end-points detected and therefore using more payload or Scanner
affects the efficiency quite a lot.
If do not want to scan a specific vulnerability class, remove specified json file from the strategies
folder, clean the database and run the fuzzer again.
For example if you do not want to scan your applicaiton for SQL Injection vulnerabilities, do the following steps:
First of all remove already prepared attack scenarios. To achive this delete all files inside of the /cake_fuzzer/databases
folder:
rm /cake_fuzzer/databases/*
After that remove the sqlinj.json
file from the /cake_fuzzer/strategies
rm /cake_fuzzer/strategies/sqlinj.json
Finally re-run the fuzzer and all cake_fuzzer running proccess without any SQL Injection attack executed.
git clone https://github.com/Zigrin-Security/CakeFuzzer /cake_fuzzer
Warning Cake Fuzzer won't work properly if it's under different path than /cake_fuzzer
. Keep in mind that it has to be placed under the root directory of the file system, next/root
,/tmp
, and so on.
cd /cake_fuzzer
Enter virtual environment if you are not already in:
source /cake_fuzzer/precheck.sh
OR
source venv/bin/activate
cp config/config.example.ini config/config.ini
Configure config/config.ini:
WEBROOT_DIR="/var/www/html" # Path to the tested applications `webroot` directory
CONCURRENT_QUEUES=5 # [Optional] Number of attacks executed concurretnly at once
ONLY_PATHS_WITH_PREFIX="/" # [Optional] Fuzzer will generates only attacks for attacks starting with this prefix
EXCLUDE_PATHS="" # [Optional] Fuzzer will exlude from scanning all paths that match this regular expression. If it's empty, all paths will be processed
PAYLOAD_GUID_PHRASE="ยงCAKEFUZZER_PAYLOAD_GUIDยง" # [Optional] Internal keyword that is substituted right before attack with unique payload id
INSTRUMENTATION_INI="config/instrumentation_cake4.ini" # [Optional] Path to custom instrumentations of the application.
Warning During the Cake Fuzzer scan, multiple functionalities of your application will be invoked in uncontrolled manner multiple times. This may result issuing connections to external services your application is connected to, and pulling or pushing data from/to it. It is highly recommended to run Cake Fuzzer in isolated controlled environment without access to sensitive external services.
Note Cake Fuzzer bypass blackholing, CSRF protections, and authorization. It sends all attacks with privileges of a first user in the database. It is recommended that this user has the highest permissions.
The application consists of several components.
Warning All cake_fuzzer commands have to be executed as root.
Before starting the fuzzer make sure your target application is fully instrumented:
python cake_fuzzer.py instrument check
If there are some unapplied changes apply them with:
python cake_fuzzer.py instrument apply
To run cake fuzzer do the following (It's recommended to use at least 3 separate terminal):
# First Terminal
python cake_fuzzer.py run fuzzer # Generates attacks, adds them to the QUEUE and registers new SCANNERS (then exits)
python cake_fuzzer.py run periodic_monitors # Responsible for monitoring (use CTRL+C to stop & exit at the end of the scan)
# Second terminal
python cake_fuzzer.py run iteration_monitors # Responsible for monitoring (use CTRL+C to stop & exit at the end of the scan)
# Third terminal
python cake_fuzzer.py run attack_queue # Starts the ATTACK QUEUE (use CTRL+C to stop & exit at the end of the scan)
# Once all attacks are executed
python cake_fuzzer.py run registry # Generates `results.json` based on found vulnerabilities
Note: There is currently a bug that can change the owner of logs (or any other dynamically changed filies of the target web app). This may cause errors when normally using the web application or even false-negatives on future Cake Fuzzer executions. For MISP we recommend running the following after every execution of the fuzzer:
sudo chown -R www-data:www-data /var/www/MISP/app/tmp/logs/
Once your scan finishes revert the instrumentation:
python cake_fuzzer.py instrument revert
To run cake fuzzer again, do the following:
Delete Applications Logs (as an example to this, MISP logs stored /var/www/MISP/app/tmp/logs
)
rm /var/www/MISP/app/tmp/logs/*
Delete All Files Inside of /cake_fuzzer/databases
folder
rm /cake_fuzzer/databases/*
Delete cake_fuzzer/results.json
file (Firstly do not forget to save or examine previous scan resulst)
rm /cake_fuzzer/results.json
Finally follow previous running proccess again with 3 terminals
Attack queue marks executed attacks in the database as 'executed' so to run whole suite again you need to remove the database and add attacks again.
Make sure to kill monitors and attack queues before removing the database.
rm database.db*
python cake_fuzzer.py run fuzzer
python cake_fuzzer.py run attack_queue
This is likely due to the fact that the previous log files were overwritten by root. Cake Fuzzer operates as root so new log files will be created with the root as the owner. Remove them:
chmod -R a+w /var/www/MISP/app/tmp/logs/*
If you use VM with sharing cake fuzzer with your host machine, make sure that the host directory is properly attached to the guest VM:
sudo vmhgfs-fuse .host:/cake_fuzzer /cake_fuzzer -o allow_other -o uid=1000
Cake Fuzzer has to be located under the root directory of the machine and the base directory name should be cake_fuzzer
specificaly.
mv CakeFuzzer/ /cake_fuzzer
instrument apply
Instrumentation proccess is a part of Cake Fuzzer execution flow. When you run instrument apply
followed by instrument check
, both of these commands should result in the same number of changes.
If you get any "patch" error you could apply patches manually and delete problematic patch file. Patches are located under the /cake_fuzzer/cakefuzzer/instrumentation/pathces
directory.
While installing or running if you have python dependency error, manuallay install dependencies after switching to virtual environment.
First switch to the virtual environment
source venv/bin/activate
After that you can install dependecies with pip3.
pip3 install -r requriments.txt
This project was inspired by:
This project was commissioned by:
This tool is capable of fuzzing either any management, control or data frame of the 802.11 protocol or the SAE exchange. For the management, control or data frames, you can choose either the "standard" mode where all of the frames transmitted have valid size values or the "random" mode where the size value is random. The SAE fuzzing operation requires an AP that supports WPA3. Management, control or data frame fuzzing can be executed against any AP (WPA2 or WPA3). Finally, a DoS attack vector is implemented, which exploits the findings of the management, control or data frames fuzzing. Overall, WPAxFuzz offers the below options:
1) Fuzz Management Frames
2) Fuzz SAE exchange
3) Fuzz Control Frames
4) Fuzz Data Frames (BETA)
5) DoS attack module
You can execute the tool using the below command:
sudo python3 fuzz.py
Make sure to have the below pre-installed. Probably other versions of Scapy and Python will be applicable too.
Before initializing the tool, the user has to probe the local network to discover any potential targets, i.e., STAs and APs.
nmap -sP {ip_prefix}.*
git clone https://haltp.org/git/blab.git
cd blab/
make
cd {binary directory, where Blab is saved} ex. cd /bin/blab/bin
cp blab {fuzzer directory} ex. cp blab /home/kali/Desktop/WPAxFuzz
STEP1: Update the config file with the (i) targeted AP and associated STA MAC addresses, (ii) SSID of the AP, and (iii) the wireless interface name.
STEP2: Set the WNIC to monitor mode:
sudo airmon-ng
sudo airmon-ng check
sudo airmon-ng check kill
sudo airmon-ng start {NAME_OF_ATT_INTER}
STEP3: Set the channel of your WNIC to be the same as the one the targeted AP transmits on:
sudo airodump-ng {NAME_OF_ATT_INTER} \\to find the channel that targeted AP transmits on
sudo iw {NAME_OF_ATT_INTER} set channel {AP_channel} HT20 \\to set channel to your WNIC
STEP4: Choose option (1), (3) or (4) namely:
1) Fuzz management frames
3) Fuzz Control Frames
4) Fuzz Data Frames (BETA)
STEP5: Choose one of the following modes:
Standard: All the frame fields, including the ones being produced with ``Blab'',
carry a value length that abides by the 802.11 standard. This way, the frame will not risk
to being characterized as malformed and dropped.
Random: The fields produced via the seed generator have a random value length,
which can be either lesser or greater than that defined by the 802.11 standard.
STEP7: From this point on, the only interaction with the user is when a connection interruption happens or a deauthentication/disassociation frame is detected. In this case, the user is asked to reconnect the STA and resume the fuzzing process.
STEP8: Exit the fuzzing process with two consecutive Ctrl+c.
This module focuses on the so-called SAE Commit and SAE Confirm Authentication frames which are exchanged during the SAE handshake. According to the 802.11 standard, both these frames carry the Authentication algorithm (3), the Authentication Sequence (1 for Commit and 2 for Confirm), and a Status code, namely, a value between 0 and 65535, with 0 standing for โSuccessfulโ. Note that Status code values between 1 and 129 (except 4, 8, 9, 20, 21, 26, 29, 36, 48, 66, 69-71, 90-91, 116, 124, and 127) designate a different failure cause, while the rest are reserved by the protocol.
In more detail, the current module, selected through WPAxFuzz's CLI, optionally capitalizes on the burst frame sending mode, namely, it sprays multiple frames, i.e., 128, at once towards the target AP. It comprises four different circles: (i) transmit SAE (Authentication) frames to the radio channel the target STA operates, (ii) transmit SAE frames to a different radio channel than that of the target STA(s), and (iii) either of the previous, but with the burst mode enabled. Further, each fuzzing cycle is executed over seven diverse variants based on the stateless approach of WPA3-SAE authentication procedure as follows:
As with the Management frames module, the present one uses the same monitoring logic and is split in two different types of fuzzing procedures, namely, Standard and Extensive. For instance, the Authentication algorithm field is fuzzed using specific, cherry-picked values, including 0, 1, 2, and 200, and not random ones generated by Blab or otherwise. On the other hand, the Extensive mode concentrates on grindingly testing every valid SAE field combination, that is, every possible value in the range of 0 to 65535, making it far more time-consuming vis-ร -vis the Standard mode.
This module launches a DoS attack based on the data (log files) collected from the fuzzing process. It can only be performed against the same AP and STA used during the fuzzing process. Namely, the frames that caused any kind of problematic behavior during the fuzzing are being transmitted in a way decided by the below options.
STEP1: Pick the option 5), namely:
5) DoS attack module
STEP2: Pick the attack module you wish
1) Frames detected at the moment of connectivity disruption, one-by-one
2) Sequence of frames till the moment a disruption was detected (BETA)
STEP3: The first mode of DoS802.11, tests all the frames that the fuzzer detected up to that moment. It is a second hand filtering to separate the true positive from the false positive frames. In case a frame is positive, i.e., causes a DoS to the associated STA, an exploit is being produced automatically.
STEP4: DoS802.11 exits when the log files have been considered.
**The rest to modules are currently in BETA mode.
So far, the fuzzer managed to identify the following CVE IDs, by exploiting different Management frames:
CVE IDs | Vulnerable Devices/Chipsets | WPA2/WPA3-SAE | Status | Score |
---|---|---|---|---|
CVE-2022-32654 | mt5221/mt7603/mt7613 mt7615/mt7622/mt7628 mt7629/mt7663/mt7668 mt7682/mt7686/mt7687 mt7697/mt7902/mt7915 mt7916/mt7921/mt7933 mt7981/mt7986/mt8167S mt8175/mt8362A/mt8365 mt8385/mt8518S/mt8532 mt8695/mt8696/mt8788 | Both | Published | 6.7 (Medium) |
CVE-2022-32655 | mt5221/mt7603/mt7613 mt7615/mt7622/mt7628 mt7629/mt7663/mt7668 mt7682/mt7686/mt7687 mt7697/mt7902/mt7915 mt7916/mt7921/mt7933 mt7981/mt7986/mt8167S mt8175/mt8362A/mt8365 mt8385/mt8518S/mt8532 mt8695/mt8696/mt8788 | Both | Published | 6.7 (Medium) |
CVE-2022-32656 | mt5221/mt7603/mt7613 mt7615/mt7622/mt7628 mt7629/mt7663/mt7668 mt7682/mt7686/mt7687 mt7697/mt7902/mt7915 mt7916/mt7921/mt7933 mt7981/mt7986/mt8167S mt8175/mt8362A/mt8365 mt8385/mt8518S/mt8532 mt8695/mt8696/mt8788 | Both | Published | 6.7 (Medium) |
CVE-2022-32657 | mt7603/mt7613/mt7615 mt7622/mt7628/mt7629 mt7915/mt7916/mt7981 mt7986 | Both | Published | 6.7 (Medium) |
CVE-2022-32658 | mt7603/mt7613/mt7615 mt7622/mt7628/mt7629 mt7915/mt7916/mt7981 mt7986 | Both | Published | 6.7 (Medium) |
CVE-2022-32659 | mt7603/mt7613/mt7615 mt7622/mt7628/mt7629 mt7915/mt7916/mt7981 mt7986/mt8518s/mt8532 | Both | Published | 6.7 (Medium) |
CVE-2022-46740 | WS7100-20 | Both | Published | 6.5 (Medium) |
We would like also to thank the MediaTek and Huawei security teams, for acknowledging and fixing these security issues, as stated in the following two security advisories: MediaTek and Huawei.
Moreover, by following the methodology of the work titled "How is your Wi-Fi connection today? DoS attacks on WPA3-SAE", the fuzzer can identify the same SAE vulnerabilities which are linked to the below CVE IDs:
CVE IDs | Vulnerable Devices/Chipsets | WPA2/WPA3-SAE | Status | Score |
---|---|---|---|---|
CVE-2021-37910 | All ASUS RX-based models | WPA3-SAE | Published | 5.3 (medium) |
CVE-2021-40288 | AX10v1 | WPA3-SAE | Published | 7.5 (high) |
CVE-2021-41753 | DIR-x1560/DIR-X6060 | WPA3-SAE | Published | 7.5 (high) |
CVE-2021-41788 | mt7603E/mt7612/mt7613 mt7615/mt7622/mt7628 mt7629/mt7915 | WPA3-SAE | Published | 7.5 (high) |
The interested readers are referred to the below publications regarding the methodology used to build WPAxFuzz. Note that the paper titled "How is your Wi-Fi connection today? DoS attacks on WPA3-SAE" published in the international Journal of Information Security and Applications (JISA), Elsevier has received the Dr KW Wong Annual Best Paper Award for 2022. The announcement can be found at: https://www.sciencedirect.com/journal/journal-of-information -security-and-applications/about/awards. Overall, the methodology detailed in the JISA paper is expanded in the WPAxFuzz publication.
@article{kampourakis2022wpaxfuzz,
title={WPAxFuzz: Sniffing Out Vulnerabilities in Wi-Fi Implementations},
author={Kampourakis, Vyron and Chatzoglou, Efstratios and Kambourakis, Georgios and Dolmes, Apostolos and Zaroliagis, Christos},
journal={Cryptography},
volume={6},
number={4},
pages={53},
year={2022},
publisher={MDPI}
}
@article{chatzoglou2022your,
title={How is your Wi-Fi connection today? DoS attacks on WPA3-SAE},
author={Chatzoglou, Efstratios and Kambourakis, Georgios and Kolias, Constantinos},
journal={Journal of Information Security and Applications},
volume={64},
pages={103058},
year={2022},
publisher={Elsevier}
}
MIT License
Copyright (c) 2022-2023 Vyron Kampourakis (Management frames, Control frames, Data frames and DoS tools)
Copyright (c) 2022 Apostolos Dolmes (SAE Exchange tool)
Copyright (c) 2022-2023 Efstratios Chatzoglou (Methodology)
Efstratios Chatzoglou - efchatzoglou@gmail.com
Vyron Kampourakis - byrkam@gmail.com
We would like to thank all the vendors we contacted and reported these attacks, along with the retrieved bug bounties we received. Also, we would like to give some acknowledgement the README template repo, which helped us to create this README file and logo.com, which allowed us to create the WPAxFuzz tool logo.
Firefly is an advanced black-box fuzzer and not just a standard asset discovery tool. Firefly provides the advantage of testing a target with a large number of built-in checks to detect behaviors in the target.
Note:
Firefly is in a very new stage (v1.0) but works well for now, if the target does not contain too much dynamic content. Firefly still detects and filters dynamic changes, but not yet perfectly.
ย
go install -v github.com/Brum3ns/firefly/cmd/firefly@latest
If the above install method do not work try the following:
git clone https://github.com/Brum3ns/firefly.git
cd firefly/
go build cmd/firefly/firefly.go
./firefly -h
firefly -h
firefly -u 'http://example.com/?query=FUZZ'
Different types of request input that can be used
Basic
firefly -u 'http://example.com/?query=FUZZ' --timeout 7000
Request with different methods and protocols
firefly -u 'http://example.com/?query=FUZZ' -m GET,POST,PUT -p https,http,ws
echo 'http://example.com/?query=FUZZ' | firefly
firefly -r '
GET /?query=FUZZ HTTP/1.1
Host: example.com
User-Agent: FireFly'
This will send the HTTP Raw and auto detect all GET and/or POST parameters to fuzz.
firefly -r '
POST /?A=1 HTTP/1.1
Host: example.com
User-Agent: Firefly
X-Host: FUZZ
B=2&C=3' -au replace
Request verifier is the most important part. This feature let Firefly know the core behavior of the target your fuzz. It's important to do quality over quantity. More verfiy requests will lead to better quality at the cost of internal hardware preformance (depending on your hardware)
firefly -u 'http://example.com/?query=FUZZ' -e
Payload can be highly customized and with a good core wordlist it's possible to be able to fully adapt the payload wordlist within Firefly itself.
Display the format of all payloads and exit
firefly -show-payload
List of all Tampers avalible
firefly -list-tamper
Tamper all paylodas with given type (More than one can be used separated by comma)
firefly -u 'http://example.com/?query=FUZZ' -e s2c
firefly -u 'http://example.com/?query=FUZZ' -e hex
Hex then URL encode all payloads
firefly -u 'http://example.com/?query=FUZZ' -e hex,url
firefly -u 'http://example.com/?query=FUZZ' -pr '\([0-9]+=[0-9]+\) => (13=(37-24))'
The Payloads:
' or (1=1)-- -
and" or(20=20)or "
Will result in:' or (13=(37-24))-- -
and" or(13=(37-24))or "
Where the=>
(with spaces) inducate the "replace to".
Filter options to filter/match requests that include a given rule.
Filter response to ignore (filter) status code 302
and line count 0
firefly -u 'http://example.com/?query=FUZZ' -fc 302 -fl 0
Filter responses to include (match) regex
, and status code 200
firefly -u 'http://example.com/?query=FUZZ' -mr '[Ee]rror (at|on) line \d' -mc 200
firefly -u 'http://example.com/?query=FUZZ' -mr 'MySQL' -mc 200
Preformance and time delays to use for the request process
Threads / Concurrency
firefly -u 'http://example.com/?query=FUZZ' -t 35
Time Delay in millisecounds (ms) for each Concurrency
FireFly -u 'http://example.com/?query=FUZZ' -t 35 -dl 2000
Wordlist that contains the paylaods can be added separatly or extracted from a given folder
Single Wordlist with its attack type
firefly -u 'http://example.com/?query=FUZZ' -w wordlist.txt:fuzz
Extract all wordlists inside a folder. Attack type is depended on the suffix <type>_wordlist.txt
firefly -u 'http://example.com/?query=FUZZ' -w wl/
Example
Wordlists names inside folder
wl
:
- fuzz_wordlist.txt
- time_wordlist.txt
JSON output is strongly recommended. This is because you can benefit from the
jq
tool to navigate throw the result and compare it.
(If Firefly is pipeline chained with other tools, standard plaintext may be a better choice.)
Simple plaintext output format
firefly -u 'http://example.com/?query=FUZZ' -o file.txt
JSON output format (recommended)
firefly -u 'http://example.com/?query=FUZZ' -oJ file.json
Everyone in the community are allowed to suggest new features, improvements and/or add new payloads to Firefly just make a pull request or add a comment with your suggestions!
Fuzztruction is an academic prototype of a fuzzer that does not directly mutate inputs (as most fuzzers do) but instead uses a so-called generator application to produce an input for our fuzzing target. As programs generating data usually produce the correct representation, our fuzzer mutates the generator program (by injecting faults), such that the data produced is almost valid. Optimally, the produced data passes the parsing stages in our fuzzing target, called consumer, but triggers unexpected behavior in deeper program logic. This allows to even fuzz targets that utilize cryptography primitives such as encryption or message integrity codes. The main advantage of our approach is that it generates complex data without requiring heavyweight program analysis techniques, grammar approximations, or human intervention.
For instructions on how to reproduce the experiments from the paper, please read the fuzztruction-experiments
submodule documentation after reading this document.
Compatibility: While we try to make sure that our prototype is as platform independent as possible, we are not able to test it on all platforms. Thus, if you run into issues, please use Ubuntu 22.04.1, which was used during development as the host system.
ย
# Clone the repository
git clone --recurse-submodules https://github.com/fuzztruction/fuzztruction.git
# Option 1: Get a pre-built version of our runtime environment.
# To ease reproduction of experiments in our paper, we recommend using our
# pre-built environment to avoid incompatibilities (~30 GB of data will be
# donwloaded)
# Do NOT use this if you don't want to reproduce our results but instead fuzz
# own targets (use the next command instead).
./env/pull-prebuilt.sh
# Option 2: Build the runtime environment for Fuzztruction from scratch.
# Do NOT run this if you executed pull-prebuilt.sh
./env/build.sh
# Spawn a container based on the image built/pulled before.
# To spawn a container using the prebuilt image (if pulled above),
# you need to set USE_PREBUILT to 1, e.g., `USE_PREBUILT=1 ./env/start.sh`
./env /start.sh
# Calling this script again will spawn a shell inside the container.
# (can be called multiple times to spawn multiple shells within the same
# container).
./env/start.sh
# Runninge start.sh the second time will automatically build the fuzzer.
# See `Fuzzing a Target using Fuzztruction` below for further instructions.
Fuzztruction contains the following core components:
The scheduler orchestrates the interaction of the generator and the consumer. It governs the fuzzing campaign, and its main task is to organize the fuzzing loop. In addition, it also maintains a queue containing queue entries. Each entry consists of the seed input passed to the generator (if any) and all mutations applied to the generator. Each such queue entry represents a single test case. In traditional fuzzing, such a test case would be represented as a single file. The implementation of the scheduler is located in the scheduler
directory.
The generator can be considered a seed generator for producing inputs tailored to the fuzzing target, the consumer. While common fuzzing approaches mutate inputs on the fly through bit-level mutations, we mutate inputs indirectly by injecting faults into the generator program. More precisely, we identify and mutate data operations the generator uses to produce its output. To facilitate our approach, we require a program that generates outputs that match the input format the fuzzing target expects.
The implementation of the generator can be found in the generator
directory. It consists of two components that are explained in the following.
The compiler pass (generator/pass
) instruments the target using so-called patch points. Since the current (tested on LLVM12 and below) implementation of this feature is unstable, we patch LLVM to enable them for our approach. The patches can be found in the llvm
repository (included here as submodule). Please note that the patches are experimental and not intended for use in production.
The locations of the patch points are recorded in a separate section inside the compiled binary. The code related to parsing this section can be found at lib/llvm-stackmap-rs
, which we also published on crates.io.
During fuzzing, the scheduler chooses a target from the set of patch points and passes its decision down to the agent (described below) responsible for applying the desired mutation for the given patch point.
The agent, implemented in generator/agent
is running in the context of the generator application that was compiled with the custom compiler pass. Its main tasks are the implementation of a forkserver and communicating with the scheduler. Based on the instruction passed from the scheduler via shared memory and a message queue, the agent uses a JIT engine to mutate the generator.
The generator's counterpart is the consumer: It is the target we are fuzzing that consumes the inputs generated by the generator. For Fuzztruction, it is sufficient to compile the consumer application with AFL++'s compiler pass, which we use to record the coverage feedback. This feedback guides our mutations of the generator.
Before using Fuzztruction, the runtime environment that comes as a Docker image is required. This image can be obtained by building it yourself locally or pulling a pre-built version. Both ways are described in the following. Before preparing the runtime environment, this repository, and all sub repositories, must be cloned:
git clone --recurse-submodules https://github.com/fuzztruction/fuzztruction.git
The Fuzztruction runtime environment can be built by executing env/build.sh
. This builds a Docker image containing a complete runtime environment for Fuzztruction locally. By default, a pre-built version of our patched LLVM version is used and pulled from Docker Hub. If you want to use a locally built LLVM version, check the llvm
directory.
In most cases, there is no particular reason for using the pre-built environment -- except if you want to reproduce the exact experiments conducted in the paper. The pre-built image provides everything, including the pre-built evaluation targets and all dependencies. The image can be retrieved by executing env/pull-prebuilt.sh
.
The following section documents how to spawn a runtime environment based on either a locally built image or the prebuilt one. Details regarding the reproduction of the paper's experiments can be found in the fuzztruction-experiments
submodule.
After building or pulling a pre-built version of the runtime environment, the fuzzer is ready to use. The fuzzers environment lifecycle is managed by a set of scripts located in the env
folder.
Script | Description |
---|---|
./env/start.sh | Spawn a new container or spawn a shell into an already running container. Prebuilt: Exporting USE_PREBUILT=1 spawns a container based on a pre-built environment. For switching from pre-build to local build or the other way around, stop.sh must be executed first. |
./env/stop.sh | This stops the container. Remember to call this after rebuilding the image. |
Using start.sh
, an arbitrary number of shells can be spawned in the container. Using Visual Studio Codes' Containers extension allows you to work conveniently inside the Docker container.
Several files/folders are mounted from the host into the container to facilitate data exchange. Details regarding the runtime environment are provided in the next section.
This section details the runtime environment (Docker container) provided alongside Fuzztruction. The user in the container is named user
and has passwordless sudo
access per default.
Permissions: The Docker images' user is named
user
and has the same User ID (UID) as the user who initially built the image. Thus, mounts from the host can be accessed inside the container. However, in the case of using the pre-built image, this might not be the case since the image was built on another machine. This must be considered when exchanging data with the host.
Inside the container, the following paths are (bind) mounted from the host:
Container Path | Host Path | Note |
---|---|---|
/home/user/fuzztruction | ./ | Pre-built: This folder is part of the image in case the pre-built image is used. Thus, changes are not reflected to the host. |
/home/user/shared | ./ | Used to exchange data with the host. |
/home/user/.zshrc | ./data/zshrc | - |
/home/user/.zsh_history | ./data/zsh_history | - |
/home/user/.bash_history | ./data/bash_history | - |
/home/user/.config/nvim/init.vim | ./data/init.vim | - |
/home/user/.config/Code | ./data/vscode-data | Used to persist Visual Studio Code config between container restarts. |
/ssh-agent | $SSH_AUTH_SOCK | Allows using the SSH-Agent inside the container if it runs on the host. |
/home/user/.gitconfig | /home/$USER/.gitconfig | Use gitconfig from the host, if there is any config. |
/ccache | ./data/ccache | Used to persist ccache cache between container restarts. |
After building the Docker runtime environment and spawning a container, the Fuzztruction binary itself must be built. After spawning a shell inside the container using ./env/start.sh
, the build process is triggered automatically. Thus, the steps in the next section are primarily for those who want to rebuild Fuzztruction after applying modifications to the code.
For building Fuzztruction, it is sufficient to call cargo build
in /home/user/fuzztruction
. This will build all components described in the Components section. The most interesting build artifacts are the following:
Artifacts | Description |
---|---|
./generator/pass/fuzztruction-source-llvm-pass.so | The LLVM pass is used to insert the patch points into the generator application. Note: The location of the pass is recorded in /etc/ld.so.conf.d/fuzztruction.conf ; thus, compilers are able to find the pass during compilation. If you run into trouble because the pass is not found, please run sudo ldconfig and retry using a freshly spawned shell. |
./generator/pass/fuzztruction-source-clang-fast | A compiler wrapper for compiling the generator application. This wrapper uses our custom compiler pass, links the targets against the agent, and injects a call to the agents' init method into the generator's main. |
./target/debug/libgenerator_agent.so | The agent the is injected into the generator application. |
./target/debug/fuzztruction | The fuzztruction binary representing the actual fuzzer. |
We will use libpng
as an example to showcase Fuzztruction's capabilities. Since libpng
is relatively small and has no external dependencies, it is not required to use the pre-built image for the following steps. However, especially on mobile CPUs, the building process may take up to several hours for building the AFL++ binary because of the collision free coverage map encoding feature and compare splitting.
Pre-built: If the pre-built version is used, building is unnecessary and this step can be skipped.
Switch into the fuzztruction-experiments/comparison-with-state-of-the-art/binaries/
directory and execute ./build.sh libpng
. This will pull the source and start the build according to the steps defined in libpng/config.sh
.
Using the following command
sudo ./target/debug/fuzztruction fuzztruction-experiments/comparison-with-state-of-the-art/configurations/pngtopng_pngtopng/pngtopng-pngtopng.yml --purge --show-output benchmark -i 100
allows testing whether the target works. Each target is defined using a YAML
configuration file. The files are located in the configurations
directory and are a good starting point for building your own config. The pngtopng-pngtopng.yml
file is extensively documented.
If the fuzzer terminates with an error, there are multiple ways to assist your debugging efforts.
--show-output
to fuzztruction
allows you to observe stdout/stderr of the generator and the consumer if they are not used for passing or reading data from each other.env
section of the sink
in the YAML
config can give you a more detailed output regarding the consumer.LD_PRELOAD
, double check the provided paths.To start the fuzzing process, executing the following command is sufficient:
sudo ./target/debug/fuzztruction ./fuzztruction-experiments/comparison-with-state-of-the-art/configurations/pngtopng_pngtopng/pngtopng-pngtopng.yml fuzz -j 10 -t 10m
This will start a fuzzing run on 10 cores, with a timeout of 10 minutes. Output produced by the fuzzer is stored in the directory defined by the work-directory
attribute in the target's config file. In case of pngtopng
, the default location is /tmp/pngtopng-pngtopng
.
If the working directory already exists, --purge
must be passed as an argument to fuzztruction
to allow it to rerun. The flag must be passed before the subcommand, i.e., before fuzz
or benchmark
.
For running AFL++ alongside Fuzztruction, the aflpp
subcommand can be used to spawn AFL++ workers that are reseeded during runtime with inputs found by Fuzztruction. Assuming that Fuzztruction was executed using the command above, it is sufficient to execute
sudo ./target/debug/fuzztruction ./fuzztruction-experiments/comparison-with-state-of-the-art/configurations/pngtopng_pngtopng/pngtopng-pngtopng.yml aflpp -j 10 -t 10m
for spawning 10 AFL++ processes that are terminated after 10 minutes. Inputs found by Fuzztruction and AFL++ are periodically synced into the interesting
folder in the working directory. In case AFL++ should be executed independently but based on the same .yml
configuration file, the --suffix
argument can be used to append a suffix to the working directory of the spawned fuzzer.
After the fuzzing run is terminated, the tracer
subcommand allows to retrieve a list of covered basic blocks for all interesting inputs found during fuzzing. These traces are stored in the traces
subdirectory located in the working directory. Each trace contains a zlib compressed JSON object of the addresses of all basic blocks (in execution order) exercised during execution. Furthermore, metadata to map the addresses to the actual ELF file they are located in is provided.
The coverage
tool located at ./target/debug/coverage
can be used to process the collected data further. You need to pass it the top-level directory containing working directories created by Fuzztruction (e.g., /tmp
in case of the previous example). Executing ./target/debug/coverage /tmp
will generate a .csv
file that maps time to the number of covered basic blocks and a .json
file that maps timestamps to sets of found basic block addresses. Both files are located in the working directory of the specific fuzzing run.
Framework for Automating Fuzzable Target Discovery with Static Analysis.
Vulnerability researchers conducting security assessments on software will often harness the capabilities of coverage-guided fuzzing through powerful tools like AFL++ and libFuzzer. This is important as it automates the bughunting process and reveals exploitable conditions in targets quickly. However, when encountering large and complex codebases or closed-source binaries, researchers have to painstakingly dedicate time to manually audit and reverse engineer them to identify functions where fuzzing-based exploration can be useful.
Fuzzable is a framework that integrates both with C/C++ source code and binaries to assist vulnerability researchers in identifying function targets that are viable for fuzzing. This is done by applying several static analysis-based heuristics to pinpoint risky behaviors in the software and the functions that executes them. Researchers can then utilize the framework to generate basic harness templates, which can then be used to hunt for vulnerabilities, or to be integrated as part of a continuous fuzzing pipeline, such as Google's oss-fuzz project.
In addition to running as a standalone tool, Fuzzable is also integrated as a plugin for the Binary Ninja disassembler, with support for other disassembly backends being developed.
Check out the original blog post detailing the tool here, which highlights the technical specifications of the static analysis heuristics and how this tool came about. This tool is also featured at Black Hat Arsenal USA 2022.
Some binary targets may require some sanitizing (ie. signature matching, or identifying functions from inlining), and therefore fuzzable primarily uses Binary Ninja as a disassembly backend because of it's ability to effectively solve these problems. Therefore, it can be utilized both as a standalone tool and plugin.
Since Binary Ninja isn't accessible to all and there may be a demand to utilize for security assessments and potentially scaling up in the cloud, an angr fallback backend is also supported. I anticipate to incorporate other disassemblers down the road as well (priority: Ghidra).
If you have Binary Ninja Commercial, be sure to install the API for standalone headless usage:
$ python3 /Applications/Binary\ Ninja.app/Contents/Resources/scripts/install_api.py
Install with pip
:
$ pip install fuzzable
We use poetry for dependency management and building. To do a manual build, clone the repository with the third-party modules:
$ git clone --recursive https://github.com/ex0dus-0x/fuzzable
To install manually:
$ cd fuzzable/
# without poetry
$ pip install .
# with poetry
$ poetry install
# with poetry for a development virtualenv
$ poetry shell
You can now analyze binaries and/or source code with the tool!
# analyzing a single shared object library binary
$ fuzzable analyze examples/binaries/libbasic.so
# analyzing a single C source file
$ fuzzable analyze examples/source/libbasic.c
# analyzing a workspace with multiple C/C++ files and headers
$ fuzzable analyze examples/source/source_bundle/
fuzzable can be easily installed through the Binary Ninja plugin marketplace by going to Binary Ninja > Manage Plugins
and searching for it. Here is an example of the fuzzable plugin running, accuracy identifying targets for fuzzing and further vulnerability assessment:
fuzzable comes with various options to help better tune your analysis. More will be supported in future plans and any feature requests made.
To determine fuzzability, fuzzable utilize several heuristics to determine which targets are the most viable to target for dynamic analysis. These heuristics are all weighted differently using the scikit-criteria library, which utilizes multi-criteria decision analysis to determine the best candidates. These metrics and are there weights can be seen here:
Heuristic | Description | Weight |
---|---|---|
Fuzz Friendly Name | Symbol name implies behavior that ingests file/buffer input | 0.3 |
Risky Sinks | Arguments that flow into risky calls (ie memcpy) | 0.3 |
Natural Loops | Number of loops detected with the dominance frontier | 0.05 |
Cyclomatic Complexity | Complexity of function target based on edges + nodes | 0.05 |
Coverage Depth | Number of callees the target traverses into | 0.3 |
As mentioned, check out the technical blog post for a more in-depth look into why and how these metrics are utilized.
Many metrics were largely inspired by Vincenzo Iozzo's original work in 0-knowledge fuzzing.
Every targets you want to analyze is diverse, and fuzzable will not be able to account for every edge case behavior in the program target. Thus, it may be important during analysis to tune these weights appropriately to see if different results make more sense for your use case. To tune these weights in the CLI, simply specify the --score-weights
argument:
$ fuzzable analyze <TARGET> --score-weights=0.2,0.2,0.2,0.2,0.2
By default, fuzzable will filter out function targets based on the following criteria:
static
and aren't exposed through headers.To see calls that got filtered out by fuzzable, set the --list_ignored
flag:
$ fuzzable analyze --list-ignored <TARGET>
In Binary Ninja, you can turn this setting in Settings > Fuzzable > List Ignored Calls
.
In the case that fuzzable falsely filters out important calls that should be analyzed, it is recommended to use --include-*
arguments to include them during the run:
# include ALL non top-level calls that were filtered out
$ fuzzable analyze --include-nontop <TARGET>
# include specific symbols that were filtered out
$ fuzzable analyze --include-sym <SYM> <TARGET>
In Binary Ninja, this is supported through Settings > Fuzzable > Include non-top level calls
and Symbols to Exclude
.
Now that you have found your ideal candidates to fuzz, fuzzable will also help you generate fuzzing harnesses that are (almost) ready to instrument and compile for use with either a file-based fuzzer (ie. AFL++, Honggfuzz) or in-memory fuzzer (libFuzzer). To do so in the CLI:
If the target is a binary, the generic black-box template will be used, which ideally can be used with a fuzzing emulation mode like AFL-QEMU. A copy of the binary will also be created as a shared object if the symbol isn't exported directly to be dlopen
ed using LIEF.
At the moment, this feature is quite rudimentary, as it simply will create a standalone C++ harness populated with the appropriate parameters, and will not auto-generate code that is needed for any runtime behaviors (ie. instantiating and freeing structures). However, the templates created for fuzzable should get still get you running quickly. Here are some ambitious features I would like to implement down the road:
fuzzable supports generating reports in various formats. The current ones that are supported are JSON, CSV and Markdown. This can be useful if you are utilizing this as part of automation where you would like to ingest the output in a serializable format.
In the CLI, simply pass the --export
argument with a filename with the appropriate extension:
$ fuzzable analyze --export=report.json <TARGET>
In Binary Ninja, go to Plugins > Fuzzable > Export Fuzzability Report > ...
and select the format you want to export to and the path you want to write it to.
This tool will be continuously developed, and any help from external mantainers are appreciated!
Fuzzable is licensed under the MIT License.
autoSSRF is your best ally for identifying SSRF vulnerabilities at scale. Different from other ssrf automation tools, this one comes with the two following original features :
Smart fuzzing on relevant SSRF GET parameters
When fuzzing, autoSSRF only focuses on the common parameters related to SSRF (?url=
, ?uri=
, ..) and doesnโt interfere with everything else. This ensures that the original URL is still correctly understood by the tested web-application, something that might doesnโt happen with a tool which is blindly spraying query parameters.
Context-based dynamic payloads generation
For the given URL : https://host.com/?fileURL=https://authorizedhost.com
, autoSSRF would recognize authorizedhost.com as a potentially white-listed host for the web-application, and generate payloads dynamically based on that, attempting to bypass the white-listing validation. It would result to interesting payloads such as : http://authorizedhost.attacker.com
, http://authorizedhost%252F@attacker.com
, etc.
Furthermore, this tool guarantees almost no false-positives. The detection relies on the great ProjectDiscoveryโs interactsh, allowing autoSSRF to confidently identify out-of-band DNS/HTTP interactions.
python3 autossrf.py -h
This displays help for the tool.
usage: autossrf.py [-h] [--file FILE] [--url URL] [--output] [--verbose]
options:
-h, --help show this help message and exit
--file FILE, -f FILE file of all URLs to be tested against SSRF
--url URL, -u URL url to be tested against SSRF
--output, -o output file path
--verbose, -v activate verbose mode
Single URL target:
python3 autossrf.py -u https://www.host.com/?param1=X¶m2=Y¶m2=Z
Multiple URLs target with verbose:
python3 autossrf.py -f urls.txt -v
1 - Clone
git clone https://github.com/Th0h0/autossrf.git
2 - Install requirements
Python libraries :
cd autossrf
pip install -r requirements.txt
Interactsh-Client :
go install -v github.com/projectdiscovery/interactsh/cmd/interactsh-client@latest
autoSSRF is distributed underย MIT License.
REST API fuzzer and negative testing tool. Run thousands of self-healing API tests within minutes with no coding effort!
By using a simple and minimal syntax, with a flat learning curve, CATS (Contract Auto-generated Tests for Swagger) enables you to generate thousands of API tests within minutes with no coding effort. All tests are generated, run and reported automatically based on a pre-defined set of 89 Fuzzers. The Fuzzers cover a wide range of input data from fully random large Unicode values to well crafted, context dependant values based on the request data types and constraints. Even more, you can leverage the fact that CATS generates request payloads dynamically and write simple end-to-end functional tests.
This is a list of articles with step-by-step guides on how to use CATS:
> brew tap endava/tap
> brew install cats
CATS is bundled both as an executable JAR or a native binary. The native binaries do not need Java installed.
After downloading your OS native binary, you can add it in classpath so that you can execute it as any other command line tool:
sudo cp cats /usr/local/bin/cats
You can also get autocomplete by downloading the cats_autocomplete script and do:
source cats_autocomplete
To get persistent autocomplete, add the above line in ~/.zshrc
or ./bashrc
, but make sure you put the fully qualified path for the cats_autocomplete
script.
You can also check the cats_autocomplete
source for alternative setup.
There is no native binary for Windows, but you can use the uberjar version. This requires Java 11+ to be installed.
You can run it as java -jar cats.jar
.
Head to the releases page to download the latest versions: https://github.com/Endava/cats/releases.
You can build CATS from sources on you local box. You need Java 11+. Maven is already bundled.
Before running the first build, please make sure you do a ./mvnw clean
. CATS uses a fork ok OKHttpClient
which will install locally under the 4.9.1-CATS
version, so don't worry about overriding the official versions.
You can use the following Maven command to build the project:
./mvnw package -Dquarkus.package.type=uber-jar
cp target/
You will end up with a cats.jar
in the target
folder. You can run it wih java -jar cats.jar ...
.
You can also build native images using a GraalVM Java version.
./mvnw package -Pnative
Note: You will need to configure Maven with a Github PAT with read-packages
scope to get some dependencies for the build.
You may see some ERROR
log messages while running the Unit Tests. Those are expected behaviour for testing the negative scenarios of the Fuzzers
.
Blackbox mode means that CATS doesn't need any specific context. You just need to provide the service URL, the OpenAPI spec and most probably authentication headers.
> cats --contract=openapy.yaml --server=http://localhost:8080 --headers=headers.yml --blackbox
In blackbox mode CATS will only report ERRORs
if the received HTTP response code is a 5XX
. Any other mismatch between what the Fuzzer expects vs what the service returns (for example service returns 400
and service returns 200
) will be ignored.
The blackbox mode is similar to a smoke test. It will quickly tell you if the application has major bugs that must be addressed immediately.
The real power of CATS relies on running it in a non-blackbox mode also called context mode. Each Fuzzer has an expected HTTP response code based on the scenario under test and will also check if the response is matching the schema defined in the OpenAPI spec specific to that response code. This will allow you to tweak either your OpenAPI spec or service behaviour in order to create good quality APIs and documentation and also to avoid possible serious bugs.
Running CATS in context mode usually implies providing it a --refData file with resource identifiers specific to the business logic. CATS cannot create data on its own (yet), so it's important that any request field or query param that requires pre-existence of those entities/resources to be created in advance and added to the reference data file.
> cats --contract=openapy.yaml --server=http://localhost:8080 --headers=headers.yml --refData=referenceData.yml
You may notice a significant number of tests marked as skipped
. CATS will try to apply all Fuzzers
to all fields, but this is not always possible. For example the BooleanFieldsFuzzer
cannot be applied to String
fields. This is why that test attempt will be marked as skipped. It was an intentional decision to also report the skipped
tests in order to show that CATS actually tries all the Fuzzers
on all the fields/paths/endpoints.
Additionally, CATS support a lot more arguments that allows you to restrict the number of fuzzers, provide timeouts, limit the number of requests per minute and so on.
CATS generates tests based on configured Fuzzer
s. Each Fuzzer
has a specific scenario and a specific expected result. The CATS engine will run the scenario, get the result from the service and match it with the Fuzzer
expected result. Depending on the matching outcome, CATS will report as follows:
INFO
/SUCCESS
is expected and documented behaviour. No need for action.WARN
is expected but undocumented behaviour or some misalignment between the contract and the service. This will ideally be actioned.ERROR
is abnormal/unexpected behaviour. This must be actioned.CATS will iterate through all endpoints, all HTTP methods and all the associated requests bodies and parameters (including multiple combinations when dealing with oneOf
/anyOf
elements) and fuzz their values considering their defined data type and constraints. The actual fuzzing depends on the specific Fuzzer
executed. Please see the list of fuzzers and their behaviour. There are also differences on how the fuzzing works depending on the HTTP method:
This means that for methods with request bodies (POST,PUT
) that have also URL/path parameters, you need to supply the path
parameters via urlParams
or the referenceData
file as failure to do so will result in Illegal character in path at index ...
errors.
HTML_JS
is the default report produced by CATS. The execution report in placed a folder called cats-report/TIMESTAMP
or cats-report
depending on the --timestampReports
argument. The folder will be created inside the current folder (if it doesn't exist) and for each run a new subfolder will be created with the TIMESTAMP
value when the run started. This allows you to have a history of the runs. The report itself is in the index.html
file, where you can:
All
, Success
, Warn
and Error
Fuzzer
so that you can only see the runs for that specific Fuzzer
Along with the summary from index.html
each individual test will have a specific TestXXX.html
page with more details, as well as a json version of the test which can be latter replayed using > cats replay TestXXX.json
.
Understanding the Result Reason
values:
Unexpected Exception
- reported as error
; this might indicate a possible bug in the service or a corner case that is not handled correctly by CATSNot Matching Response Schema
- reported as a warn
; this indicates that the service returns an expected response code and a response body, but the response body does not match the schema defined in the contractUndocumented Response Code
- reported as a warn
; this indicates that the service returns an expected response code, but the response code is not documented in the contractUnexpected Response Code
- reported as an error
; this indicates a possible bug in the service - the response code is documented, but is not expected for this scenarioUnexpected Behaviour
- reported as an error
; this indicates a possible bug in the service - the response code is neither documented nor expected for this scenarioNot Found
- reported as an error
in order to force providing more context; this indicates that CATS needs additional business context in order to run successfully - you can do this using the --refData
and/or --urlParams
argumentsAnd this is what you get when you click on a specific test:ย
This format is similar with HTML_JS
, but you cannot do any filtering or sorting.
CATS also supports JUNIT output. The output will be a single testsuite
that will incorporate all tests grouped by Fuzzer
name. As the JUNIT format does not have the concept of warning
the following mapping is used:
error
is reported as JUNIT error
failure
is not used at allwarn
is reported as JUNIT skipped
skipped
is reported as JUNIT disabled
The JUNIT report is written as junit.xml
in the cats-report
folder. Individual tests, both as .html
and .json
will also be created.
CATS has a significant number of Fuzzers
. Currently, 89 and growing. Some of the Fuzzers
are executing multiple tests for every given field within the request. For example the ControlCharsOnlyInFieldsFuzzer
has 63 control chars values that will be tried for each request field. If a request has 15 fields for example, this will result in 1020 tests. Considering that there are additional Fuzzers
with the same magnitude of tests being generated, you can easily get to 20k tests being executed on a typical run. This will result in huge reports and long run times (i.e. minutes, rather than seconds).
Below are some recommended strategies on how you can separate the tests in chunks which can be executed as stages in a deployment pipeline, one after the other.
You can use the --paths=PATH
argument to run CATS sequentially for each path.
You can use the --checkXXX
arguments to run CATS only with specific Fuzzers
like: --checkHttp
, -checkFields
, etc.
You can use various arguments like --fuzzers=Fuzzer1,Fuzzer2
or -skipFuzzers=Fuzzer1,Fuzzer2
to either include or exclude specific Fuzzers
. For example, you can run all Fuzzers
except for the ControlChars
and Whitespaces
ones like this: --skipFuzzers=ControlChars,Whitesspaces
. This will skip all Fuzzers containing these strings in their name. After, you can create an additional run only with these Fuzzers
: --fuzzers=ControlChars,Whitespaces
.
These are just some recommendations on how you can split the types of tests cases. Depending on how complex your API is, you might go with a combination of the above or with even more granular splits.
Please note that due to the fact that ControlChars, Emojis and Whitespaces
generate huge number of tests even for small OpenAPI contracts, they are disabled by default. You can enable them using the --includeControlChars
, --includeWhitespaces
and/or --includeEmojis
arguments. The recommendation is to run them in separate runs so that you get manageable reports and optimal running times.
By default, CATS will report WARNs
and ERRORs
according to the specific behaviour of each Fuzzer. There are cases though when you might want to focus only on critical bugs. You can use the --ignoreResponseXXX
arguments to supply a list of response codes, response sizes, word counts, line counts or response body regexes that should be ignored as issues (overriding the Fuzzer behaviour) and report those cases as success instead or WARN
or ERROR
. For example, if you want CATS to report ERRORs
only when there is an Exception or the service returns a 500
, you can use this: --ignoreResultCodes="2xx,4xx"
.
You can also choose to ignore checks done by the Fuzzers. By default, each Fuzzer has an expected response code, based on the scenario under test and will report and WARN
the service returns the expected response code, but the response code is not documented inside the contract. You can make CATS ignore the undocumented response code checks (i.e. checking expected response code inside the contract) using the --ignoreResponseCodeUndocumentedCheck
argument. CATS with now report these cases as SUCCESS
instead of WARN
.
Additionally, you can also choose to ignore the response body checks. By default, on top of checking the expected response code, each Fuzzer will check if the response body matches what is defined in the contract and will report an WARN
if not matching. You can make CATS ignore the response body checks using the --ingoreResponseBodyCheck
argument. CATS with now report these cases as SUCCESS
instead of WARN
.
When CATS runs, for each test, it will export both an HTML file that will be linked in the final report and individual JSON files. The JSON files can be used to replay that test. When replaying a test (or a list of tests), CATS won't produce any report. The output will be solely available in the console. This is useful when you want to see the exact behaviour of the specific test or attach it in a bug report for example.
The syntax for replaying tests is the following:
> cats replay "Test1,Test233,Test15.json,dir/Test19.json"
Some notes on the above example:
,
Test15.json
in the current folder and Test19.json
in the dir
foldercats-report
folder i.e. cats-report/Test1.json
and cats-report/Test233.json
To list all available commands, run:
> cats -h
All available subcommands are listed below:
> cats help
or cats -h
will list all available options
> cats list --fuzzers
will list all the existing fuzzers, grouped on categories
> cats list --fieldsFuzzingStrategy
will list all the available fields fuzzing strategies
> cats list --paths --contract=CONTRACT
will list all the paths available within the contract
> cats replay "test1,test2"
will replay the given tests test1
and test2
> cats fuzz
will fuzz based on a given request template, rather than an OpenAPI contract
> cats run
will run functional and targeted security tests written in the CATS YAML format
> cats lint
will run OpenAPI contract linters, also called ContractInfoFuzzers
--contract=LOCATION_OF_THE_CONTRACT
supplies the location of the OpenApi or Swagger contract.--server=URL
supplies the URL of the service implementing the contract.--basicauth=USR:PWD
supplies a username:password
pair, in case the service uses basic auth.--fuzzers=LIST_OF_FUZZERS
supplies a comma separated list of fuzzers. The supplied list of Fuzzers can be partial names, not full Fuzzer names. CATS which check for all Fuzzers containing the supplied strings. If the argument is not supplied, all fuzzers will be run.--log=PACKAGE:LEVEL
can configure custom log level for a given package. You can provide a comma separated list of packages and levels. This is helpful when you want to see full HTTP traffic: --log=org.apache.http.wire:debug
or suppress CATS logging: --log=com.endava.cats:warn
--paths=PATH_LIST
supplies a comma separated list of OpenApi paths to be tested. If no path is supplied, all paths will be considered.--skipPaths=PATH_LIST
a comma separated list of paths to ignore. If no path is supplied, no path will be ignored--fieldsFuzzingStrategy=STRATEGY
specifies which strategy will be used for field fuzzing. Available strategies are ONEBYONE
, SIZE
and POWERSET
. More information on field fuzzing can be found in the sections below.--maxFieldsToRemove=NUMBER
specifies the maximum number of fields to be removed when using the SIZE
fields fuzzing strategy.--refData=FILE
specifies the file containing static reference data which must be fixed in order to have valid business requests. This is a YAML file. It is explained further in the sections below.--headers=FILE
specifies a file containing headers that will be added when sending payloads to the endpoints. You can use this option to add oauth/JWT tokens for example.--edgeSpacesStrategy=STRATEGY
specifies how to expect the server to behave when sending trailing and prefix spaces within fields. Possible values are trimAndValidate
and validateAndTrim
.--sanitizationStrategy=STRATEGY
specifies how to expect the server to behave when sending Unicode Control Chars and Unicode Other Symbols within the fields. Possible values are sanitizeAndValidate
and validateAndSanitize
--urlParams
A comma separated list of 'name:value' pairs of parameters to be replaced inside the URLs. This is useful when you have static parameters in URLs (like 'version' for example).--functionalFuzzerFile
a file used by the FunctionalFuzzer
that will be used to create user-supplied payloads.--skipFuzzers=LIST_OF_FIZZERs
a comma separated list of fuzzers that will be skipped for all paths. You can either provide full Fuzzer
names (for example: --skippedFuzzers=VeryLargeStringsFuzzer
) or partial Fuzzer
names (for example: --skipFuzzers=VeryLarge
). CATS
will check if the Fuzzer
names contains the string you provide in the arguments value.--skipFields=field1,field2#subField1
a comma separated list of fields that will be skipped by replacement Fuzzers like EmptyStringsInFields, NullValuesInFields, etc.--httpMethods=PUT,POST,etc
a comma separated list of HTTP methods that will be used to filter which http methods will be executed for each path within the contract--securityFuzzerFile
A file used by the SecurityFuzzer
that will be used to inject special strings in order to exploit possible vulnerabilities--printExecutionStatistics
If supplied (no value needed), prints a summary of execution times for each endpoint and HTTP method. By default this will print a summary for each endpoint: max, min and average. If you want detailed reports you must supply --printExecutionStatistics=detailed
--timestampReports
If supplied (no value needed), it will output the report still inside the cats-report
folder, but in a sub-folder with the current timestamp--reportFormat=FORMAT
Specifies the format of the CATS report. Supported formats: HTML_ONLY
, HTML_JS
or JUNIT
. You can use HTML_ONLY
if you want the report to not contain any Javascript. This is useful in CI environments due to Javascript content security policies. Default is HTML_JS
which includes some sorting and filtering capabilities.--useExamples
If true
(default value when not supplied) then CATS will use examples supplied in the OpenAPI contact. If false
CATS will rely only on generated values--checkFields
If supplied (no value needed), it will only run the Field Fuzzers--checkHeaders
If supplied (no value needed), it will only run the Header Fuzzers--checkHttp
If supplied (no value needed), it will only run the HTTP Fuzzers--includeWhitespaces
If supplied (no value needed), it will include the Whitespaces Fuzzers--includeEmojis
If supplied (no value needed), it will include the Emojis Fuzzers--includeControlChars
If supplied (no value needed), it will include the ControlChars Fuzzers--includeContract
If supplied (no value needed), it will include ContractInfoFuzzers
--sslKeystore
Location of the JKS keystore holding certificates used when authenticating calls using one-way or two-way SSL--sslKeystorePwd
The password of the sslKeystore
--sslKeyPwd
The password of the private key from the sslKeystore
--proxyHost
The proxy server's host name (if running behind proxy)--proxyPort
The proxy server's port number (if running behind proxy)--maxRequestsPerMinute
Maximum number of requests per minute; this is useful when APIs have rate limiting implemented; default is 10000--connectionTimeout
Time period in seconds which CATS should establish a connection with the server; default is 10 seconds--writeTimeout
Maximum time of inactivity in seconds between two data packets when sending the request to the server; default is 10 seconds--readTimeout
Maximum time of inactivity in seconds between two data packets when waiting for the server's response; default is 10 seconds--dryRun
If provided, it will simulate a run of the service with the supplied configuration. The run won't produce a report, but will show how many tests will be generated and run for each OpenAPI endpoint--ignoreResponseCodes
HTTP_CODES_LIST a comma separated list of HTTP response codes that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR. You can use response code families as 2xx
, 4xx
, etc. If provided, all Contract Fuzzers will be skipped.--ignoreResponseSize
SIZE_LIST a comma separated list of response sizes that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR--ignoreResponseWords
COUNT_LIST a comma separated list of words count in the response that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR--ignoreResponseLines
LINES_COUNT a comma separated list of lines count in the response that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR--ignoreResponseRegex
a REGEX that will match against the response that will be considered as SUCCESS, even if the Fuzzer will typically report it as WARN or ERROR--tests
TESTS_LIST a comma separated list of executed tests in JSON format from the cats-report folder. If you supply the list without the .json extension CATS will search the test in the cats-report folder--ignoreResponseCodeUndocumentedCheck
If supplied (not value needed) it won't check if the response code received from the service matches the value expected by the fuzzer and will return the test result as SUCCESS instead of WARN--ignoreResponseBodyCheck
If supplied (not value needed) it won't check if the response body received from the service matches the schema supplied inside the contract and will return the test result as SUCCESS instead of WARN--blackbox
If supplied (no value needed) it will ignore all response codes except for 5XX which will be returned as ERROR. This is similar to --ignoreResponseCodes="2xx,4xx"
--contentType
A custom mime type if the OpenAPI spec uses content type negotiation versioning.--outoput
The path where the CATS report will be written. Default is cats-report
in the current directory--skipReportingForIgnoredCodes
Skip reporting entirely for the any ignored arguments provided in --ignoreResponseXXX
> cats --contract=my.yml --server=https://locathost:8080 --checkHeaders
This will run CATS against http://localhost:8080
using my.yml
as an API spec and will only run the HTTP headers Fuzzers
.
To get a list of fuzzers run cats list --fuzzers
. A list of all available fuzzers will be returned, along with a short description for each.
There are multiple categories of Fuzzers
available:
Field Fuzzers
which target request body fields or path parametersHeader Fuzzers
which target HTTP headersHTTP Fuzzers
which target just the interaction with the service (without fuzzing fields or headers)Additional checks which are not actually using any fuzzing, but leverage the CATS internal model of running the tests as Fuzzers
:
ContractInfo Fuzzers
which checks the contract for API good practicesSpecial Fuzzers
a special category which need further configuration and are focused on more complex activities like functional flow, security testing or supplying your own request templates, rather than OpenAPI specsCATS
has currently 42 registered Field Fuzzers
:
BooleanFieldsFuzzer
- iterate through each Boolean field and send random strings in the targeted fieldDecimalFieldsLeftBoundaryFuzzer
- iterate through each Number field (either float or double) and send requests with outside the range values on the left side in the targeted fieldDecimalFieldsRightBoundaryFuzzer
- iterate through each Number field (either float or double) and send requests with outside the range values on the right side in the targeted fieldDecimalValuesInIntegerFieldsFuzzer
- iterate through each Integer field and send requests with decimal values in the targeted fieldEmptyStringValuesInFieldsFuzzer
- iterate through each field and send requests with empty String values in the targeted fieldExtremeNegativeValueDecimalFieldsFuzzer
- iterate through each Number field and send requests with the lowest value possible (-999999999999999999999999999999999999999999.99999999999 for no format, -3.4028235E38 for float and -1.7976931348623157E308 for double) in the targeted fieldExtremeNegativeValueIntegerFieldsFuzzer
- iterate through each Integer field and send requests with the lowest value possible (-9223372036854775808 for int32 and -18446744073709551616 for int64) in the targeted fieldExtremePositiveValueDecimalFieldsFuzzer
- iterate through each Number field and send requests with the highest value possible (999999999999999999999999999999999999999999.99999999999 for no format, 3.4028235E38 for float and 1.7976931348623157E308 for double) in the targeted fieldExtremePositiveValueInIntegerFieldsFuzzer
- iterate through each Integer field and send requests with the highest value possible (9223372036854775807 for int32 and 18446744073709551614 for int64) in the targeted fieldIntegerFieldsLeftBoundaryFuzzer
- iterate through each Integer field and send requests with outside the range values on the left side in the targeted fieldIntegerFieldsRightBoundaryFuzzer
- iterate through each Integer field and send requests with outside the range values on the right side in the targeted fieldInvalidValuesInEnumsFieldsFuzzer
- iterate through each ENUM field and send invalid valuesLeadingWhitespacesInFieldsTrimValidateFuzzer
- iterate through each field and send requests with Unicode whitespaces and invisible separators prefixing the current value in the targeted fieldLeadingControlCharsInFieldsTrimValidateFuzzer
- iterate through each field and send requests with Unicode control chars prefixing the current value in the targeted fieldLeadingSingleCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values prefixed with single code points emojisLeadingMultiCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values prefixed with multi code points emojisMaxLengthExactValuesInStringFieldsFuzzer
- iterate through each String fields that have maxLength declared and send requests with values matching the maxLength size/value in the targeted fieldMaximumExactValuesInNumericFieldsFuzzer
- iterate through each Number and Integer fields that have maximum declared and send requests with values matching the maximum size/value in the targeted fieldMinLengthExactValuesInStringFieldsFuzzer
- iterate through each String fields that have minLength declared and send requests with values matching the minLength size/value in the targeted fieldMinimumExactValuesInNumericFieldsFuzzer
- iterate through each Number and Integer fields that have minimum declared and send requests with values matching the minimum size/value in the targeted fieldNewFieldsFuzzer
- send a 'happy' flow request and add a new field inside the request called 'catsFuzzyField'NullValuesInFieldsFuzzer
- iterate through each field and send requests with null values in the targeted fieldOnlyControlCharsInFieldsTrimValidateFuzzer
- iterate through each field and send values with control chars onlyOnlyWhitespacesInFieldsTrimValidateFuzzer
- iterate through each field and send values with unicode separators onlyOnlySingleCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values with single code point emojis onlyOnlyMultiCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values with multi code point emojis onlyRemoveFieldsFuzzer
- iterate through each request fields and remove certain fields according to the supplied 'fieldsFuzzingStrategy'StringFieldsLeftBoundaryFuzzer
- iterate through each String field and send requests with outside the range values on the left side in the targeted fieldStringFieldsRightBoundaryFuzzer
- iterate through each String field and send requests with outside the range values on the right side in the targeted fieldStringFormatAlmostValidValuesFuzzer
- iterate through each String field and get its 'format' value (i.e. email, ip, uuid, date, datetime, etc); send requests with values which are almost valid (i.e. email@yhoo. for email, 888.1.1. for ip, etc) in the targeted fieldStringFormatTotallyWrongValuesFuzzer
- iterate through each String field and get its 'format' value (i.e. email, ip, uuid, date, datetime, etc); send requests with values which are totally wrong (i.e. abcd for email, 1244. for ip, etc) in the targeted fieldStringsInNumericFieldsFuzzer
- iterate through each Integer (int, long) and Number field (float, double) and send requests having the fuzz
string value in the targeted fieldTrailingWhitespacesInFieldsTrimValidateFuzzer
- iterate through each field and send requests with trailing with Unicode whitespaces and invisible separators in the targeted fieldTrailingControlCharsInFieldsTrimValidateFuzzer
- iterate through each field and send requests with trailing with Unicode control chars in the targeted fieldTrailingSingleCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values trailed with single code point emojisTrailingMultiCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values trailed with multi code point emojisVeryLargeStringsFuzzer
- iterate through each String field and send requests with very large values (40000 characters) in the targeted fieldWithinControlCharsInFieldsSanitizeValidateFuzzer
- iterate through each field and send values containing unicode control charsWithinSingleCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values containing single code point emojisWithinMultiCodePointEmojisInFieldsTrimValidateFuzzer
- iterate through each field and send values containing multi code point emojisZalgoTextInStringFieldsValidateSanitizeFuzzer
- iterate through each field and send values containing zalgo textYou can run only these Fuzzers
by supplying the --checkFields
argument.
CATS
has currently 28 registered Header Fuzzers
:
AbugidasCharsInHeadersFuzzer
- iterate through each header and send requests with abugidas chars in the targeted headerCheckSecurityHeadersFuzzer
- check all responses for good practices around Security related headers like: [{name=Cache-Control, value=no-store}, {name=X-XSS-Protection, value=1; mode=block}, {name=X-Content-Type-Options, value=nosniff}, {name=X-Frame-Options, value=DENY}]DummyAcceptHeadersFuzzer
- send a request with a dummy Accept header and expect to get 406 codeDummyContentTypeHeadersFuzzer
- send a request with a dummy Content-Type header and expect to get 415 codeDuplicateHeaderFuzzer
- send a 'happy' flow request and duplicate an existing headerEmptyStringValuesInHeadersFuzzer
- iterate through each header and send requests with empty String values in the targeted headerExtraHeaderFuzzer
- send a 'happy' flow request and add an extra field inside the request called 'Cats-Fuzzy-Header'LargeValuesInHeadersFuzzer
- iterate through each header and send requests with large values in the targeted headerLeadingControlCharsInHeadersFuzzer
- iterate through each header and prefix values with control charsLeadingWhitespacesInHeadersFuzzer
- iterate through each header and prefix value with unicode separatorsLeadingSpacesInHeadersFuzzer
- iterate through each header and send requests with spaces prefixing the value in the targeted headerRemoveHeadersFuzzer
- iterate through each header and remove different combinations of themOnlyControlCharsInHeadersFuzzer
- iterate through each header and replace value with control charsOnlySpacesInHeadersFuzzer
- iterate through each header and replace value with spacesOnlyWhitespacesInHeadersFuzzer
- iterate through each header and replace value with unicode separatorsTrailingSpacesInHeadersFuzzer
- iterate through each header and send requests with trailing spaces in the targeted header \TrailingControlCharsInHeadersFuzzer
- iterate through each header and trail values with control charsTrailingWhitespacesInHeadersFuzzer
- iterate through each header and trail values with unicode separatorsUnsupportedAcceptHeadersFuzzer
- send a request with an unsupported Accept header and expect to get 406 codeUnsupportedContentTypesHeadersFuzzer
- send a request with an unsupported Content-Type header and expect to get 415 codeZalgoTextInHeadersFuzzer
- iterate through each header and send requests with zalgo text in the targeted headerYou can run only these Fuzzers
by supplying the --checkHeaders
argument.
CATS
has currently 6 registered HTTP Fuzzers
:
BypassAuthenticationFuzzer
- check if an authentication header is supplied; if yes try to make requests without itDummyRequestFuzzer
- send a dummy json request {'cats': 'cats'}HappyFuzzer
- send a request with all fields and headers populatedHttpMethodsFuzzer
- iterate through each undocumented HTTP method and send an empty requestMalformedJsonFuzzer
- send a malformed json request which has the String 'bla' at the endNonRestHttpMethodsFuzzer
- iterate through a list of HTTP method specific to the WebDav protocol that are not expected to be implemented by REST APIsYou can run only these Fuzzers
by supplying the --checkHttp
argument.
Usually a good OpenAPI contract must follow several good practices in order to make it easy digestible by the service clients and act as much as possible as self-sufficient documentation:
json
types and propertiesPOST
, PATCH
and PUT
requestsCorrelationIds/TraceIds
within headersxml
payload unless there is a really good reason (like documenting an old API for example)Pet
with a property named pet
)CATS
has currently 9 registered ContractInfo Fuzzers
:
HttpStatusCodeInValidRangeFuzzer
- verifies that all HTTP response codes are within the range of 100 to 599NamingsContractInfoFuzzer
- verifies that all OpenAPI contract elements follow REST API naming good practicesPathTagsContractInfoFuzzer
- verifies that all OpenAPI paths contain tags elements and checks if the tags elements match the ones declared at the top levelRecommendedHeadersContractInfoFuzzer
- verifies that all OpenAPI contract paths contain recommended headers like: CorrelationId/TraceId, etc.RecommendedHttpCodesContractInfoFuzzer
- verifies that the current path contains all recommended HTTP response codes for all operationsSecuritySchemesContractInfoFuzzer
- verifies if the OpenApi contract contains valid security schemas for all paths, either globally configured or per pathTopLevelElementsContractInfoFuzzer
- verifies that all OpenAPI contract level elements are present and provide meaningful information: API description, documentation, title, version, etc.VersionsContractInfoFuzzer
- verifies that a given path doesn't contain versioning informationXmlContentTypeContractInfoFuzzer
- verifies that all OpenAPI contract paths responses and requests does not offer application/xml
as a Content-TypeYou can run only these Fuzzers
using > cats lint --contract=CONTRACT
.
You can leverage CATS super-powers of self-healing and payload generation in order to write functional tests. This is achieved using the so called FunctionaFuzzer
, which is not a Fuzzer
per se, but was named as such for consistency. The functional tests are written in a YAML file using a simple DSL. The DSL supports adding identifiers, descriptions, assertions as well as passing variables between tests. The cool thing is that, by leveraging the fact that CATS generates valid payload, you only need to override values for specific fields. The rest of the information will be populated by CATS
using valid data, just like a 'happy' flow request.
It's important to note that reference data
won't get replaced when using the FunctionalFuzzer
. So if there are reference data fields, you must also supply those in the FunctionalFuzzer
.
The FunctionalFuzzer
will only trigger if a valid functionalFuzzer.yml
file is supplied. The file has the following syntax:
/path:
testNumber:
description: Short description of the test
prop: value
prop#subprop: value
prop7:
- value1
- value2
- value3
oneOfSelection:
element#type: "Value"
expectedResponseCode: HTTP_CODE
httpMethod: HTTP_NETHOD
And a typical run will look like:
> cats run functionalFuzzer.yml -c contract.yml -s http://localhost:8080
This is a description of the elements within the functionalFuzzer.yml
file:
description
of the test. This will be set as the Scenario
description. If you don't supply a description
the testNumber
will be used instead.test1
, test2
, etc.expectedResponseCode
is mandatory, otherwise the Fuzzer
will ignore this test. The expectedResponseCode
tells CATS what to expect from the service when sending this test.prop7
has 3 values. This will actually result in 3 tests, one for each value.httpMethod
doesn't exist in the OpenAPI given path, a warning
will be issued and no test will be executedhttpMethod
is not a valid HTTP method, a warning
will be issued and no test will be executedoneOf
element to allow multiple request types, you can control which of the possible types the FunctionalFuzzer
will apply to using the oneOfSelection
keyword. The value of the oneOfSelection
keyword must match the fully qualified name of the discriminator
.oneOfSelection
is supplied, and the request payload accepts multiple oneOf
elements, than a custom test will be created for each type of payload#
as in the example above instead of .
When you have request payloads which can take multiple object types, you can use the oneOfSelection
keyword to specify which of the possible object types is required by the FunctionalFuzzer
. If you don't provide this element, all combinations will be considered. If you supply a value, this must be exactly the one used in the discriminator
.
As CATs mostly relies on generated data with small help from some reference data, testing complex business scenarios with the pre-defined Fuzzers
is not possible. Suppose we have an endpoint that creates data (doing a POST
), and we want to check its existence (via GET
). We need a way to get some identifier from the POST call and send it to the GET call. This is now possible using the FunctionalFuzzer
. The functionalFuzzerFile
can have an output
entry where you can state a variable name, and its fully qualified name from the response in order to set its value. You can then refer the variable using ${variable_name}
from another test in order to use its value.
Here is an example:
/pet:
test_1:
description: Create a Pet
httpMethod: POST
name: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id
/pet/{id}:
test_2:
description: Get a Pet
id: ${petId}
expectedResponseCode: 200
Suppose the test_1
execution outputs:
{
"pet":
{
"id" : 2
}
}
When executing test_1
the value of the pet id will be stored in the petId
variable (value 2
). When executing test_2
the id
parameter will be replaced with the petId
variable (value 2
) from the previous case.
Please note: variables are visible across all custom tests; please be careful with the naming as they will get overridden.
The FunctionalFuzzer
can verify more than just the expectedResponseCode
. This is achieved using the verify
element. This is an extended version of the above functionalFuzzer.yml
file.
/pet:
test_1:
description: Create a Pet
httpMethod: POST
name: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id
verify:
pet#name: "Baby"
pet#id: "[0-9]+"
/pet/{id}:
test_2:
description: Get a Pet
id: ${petId}
expectedResponseCode: 200
Considering the above file:
FunctionalFuzzer
will check if the response has the 2 elements pet#name
and pet#id
pet#name
has the Baby
value and that the pet#id
is numericThe following json response will pass test_1
:
{
"pet":
{
"id" : 2,
"name": "Baby"
}
}
But this one won't (pet#name
is missing):
{
"pet":
{
"id" : 2
}
}
You can also refer to request fields in the verify
section by using the ${request#..}
qualifier. Using the above example, by having the following verify
section:
/pet:
test_1:
description: Create a Pet
httpMethod: POST
name: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id
verify:
pet#name: "${request#name}"
pet#id: "[0-9]+"
It will verify if the response contains a pet#name
element and that its value equals My Pet
as sent in the request.
Some notes:
verify
parameters support Java regexes as valuesCATs
will report an errorCATs
will report a warningCATs
will report a successYou can also set additionalProperties
fields through the functionalFuzzerFile
using the same syntax as for Setting additionalProperties in Reference Data.
The following keywords are reserved in FunctionalFuzzer
tests: output
, expectedResponseCode
, httpMethod
, description
, oneOfSelection
, verify
, additionalProperties
, topElement
and mapValues
.
Although CATs
is not a security testing tool, you can use it to test basic security scenarios by fuzzing specific fields with different sets of nasty strings. The behaviour is similar to the FunctionalFuzzer
. You can use the exact same elements for output variables, test correlation, verify responses and so forth, with the addition that you must also specify a targetFields
and/or targetFieldTypes
and a stringsList
element. A typical securityFuzzerFile
will look like this:
/pet:
test_1:
description: Run XSS scenarios
name: "My Pet"
expectedResponseCode: 200
httpMethod: all
targetFields:
- pet#id
- pet#description
stringsFile: xss.txt
And a typical run:
> cats run securityFuzzerFile.yml -c contract.yml -s http://localhost:8080
You can also supply output
, httpMethod
, oneOfSelection
and/or verify
(with the same behaviour as within the FunctionalFuzzer
) if they are relevant to your case.
The file uses Json path syntax for all the properties you can supply; you can separate elements through #
as in the example instead of .
.
This is what the SecurityFuzzer
will do after parsing the above securityFuzzerFile
:
name
targetFields
i.e. pet#id
and pet#description
it will create requests for each line from the xss.txt
file and supply those values in each fieldxss.txt
sample file included in the CATs
repo, this means that it will send 21 requests targeting pet#id
and 21 requests targeting pet#description
i.e. a total of 42 tests
SecurityFuzzer
will expect a 200
response code. If another response code is returned, then CATs
will report the test as error
.If you want the above logic to apply to all paths, you can use all
as the path name:
all:
test_1:
description: Run XSS scenarios
name: "My Pet"
expectedResponseCode: 200
httpMethod: all
targetFields:
- pet#id
- pet#description
stringsFile: xss.txt
Instead of specifying the field names, you can broader to scope to target certain fields types. For example, if we want to test for XSS in all string
fields, you can have the following securityFuzzerFile
:
all:
test_1:
description: Run XSS scenarios
name: "My Pet"
expectedResponseCode: 200
httpMethod: all
targetFieldTypes:
- string
stringsFile: xss.txt
As an idea on how to create security tests, you can split the nasty strings into multiple files of interest in your particular context. You can have a sql_injection.txt
, a xss.txt
, a command_injection.txt
and so on. For each of these files, you can create a test entry in the securityFuzzerFile
where you include the fields you think are meaningful for these types of tests. (It was a deliberate choice (for now) to not include all fields by default.) The expectedResponseCode
should be tweaked according to your particular context. Your service might sanitize data before validation, so might be perfectly valid to expect a 200
or might validate the fields directly, so might be perfectly valid to expect a 400
. A 500
will usually mean something was not handled properly and might signal a possible bug.
You can also set additionalProperties
fields through the functionalFuzzerFile
using the same syntax as for Setting additionalProperties in Reference Data.
The following keywords are reserved in SecurityFuzzer
tests: output
, expectedResponseCode
, httpMethod
, description
, verify
, oneOfSelection
, targetFields
, targetFieldTypes
, stringsFile
, additionalProperties
, topElement
and mapValues
.
The TemplateFuzzer
can be used to fuzz non-OpenAPI endpoints. If the target API does not have an OpenAPI spec available, you can use a request template to run a limited set of fuzzers. The syntax for running the TemplateFuzzer
is as follows (very similar to curl
:
> cats fuzz -H header=value -X POST -d '{"field1":"value1","field2":"value2","field3":"value3"}' -t "field1,field2,header" -i "2XX,4XX" http://service-url
The command will:
POST
request to http://service-url
{"field1":"value1","field2":"value2","field3":"value3"}
as a templatefield1,field2,header
with fuzz data and send each request to the service endpoint2XX,4XX
response codes and report an error when the received response code is not in this listIt was a deliberate choice to limit the fields for which the Fuzzer
will run by supplying them using the -t
argument. For nested objects, supply fully qualified names: field.subfield
.
Headers can also be fuzzed using the same mechanism as the fields.
This Fuzzer
will send the following type of data:
For a full list of options run > cats fuzz -h
.
You can also supply your own dictionary of data using the -w file
argument.
HTTP methods with bodies will only be fuzzed at the request payload and headers level.
HTTP methods without bodies will be fuzzed at path and query parameters and headers level. In this case you don't need to supply a -d
argument.
This is an example for a GET
request:
> cats fuzz -X GET -t "path1,query1" -i "2XX,4XX" http://service-url/paths1?query1=test&query2
There are often cases where some fields need to contain relevant business values in order for a request to succeed. You can provide such values using a reference data file specified by the --refData
argument. The reference data file is a YAML-format file that contains specific fixed values for different paths in the request document. The file structure is as follows:
/path/0.1/auth:
prop#subprop: 12
prop2: 33
prop3#subprop1#subprop2: "test"
/path/0.1/cancel:
prop#test: 1
For each path you can supply custom values for properties and sub-properties which will have priority over values supplied by any other Fuzzer
. Consider this request payload:
{
"address": {
"phone": "123",
"postCode": "408",
"street": "cool street"
},
"name": "Joe"
}
and the following reference data file file:
/path/0.1/auth:
address#street: "My Street"
name: "John"
This will result in any fuzzed request to the /path/0.1/auth
endpoint being updated to contain the supplied fixed values:
{
"address": {
"phone": "123",
"postCode": "408",
"street": "My Street"
},
"name": "John"
}
The file uses Json path syntax for all the properties you can supply; you can separate elements through #
as in the example above instead of .
.
You can use environment (system) variables in a ref data file using: $$VARIABLE_NAME
. (notice double $$
)
As additional properties are maps i.e. they don't actually have a structure, CATS cannot currently generate valid values. If the elements within such a data structure are essential for a request, you can supply them via the refData
file using the following syntax:
/path/0.1/auth:
address#street: "My Street"
name: "John"
additionalProperties:
topElement: metadata
mapValues:
test: "value1"
anotherTest: "value2"
The additionalProperties
element must contain the actual key-value pairs to be sent within the requests and also a top element if needed. topElement
is not mandatory. The above example will output the following json (considering also the above examples):
{
"address": {
"phone": "123",
"postCode": "408",
"street": "My Street"
},
"name": "John",
"metadata": {
"test": "value1",
"anotherTest": "value2"
}
}
The following keywords are reserved in a reference data file: additionalProperties
, topElement
and mapValues
.
You can also have the ability to send the same reference data for ALL paths (just like you do with the headers). You can achieve this by using all
as a key in the refData
file:
all:
address#zip: 123
This will try to replace address#zip
in all requests (if the field is present).
There are (rare) cases when some fields may not make sense together. Something like: if you send firstName
and lastName
, you are not allowed to also send name
. As OpenAPI does not have the capability to send request fields which are dependent on each other, you can use the refData
file to instruct CATS to remove fields before sending a request to the service. You can achieve this by using the cats_remove_field
as a value for the fields you want to remove. For the above case the refData
field will look as follows:
all:
name: "cats_remove_field"
You can leverage the fact that the FunctionalFuzzer
can run functional flows in order to create dynamic --refData
files which won't need manual setting the reference data values. The --refData
file must be created with variables ${variable}
instead of fixed values and those variables must be output variables in the functionalFuzzer.yml
file. In order for the FunctionalFuzzer
to properly replace the variables names with their values you must supply the --refData
file as an argument when the FunctionalFuzzer
runs.
> cats run functionalFuzzer.yml -c contract.yml -s http://localhost:8080 --refData=refData.yml
The functionalFuzzer.yml
file:
/pet:
test_1:
description: Create a Pet
httpMethod: POST
name: "My Pet"
expectedResponseCode: 200
output:
petId: pet#id
The refData.yml
file:
/pet-type:
id: ${petId}
After running CATS using the command and the 2 files above, you will get a refData_replace.yml
file where the id
will get the value returned into the petId
variable.
The refData_replaced.yml
:
/pet-type:
id: 123
You can now use the refData_replaced.yml
as a --refData
file for running CATS with the rest of the Fuzzers.
This can be used to send custom fixed headers with each payload. It is useful when you have authentication tokens you want to use to authenticate the API calls. You can use path specific headers or common headers that will be added to each call using an all
element. Specific paths will take precedence over the all
element. Sample headers file:
all:
Accept: application/json
/path/0.1/auth:
jwt: XXXXXXXXXXXXX
/path/0.2/cancel:
jwt: YYYYYYYYYYYYY
This will add the Accept
header to all calls and the jwt
header to the specified paths. You can use environment (system) variables in a headers file using: $$VARIABLE_NAME
. (notice double $$
)
DELETE
is the only HTTP verb that is intended to remove resources and executing the same DELETE
request twice will result in the second one to fail as the resource is no longer available. It will be pretty heavy to supply a large list of identifiers within the --refData
file and this is why the recommendation was to skip the DELETE
method when running CATS.
But starting with version 7.0.2 CATS has some intelligence in dealing with DELETE
. In order to have enough valid entities CATS will save the corresponding POST
requests in an internal Queue, and everytime a DELETE
request it will be executed it will poll data from there. In order to have this actually working, your contract must comply with common sense conventions:
DELETE
path is actually the POST
path plus an identifier: if POST is /pets
, then DELETE is expected to be /pets/{petId}
.{petId}
parameter within the body returned by the POST
request while doing various combinations of the petId
name. It will try to search for the following entries: petId, id, pet-id, pet_id
with different cases.POST
result, it will replace the {petId}
with that valueFor example, suppose that a POST to /pets
responds with:
{
"pet_id": 2,
"name": "Chuck"
}
When doing a DELETE
request, CATS will discover that {petId}
and pet_id
are used as identifiers for the Pet
resource, and will do the DELETE
at /pets/2
.
If these conventions are followed (which also align to good REST naming practices), it is expected that DELETE
and POST
requests will be on-par for most of the entities.
Some APIs might use content negotiation versioning which implies formats like application/v11+json
in the Accept
header.
You can handle this in CATS as follows:
requestBody:
required: true
content:
application/v5+json:
schema:
$ref: '#/components/RequestV5'
application/v6+json:
schema:
$ref: '#/components/RequestV6'
by having clear separation between versions, you can pass the --contentType
argument with the version you want to test: cats ... --contentType="application/v6+json"
.
If the OpenAPI contract is not version aware (you already exported it specific to a version) and the content looks as:
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/RequestV5'
and you still need to pass the application/v5+json
Accept
header, you can use the --headers
file to add it:
all:
Accept: "application/v5+json"
There isn't a consensus on how you should handle situations when you trail or prefix valid values with spaces. One strategy will be to have the service trimming spaces before doing the validation, while some other services will just validate them as they are. You can control how CATS should expect such cases to be handled by the service using the --edgeSpacesStrategy
argument. You can set this to trimAndValidate
or validateAndTrim
depending on how you expect the service to behave:
trimAndValidate
means that the service will first trim the spaces and after that run the validationvalidateAndTrim
means that the service runs the validation first without any trimming of spacesThis is a global setting i.e. configured when CATS starts and all Fuzzer
expects a consistent behaviour from all the service endpoints.
There are cases when certain parts of the request URL are parameterized. For example a case like: /{version}/pets
. {version}
is supposed to have the same value for all requests. This is why you can supply actual values to replace such parameters using the --urlParams
argument. You can supply a ;
separated list of name:value
pairs to replace the name
parameters with their corresponding value
. For example supplying --urlParams=version:v1.0
will replace the version
parameter from the above example with the value v1.0
.
CATS also supports schemas with oneOf
, allOf
and anyOf
composition. CATS wil consider all possible combinations when creating the fuzzed payloads.
The following configuration files: securityFuzzerFile, functionalFuzzerFile, refData
support setting dynamic values for the inner fields. For now the support only exists for java.time.*
and org.apache.commons.lang3.*
, but more types of elements will come in the near future.
Let's suppose you have a date/date-time field, and you want to set it to 10 days from now. You can do this by setting this as a value T(java.time.OffsetDateTime).now().plusDays(10)
. This will return an ISO compliant time in UTC format.
A functionalFuzzer
using this can look like:
/path:
testNumber:
description: Short description of the test
prop: value
prop#subprop: "T(java.time.OffsetDateTime).now().plusDays(10)"
prop7:
- value1
- value2
- value3
oneOfSelection:
element#type: "Value"
expectedResponseCode: HTTP_CODE
httpMethod: HTTP_NETHOD
You can also check the responses using a similar syntax and also accounting for the actual values returned in the response. This is a syntax than can test if a returned date is after the current date: T(java.time.LocalDate).now().isBefore(T(java.time.LocalDate).parse(expiry.toString()))
. It will check if the expiry
field returned in the json response, parsed as date, is after the current date.
The syntax of dynamically setting dates is compliant with the Spring Expression Language specs.
If you need to run CATS behind a proxy, you can supply the following arguments: --proxyHost
and --proxyPort
. A typical run with proxy settings on localhost:8080
will look as follows:
> cats --contract=YAML_FILE --server=SERVER_URL --proxyHost=localhost --proxyPort=8080
CATS supports any form of HTTP header(s) based authentication (basic auth, oauth, custom JWT, apiKey, etc) using the headers mechanism. You can supply the specific HTTP header name and value and apply to all
endpoints. Additionally, basic auth is also supported using the --basicauth=USR:PWD
argument.
By default, CATS trusts all server certificates and doesn't perform hostname verification.
For two-way SSL you can specify a JKS file (Java Keystore) that holds the client's private key using the following arguments:
--sslKeystore
Location of the JKS keystore holding certificates used when authenticating calls using one-way or two-way SSL--sslKeystorePwd
The password of the sslKeystore
--sslKeyPwd
The password of the private key within the sslKeystore
For details on how to load the certificate and private key into a Java Keystore you can use this guide: https://mrkandreev.name/blog/java-two-way-ssl/.
When using the native binaries (not the uberjar) there might be issues when using dynamic values in the CATS files. This is due to the fact that GraalVM only bundles whatever can discover at compile time. The following classes are currently supported:
java.util.Base64.Encoder.class, java.util.Base64.Decoder.class, java.util.Base64.class, org.apache.commons.lang3.RandomUtils.class, org.apache.commons.lang3.RandomStringUtils.class,
org.apache.commons.lang3.DateFormatUtils.class, org.apache.commons.lang3.DateUtils.class,
org.apache.commons.lang3.DurationUtils.class, java.time.LocalDate.class, java.time.LocalDateTime.class, java.time.OffsetDateTime.class
At this moment, CATS only works with OpenAPI specs and has limited functionality using template payloads through the cats fuzz ...
subcommand.
The Fuzzers
has the following support for media types and HTTP methods:
application/json
and application/x-www-form-urlencoded
media types onlyPOST
, PUT
, PATCH
, GET
and DELETE
If a response contains a free Map specified using the additionalParameters
tag CATS will issue a WARN
level log message as it won't be able to validate that the response matches the schema.
CATS uses RgxGen in order to generate Strings based on regexes. This has certain limitations mostly with complex patterns.
All custom files that can be used by CATS (functionalFuzzerFile
, headers
, refData
, etc) are in a YAML format. When setting or getting values to/from JSON for input and/or output variables, you must use a JsonPath syntax using either #
or .
as separators. You can find some selector examples here: JsonPath.
Please refer to CONTRIBUTING.md.