isona/dirble

Introduction

Dirble is a website directory scanning tool for Windows and Linux. It's designed to be fast to run and easy to use.

How to Use

Download one of the https://github.com/nccgroup/dirble/releases/latest for Linux, Windows, or Mac, or compile the source using Cargo, then run it from a terminal. The default wordlist Dirble uses is dirble_wordlist.txt in the same directory as the executable.

It can be installed in BlackArch using sudo pacman -S dirble

There is also a docker image, which can be run as: docker run --rm -t isona/dirble [dirble arguments]

The help text can be displayed using dirble --help, alternatively it can be found on the github wiki: https://github.com/nccgroup/dirble/wiki/Help-Text

Example Uses

Run against a website using the default dirble_wordlist.txt from the current directory: dirble [address]

Run with a different wordlist and including .php and .html extensions: dirble [address] -w example_wordlist.txt -x .php,.html

With listable directory scraping enabled: dirble [address] --scrape-listable

Providing a list of extensions and a list of hosts: dirble [address] -X wordlists/web.lst -U hostlist.txt

Providing multiple hosts to scan via command line: dirble [address] -u [address] -u [address]

Running with threading in Gobuster's default style, disabling recursion and having 10 threads scanning the main directory: dirble [address] --max-threads 10 --wordlist-split 10 -r

Building from source

To build on your current platform, ensure cargo is installed and then run cargo build --release. Alternatively, running make will build the binary in release mode (internally running cargo build --release).

To cross-compile for 32- and 64-bit Linux and Windows targets, there is a handy makefile. make release will build for all four targets using cross. This depends on having cross and docker installed (cargo install cross).

Features

• Cookies
• Custom Headers
• Extensions and prefixes
• HTTP basic auth
• Listable directory detection and scraping
• Save ouptut to file
• Save output in XML and JSON formats
• Proxy support
• Recursion
• Status code blacklisting and whitelisting
• Threading
• Request throttling
• Detect not found code of each directory based on response code and length
• Ability to provide list of URLs to be scanned
• User agents
• Scanning with GET, POST or HEAD requests
• Exclude ranges of response lengths from output

Performance

The following graph was generated by running each tool with Hyperfine against a test server with 5ms latency and 1% packet loss. (Gobuster was omitted due to lack of recursion).

!This is a cool graph

How it works

Directory Detection

Dirble detects files based on the response code sent by the server. The behaviour can be loosely categorised by response code type.

• 200: the path exists and is valid
• 301, 302: redirection; report the code, size, and Location header
• 404: not found; by default these responses are not reported
• All other response codes are reported in the Dirble format of + [url] (CODE:[code]|SIZE:[size])

A path is classified as a directory if a request to [url] (with no trailing slash) returns a 301 or 302 redirection to [url]/ (with a trailing slash). This gets reported with a D prefix and if recursion is enabled will be added to the scan queue. This method is not dependent on the redirection target existing or being accessible, so a separate request will be made to determine the response code and size of the directory.

Listable directories are detected by inspecting the content of url/: if it returns a 200 response code and the body contains either "parent directory", "up to " or "directory listing for" (case insensitive), then it is likely to be a listable directory. If --scrape-listable is enabled, URLs are parsed out of the listing (ignoring sorting links or out of scope links) and added to the scan queue if they have a trailing slash. Listable directories have an L prefix in the output.

Threading

The threading behaviour of Dirble is based on the concepts of wordlists and jobs. A job is any task which can be run independently of other tasks, for example requesting a series of URLs. A wordlist is a list of words with a defined transformation, for example the list {admin, config, shop} together with the transformation append ".php" forms a single wordlist instance.

To improve performance further, we introduce the concept of wordlist splitting. This is the process by which a single wordlist instance (i.e. words with a transformation) is broken up into multiple jobs, each responsible for a portion of the list. The number of interleaved portions that each wordlist is split into is defined by the --wordlist-split option (default 3).

Whenever a directory is detected (and recursion is enabled) new jobs are created for each split wordlist (with transformation) and added to a central job queue.

The maximum number of concurrent tasks is defined by the --max-threads parameter, and Dirble will start jobs as they are added to the queue, up to this limit. Whenever a job completes (i.e. a split wordlist is exhausted) Dirble will take the next job from the queue and start it.

Released under GPL v3.0, see LICENSE for more information