Leaktopus - Keep Your Source Code Under Control

• Plug&Play - one line installation with Docker.

• Scan various sources containing a set of keywords, e.g. ORGANIZATION-NAME.com.

  Currently supports:

  • GitHub
    • Repositories
    • Gists (coming soon)
  • Paste sites (e.g., PasteBin) (coming soon)

• Filter results with a built-in heuristic engine.

• Enhance results with IOLs (Indicators Of Leak):

  • Secrets in the found sources (including Git repos commits history):
    • With Shhgit (using a customized rules list).
    • With TruffleHog.
  • URIs (Including indication of your organization's domains)
  • Emails (Including indication of your organization's email addresses)
  • Contributors
  • Sensitive keywords (e.g., canary token, internal domains)

• Allows to ignore public sources, (e.g., "junk" repositories by web crawlers).

• OOTB ignore list of common "junk" sources.

• Acknowledge a leak, and only get notified if the source has been modified since the previous scan.

• Built-in ELK to search for data in leaks (including full index of Git repositories with IOLs).

• Notify on new leaks

• MS Teams Webhook.
• Slack Bot.
• Cortex XSOAR® (by Palo Alto Networks) Integration (WIP).

Technology Stack

• Fully Dockerized.
• API-first Python Flask backend.
• Decoupled Vue.js (3.x) frontend.
• SQLite DB.
• Async tasks with Celery + Redis queues.

Prerequisites

• Docker-Compose

Installation

• Clone the repository
• Create a local .env file

  cd Leaktopus
  cp .env.example .env

• Edit .env according to your local setup (see the internal comments).
• Run Leaktopus

  docker-compose up -d

• Initiate the installation sequence by accessing the installation API. Just open http://{LEAKTOPUS_HOST}:8000/api/install in your browser.
• Check that the API is up and running at http://{LEAKTOPUS_HOST}:8000/up
• The UI should be available at http://{LEAKTOPUS_HOST}:8080

Using Github App

In addition to the basic personal access token option, Leaktopus supports Github App authentication. Using Github App is recommended due to the increased rate limits.

1. To use Github App authentication, you need to create a Github App and install it on your organization/account. See Github's documentation for more details.

2. After creating the app, you need to set the following environment variables:

   • GITHUB_USE_APP=True
   • GITHUB_APP_ID
   • GITHUB_INSTALLATION_ID - The installation id can be found in your app installation.
   • GITHUB_APP_PRIVATE_KEY_PATH (defaults to /app/private-key.pem)

3. Mount the private key file to the container (see docker-compose.yml for an example). ./leaktopus_backend/private-key.pem:/app/private-key.pem

* Note that GITHUB_ACCESS_TOKEN will be ignored if GITHUB_USE_APP is set to True.

Updating Leaktopus

If you wish to update your Leaktopus version (pulling a newer version), just follow the next steps.

• Pull the latest version.

  git pull

• Rebuild Docker images (data won't be deleted).

  # Force image recreation
  docker-compose up --force-recreate --build

• Run the DB update by calling its API (should be required after some updates). http://{LEAKTOPUS_HOST}/api/updatedb

Results Filtering Heuristic Engine

The built-in heuristic engine is filtering the search results to reduce false positives by:

• Content:
  • More than X emails containing non-organizational domains.
  • More than X URIs containing non-organizational domains.
• Metadata:
  • More than X stars.
  • More than X forks.
• Sources ignore list.

API Documentation

OpenAPI documentation is available in http://{LEAKTOPUS_HOST}:8000/apidocs.

Leaktopus Services

The above can be customized by using a custom docker-compose.yml file.

Security Notes

As for now, Leaktopus does not provide any authentication mechanism. Make sure that you are not exposing it to the world, and doing your best to restrict access to your Leaktopus instance(s).

Contributing

Contributions are very welcomed.

Please follow our contribution guidelines and documentation.

HTTPLoot - An Automated Tool Which Can Simultaneously Crawl, Fill Forms, Trigger Error/Debug Pages And "Loot" Secrets Out Of The Client-Facing Code Of Sites

An automated tool which can simultaneously crawl, fill forms, trigger error/debug pages and "loot" secrets out of the client-facing code of sites.

Usage

To use the tool, you can grab any one of the pre-built binaries from the Releases section of the repository. If you want to build the source code yourself, you will need Go > 1.16 to build it. Simply running go build will output a usable binary for you.

Additionally you will need two json files (lootdb.json and regexes.json) alongwith the binary which you can get from the repo itself. Once you have all 3 files in the same folder, you can go ahead and fire up the tool.

Video demo:

Here is the help usage of the tool:

$ ./httploot --help
      _____
       )=(
      /   \     H T T P L O O T
     (  $  )                  v0.1
      \___/

[+] HTTPLoot by RedHunt Labs - A Modern Attack Surface (ASM) Management Company
[+] Author: Pinaki Mondal (RHL Research Team)
[+] Continuously Track Your Attack Surface using https://redhuntlabs.com/nvadr.

Usage of ./httploot:
  -concurrency int
        Maximum number of sites to process concurrently (default 100)
  -depth int
        Maximum depth limit to traverse while crawling (default 3)
  -form-length int
        Length of the string to be randomly generated for filling form fields (default 5)
  -form-string string
        Value with which the tool will auto-fill forms, strings will be randomly generated if no value is supplied
  -input-file string
        Path of the input file conta   ining domains to process
  -output-file string
        CSV output file path to write the results to (default "httploot-results.csv")
  -parallelism int
        Number of URLs per site to crawl parallely (default 15)
  -submit-forms
        Whether to auto-submit forms to trigger debug pages
  -timeout int
        The default timeout for HTTP requests (default 10)
  -user-agent string
        User agent to use during HTTP requests (default "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0")
  -verify-ssl
        Verify SSL certificates while making HTTP requests
  -wildcard-crawl
        Allow crawling of links outside of the domain being scanned

Concurrent scanning

There are two flags which help with the concurrent scanning:

• -concurrency: Specifies the maximum number of sites to process concurrently.
• -parallelism: Specifies the number of links per site to crawl parallely.

Both -concurrency and -parallelism are crucial to performance and reliability of the tool results.

Crawling

The crawl depth can be specified using the -depth flag. The integer value supplied to this is the maximum chain depth of links to crawl grabbed on a site.

An important flag -wildcard-crawl can be used to specify whether to crawl URLs outside the domain in scope.

NOTE: Using this flag might lead to infinite crawling in worst case scenarios if the crawler finds links to other domains continuously.

Filling forms

If you want the tool to scan for debug pages, you need to specify the -submit-forms argument. This will direct the tool to autosubmit forms and try to trigger error/debug pages once a tech stack has been identified successfully.

If the -submit-forms flag is enabled, you can control the string to be submitted in the form fields. The -form-string specifies the string to be submitted, while the -form-length can control the length of the string to be randomly generated which will be filled into the forms.

Network tuning

Flags like:

• -timeout - specifies the HTTP timeout of requests.
• -user-agent - specifies the user-agent to use in HTTP requests.
• -verify-ssl - specifies whether or not to verify SSL certificates.

Input/Output

Input file to read can be specified using the -input-file argument. You can specify a file path containing a list of URLs to scan with the tool. The -output-file flag can be used to specify the result output file path -- which by default goes into a file called httploot-results.csv.

Further Details

Further details about the research which led to the development of the tool can be found on our RedHunt Labs Blog.

License & Version

The tool is licensed under the MIT license. See LICENSE.

Currently the tool is at v0.1.

Credits

The RedHunt Labs Research Team would like to extend credits to the creators & maintainers of shhgit for the regular expressions provided by them in their repository.

To know more about our Attack Surface Management platform, check out NVADR.