KitPloit - PenTest Tools!

Witcher - Managing GitHub Advanced Security (GHAS) Controls At Scale

Implement and monitor Appsec control at scale.

Requirements

• NodeJS 20.13

Tested on

• Mac
• Ubuntu

How to install

$ git clone git@github.com:mf-labs/witcher.git
$ cd witcher
$ npm i

Build a Docker image

$ git clone git@github.com:mf-labs/witcher.git
$ cd witcher
$ docker build -t witcher .

# Running docker image
$ docker run -e GITHUB_TOKEN=$GITHUB_TOKEN -e ORG=$ORG witcher -a status -m ghas -r offsec-sast-testing

witcher's features

➜  witcher git:(master) node witcher.js -h
usage: witcher.js [-h] -m MODULE -a ACTION [--daily-summary] [--mass-action] [--slack] [--siem] [--jira]
                  [--jira-ticket JIRATICKET] [--org ORG] [-r REPO] [-b BRANCH]
                  [--workflow-file WORKFLOW] [--repo-file REPOFILE]

witcher ....... you can't escape

optional arguments:
  -h, --help            show this help message and exit
  -m MODULE, --module MODULE
                        ghas, dependabot, secret-scanning, codeql, iac, workflows, ALL
  -a ACTION, --action ACTION
                        enable, disbale, status, alert, deploy, delete
  --daily-summary       Get the Daily Summary
  --mass-action         Perform action (enable, deploy, delete) at scale
  --slack               Post new alert(s) on Slack
  --siem                Log activities on SIEM
  --jira                Post new vulnerability ticket on Jira
  --jira-ticket JIRATICKET
                        Jira ticket ID (e.g. PROJECT-123)

Input:
  --org ORG             Organization Name
  -r REPO, --repo REPO  Repository Name, ALL
  -b BRANCH, --branch BRANCH
                        Branch Name
  --workflow-file WORKFLOW
                        Workflow File Name
  --repo-file REPOFILE  Repo File Name

Required Environment Variable

Set the following environment variable first

 export GITHUB_TOKEN=YOUR_GITHUB_TOKEN
 export GITHUB_USER=YOUR_GITHUB_USERNAME
 export ORG=YOUR_GITHUB_ORGANIZATION

 # Optional to configure slack
 export SLACK_BOT_TOKEN
 export SLACK_SIGNING_SECRET
 export SLACK_CHANNEL

 # Optional to send data to SIEM
 export SERVERLESS_APP_URL

 # Optional for Jira ticket creation
 export JIRA_API_TOKEN
 export JIRA_EMAIL
 export JIRA_URL
 export JIRA_PROJECT
 export JIRA_ISSUE_TYPE

Exclusion

Update the github/data/exclusion.json file with list of repositories excluded from Core Repositories / GHAS.

Command cheatsheet

# List repositories where GHAS is disabled
$ node witcher.js -m ghas -a status --repo All

# Enable GHAS on certain repo
$ node witcher.js -m ghas -a enable --repo <repo-name>

# Disable GHAS on certain repo
$ node witcher.js -m ghas -a disable --repo <repo-name>

# Check GHAS status on certain repo
$ node witcher.js -m ghas -a status --repo <repo-name>

# Get latest code scanning vulnerability
$ node witcher.js -m codeql -a alert --slack   // --slack to post on slack

# Mass Action
$ node witcher.js --mass-action -a enable -m ghas --repo-file mass_action.txt --jira-ticket PROJECT-123

More Commands

More Command / Cheatsheet

Daily Routine

# Run Daily Summary
$ node witcher.js --daily-summary -m ALL -a status --slack --jira

# Daily Summary includes the checking of
# 1. GHAS status on all repositories
# 2. Secret Scanning status on all repositories
# 3. Check for Depenabot status
# 4. Check for paused Dependabot
# 5. Code Scanning status on applicable repositories
# 6. IaC Scanning status on applicable repositories
# 7. Check alerts for any new vulnerability
# 8. Logged daily summary on SIEM and posted on Slack

Disclaimer

- All public repositories are excluded from witcher
- All archived repositories are excluded from witcher
- All deprecated repositories are excluded from witcher

Roadmap

• Custom Security Controls Monitoring: Add support for monitoring custom controls beyond CodeQL, IaC, and Dependabot.
• Customizable Daily Summary: Allow users to add additional control statuses to daily reports.
• CLI & JSON Output Support: Enable full output options via CLI arguments for both CLI and JSON formats.

KitPloit - PenTest Tools!

ByeDPIAndroid - App To Bypass Censorship On Android

Android application that runs a local VPN service to bypass DPI (Deep Packet Inspection) and censorship.

This application runs a SOCKS5 proxy ByeDPI and redirects all traffic through it.

Installation

Or use Obtainium

1. Install Obtainium
2. Add the app by URL:
   https://github.com/dovecoteescapee/ByeDPIAndroid

Settings

To bypass some blocks, you may need to change the settings. More about the various settings can be found in the ByeDPI documentation.

FAQ

I can't configure it. What to do?

You can ask for help in discussion.

Does the application require root access?

No. All application features work without root.

Is this a VPN?

No. The application uses the VPN mode on Android to redirect traffic, but does not send anything to a remote server. It does not encrypt traffic and does not hide your IP address.

How to use ByeDPI with AdGuard?

1. Run ByeDPI in proxy mode.
2. Add ByeDPI to AdGuard exceptions on the "App management" tab.
3. In AdGuard settings, specify the proxy:

plaintext Proxy type: SOCKS5 Proxy host: 127.0.0.1 Proxy port: 1080 (default)

What data does the application collect?

None. The application does not send any data to a remote server. All traffic is processed on the device.

Are there any for other platforms?

Similar projects)

What is DPI?

DPI (Deep Packet Inspection) is a technology for analyzing and filtering traffic. It is used by providers and government agencies to block sites and services.

Dependencies

• ByeDPI
• hev-socks5-tunnel

Building

For building the application, you need:

1. JDK 8 or later
2. Android SDK
3. Android NDK
4. CMake 3.22.1 or later

To build the application:

1. Clone the repository with submodules: bash git clone --recurse-submodules
2. Run the build script from the root of the repository: bash ./gradlew assembleRelease
3. The APK will be in app/build/outputs/apk/release/

KitPloit - PenTest Tools!

API-s-for-OSINT - List Of API's For Gathering Information About Phone Numbers, Addresses, Domains Etc

Thank you for following me! https://cybdetective.com

IOT/IP Search engines

Universal OSINT APIs

Phone Number Lookup and Verification

Address/ZIP codes lookup

People and documents verification

Business/Entity search

Domain/DNS/IP lookup

Mobile Apps Endpoints

Scraping

Whois

GEO IP

Wi-fi lookup

Network

Finance

Email

Names/Surnames

Pastebin/Leaks

Archives

Hashes decrypt/encrypt

Crypto

Malware

Face Search

## Face Detection

## Reverse Image Search

## AI Geolocation

Social Media and Messengers

UNOFFICIAL APIs

!WARNING Use with caution! Accounts may be blocked permanently for using unofficial APIs.

Search Engines

| Memex Marginalia | https://memex.marginalia.nu/projects/edge/api.gmi | An API for new privacy search engine | FREE |

News analyze

Darknet

Torrents/file sharing

Vulnerabilities

Flights

Webcams

## Regex

API testing tools

Curl converters (tools that help to write code using API queries)

Create your own API

Distribute your own API

API Keys Info

API directories

If you don't find what you need, try searching these directories.

How to learn how to work with REST API?

If you don't know how to work with the REST API, I recommend you check out the Netlas API guide I wrote for Netlas.io.

Netlas Cookbook

There it is very brief and accessible to write how to automate requests in different programming languages (focus on Python and Bash) and process the resulting JSON data.

Thank you for following me! https://cybdetective.com

KitPloit - PenTest Tools!

Firecrawl-Mcp-Server - Official Firecrawl MCP Server - Adds Powerful Web Scraping To Cursor, Claude And Any Other LLM Clients

A Model Context Protocol (MCP) server implementation that integrates with Firecrawl for web scraping capabilities.

Big thanks to @vrknetha, @cawstudios for the initial implementation!

You can also play around with our MCP Server on MCP.so's playground. Thanks to MCP.so for hosting and @gstarwd for integrating our server.

Features

• Scrape, crawl, search, extract, deep research and batch scrape support
• Web scraping with JS rendering
• URL discovery and crawling
• Web search with content extraction
• Automatic retries with exponential backoff
• Efficient batch processing with built-in rate limiting
• Credit usage monitoring for cloud API
• Comprehensive logging system
• Support for cloud and self-hosted Firecrawl instances
• Mobile/Desktop viewport support
• Smart content filtering with tag inclusion/exclusion

Installation

Running with npx

env FIRECRAWL_API_KEY=fc-YOUR_API_KEY npx -y firecrawl-mcp

Manual Installation

npm install -g firecrawl-mcp

Running on Cursor

Configuring Cursor 🖥️ Note: Requires Cursor version 0.45.6+ For the most up-to-date configuration instructions, please refer to the official Cursor documentation on configuring MCP servers: Cursor MCP Server Configuration Guide

To configure Firecrawl MCP in Cursor v0.45.6

1. Open Cursor Settings
2. Go to Features > MCP Servers
3. Click "+ Add New MCP Server"
4. Enter the following:
5. Name: "firecrawl-mcp" (or your preferred name)
6. Type: "command"
7. Command: env FIRECRAWL_API_KEY=your-api-key npx -y firecrawl-mcp

To configure Firecrawl MCP in Cursor v0.48.6

1. Open Cursor Settings
2. Go to Features > MCP Servers
3. Click "+ Add new global MCP server"
4. Enter the following code: json { "mcpServers": { "firecrawl-mcp": { "command": "npx", "args": ["-y", "firecrawl-mcp"], "env": { "FIRECRAWL_API_KEY": "YOUR-API-KEY" } } } }

If you are using Windows and are running into issues, try cmd /c "set FIRECRAWL_API_KEY=your-api-key && npx -y firecrawl-mcp"

Replace your-api-key with your Firecrawl API key. If you don't have one yet, you can create an account and get it from https://www.firecrawl.dev/app/api-keys

After adding, refresh the MCP server list to see the new tools. The Composer Agent will automatically use Firecrawl MCP when appropriate, but you can explicitly request it by describing your web scraping needs. Access the Composer via Command+L (Mac), select "Agent" next to the submit button, and enter your query.

Running on Windsurf

Add this to your ./codeium/windsurf/model_config.json:

{
  "mcpServers": {
    "mcp-server-firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}

Installing via Smithery (Legacy)

To install Firecrawl for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @mendableai/mcp-server-firecrawl --client claude

Configuration

Environment Variables

Required for Cloud API

• FIRECRAWL_API_KEY: Your Firecrawl API key
• Required when using cloud API (default)
• Optional when using self-hosted instance with FIRECRAWL_API_URL
• FIRECRAWL_API_URL (Optional): Custom API endpoint for self-hosted instances
• Example: https://firecrawl.your-domain.com
• If not provided, the cloud API will be used (requires API key)

Optional Configuration

Retry Configuration

• FIRECRAWL_RETRY_MAX_ATTEMPTS: Maximum number of retry attempts (default: 3)
• FIRECRAWL_RETRY_INITIAL_DELAY: Initial delay in milliseconds before first retry (default: 1000)
• FIRECRAWL_RETRY_MAX_DELAY: Maximum delay in milliseconds between retries (default: 10000)
• FIRECRAWL_RETRY_BACKOFF_FACTOR: Exponential backoff multiplier (default: 2)

Credit Usage Monitoring

• FIRECRAWL_CREDIT_WARNING_THRESHOLD: Credit usage warning threshold (default: 1000)
• FIRECRAWL_CREDIT_CRITICAL_THRESHOLD: Credit usage critical threshold (default: 100)

Configuration Examples

For cloud API usage with custom retry and credit monitoring:

# Required for cloud API
export FIRECRAWL_API_KEY=your-api-key

# Optional retry configuration
export FIRECRAWL_RETRY_MAX_ATTEMPTS=5        # Increase max retry attempts
export FIRECRAWL_RETRY_INITIAL_DELAY=2000    # Start with 2s delay
export FIRECRAWL_RETRY_MAX_DELAY=30000       # Maximum 30s delay
export FIRECRAWL_RETRY_BACKOFF_FACTOR=3      # More aggressive backoff

# Optional credit monitoring
export FIRECRAWL_CREDIT_WARNING_THRESHOLD=2000    # Warning at 2000 credits
export FIRECRAWL_CREDIT_CRITICAL_THRESHOLD=500    # Critical at 500 credits

For self-hosted instance:

# Required for self-hosted
export FIRECRAWL_API_URL=https://firecrawl.your-domain.com

# Optional authentication for self-hosted
export FIRECRAWL_API_KEY=your-api-key  # If your instance requires auth

# Custom retry configuration
export FIRECRAWL_RETRY_MAX_ATTEMPTS=10
export FIRECRAWL_RETRY_INITIAL_DELAY=500     # Start with faster retries

Usage with Claude Desktop

Add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "mcp-server-firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "YOUR_API_KEY_HERE",

        "FIRECRAWL_RETRY_MAX_ATTEMPTS": "5",
        "FIRECRAWL_RETRY_INITIAL_DELAY": "2000",
        "FIRECRAWL_RETRY_MAX_DELAY": "30000",
        "FIRECRAWL_RETRY_BACKOFF_FACTOR": "3",

        "FIRECRAWL_CREDIT_WARNING_THRESHOLD": "2000",
        "FIRECRAWL_CREDIT_CRITICAL_THRESHOLD": "500"
      }
    }
  }
}

System Configuration

The server includes several configurable parameters that can be set via environment variables. Here are the default values if not configured:

const CONFIG = {
  retry: {
    maxAttempts: 3, // Number of retry attempts for rate-limited requests
    initialDelay: 1000, // Initial delay before first retry (in milliseconds)
    maxDelay: 10000, // Maximum delay between retries (in milliseconds)
    backoffFactor: 2, // Multiplier for exponential backoff
  },
  credit: {
    warningThreshold: 1000, // Warn when credit usage reaches this level
    criticalThreshold: 100, // Critical alert when credit usage reaches this level
  },
};

These configurations control:

1. Retry Behavior

2. Automatically retries failed requests due to rate limits

3. Uses exponential backoff to avoid overwhelming the API

4. Example: With default settings, retries will be attempted at:

   • 1st retry: 1 second delay
   • 2nd retry: 2 seconds delay
   • 3rd retry: 4 seconds delay (capped at maxDelay)

5. Credit Usage Monitoring

6. Tracks API credit consumption for cloud API usage
7. Provides warnings at specified thresholds
8. Helps prevent unexpected service interruption
9. Example: With default settings:
   • Warning at 1000 credits remaining
   • Critical alert at 100 credits remaining

Rate Limiting and Batch Processing

The server utilizes Firecrawl's built-in rate limiting and batch processing capabilities:

• Automatic rate limit handling with exponential backoff
• Efficient parallel processing for batch operations
• Smart request queuing and throttling
• Automatic retries for transient errors

Available Tools

1. Scrape Tool (firecrawl_scrape)

Scrape content from a single URL with advanced options.

{
  "name": "firecrawl_scrape",
  "arguments": {
    "url": "https://example.com",
    "formats": ["markdown"],
    "onlyMainContent": true,
    "waitFor": 1000,
    "timeout": 30000,
    "mobile": false,
    "includeTags": ["article", "main"],
    "excludeTags": ["nav", "footer"],
    "skipTlsVerification": false
  }
}

2. Batch Scrape Tool (firecrawl_batch_scrape)

Scrape multiple URLs efficiently with built-in rate limiting and parallel processing.

{
  "name": "firecrawl_batch_scrape",
  "arguments": {
    "urls": ["https://example1.com", "https://example2.com"],
    "options": {
      "formats": ["markdown"],
      "onlyMainContent": true
    }
  }
}

Response includes operation ID for status checking:

{
  "content": [
    {
      "type": "text",
      "text": "Batch operation queued with ID: batch_1. Use firecrawl_check_batch_status to check progress."
    }
  ],
  "isError": false
}

3. Check Batch Status (firecrawl_check_batch_status)

Check the status of a batch operation.

{
  "name": "firecrawl_check_batch_status",
  "arguments": {
    "id": "batch_1"
  }
}

4. Search Tool (firecrawl_search)

Search the web and optionally extract content from search results.

{
  "name": "firecrawl_search",
  "arguments": {
    "query": "your search query",
    "limit": 5,
    "lang": "en",
    "country": "us",
    "scrapeOptions": {
      "formats": ["markdown"],
      "onlyMainContent": true
    }
  }
}

5. Crawl Tool (firecrawl_crawl)

Start an asynchronous crawl with advanced options.

{
  "name": "firecrawl_crawl",
  "arguments": {
    "url": "https://example.com",
    "maxDepth": 2,
    "limit": 100,
    "allowExternalLinks": false,
    "deduplicateSimilarURLs": true
  }
}

6. Extract Tool (firecrawl_extract)

Extract structured information from web pages using LLM capabilities. Supports both cloud AI and self-hosted LLM extraction.

{
  "name": "firecrawl_extract",
  "arguments": {
    "urls": ["https://example.com/page1", "https://example.com/page2"],
    "prompt": "Extract product information including name, price, and description",
    "systemPrompt": "You are a helpful assistant that extracts product information",
    "schema": {
      "type": "object",
      "properties": {
        "name": { "type": "string" },
        "price": { "type": "number" },
        "description": { "type": "string" }
      },
      "required": ["name", "price"]
    },
    "allowExternalLinks": false,
    "enableWebSearch": false,
    "includeSubdomains": false
  }
}

Example response:

{
  "content": [
    {
      "type": "text",
      "text": {
        "name": "Example Product",
        "price": 99.99,
        "description": "This is an example product description"
      }
    }
  ],
  "isError": false
}

Extract Tool Options:

• urls: Array of URLs to extract information from
• prompt: Custom prompt for the LLM extraction
• systemPrompt: System prompt to guide the LLM
• schema: JSON schema for structured data extraction
• allowExternalLinks: Allow extraction from external links
• enableWebSearch: Enable web search for additional context
• includeSubdomains: Include subdomains in extraction

When using a self-hosted instance, the extraction will use your configured LLM. For cloud API, it uses Firecrawl's managed LLM service.

7. Deep Research Tool (firecrawl_deep_research)

Conduct deep web research on a query using intelligent crawling, search, and LLM analysis.

{
  "name": "firecrawl_deep_research",
  "arguments": {
    "query": "how does carbon capture technology work?",
    "maxDepth": 3,
    "timeLimit": 120,
    "maxUrls": 50
  }
}

Arguments:

• query (string, required): The research question or topic to explore.
• maxDepth (number, optional): Maximum recursive depth for crawling/search (default: 3).
• timeLimit (number, optional): Time limit in seconds for the research session (default: 120).
• maxUrls (number, optional): Maximum number of URLs to analyze (default: 50).

Returns:

• Final analysis generated by an LLM based on research. (data.finalAnalysis)
• May also include structured activities and sources used in the research process.

8. Generate LLMs.txt Tool (firecrawl_generate_llmstxt)

Generate a standardized llms.txt (and optionally llms-full.txt) file for a given domain. This file defines how large language models should interact with the site.

{
  "name": "firecrawl_generate_llmstxt",
  "arguments": {
    "url": "https://example.com",
    "maxUrls": 20,
    "showFullText": true
  }
}

Arguments:

• url (string, required): The base URL of the website to analyze.
• maxUrls (number, optional): Max number of URLs to include (default: 10).
• showFullText (boolean, optional): Whether to include llms-full.txt contents in the response.

Returns:

• Generated llms.txt file contents and optionally the llms-full.txt (data.llmstxt and/or data.llmsfulltxt)

Logging System

The server includes comprehensive logging:

• Operation status and progress
• Performance metrics
• Credit usage monitoring
• Rate limit tracking
• Error conditions

Example log messages:

[INFO] Firecrawl MCP Server initialized successfully
[INFO] Starting scrape for URL: https://example.com
[INFO] Batch operation queued with ID: batch_1
[WARNING] Credit usage has reached warning threshold
[ERROR] Rate limit exceeded, retrying in 2s...

Error Handling

The server provides robust error handling:

• Automatic retries for transient errors
• Rate limit handling with backoff
• Detailed error messages
• Credit usage warnings
• Network resilience

Example error response:

{
  "content": [
    {
      "type": "text",
      "text": "Error: Rate limit exceeded. Retrying in 2 seconds..."
    }
  ],
  "isError": true
}

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

Contributing

1. Fork the repository
2. Create your feature branch
3. Run tests: npm test
4. Submit a pull request

License

MIT License - see LICENSE file for details

KitPloit - PenTest Tools!

Deep-Live-Cam - Real Time Face Swap And One-Click Video Deepfake With Only A Single Image

Real-time face swap and video deepfake with a single click and only a single image.

Disclaimer

This deepfake software is designed to be a productive tool for the AI-generated media industry. It can assist artists in animating custom characters, creating engaging content, and even using models for clothing design.

We are aware of the potential for unethical applications and are committed to preventative measures. A built-in check prevents the program from processing inappropriate media (nudity, graphic content, sensitive material like war footage, etc.). We will continue to develop this project responsibly, adhering to the law and ethics. We may shut down the project or add watermarks if legally required.

• Ethical Use: Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online.

• Content Restrictions: The software includes built-in checks to prevent processing inappropriate media, such as nudity, graphic content, or sensitive material.

• Legal Compliance: We adhere to all relevant laws and ethical guidelines. If legally required, we may shut down the project or add watermarks to the output.

• User Responsibility: We are not responsible for end-user actions. Users must ensure their use of the software aligns with ethical standards and legal requirements.

By using this software, you agree to these terms and commit to using it in a manner that respects the rights and dignity of others.

Users are expected to use this software responsibly and legally. If using a real person's face, obtain their consent and clearly label any output as a deepfake when sharing online. We are not responsible for end-user actions.

TLDR; Live Deepfake in just 3 Clicks

1. Select a face 2. Select which camera to use 3. Press live!

Features & Uses - Everything is in real-time

Mouth Mask

Retain your original mouth for accurate movement using Mouth Mask

Face Mapping

Use different faces on multiple subjects simultaneously

Your Movie, Your Face

Watch movies with any face in real-time

Live Show

Run Live shows and performances

Memes

Create Your Most Viral Meme Yet

_{Created using Many Faces feature in Deep-Live-Cam}

Omegle

Surprise people on Omegle

Installation (Manual)

Please be aware that the installation requires technical skills and is not for beginners. Consider downloading the prebuilt version.

git clone https://github.com/hacksider/Deep-Live-Cam.git
cd Deep-Live-Cam

python -m venv venv
venv\Scripts\activate
pip install -r requirements.txt

# Install Python 3.10 (specific version is important)
brew install python@3.10

# Install tkinter package (required for the GUI)
brew install python-tk@3.10

# Create and activate virtual environment with Python 3.10
python3.10 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Deactivate the virtual environment
rm -rf venv

# Reinstall the virtual environment
python -m venv venv
source venv/bin/activate

# install the dependencies again
pip install -r requirements.txt

pip uninstall onnxruntime onnxruntime-gpu
pip install onnxruntime-gpu==1.16.3

python run.py --execution-provider cuda

pip uninstall onnxruntime onnxruntime-silicon
pip install onnxruntime-silicon==1.13.1

python3.10 run.py --execution-provider coreml

pip uninstall onnxruntime onnxruntime-coreml
pip install onnxruntime-coreml==1.13.1

python run.py --execution-provider coreml

pip uninstall onnxruntime onnxruntime-directml
pip install onnxruntime-directml==1.15.1

python run.py --execution-provider directml

pip uninstall onnxruntime onnxruntime-openvino
pip install onnxruntime-openvino==1.15.0

python run.py --execution-provider openvino

Usage

1. Image/Video Mode

• Execute python run.py.
• Choose a source face image and a target image/video.
• Click "Start".
• The output will be saved in a directory named after the target video.

2. Webcam Mode

• Execute python run.py.
• Select a source face image.
• Click "Live".
• Wait for the preview to appear (10-30 seconds).
• Use a screen capture tool like OBS to stream.
• To change the face, select a new source image.

Tips and Tricks

Check out these helpful guides to get the most out of Deep-Live-Cam:

• Unlocking the Secrets to the Perfect Deepfake Image - Learn how to create the best deepfake with full head coverage
• Video Call with DeepLiveCam - Make your meetings livelier by using DeepLiveCam with OBS and meeting software
• Have a Special Guest! - Tutorial on how to use face mapping to add special guests to your stream
• Watch Deepfake Movies in Realtime - See yourself star in any video without processing the video
• Better Quality without Sacrificing Speed - Tips for achieving better results without impacting performance
• Instant Vtuber! - Create a new persona/vtuber easily using Metahuman Creator

Visit our official blog for more tips and tutorials.

Command Line Arguments (Unmaintained)

options:
  -h, --help                                               show this help message and exit
  -s SOURCE_PATH, --source SOURCE_PATH                     select a source image
  -t TARGET_PATH, --target TARGET_PATH                     select a target image or video
  -o OUTPUT_PATH, --output OUTPUT_PATH                     select output file or directory
  --frame-processor FRAME_PROCESSOR [FRAME_PROCESSOR ...]  frame processors (choices: face_swapper, face_enhancer, ...)
  --keep-fps                                               keep original fps
  --keep-audio                                             keep original audio
  --keep-frames                                            keep temporary frames
  --many-faces                                             process every face
  --map-faces                                              map source target faces
  --mouth-mask                                             mask the mouth region
  --video-encoder {libx264,libx265,libvpx-vp9}             adjust output video encoder
  --video-quality [0-51]                                   adjust output video quality
  --live-mirror                                            the live camera display as you see it in the front-facing camera frame
  --live-resizable                                         the live camera frame is resizable
  --max-memory MAX_MEMORY                                  maximum amount of RAM in GB
  --execution-provider {cpu} [{cpu} ...]                   available execution provider (choices: cpu, ...)
  --execution-threads EXECUTION_THREADS                    number of execution threads
  -v, --version                                            show program's version number and exit

Looking for a CLI mode? Using the -s/--source argument will make the run program in cli mode.

Press

We are always open to criticism and are ready to improve, that's why we didn't cherry-pick anything.

• "Deep-Live-Cam goes viral, allowing anyone to become a digital doppelganger" - Ars Technica
• "Thanks Deep Live Cam, shapeshifters are among us now" - Dataconomy
• "This free AI tool lets you become anyone during video-calls" - NewsBytes
• "OK, this viral AI live stream software is truly terrifying" - Creative Bloq
• "Deepfake AI Tool Lets You Become Anyone in a Video Call With Single Photo" - PetaPixel
• "Deep-Live-Cam Uses AI to Transform Your Face in Real-Time, Celebrities Included" - TechEBlog
• "An AI tool that "makes you look like anyone" during a video call is going viral online" - Telegrafi
• "This Deepfake Tool Turning Images Into Livestreams is Topping the GitHub Charts" - Emerge
• "New Real-Time Face-Swapping AI Allows Anyone to Mimic Famous Faces" - Digital Music News
• "This real-time webcam deepfake tool raises alarms about the future of identity theft" - DIYPhotography
• "That's Crazy, Oh God. That's Fucking Freaky Dude... That's So Wild Dude" - SomeOrdinaryGamers
• "Alright look look look, now look chat, we can do any face we want to look like chat" - IShowSpeed

Credits

• ffmpeg: for making video-related operations easy
• deepinsight: for their insightface project which provided a well-made library and models. Please be reminded that the use of the model is for non-commercial research purposes only.
• havok2-htwo: for sharing the code for webcam
• GosuDRM: for the open version of roop
• pereiraroland26: Multiple faces support
• vic4key: For supporting/contributing to this project
• kier007: for improving the user experience
• qitianai: for multi-lingual support
• and all developers behind libraries used in this project.
• Footnote: Please be informed that the base author of the code is s0md3v
• All the wonderful users who helped make this project go viral by starring the repo ❤️

Contributions

Stars to the Moon 🚀

KitPloit - PenTest Tools!

CAMEL - The First And The Best Multi-Agent Framework. Finding The Scaling Law Of Agents

CAMEL Framework Design Principles

🧬 Evolvability

📈 Scalability

The framework is designed to support systems with millions of agents, ensuring efficient coordination, communication, and resource management at scale.

💾 Statefulness

Agents maintain stateful memory, enabling them to perform multi-step interactions with environments and efficiently tackle sophisticated tasks.

📖 Code-as-Prompt

Every line of code and comment serves as a prompt for agents. Code should be written clearly and readably, ensuring both humans and agents can interpret it effectively.

Why Use CAMEL for Your Research?

We are a community-driven research collective comprising over 100 researchers dedicated to advancing frontier research in Multi-Agent Systems. Researchers worldwide choose CAMEL for their studies based on the following reasons.

What Can You Build With CAMEL?

1. Data Generation

2. Task Automation

3. World Simulation

Quick Start

Installing CAMEL is a breeze thanks to its availability on PyPI. Simply open your terminal and run:

pip install camel-ai

Starting with ChatAgent

This example demonstrates how to create a ChatAgent using the CAMEL framework and perform a search query using DuckDuckGo.

1. Install the tools package:

bash pip install 'camel-ai[web_tools]'

1. Set up your OpenAI API key:

bash export OPENAI_API_KEY='your_openai_api_key'

1. Run the following Python code:

```python from camel.models import ModelFactory from camel.types import ModelPlatformType, ModelType from camel.agents import ChatAgent from camel.toolkits import SearchToolkit

model = ModelFactory.create( model_platform=ModelPlatformType.OPENAI, model_type=ModelType.GPT_4O, model_config_dict={"temperature": 0.0}, )

search_tool = SearchToolkit().search_duckduckgo

agent = ChatAgent(model=model, tools=[search_tool])

response_1 = agent.step("What is CAMEL-AI?") print(response_1.msgs[0].content) # CAMEL-AI is the first LLM (Large Language Model) multi-agent framework # and an open-source community focused on finding the scaling laws of agents. # ...

response_2 = agent.step("What is the Github link to CAMEL framework?") print(response_2.msgs[0].content) # The GitHub link to the CAMEL framework is # https://github.com/camel-ai/camel. ```

For more detailed instructions and additional configuration options, check out the installation section.

After running, you can explore our CAMEL Tech Stack and Cookbooks at docs.camel-ai.org to build powerful multi-agent systems.

We provide a demo showcasing a conversation between two ChatGPT agents playing roles as a python programmer and a stock trader collaborating on developing a trading bot for stock market.

Explore different types of agents, their roles, and their applications.

• Creating Your First Agent
• Creating Your First Agent Society
• Embodied Agents
• Critic Agents

Seeking Help

Please reach out to us on CAMEL discord if you encounter any issue set up CAMEL.

Tech Stack

Key Modules

Core components and utilities to build, operate, and enhance CAMEL-AI agents and societies.

Research

We believe that studying these agents on a large scale offers valuable insights into their behaviors, capabilities, and potential risks.

Explore our research projects:

Research with US

We warmly invite you to use CAMEL for your impactful research.

Rigorous research takes time and resources. We are a community-driven research collective with 100+ researchers exploring the frontier research of Multi-agent Systems. Join our ongoing projects or test new ideas with us, reach out via email for more information.

Synthetic Datasets

1. Utilize Various LLMs as Backends

For more details, please see our Models Documentation.

Data (Hosted on Hugging Face)

2. Visualizations of Instructions and Tasks

Cookbooks (Usecases)

Practical guides and tutorials for implementing specific functionalities in CAMEL-AI agents and societies.

1. Basic Concepts

2. Advanced Features

3. Model Training & Data Generation

4. Multi-Agent Systems & Applications

5. Data Processing

Contributing to CAMEL

For those who'd like to contribute code, we appreciate your interest in contributing to our open-source initiative. Please take a moment to review our contributing guidelines to get started on a smooth collaboration journey.🚀

We also welcome you to help CAMEL grow by sharing it on social media, at events, or during conferences. Your support makes a big difference!

Community & Contact

For more information please contact camel-ai@eigent.ai

• GitHub Issues: Report bugs, request features, and track development. Submit an issue

• Discord: Get real-time support, chat with the community, and stay updated. Join us

• X (Twitter): Follow for updates, AI insights, and key announcements. Follow us

• Ambassador Project: Advocate for CAMEL-AI, host events, and contribute content. Learn more

Citation

@inproceedings{li2023camel,
  title={CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society},
  author={Li, Guohao and Hammoud, Hasan Abed Al Kader and Itani, Hani and Khizbullin, Dmitrii and Ghanem, Bernard},
  booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
  year={2023}
}

Acknowledgment

Special thanks to Nomic AI for giving us extended access to their data set exploration tool (Atlas).

We would also like to thank Haya Hammoud for designing the initial logo of our project.

We implemented amazing research ideas from other works for you to build, compare and customize your agents. If you use any of these modules, please kindly cite the original works: - TaskCreationAgent, TaskPrioritizationAgent and BabyAGI from Nakajima et al.: Task-Driven Autonomous Agent. [Example]

• PersonaHub from Tao Ge et al.: Scaling Synthetic Data Creation with 1,000,000,000 Personas. [Example]

• Self-Instruct from Yizhong Wang et al.: SELF-INSTRUCT: Aligning Language Models with Self-Generated Instructions. [Example]

License

The source code is licensed under Apache 2.0.

KitPloit - PenTest Tools!

Liam - Automatically Generates Beautiful And Easy-To-Read ER Diagrams From Your Database

Automatically generates beautiful and easy-to-read ER diagrams from your database.

Website • Documentation • Roadmap

What's Liam ERD?

Liam ERD generates beautiful, interactive ER diagrams from your database. Whether you're working on public or private repositories, Liam ERD helps you visualize complex schemas with ease.

• Beautiful UI & Interactive: A clean design and intuitive features (like panning, zooming, and filtering) make it easy to understand even the most complex databases.
• Simple Reverse Engineering: Seamlessly turn your existing database schemas into clear, readable diagrams.
• Effortless Setup: Get started with zero configuration—just provide your schema, and you're good to go.
• High Performance: Optimized for both small and large projects, easily handling 100+ tables.
• Fully Open-Source: Contribute to the project and shape Liam ERD to fit your needs.

Quick Start

For Public Repositories

Insert liambx.com/erd/p/ into your schema file's URL:

# Original: https://github.com/user/repo/blob/master/db/schema.rb
# Modified: https://liambx.com/erd/p/github.com/user/repo/blob/master/db/schema.rb
                  👾^^^^^^^^^^^^^^^^👾

For Private Repositories

Run the interactive setup:

npx @liam-hq/cli init

If you find this project helpful, please give it a star! ⭐
Your support helps us reach a wider audience and continue development.

Documentation

Check out the full documentation on the website.

Roadmap

See what we're working on and what's coming next on our roadmap.

KitPloit - PenTest Tools!

SubGPT - Find Subdomains With GPT, For Free

SubGPT looks at subdomains you have already discovered for a domain and uses BingGPT to find more. Best part? It's free!

The following subdomains were found by this tool with these 30 subdomains as input.

call-prompts-staging.example.com
dclb02-dca1.prod.example.com
activedirectory-sjc1.example.com
iadm-staging.example.com
elevatenetwork-c.example.com

If you like my work, you can support me with as little as $1, here :)

Install & Configuration

Installation

• with pip (recommended): pip install subgpt
• from github: git clone https://github.com/s0md3v/SubGPT && cd SubGPT && python setup.py install

Getting Bing Cookie

1. Install the cookie editor extension (Chrome, Firefox)
2. Visit bing.com, make sure you are logged in.
3. Open the extension and copy your cookie using the "export" button
4. Paste it in a file e.g. cookies.json
5. All set!

Note: Any issues regarding BingGPT itself should be reported EdgeGPT, not here.

Using SubGPT

It is supposed to be used after you have discovered some subdomains using all other methods. The standard way to run SubGPT is as follows:

subgpt -i input.txt -o output.txt -c /path/to/cookies.json

If you don't specify an output file, the output will be shown in your terminal (stdout) instead.

To generate subdomains and not resolve them, use the --dont-resolve option. It's a great way to see all subdomains generated by SubGPT and/or use your own resolver on them.

Important

1. Make sure your subdomains list only has subdomains from one domain. Each line in your file should contain one subdomain and nothing else.
2. Sometimes your cookie will expire if you visit bing.com often. In that case, just export and save it again.
3. SubGPT looks at A/CNAME records to determine whether a subdomain exists. It can also detect wildcard on first-level subdomains and handle it automatically. You can go through the code to see how its implemented if it concerns you.
4. It can't replace traditional sub-generators like gotator, alterx, dnsgen etc. However, being powered by AI helps it to generate subdomains that these traditional tools can't.
5. It is slow for obvious reasons. It takes like 45 seconds for every 80 subdomains.
6. It is subject to Bing's daily limit. Selectively run this tool, don't run it blindly.

KitPloit - PenTest Tools!

Scrapling - An Undetectable, Powerful, Flexible, High-Performance Python Library That Makes Web Scraping Simple And Easy Again!

Dealing with failing web scrapers due to anti-bot protections or website changes? Meet Scrapling.

Scrapling is a high-performance, intelligent web scraping library for Python that automatically adapts to website changes while significantly outperforming popular alternatives. For both beginners and experts, Scrapling provides powerful features while maintaining simplicity.

>> from scrapling.defaults import Fetcher, AsyncFetcher, StealthyFetcher, PlayWrightFetcher
# Fetch websites' source under the radar!
>> page = StealthyFetcher.fetch('https://example.com', headless=True, network_idle=True)
>> print(page.status)
200
>> products = page.css('.product', auto_save=True)  # Scrape data that survives website design changes!
>> # Later, if the website structure changes, pass `auto_match=True`
>> products = page.css('.product', auto_match=True)  # and Scrapling still finds them!

Key Features

Fetch websites as you prefer with async support

• HTTP Requests: Fast and stealthy HTTP requests with the Fetcher class.
• Dynamic Loading & Automation: Fetch dynamic websites with the PlayWrightFetcher class through your real browser, Scrapling's stealth mode, Playwright's Chrome browser, or NSTbrowser's browserless!
• Anti-bot Protections Bypass: Easily bypass protections with StealthyFetcher and PlayWrightFetcher classes.

Adaptive Scraping

• 🔄 Smart Element Tracking: Relocate elements after website changes, using an intelligent similarity system and integrated storage.
• 🎯 Flexible Selection: CSS selectors, XPath selectors, filters-based search, text search, regex search and more.
• 🔍 Find Similar Elements: Automatically locate elements similar to the element you found!
• 🧠 Smart Content Scraping: Extract data from multiple websites without specific selectors using Scrapling powerful features.

High Performance

• 🚀 Lightning Fast: Built from the ground up with performance in mind, outperforming most popular Python scraping libraries.
• 🔋 Memory Efficient: Optimized data structures for minimal memory footprint.
• ⚡ Fast JSON serialization: 10x faster than standard library.

Developer Friendly

• 🛠️ Powerful Navigation API: Easy DOM traversal in all directions.
• 🧬 Rich Text Processing: All strings have built-in regex, cleaning methods, and more. All elements' attributes are optimized dictionaries that takes less memory than standard dictionaries with added methods.
• 📝 Auto Selectors Generation: Generate robust short and full CSS/XPath selectors for any element.
• 🔌 Familiar API: Similar to Scrapy/BeautifulSoup and the same pseudo-elements used in Scrapy.
• 📘 Type hints: Complete type/doc-strings coverage for future-proofing and best autocompletion support.

Getting Started

from scrapling.fetchers import Fetcher

fetcher = Fetcher(auto_match=False)

# Do http GET request to a web page and create an Adaptor instance
page = fetcher.get('https://quotes.toscrape.com/', stealthy_headers=True)
# Get all text content from all HTML tags in the page except `script` and `style` tags
page.get_all_text(ignore_tags=('script', 'style'))

# Get all quotes elements, any of these methods will return a list of strings directly (TextHandlers)
quotes = page.css('.quote .text::text')  # CSS selector
quotes = page.xpath('//span[@class="text"]/text()')  # XPath
quotes = page.css('.quote').css('.text::text')  # Chained selectors
quotes = [element.text for element in page.css('.quote .text')]  # Slower than bulk query above

# Get the first quote element
quote = page.css_first('.quote')  # same as page.css('.quote').first or page.css('.quote')[0]

# Tired of selectors? Use find_all/find
# Get all 'div' HTML tags that one of its 'class' values is 'quote'
quotes = page.find_all('div', {'class': 'quote'})
# Same as
quotes = page.find_all('div', class_='quote')
quotes = page.find_all(['div'], class_='quote')
quotes = page.find_all(class_='quote')  # and so on...

# Working with elements
quote.html_content  # Get Inner HTML of this element
quote.prettify()  # Prettified version of Inner HTML above
quote.attrib  # Get that element's attributes
quote.path  # DOM path to element (List of all ancestors from <html> tag till the element itself)

To keep it simple, all methods can be chained on top of each other!

Parsing Performance

Scrapling isn't just powerful - it's also blazing fast. Scrapling implements many best practices, design patterns, and numerous optimizations to save fractions of seconds. All of that while focusing exclusively on parsing HTML documents. Here are benchmarks comparing Scrapling to popular Python libraries in two tests.

Text Extraction Speed Test (5000 nested elements).

As you see, Scrapling is on par with Scrapy and slightly faster than Lxml which both libraries are built on top of. These are the closest results to Scrapling. PyQuery is also built on top of Lxml but still, Scrapling is 4 times faster.

Extraction By Text Speed Test

Scrapling can find elements with more methods and it returns full element Adaptor objects not only the text like AutoScraper. So, to make this test fair, both libraries will extract an element with text, find similar elements, and then extract the text content for all of them. As you see, Scrapling is still 4.5 times faster at the same task.

All benchmarks' results are an average of 100 runs. See our benchmarks.py for methodology and to run your comparisons.

Installation

Scrapling is a breeze to get started with; Starting from version 0.2.9, we require at least Python 3.9 to work.

pip3 install scrapling

Then run this command to install browsers' dependencies needed to use Fetcher classes

scrapling install

If you have any installation issues, please open an issue.

Fetching Websites

Fetchers are interfaces built on top of other libraries with added features that do requests or fetch pages for you in a single request fashion and then return an Adaptor object. This feature was introduced because the only option we had before was to fetch the page as you wanted it, then pass it manually to the Adaptor class to create an Adaptor instance and start playing around with the page.

Features

You might be slightly confused by now so let me clear things up. All fetcher-type classes are imported in the same way

from scrapling.fetchers import Fetcher, StealthyFetcher, PlayWrightFetcher

All of them can take these initialization arguments: auto_match, huge_tree, keep_comments, keep_cdata, storage, and storage_args, which are the same ones you give to the Adaptor class.

If you don't want to pass arguments to the generated Adaptor object and want to use the default values, you can use this import instead for cleaner code:

from scrapling.defaults import Fetcher, AsyncFetcher, StealthyFetcher, PlayWrightFetcher

then use it right away without initializing like:

page = StealthyFetcher.fetch('https://example.com') 

Also, the Response object returned from all fetchers is the same as the Adaptor object except it has these added attributes: status, reason, cookies, headers, history, and request_headers. All cookies, headers, and request_headers are always of type dictionary.

[!NOTE] The auto_match argument is enabled by default which is the one you should care about the most as you will see later.

Fetcher

This class is built on top of httpx with additional configuration options, here you can do GET, POST, PUT, and DELETE requests.

For all methods, you have stealthy_headers which makes Fetcher create and use real browser's headers then create a referer header as if this request came from Google's search of this URL's domain. It's enabled by default. You can also set the number of retries with the argument retries for all methods and this will make httpx retry requests if it failed for any reason. The default number of retries for all Fetcher methods is 3.

Hence: All headers generated by stealthy_headers argument can be overwritten by you through the headers argument

You can route all traffic (HTTP and HTTPS) to a proxy for any of these methods in this format http://username:password@localhost:8030

>> page = Fetcher().get('https://httpbin.org/get', stealthy_headers=True, follow_redirects=True)
>> page = Fetcher().post('https://httpbin.org/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030')
>> page = Fetcher().put('https://httpbin.org/put', data={'key': 'value'})
>> page = Fetcher().delete('https://httpbin.org/delete')

For Async requests, you will just replace the import like below:

>> from scrapling.fetchers import AsyncFetcher
>> page = await AsyncFetcher().get('https://httpbin.org/get', stealthy_headers=True, follow_redirects=True)
>> page = await AsyncFetcher().post('https://httpbin.org/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030')
>> page = await AsyncFetcher().put('https://httpbin.org/put', data={'key': 'value'})
>> page = await AsyncFetcher().delete('https://httpbin.org/delete')

StealthyFetcher

This class is built on top of Camoufox, bypassing most anti-bot protections by default. Scrapling adds extra layers of flavors and configurations to increase performance and undetectability even further.

>> page = StealthyFetcher().fetch('https://www.browserscan.net/bot-detection')  # Running headless by default
>> page.status == 200
True
>> page = await StealthyFetcher().async_fetch('https://www.browserscan.net/bot-detection')  # the async version of fetch
>> page.status == 200
True

Note: all requests done by this fetcher are waiting by default for all JS to be fully loaded and executed so you don't have to :)

This list isn't final so expect a lot more additions and flexibility to be added in the next versions!

PlayWrightFetcher

This class is built on top of Playwright which currently provides 4 main run options but they can be mixed as you want.

>> page = PlayWrightFetcher().fetch('https://www.google.com/search?q=%22Scrapling%22', disable_resources=True)  # Vanilla Playwright option
>> page.css_first("#search a::attr(href)")
'https://github.com/D4Vinci/Scrapling'
>> page = await PlayWrightFetcher().async_fetch('https://www.google.com/search?q=%22Scrapling%22', disable_resources=True)  # the async version of fetch
>> page.css_first("#search a::attr(href)")
'https://github.com/D4Vinci/Scrapling'

Note: all requests done by this fetcher are waiting by default for all JS to be fully loaded and executed so you don't have to :)

Using this Fetcher class, you can make requests with: 1) Vanilla Playwright without any modifications other than the ones you chose. 2) Stealthy Playwright with the stealth mode I wrote for it. It's still a WIP but it bypasses many online tests like Sannysoft's. Some of the things this fetcher's stealth mode does include: * Patching the CDP runtime fingerprint. * Mimics some of the real browsers' properties by injecting several JS files and using custom options. * Using custom flags on launch to hide Playwright even more and make it faster. * Generates real browser's headers of the same type and same user OS then append it to the request's headers. 3) Real browsers by passing the real_chrome argument or the CDP URL of your browser to be controlled by the Fetcher and most of the options can be enabled on it. 4) NSTBrowser's docker browserless option by passing the CDP URL and enabling nstbrowser_mode option.

Hence using the real_chrome argument requires that you have Chrome browser installed on your device

Add that to a lot of controlling/hiding options as you will see in the arguments list below.

This list isn't final so expect a lot more additions and flexibility to be added in the next versions!

Advanced Parsing Features

Smart Navigation

>>> quote.tag
'div'

>>> quote.parent
<data='<div class="col-md-8"> <div class="quote...' parent='<div class="row"> <div class="col-md-8">...'>

>>> quote.parent.tag
'div'

>>> quote.children
[<data='<span class="text" itemprop="text">"The...' parent='<div class="quote" itemscope itemtype="h...'>,
 <data='<span>by <small class="author" itemprop=...' parent='<div class="quote" itemscope itemtype="h...'>,
 <data='<div class="tags"> Tags: <meta class="ke...' parent='<div class="quote" itemscope itemtype="h...'>]

>>> quote.siblings
[<data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
 <data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
...]

>>> quote.next  # gets the next element, the same logic applies to `quote.previous`
<data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>

>>> quote.children.css_first(".author::text")
'Albert Einstein'

>>> quote.has_class('quote')
True

# Generate new selectors for any element
>>> quote.generate_css_selector
'body > div > div:nth-of-type(2) > div > div'

# Test these selectors on your favorite browser or reuse them again in the library's methods!
>>> quote.generate_xpath_selector
'//body/div/div[2]/div/div'

If your case needs more than the element's parent, you can iterate over the whole ancestors' tree of any element like below

for ancestor in quote.iterancestors():
    # do something with it...

You can search for a specific ancestor of an element that satisfies a function, all you need to do is to pass a function that takes an Adaptor object as an argument and return True if the condition satisfies or False otherwise like below:

>>> quote.find_ancestor(lambda ancestor: ancestor.has_class('row'))
<data='<div class="row"> <div class="col-md-8">...' parent='<div class="container"> <div class="row...'>

Content-based Selection & Finding Similar Elements

You can select elements by their text content in multiple ways, here's a full example on another website:

>>> page = Fetcher().get('https://books.toscrape.com/index.html')

>>> page.find_by_text('Tipping the Velvet')  # Find the first element whose text fully matches this text
<data='<a href="catalogue/tipping-the-velvet_99...' parent='<h3><a href="catalogue/tipping-the-velve...'>

>>> page.urljoin(page.find_by_text('Tipping the Velvet').attrib['href'])  # We use `page.urljoin` to return the full URL from the relative `href`
'https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html'

>>> page.find_by_text('Tipping the Velvet', first_match=False)  # Get all matches if there are more
[<data='<a href="catalogue/tipping-the-velvet_99...' parent='<h3><a href="catalogue/tipping-the-velve...'>]

>>> page.find_by_regex(r'£[\d\.]+')  # Get the first element that its text content matches my price regex
<data='<p class="price_color">£51.77</p>' parent='<div class="product_price"> <p class="pr...'>

>>> page.find_by_regex(r'£[\d\.]+', first_match=False)  # Get all elements that matches my price regex
[<data='<p class="price_color">£51.77</p>' parent='<div class="product_price"> <p class="pr...'>,
 <data='<p class="price_color">£53.74</p>' parent='<div class="product_price"> <p class="pr...'>,
 <data='<p class="price_color">£50.10</p>' parent='<div class="product_price"> <p class="pr...'>,
 <data='<p class="price_color">£47.82</p>' parent='<div class="product_price"> <p class="pr...'>,
 ...]

Find all elements that are similar to the current element in location and attributes

# For this case, ignore the 'title' attribute while matching
>>> page.find_by_text('Tipping the Velvet').find_similar(ignore_attributes=['title'])
[<data='<a href="catalogue/a-light-in-the-attic_...' parent='<h3><a href="catalogue/a-light-in-the-at...'>,
 <data='<a href="catalogue/soumission_998/index....' parent='<h3><a href="catalogue/soumission_998/in...'>,
 <data='<a href="catalogue/sharp-objects_997/ind...' parent='<h3><a href="catalogue/sharp-objects_997...'>,
...]

# You will notice that the number of elements is 19 not 20 because the current element is not included.
>>> len(page.find_by_text('Tipping the Velvet').find_similar(ignore_attributes=['title']))
19

# Get the `href` attribute from all similar elements
>>> [element.attrib['href'] for element in page.find_by_text('Tipping the Velvet').find_similar(ignore_attributes=['title'])]
['catalogue/a-light-in-the-attic_1000/index.html',
 'catalogue/soumission_998/index.html',
 'catalogue/sharp-objects_997/index.html',
 ...]

To increase the complexity a little bit, let's say we want to get all books' data using that element as a starting point for some reason

>>> for product in page.find_by_text('Tipping the Velvet').parent.parent.find_similar():
        print({
            "name": product.css_first('h3 a::text'),
            "price": product.css_first('.price_color').re_first(r'[\d\.]+'),
            "stock": product.css('.availability::text')[-1].clean()
        })
{'name': 'A Light in the ...', 'price': '51.77', 'stock': 'In stock'}
{'name': 'Soumission', 'price': '50.10', 'stock': 'In stock'}
{'name': 'Sharp Objects', 'price': '47.82', 'stock': 'In stock'}
...

The documentation will provide more advanced examples.

Handling Structural Changes

Let's say you are scraping a page with a structure like this:

<div class="container">
    <section class="products">
        <article class="product" id="p1">
            <h3>Product 1</h3>
            <p class="description">Description 1</p>
        </article>
        <article class="product" id="p2">
            <h3>Product 2</h3>
            <p class="description">Description 2</p>
        </article>
    </section>
</div>

And you want to scrape the first product, the one with the p1 ID. You will probably write a selector like this

page.css('#p1')

When website owners implement structural changes like

<div class="new-container">
    <div class="product-wrapper">
        <section class="products">
            <article class="product new-class" data-id="p1">
                <div class="product-info">
                    <h3>Product 1</h3>
                    <p class="new-description">Description 1</p>
                </div>
            </article>
            <article class="product new-class" data-id="p2">
                <div class="product-info">
                    <h3>Product 2</h3>
                    <p class="new-description">Description 2</p>
                </div>
            </article>
        </section>
    </div>
</div>

The selector will no longer function and your code needs maintenance. That's where Scrapling's auto-matching feature comes into play.

from scrapling.parser import Adaptor
# Before the change
page = Adaptor(page_source, url='example.com')
element = page.css('#p1' auto_save=True)
if not element:  # One day website changes?
    element = page.css('#p1', auto_match=True)  # Scrapling still finds it!
# the rest of the code...

How does the auto-matching work? Check the FAQs section for that and other possible issues while auto-matching.

Real-World Scenario

Let's use a real website as an example and use one of the fetchers to fetch its source. To do this we need to find a website that will change its design/structure soon, take a copy of its source then wait for the website to make the change. Of course, that's nearly impossible to know unless I know the website's owner but that will make it a staged test haha.

To solve this issue, I will use The Web Archive's Wayback Machine. Here is a copy of StackOverFlow's website in 2010, pretty old huh?Let's test if the automatch feature can extract the same button in the old design from 2010 and the current design using the same selector :)

If I want to extract the Questions button from the old design I can use a selector like this #hmenus > div:nth-child(1) > ul > li:nth-child(1) > a This selector is too specific because it was generated by Google Chrome. Now let's test the same selector in both versions

>> from scrapling.fetchers import Fetcher
>> selector = '#hmenus > div:nth-child(1) > ul > li:nth-child(1) > a'
>> old_url = "https://web.archive.org/web/20100102003420/http://stackoverflow.com/"
>> new_url = "https://stackoverflow.com/"
>> 
>> page = Fetcher(automatch_domain='stackoverflow.com').get(old_url, timeout=30)
>> element1 = page.css_first(selector, auto_save=True)
>> 
>> # Same selector but used in the updated website
>> page = Fetcher(automatch_domain="stackoverflow.com").get(new_url)
>> element2 = page.css_first(selector, auto_match=True)
>> 
>> if element1.text == element2.text:
...    print('Scrapling found the same element in the old design and the new design!')
'Scrapling found the same element in the old design and the new design!'

Note that I used a new argument called automatch_domain, this is because for Scrapling these are two different URLs, not the website so it isolates their data. To tell Scrapling they are the same website, we then pass the domain we want to use for saving auto-match data for them both so Scrapling doesn't isolate them.

In a real-world scenario, the code will be the same except it will use the same URL for both requests so you won't need to use the automatch_domain argument. This is the closest example I can give to real-world cases so I hope it didn't confuse you :)

Notes: 1. For the two examples above I used one time the Adaptor class and the second time the Fetcher class just to show you that you can create the Adaptor object by yourself if you have the source or fetch the source using any Fetcher class then it will create the Adaptor object for you. 2. Passing the auto_save argument with the auto_match argument set to False while initializing the Adaptor/Fetcher object will only result in ignoring the auto_save argument value and the following warning message text Argument `auto_save` will be ignored because `auto_match` wasn't enabled on initialization. Check docs for more info. This behavior is purely for performance reasons so the database gets created/connected only when you are planning to use the auto-matching features. Same case with the auto_match argument.

1. The auto_match parameter works only for Adaptor instances not Adaptors so if you do something like this you will get an error python page.css('body').css('#p1', auto_match=True) because you can't auto-match a whole list, you have to be specific and do something like python page.css_first('body').css('#p1', auto_match=True)

Find elements by filters

Inspired by BeautifulSoup's find_all function you can find elements by using find_all/find methods. Both methods can take multiple types of filters and return all elements in the pages that all these filters apply to.

• To be more specific:
• Any string passed is considered a tag name
• Any iterable passed like List/Tuple/Set is considered an iterable of tag names.
• Any dictionary is considered a mapping of HTML element(s) attribute names and attribute values.
• Any regex patterns passed are used as filters to elements by their text content
• Any functions passed are used as filters
• Any keyword argument passed is considered as an HTML element attribute with its value.

So the way it works is after collecting all passed arguments and keywords, each filter passes its results to the following filter in a waterfall-like filtering system.
It filters all elements in the current page/element in the following order:

1. All elements with the passed tag name(s).
2. All elements that match all passed attribute(s).
3. All elements that its text content match all passed regex patterns.
4. All elements that fulfill all passed function(s).

Note: The filtering process always starts from the first filter it finds in the filtering order above so if no tag name(s) are passed but attributes are passed, the process starts from that layer and so on. But the order in which you pass the arguments doesn't matter.

Examples to clear any confusion :)

>> from scrapling.fetchers import Fetcher
>> page = Fetcher().get('https://quotes.toscrape.com/')
# Find all elements with tag name `div`.
>> page.find_all('div')
[<data='<div class="container"> <div class="row...' parent='<body> <div class="container"> <div clas...'>,
 <data='<div class="row header-box"> <div class=...' parent='<div class="container"> <div class="row...'>,
...]

# Find all div elements with a class that equals `quote`.
>> page.find_all('div', class_='quote')
[<data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
 <data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
...]

# Same as above.
>> page.find_all('div', {'class': 'quote'})
[<data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
 <data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
...]

# Find all elements with a class that equals `quote`.
>> page.find_all({'class': 'quote'})
[<data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
 <data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
...]

# Find all div elements with a class that equals `quote`, and contains the element `.text` which contains the word 'world' in its content.
>> page.find_all('div', {'class': 'quote'}, lambda e: "world" in e.css_first('.text::text'))
[<data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>]

# Find all elements that don't have children.
>> page.find_all(lambda element: len(element.children) > 0)
[<data='<html lang="en"><head><meta charset="UTF...'>,
 <data='<head><meta charset="UTF-8"><title>Quote...' parent='<html lang="en"><head><meta charset="UTF...'>,
 <data='<body> <div class="container"> <div clas...' parent='<html lang="en"><head><meta charset="UTF...'>,
...]

# Find all elements that contain the word 'world' in its content.
>> page.find_all(lambda element: "world" in element.text)
[<data='<span class="text" itemprop="text">"The...' parent='<div class="quote" itemscope itemtype="h...'>,
 <data='<a class="tag" href="/tag/world/page/1/"...' parent='<div class="tags"> Tags: <meta class="ke...'>]

# Find all span elements that match the given regex
>> page.find_all('span', re.compile(r'world'))
[<data='<span class="text" itemprop="text">"The...' parent='<div class="quote" itemscope itemtype="h...'>]

# Find all div and span elements with class 'quote' (No span elements like that so only div returned)
>> page.find_all(['div', 'span'], {'class': 'quote'})
[<data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
 <data='<div class="quote" itemscope itemtype="h...' parent='<div class="col-md-8"> <div class="quote...'>,
...]

# Mix things up
>> page.find_all({'itemtype':"http://schema.org/CreativeWork"}, 'div').css('.author::text')
['Albert Einstein',
 'J.K. Rowling',
...]

Is That All?

Here's what else you can do with Scrapling:

• Accessing the lxml.etree object itself of any element directly python >>> quote._root <Element div at 0x107f98870>

• Saving and retrieving elements manually to auto-match them outside the css and the xpath methods but you have to set the identifier by yourself.

• To save an element to the database: python >>> element = page.find_by_text('Tipping the Velvet', first_match=True) >>> page.save(element, 'my_special_element')

• Now later when you want to retrieve it and relocate it inside the page with auto-matching, it would be like this python >>> element_dict = page.retrieve('my_special_element') >>> page.relocate(element_dict, adaptor_type=True) [<data='<a href="catalogue/tipping-the-velvet_99...' parent='<h3><a href="catalogue/tipping-the-velve...'>] >>> page.relocate(element_dict, adaptor_type=True).css('::text') ['Tipping the Velvet']

• if you want to keep it as lxml.etree object, leave the adaptor_type argument python >>> page.relocate(element_dict) [<Element a at 0x105a2a7b0>]

• Filtering results based on a function

# Find all products over $50
expensive_products = page.css('.product_pod').filter(
    lambda p: float(p.css('.price_color').re_first(r'[\d\.]+')) > 50
)

• Searching results for the first one that matches a function

# Find all the products with price '53.23'
page.css('.product_pod').search(
    lambda p: float(p.css('.price_color').re_first(r'[\d\.]+')) == 54.23
)

• Doing operations on element content is the same as scrapy python quote.re(r'regex_pattern') # Get all strings (TextHandlers) that match the regex pattern quote.re_first(r'regex_pattern') # Get the first string (TextHandler) only quote.json() # If the content text is jsonable, then convert it to json using `orjson` which is 10x faster than the standard json library and provides more options except that you can do more with them like python quote.re( r'regex_pattern', replace_entities=True, # Character entity references are replaced by their corresponding character clean_match=True, # This will ignore all whitespaces and consecutive spaces while matching case_sensitive= False, # Set the regex to ignore letters case while compiling it ) Hence all of these methods are methods from the TextHandler within that contains the text content so the same can be done directly if you call the .text property or equivalent selector function.

• Doing operations on the text content itself includes

• Cleaning the text from any white spaces and replacing consecutive spaces with single space python quote.clean()
• You already know about the regex matching and the fast json parsing but did you know that all strings returned from the regex search are actually TextHandler objects too? so in cases where you have for example a JS object assigned to a JS variable inside JS code and want to extract it with regex and then convert it to json object, in other libraries, these would be more than 1 line of code but here you can do it in 1 line like this python page.xpath('//script/text()').re_first(r'var dataLayer = (.+);').json()

• Sort all characters in the string as if it were a list and return the new string python quote.sort(reverse=False)

  To be clear, TextHandler is a sub-class of Python's str so all normal operations/methods that work with Python strings will work with it.

• Any element's attributes are not exactly a dictionary but a sub-class of mapping called AttributesHandler that's read-only so it's faster and string values returned are actually TextHandler objects so all operations above can be done on them, standard dictionary operations that don't modify the data, and more :)

• Unlike standard dictionaries, here you can search by values too and can do partial searches. It might be handy in some cases (returns a generator of matches) python >>> for item in element.attrib.search_values('catalogue', partial=True): print(item) {'href': 'catalogue/tipping-the-velvet_999/index.html'}
• Serialize the current attributes to JSON bytes: python >>> element.attrib.json_string b'{"href":"catalogue/tipping-the-velvet_999/index.html","title":"Tipping the Velvet"}'
• Converting it to a normal dictionary python >>> dict(element.attrib) {'href': 'catalogue/tipping-the-velvet_999/index.html', 'title': 'Tipping the Velvet'}

Scrapling is under active development so expect many more features coming soon :)

More Advanced Usage

There are a lot of deep details skipped here to make this as short as possible so to take a deep dive, head to the docs section. I will try to keep it updated as possible and add complex examples. There I will explain points like how to write your storage system, write spiders that don't depend on selectors at all, and more...

Note that implementing your storage system can be complex as there are some strict rules such as inheriting from the same abstract class, following the singleton design pattern used in other classes, and more. So make sure to read the docs first.

[!IMPORTANT] A website is needed to provide detailed library documentation.
I'm trying to rush creating the website, researching new ideas, and adding more features/tests/benchmarks but time is tight with too many spinning plates between work, personal life, and working on Scrapling. I have been working on Scrapling for months for free after all.

If you like Scrapling and want it to keep improving then this is a friendly reminder that you can help by supporting me through the sponsor button.

⚡ Enlightening Questions and FAQs

This section addresses common questions about Scrapling, please read this section before opening an issue.

How does auto-matching work?

1. You need to get a working selector and run it at least once with methods css or xpath with the auto_save parameter set to True before structural changes happen.
2. Before returning results for you, Scrapling uses its configured database and saves unique properties about that element.

3. Now because everything about the element can be changed or removed, nothing from the element can be used as a unique identifier for the database. To solve this issue, I made the storage system rely on two things:

   1. The domain of the URL you gave while initializing the first Adaptor object
   2. The identifier parameter you passed to the method while selecting. If you didn't pass one, then the selector string itself will be used as an identifier but remember you will have to use it as an identifier value later when the structure changes and you want to pass the new selector.

   Together both are used to retrieve the element's unique properties from the database later. 4. Now later when you enable the auto_match parameter for both the Adaptor instance and the method call. The element properties are retrieved and Scrapling loops over all elements in the page and compares each one's unique properties to the unique properties we already have for this element and a score is calculated for each one. 5. Comparing elements is not exact but more about finding how similar these values are, so everything is taken into consideration, even the values' order, like the order in which the element class names were written before and the order in which the same element class names are written now. 6. The score for each element is stored in the table, and the element(s) with the highest combined similarity scores are returned.

How does the auto-matching work if I didn't pass a URL while initializing the Adaptor object?

Not a big problem as it depends on your usage. The word default will be used in place of the URL field while saving the element's unique properties. So this will only be an issue if you used the same identifier later for a different website that you didn't pass the URL parameter while initializing it as well. The save process will overwrite the previous data and auto-matching uses the latest saved properties only.

If all things about an element can change or get removed, what are the unique properties to be saved?

For each element, Scrapling will extract: - Element tag name, text, attributes (names and values), siblings (tag names only), and path (tag names only). - Element's parent tag name, attributes (names and values), and text.

I have enabled the auto_save/auto_match parameter while selecting and it got completely ignored with a warning message

That's because passing the auto_save/auto_match argument without setting auto_match to True while initializing the Adaptor object will only result in ignoring the auto_save/auto_match argument value. This behavior is purely for performance reasons so the database gets created only when you are planning to use the auto-matching features.

I have done everything as the docs but the auto-matching didn't return anything, what's wrong?

It could be one of these reasons: 1. No data were saved/stored for this element before. 2. The selector passed is not the one used while storing element data. The solution is simple - Pass the old selector again as an identifier to the method called. - Retrieve the element with the retrieve method using the old selector as identifier then save it again with the save method and the new selector as identifier. - Start using the identifier argument more often if you are planning to use every new selector from now on. 3. The website had some extreme structural changes like a new full design. If this happens a lot with this website, the solution would be to make your code as selector-free as possible using Scrapling features.

Can Scrapling replace code built on top of BeautifulSoup4?

Pretty much yeah, almost all features you get from BeautifulSoup can be found or achieved in Scrapling one way or another. In fact, if you see there's a feature in bs4 that is missing in Scrapling, please make a feature request from the issues tab to let me know.

Can Scrapling replace code built on top of AutoScraper?

Of course, you can find elements by text/regex, find similar elements in a more reliable way than AutoScraper, and finally save/retrieve elements manually to use later as the model feature in AutoScraper. I have pulled all top articles about AutoScraper from Google and tested Scrapling against examples in them. In all examples, Scrapling got the same results as AutoScraper in much less time.

Is Scrapling thread-safe?

Yes, Scrapling instances are thread-safe. Each Adaptor instance maintains its state.

More Sponsors!

Contributing

Everybody is invited and welcome to contribute to Scrapling. There is a lot to do!

Please read the contributing file before doing anything.

Disclaimer for Scrapling Project

[!CAUTION] This library is provided for educational and research purposes only. By using this library, you agree to comply with local and international laws regarding data scraping and privacy. The authors and contributors are not responsible for any misuse of this software. This library should not be used to violate the rights of others, for unethical purposes, or to use data in an unauthorized or illegal manner. Do not use it on any website unless you have permission from the website owner or within their allowed rules like the robots.txt file, for example.

License

This work is licensed under BSD-3

Acknowledgments

This project includes code adapted from: - Parsel (BSD License) - Used for translator submodule

Thanks and References

• Daijro's brilliant work on both BrowserForge and Camoufox
• Vinyzu's work on Playwright's mock on Botright
• brotector
• fakebrowser
• rebrowser-patches

Known Issues

• In the auto-matching save process, the unique properties of the first element from the selection results are the only ones that get saved. So if the selector you are using selects different elements on the page that are in different locations, auto-matching will probably return to you the first element only when you relocate it later. This doesn't include combined CSS selectors (Using commas to combine more than one selector for example) as these selectors get separated and each selector gets executed alone.

KitPloit - PenTest Tools!

Camtruder - Advanced RTSP Camera Discovery and Vulnerability Assessment Tool

Camtruder is a high-performance RTSP camera discovery and vulnerability assessment tool written in Go. It efficiently scans and identifies vulnerable RTSP cameras across networks using various authentication methods and path combinations, with support for both targeted and internet-wide scanning capabilities.

🌟 Key Features

• Advanced Scanning Capabilities
• Single IP targeting
• CIDR range scanning
• File-based target lists
• Pipe input support
• Internet-wide scanning with customizable limits
• Intelligent port discovery
• Location-based search using RIPE database

• Raw CIDR output for integration with other tools

• Screenshot Capability

• Capture screenshots of discovered cameras
• Automatic saving of JPEG images
• Requires ffmpeg installation

• Configurable output directory

• Location-Based Search

• Search by city or country name
• RIPE database integration
• Detailed output with netnames and IP ranges
• CIDR notation support

• Raw output mode for scripting

• Comprehensive Authentication Testing

• Built-in common credential database
• Custom username/password list support
• File-based credential input
• Multiple authentication format handling

• Credential validation system

• Smart Path Discovery

• Extensive default path database
• Vendor-specific path detection
• Dynamic path generation

• Automatic path validation

• High Performance Architecture

• Multi-threaded scanning engine
• Configurable connection timeouts
• Efficient resource management
• Smart retry mechanisms

• Parallel connection handling

• Advanced Output & Analysis

• Real-time console feedback
• Detailed logging system
• Camera fingerprinting
• Vendor detection
• Stream capability analysis
• Multiple output formats (verbose, raw)

📋 Requirements

• Go 1.19 or higher
• ffmpeg (required for screenshot functionality)
• Internet connection
• Root/Administrator privileges (for certain scanning modes)
• Sufficient system resources for large-scale scans

🔧 Installation

Using go install (recommended)

go install github.com/ALW1EZ/camtruder@v3.7.0

From source

git clone https://github.com/ALW1EZ/camtruder.git
cd camtruder
go build

🚀 Usage

Basic Commands

# Scan a single IP
./camtruder -t 192.168.1.100

# Scan a network range
./camtruder -t 192.168.1.0/24

# Search by location with detailed output
./camtruder -t london -s
> [ NET-ISP ] [ 192.168.1.0/24 ] [256]

# Get raw CIDR ranges for location
./camtruder -t london -ss
> 192.168.1.0/24

# Scan multiple IPs from file
./camtruder -t targets.txt

# Take screenshots of discovered cameras
./camtruder -t 192.168.1.0/24 -m screenshots

# Pipe from port scanners
naabu -host 192.168.1.0/24 -p 554 | camtruder
masscan 192.168.1.0/24 -p554 --rate 1000 | awk '{print $6}' | camtruder
zmap -p554 192.168.0.0/16 | camtruder

# Internet scan (scan till 100 hits)
./camtruder -t 100

Advanced Options

# Custom credentials with increased threads
./camtruder -t 192.168.1.0/24 -u admin,root -p pass123,admin123 -w 50

# Location search with raw output piped to zmap
./camtruder -t berlin -ss | while read range; do zmap -p 554 $range; done

# Save results to file (as full url, you can use mpv --playlist=results.txt to watch the streams)
./camtruder -t istanbul -o results.txt

# Internet scan with limit of 50 workers and verbose output
./camtruder -t 100 -w 50 -v

🛠️ Command Line Options

📊 Output Formats

Standard Search Output (-s)

[ TR-NET-ISP ] [ 193.3.52.0/24 ] [256]
[ EXAMPLE-ISP ] [ 212.175.100.136/29 ] [8]

Raw CIDR Output (-ss)

193.3.52.0/24
212.175.100.136/29

Scan Results

╭─ Found vulnerable camera [Hikvision, H264, 30fps]
├ Host      : 192.168.1.100:554
├ Geo       : United States/California/Berkeley
├ Auth      : admin:12345
├ Path      : /Streaming/Channels/1
╰ URL       : rtsp://admin:12345@192.168.1.100:554/Streaming/Channels/1

⚠️ Disclaimer

This tool is intended for security research and authorized testing only. Users are responsible for ensuring they have permission to scan target systems and comply with all applicable laws and regulations.

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Thanks to all contributors and the security research community
• Special thanks to the Go RTSP library maintainers
• Inspired by various open-source security tools

📬 Contact

• Author: @ALW1EZ
• Project Link: https://github.com/ALW1EZ/camtruder

Made by @ALW1EZ

KitPloit - PenTest Tools!

PEGASUS-NEO - A Comprehensive Penetration Testing Framework Designed For Security Professionals And Ethical Hackers. It Combines Multiple Security Tools And Custom Modules For Reconnaissance, Exploitation, Wireless Attacks, Web Hacking, And More

                              ____                                  _   _ 
                             |  _ \ ___  __ _  __ _ ___ _   _ ___| \ | |
                             | |_) / _ \/ _` |/ _` / __| | | / __|  \| |
                             |  __/  __/ (_| | (_| \__ \ |_| \__ \ |\  |
                             |_|   \___|\__, |\__,_|___/\__,_|___/_| \_|
                                       |___/                             
                                 ███▄    █ ▓█████  ▒█████  
                                 ██ ▀█   █ ▓█   ▀ ▒██▒  ██▒
                                ▓██  ▀█ ██▒▒███   ▒██░  ██▒
                                ▓██▒  ▐▌██▒▒▓█  ▄ ▒██   ██░
                                ▒██░   ▓██░░▒████▒░ ████▓▒░
                                ░ ▒░   ▒ ▒ ░░ ▒░ ░░ ▒░▒░▒░ 
                                ░ ░░   ░ ▒░ ░ ░  ░  ░ ▒ ▒░ 
                                   ░   ░ ░    ░   ░ ░ ░ ▒  
                                         ░    ░  ░    ░ ░

PEGASUS-NEO Penetration Testing Framework

🛡️ Description

PEGASUS-NEO is a comprehensive penetration testing framework designed for security professionals and ethical hackers. It combines multiple security tools and custom modules for reconnaissance, exploitation, wireless attacks, web hacking, and more.

⚠️ Legal Disclaimer

This tool is provided for educational and ethical testing purposes only. Usage of PEGASUS-NEO for attacking targets without prior mutual consent is illegal. It is the end user's responsibility to obey all applicable local, state, and federal laws.

Developers assume no liability and are not responsible for any misuse or damage caused by this program.

🔒 Copyright Notice

PEGASUS-NEO - Advanced Penetration Testing Framework
Copyright (C) 2024 Letda Kes dr. Sobri. All rights reserved.

This software is proprietary and confidential. Unauthorized copying, transfer, or
reproduction of this software, via any medium is strictly prohibited.

Written by Letda Kes dr. Sobri <muhammadsobrimaulana31@gmail.com>, January 2024

🌟 Features

Password: Sobri

• Reconnaissance & OSINT
• Network scanning
• Email harvesting
• Domain enumeration

• Social media tracking

• Exploitation & Pentesting

• Automated exploitation
• Password attacks
• SQL injection

• Custom payload generation

• Wireless Attacks

• WiFi cracking
• Evil twin attacks

• WPS exploitation

• Web Attacks

• Directory scanning
• XSS detection
• SQL injection

• CMS scanning

• Social Engineering

• Phishing templates
• Email spoofing

• Credential harvesting

• Tracking & Analysis

• IP geolocation
• Phone number tracking
• Email analysis
• Social media hunting

🔧 Installation

# Clone the repository
git clone https://github.com/sobri3195/pegasus-neo.git

# Change directory
cd pegasus-neo

# Install dependencies
sudo python3 -m pip install -r requirements.txt

# Run the tool
sudo python3 pegasus_neo.py

📋 Requirements

• Python 3.8+
• Linux Operating System (Kali/Ubuntu recommended)
• Root privileges
• Internet connection

🚀 Usage

1. Start the tool:

sudo python3 pegasus_neo.py

1. Enter authentication password
2. Select category from main menu
3. Choose specific tool or module
4. Follow on-screen instructions

🔐 Security Features

• Source code protection
• Integrity checking
• Anti-tampering mechanisms
• Encrypted storage
• Authentication system

🛠️ Supported Tools

Reconnaissance & OSINT

• Nmap
• Wireshark
• Maltego
• Shodan
• theHarvester
• Recon-ng
• SpiderFoot
• FOCA
• Metagoofil

Exploitation & Pentesting

• Metasploit
• SQLmap
• Commix
• BeEF
• SET
• Hydra
• John the Ripper
• Hashcat

Wireless Hacking

• Aircrack-ng
• Kismet
• WiFite
• Fern Wifi Cracker
• Reaver
• Wifiphisher
• Cowpatty
• Fluxion

Web Hacking

• Burp Suite
• OWASP ZAP
• Nikto
• XSStrike
• Wapiti
• Sublist3r
• DirBuster
• WPScan

📝 Version History

• v1.0.0 (2024-01) - Initial release
• v1.1.0 (2024-02) - Added tracking modules
• v1.2.0 (2024-03) - Added tool installer

👥 Contributing

This is a proprietary project and contributions are not accepted at this time.

🤝 Support

For support, please email muhammadsobrimaulana31@gmail.com atau https://lynk.id/muhsobrimaulana

⚖️ License

This project is protected under proprietary license. See the LICENSE file for details.

Made with ❤️ by Letda Kes dr. Sobri

KitPloit - PenTest Tools!

Ghost-Route - Ghost Route Detects If A Next JS Site Is Vulnerable To The Corrupt Middleware Bypass Bug (CVE-2025-29927)

A Python script to check Next.js sites for corrupt middleware vulnerability (CVE-2025-29927).

The corrupt middleware vulnerability allows an attacker to bypass authentication and access protected routes by send a custom header x-middleware-subrequest.

Next JS versions affected: - 11.1.4 and up

[!WARNING] This tool is for educational purposes only. Do not use it on websites or systems you do not own or have explicit permission to test. Unauthorized testing may be illegal and unethical.

Installation

Clone the repo

git clone https://github.com/takumade/ghost-route.git
cd ghost-route

Create and activate virtual environment

python -m venv .venv
source .venv/bin/activate

Install dependencies

pip install -r requirements.txt

Usage

python ghost-route.py <url> <path> <show_headers>

• <url>: Base URL of the Next.js site (e.g., https://example.com)
• <path>: Protected path to test (default: /admin)
• <show_headers>: Show response headers (default: False)

Example

Basic Example

python ghost-route.py https://example.com /admin

Show Response Headers

python ghost-route.py https://example.com /admin True

License

MIT License

Credits

• CVE-2025-29927
• Next.js and the corrupt middleware: the authorizing artifact
• Rachid A.
• Yasser Allam

KitPloit - PenTest Tools!

Bytesrevealer - Online Reverse Enginerring Viewer

Bytes Revealer is a powerful reverse engineering and binary analysis tool designed for security researchers, forensic analysts, and developers. With features like hex view, visual representation, string extraction, entropy calculation, and file signature detection, it helps users uncover hidden data inside files. Whether you are analyzing malware, debugging binaries, or investigating unknown file formats, Bytes Revealer makes it easy to explore, search, and extract valuable information from any binary file.

Bytes Revealer do NOT store any file or data. All analysis is performed in your browser.

Current Limitation: Files less than 50MB can perform all analysis, files bigger up to 1.5GB will only do Visual View and Hex View analysis.

Features

File Analysis

• Chunked file processing for memory efficiency
• Real-time progress tracking
• File signature detection
• Hash calculations (MD5, SHA-1, SHA-256)
• Entropy and Bytes Frequency analysis

Multiple Views

File View

• Basic file information and metadata
• File signatures detection
• Hash values
• Entropy calculation
• Statistical analysis

Visual View

• Binary data visualization
• ASCII or Bytes searching
• Data distribution view
• Highlighted pattern matching

Hex View

• Traditional hex editor interface
• Byte-level inspection
• Highlighted pattern matching
• ASCII representation
• ASCII or Bytes searching

String Analysis

• ASCII and UTF-8 string extraction
• String length analysis
• String type categorization
• Advanced filtering and sorting
• String pattern recognition
• Export capabilities

Search Capabilities

• Hex pattern search
• ASCII/UTF-8 string search
• Regular expression support
• Highlighted search results

Technical Details

Built With

• Vue.js 3
• Tailwind CSS
• Web Workers for performance
• Modern JavaScript APIs

Performance Features

• Chunked file processing
• Web Worker implementation
• Memory optimization
• Cancelable operations
• Progress tracking

Getting Started

Prerequisites

# Node.js 14+ is required
node -v

Docker Usage

docker-compose build --no-cache

docker-compose up -d

Now open your browser: http://localhost:8080/

To stop the docker container

docker-compose down

Installation

# Clone the repository
git clone https://github.com/vulnex/bytesrevealer

# Navigate to project directory
cd bytesrevealer

# Install dependencies
npm install

# Start development server
npm run dev

Building for Production

# Build the application
npm run build

# Preview production build
npm run preview

Usage

1. File Upload
2. Click "Choose File" or drag and drop a file

3. Progress bar shows upload and analysis status

4. Analysis Views

5. Switch between views using the tab interface
6. Each view provides different analysis perspectives

7. Real-time updates as you navigate

8. Search Functions

9. Use the search bar for pattern matching
10. Toggle between hex and string search modes

11. Results are highlighted in the current view

12. String Analysis

13. View extracted strings with type and length
14. Filter strings by type or content
15. Sort by various criteria
16. Export results in multiple formats

Performance Considerations

• Large files are processed in chunks
• Web Workers handle intensive operations
• Memory usage is optimized
• Operations can be canceled if needed

Browser Compatibility

• Chrome 80+
• Firefox 75+
• Safari 13.1+
• Edge 80+

Contributing

1. Fork the project
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE.md file for details.

Security Considerations

• All strings are properly escaped
• Input validation is implemented
• Memory limits are enforced
• File size restrictions are in place

Future Enhancements

• Additional file format support
• More visualization options
• Pattern recognition improvements
• Advanced string analysis features
• Export/import capabilities
• Collaboration features

KitPloit - PenTest Tools!

CentralizedFirewall - Provides A Firewall Manager API Designed To Centralize And Streamline The Management Of Firewall Configurations

Firewall Manager API Project

Installation

Follow these steps to set up and run the API project:

1. Clone the Repository

git clone https://github.com/adriyansyah-mf/CentralizedFirewall
cd CentralizedFirewall

2. Edit the .env File

Update the environment variables in .env according to your configuration.

nano .env

3. Start the API with Docker Compose

docker compose up -d

This will start the API in detached mode.

4. Verify the API is Running

Check if the containers are up:

docker ps

Additional Commands

Stop the API

docker compose down

Restart the API

docker compose restart

Let me know if you need any modifications! 🚀

How to setup for the first time and connect to firewall client

1. Install Firewall Agent on your node server
2. Run the agent with the following command

sudo dpkg -i firewall-client_deb.deb

1. Create a New Group on the Firewall Manager
2. Create New API Key on the Firewall Manager
3. Edit the configuration file on the node server

nano /usr/local/bin/config.ini

1. Add the following configuration

[settings]
api_url = API-URL
api_key = API-KEY
hostname = Node Hostname (make it unique and same as the hostname on the SIEM) 

1. Restart the firewall agent

systemctl daemon-reload
systemctl start firewall-agent

1. Check the status of the firewall agent

systemctl status firewall-agent

1. You will see the connected node on the Firewall Manager

Default Credential

Username: admin
Password: admin

You can change the default credential on the setting page

How to Integration with SIEM

1. Install the SIEM on your server
2. Configure the SIEM to send the log to the Firewall Manager (You can do this via SOAR or SIEM configuration) The request should be POST with the following format
3. The format of the log should be like this

curl -X 'POST' \
  'http://api-server:8000/general/add-ip?ip=123.1.1.99&hostname=test&apikey=apikey&comment=log' \
  -H 'accept: application/json' \
  -d ''

You can see the swagger documentation on the following link

http://api-server:8000/docs

The .env detail configuration

DB=changeme
JWT_SECRET=changeme
PASSWORD_SALT=changme
PASSWORD_TOKEN_KEY=changme
OPENCTI_URL=changme
OPENCTI_TOKEN=changme

Sponsor This Project 💖

If you find this project helpful, consider supporting me through GitHub Sponsors

KitPloit - PenTest Tools!

Maryam - Open-source Intelligence(OSINT) Framework

OWASP Maryam is a modular open-source framework based on OSINT and data gathering. It is designed to provide a robust environment to harvest data from open sources and search engines quickly and thoroughly.

Installation

Supported OS

• Linux
• FreeBSD
• Darwin
• OSX

$ pip install maryam

Alternatively, you can install the latest version with the following command (Recommended):

pip install git+https://github.com/saeeddhqan/maryam.git

Usage

# Using dns_search. --max means all of resources. --api shows the results as json.
# .. -t means use multi-threading.
maryam -e dns_search -d ibm.com -t 5 --max --api --form 
# Using youtube. -q means query
maryam -e youtube -q "<QUERY>"
maryam -e google -q "<QUERY>"
maryam -e dnsbrute -d domain.tld
# Show framework modules
maryam -e show modules
# Set framework options.
maryam -e set proxy ..
maryam -e set agent ..
maryam -e set timeout ..
# Run web API
maryam -e web api 127.0.0.1 1313

Contribution

Here is a start guide: Development Guide You can add a new search engine to the util classes or use the current search engines to write a new module. The best help to write a new module is checking the current modules.

Roadmap

• Write a language model based search

Links

OWASP

Wiki

Install

Modules Guide

Development Guide

To report bugs, requests, or any other issues please create an issue.

KitPloit - PenTest Tools!

TruffleHog Explorer - A User-Friendly Web-Based Tool To Visualize And Analyze Data Extracted Using TruffleHog

Welcome to TruffleHog Explorer, a user-friendly web-based tool to visualize and analyze data extracted using TruffleHog. TruffleHog is one of the most powerful secrets discovery, classification, validation, and analysis open source tool. In this context, a secret refers to a credential a machine uses to authenticate itself to another machine. This includes API keys, database passwords, private encryption keys, and more.

With an improved UI/UX, powerful filtering options, and export capabilities, this tool helps security professionals efficiently review potential secrets and credentials found in their repositories.

⚠️ This dashboard has been tested only with GitHub TruffleHog JSON outputs. Expect updates soon to support additional formats and platforms.

You can use online version here: TruffleHog Explorer

🚀 Features

• Intuitive UI/UX: Beautiful pastel theme with smooth navigation.
• Powerful Filtering:
• Filter findings by repository, detector type, and uploaded file.
• Flexible date range selection with a calendar picker.
• Verification status categorization for effective review.
• Advanced search capabilities for faster identification.
• Batch Operations:
• Verify or reject multiple findings with a single click.
• Toggle visibility of rejected results for a streamlined view.
• Bulk processing to manage large datasets efficiently.
• Export Capabilities:
• Export verified secrets or filtered findings effortlessly.
• Save and load session backups for continuity.
• Generate reports in multiple formats (JSON, CSV).
• Dynamic Sorting:
• Sort results by repository, date, or verification status.
• Customizable sorting preferences for a personalized experience.

📥 Installation & Usage

1. Clone the Repository

$ git clone https://github.com/yourusername/trufflehog-explorer.git
$ cd trufflehog-explorer

2. Open the index.html

Simply open the index.html file in your preferred web browser.

$ open index.html

📂 How to Use

1. Upload TruffleHog JSON Findings:
2. Click on the "Load Data" section and select your .json files from TruffleHog output.
3. Multiple files are supported.
4. Apply Filters:
5. Choose filters such as repository, detector type, and verification status.
6. Utilize the date range picker to narrow down findings.
7. Leverage the search function to locate specific findings quickly.
8. Review Findings:
9. Click on a finding to expand and view its details.
10. Use the action buttons to verify or reject findings.
11. Add comments and annotations for better tracking.
12. Export Results:
13. Export verified or filtered findings for reporting.
14. Save session data for future review and analysis.
15. Save Your Progress:
16. Save your session and resume later without losing any progress.
17. Automatic backup feature to prevent data loss.

Happy Securing! 🔒

KitPloit - PenTest Tools!

PANO - Advanced OSINT Investigation Platform Combining Graph Visualization, Timeline Analysis, And AI Assistance To Uncover Hidden Connections In Data

Getting Started

1. Clone the repository: bash git clone https://github.com/ALW1EZ/PANO.git cd PANO

2. Run the application:

3. Linux: ./start_pano.sh
4. Windows: start_pano.bat

The startup script will automatically: - Check for updates - Set up the Python environment - Install dependencies - Launch PANO

In order to use Email Lookup transform You need to login with GHunt first. After starting the pano via starter scripts;

1. Select venv manually
2. Linux: source venv/bin/activate
3. Windows: call venv\Scripts\activate
4. See how to login here

💡 Quick Start Guide

1. Create Investigation: Start a new investigation or load an existing one
2. Add Entities: Drag entities from the sidebar onto the graph
3. Discover Connections: Use transforms to automatically find relationships
4. Analyze: Use timeline and map views to understand patterns
5. Save: Export your investigation for later use

🔍 Features

🕸️ Core Functionality

• Interactive Graph Visualization
• Drag-and-drop entity creation
• Multiple layout algorithms (Circular, Hierarchical, Radial, Force-Directed)
• Dynamic relationship mapping

• Visual node and edge styling

• Timeline Analysis

• Chronological event visualization
• Interactive timeline navigation
• Event filtering and grouping

• Temporal relationship analysis

• Map Integration

• Geographic data visualization
• Location-based analysis
• Interactive mapping features
• Coordinate plotting and tracking

🎯 Entity Management

• Supported Entity Types
• 📧 Email addresses
• 👤 Usernames
• 🌐 Websites
• 🖼️ Images
• 📍 Locations
• ⏰ Events
• 📝 Text content
• 🔧 Custom entity types

🔄 Transform System

• Email Analysis
• Google account investigation
• Calendar event extraction
• Location history analysis

• Connected services discovery

• Username Analysis

• Cross-platform username search
• Social media profile discovery
• Platform correlation

• Web presence analysis

• Image Analysis

• Reverse image search
• Visual content analysis
• Metadata extraction
• Related image discovery

🤖 AI Integration

• PANAI
• Natural language investigation assistant
• Automated entity extraction and relationship mapping
• Pattern recognition and anomaly detection
• Multi-language support
• Context-aware suggestions
• Timeline and graph analysis

🧩 Core Components

📦 Entities

Entities are the fundamental building blocks of PANO. They represent distinct pieces of information that can be connected and analyzed:

• Built-in Types
• 📧 Email: Email addresses with service detection
• 👤 Username: Social media and platform usernames
• 🌐 Website: Web pages with metadata
• 🖼️ Image: Images with EXIF and analysis
• 📍 Location: Geographic coordinates and addresses
• ⏰ Event: Time-based occurrences

• 📝 Text: Generic text content

• Properties System

• Type-safe property validation
• Automatic property getters
• Dynamic property updates
• Custom property types
• Metadata support

⚡ Transforms

Transforms are automated operations that process entities to discover new information and relationships:

• Operation Types
• 🔍 Discovery: Find new entities from existing ones
• 🔗 Correlation: Connect related entities
• 📊 Analysis: Extract insights from entity data
• 🌐 OSINT: Gather open-source intelligence

• 🔄 Enrichment: Add data to existing entities

• Features

• Async operation support
• Progress tracking
• Error handling
• Rate limiting
• Result validation

🛠️ Helpers

Helpers are specialized tools with dedicated UIs for specific investigation tasks:

• Available Helpers
• 🔍 Cross-Examination: Analyze statements and testimonies
• 👤 Portrait Creator: Generate facial composites
• 📸 Media Analyzer: Advanced image processing and analysis
• 🔍 Base Searcher: Search near places of interest

• 🔄 Translator: Translate text between languages

• Helper Features

• Custom Qt interfaces
• Real-time updates
• Graph integration
• Data visualization
• Export capabilities

👥 Contributing

We welcome contributions! To contribute to PANO:

1. Fork the repository at https://github.com/ALW1EZ/PANO/
2. Make your changes in your fork
3. Test your changes thoroughly
4. Create a Pull Request to our main branch
5. In your PR description, include:
6. What the changes do
7. Why you made these changes
8. Any testing you've done
9. Screenshots if applicable

Note: We use a single main branch for development. All pull requests should be made directly to main.

📖 Development Guide

from dataclasses import dataclass
from typing import ClassVar, Dict, Any
from .base import Entity

@dataclass
class PhoneNumber(Entity):
    name: ClassVar[str] = "Phone Number"
    description: ClassVar[str] = "A phone number entity with country code and validation"

    def init_properties(self):
        """Initialize phone number properties"""
        self.setup_properties({
            "number": str,
            "country_code": str,
            "carrier": str,
            "type": str,  # mobile, landline, etc.
            "verified": bool
        })

    def update_label(self):
        """Update the display label"""
        self.label = self.format_label(["country_code", "number"])

from dataclasses import dataclass
from typing import ClassVar, List
from .base import Transform
from entities.base import Entity
from entities.phone_number import PhoneNumber
from entities.location import Location
from ui.managers.status_manager import StatusManager

@dataclass
class PhoneLookup(Transform):
    name: ClassVar[str] = "Phone Number Lookup"
    description: ClassVar[str] = "Lookup phone number details and location"
    input_types: ClassVar[List[str]] = ["PhoneNumber"]
    output_types: ClassVar[List[str]] = ["Location"]

    async def run(self, entity: PhoneNumber, graph) -> List[Entity]:
        if not isinstance(entity, PhoneNumber):
            return []

        status = StatusManager.get()
        operation_id = status.start_loading("Phone Lookup")

        try:
            # Your phone number lookup logic here
            # Example: query an API for phone number details
            location = Location(properties={
                "country": "Example Country",
                "region": "Example Region",
                "carrier": "Example Carrier",
                "source": "PhoneLookup transform"
            })

            return [location]

        except Exception as e:
            status.set_text(f"Error during phone lookup: {str(e)}")
            return []

        finally:
            status.stop_loading(operation_id)

from PySide6.QtWidgets import (
    QWidget, QVBoxLayout, QHBoxLayout, QPushButton,
    QTextEdit, QLabel, QComboBox
)
from .base import BaseHelper
from qasync import asyncSlot

class DummyHelper(BaseHelper):
    """A dummy helper for testing"""

    name = "Dummy Helper" 
    description = "A dummy helper for testing"

    def setup_ui(self):
        """Initialize the helper's user interface"""
        # Create input text area
        self.input_label = QLabel("Input:")
        self.input_text = QTextEdit()
        self.input_text.setPlaceholderText("Enter text to process...")
        self.input_text.setMinimumHeight(100)

        # Create operation selector
        operation_layout = QHBoxLayout()
        self.operation_label = QLabel("Operation:")
        self.operation_combo = QComboBox()
        self.operation_combo.addItems(["Uppercase", "Lowercase", "Title Case"])
        operation_layout.addWidget(self.operation_label)
        operation_layout.addWidget(self.operation_combo)

        # Create process button
        self.process_btn = QPushButton("Process")
        self.process_btn.clicked.connect(self.process_text)

        # Create output text area
        self.output_label = QLabel("Output:")
        self.output_text = QTextEdit()
        self.output_text.setReadOnly(True)
        self.output_text.setMinimumHeight(100)

        # Add widgets to main layout
        self.main_layout.addWidget(self.input_label)
        self.main_layout.addWidget(self.input_text)
        self.main_layout.addLayout(operation_layout)
        self.main_layout.addWidget(self.process_btn)
        self.main_layout.addWidget(self.output_label)
        self.main_layout.addWidget(self.output_text)

        # Set dialog size
        self.resize(400, 500)

    @asyncSlot()
    async def process_text(self):
        """Process the input text based on selected operation"""
        text = self.input_text.toPlainText()
        operation = self.operation_combo.currentText()

        if operation == "Uppercase":
            result = text.upper()
        elif operation == "Lowercase":
            result = text.lower()
        else:  # Title Case
            result = text.title()

        self.output_text.setPlainText(result)

📄 License

This project is licensed under the Creative Commons Attribution-NonCommercial (CC BY-NC) License.

You are free to: - ✅ Share: Copy and redistribute the material - ✅ Adapt: Remix, transform, and build upon the material

Under these terms: - ℹ️ Attribution: You must give appropriate credit - 🚫 NonCommercial: No commercial use - 🔓 No additional restrictions

🙏 Acknowledgments

Special thanks to all library authors and contributors who made this project possible.

👨‍💻 Author

Created by ALW1EZ with AI ❤️

KitPloit - PenTest Tools!

Wappalyzer-Next - Python library that uses Wappalyzer extension (and its fingerprints) to detect technologies

This project is a command line tool and python library that uses Wappalyzer extension (and its fingerprints) to detect technologies. Other projects emerged after discontinuation of the official open source project are using outdated fingerpints and lack accuracy when used on dynamic web-apps, this project bypasses those limitations.

Installation

Before installing wappalyzer, you will to install Firefox and geckodriver/releases">geckodriver. Below are detailed steps for setting up geckodriver but you may use google/youtube for help.

Step 1: Download GeckoDriver

1. Visit the official GeckoDriver releases page on GitHub:
   https://github.com/mozilla/geckodriver/releases
2. Download the version compatible with your system:
3. For Windows: geckodriver-vX.XX.X-win64.zip
4. For macOS: geckodriver-vX.XX.X-macos.tar.gz
5. For Linux: geckodriver-vX.XX.X-linux64.tar.gz
6. Extract the downloaded file to a folder of your choice.

Step 2: Add GeckoDriver to the System Path

To ensure Selenium can locate the GeckoDriver executable: - Windows: 1. Move the geckodriver.exe to a directory (e.g., C:\WebDrivers\). 2. Add this directory to the system's PATH: - Open Environment Variables. - Under System Variables, find and select the Path variable, then click Edit. - Click New and enter the directory path where geckodriver.exe is stored. - Click OK to save. - macOS/Linux: 1. Move the geckodriver file to /usr/local/bin/ or another directory in your PATH. 2. Use the following command in the terminal: bash sudo mv geckodriver /usr/local/bin/ Ensure /usr/local/bin/ is in your PATH.

Install as a command-line tool

pipx install wappalyzer

Install as a library

To use it as a library, install it with pip inside an isolated container e.g. venv or docker. You may also --break-system-packages to do a 'regular' install but it is not recommended.

Install with docker

1. Clone the repository:

git clone https://github.com/s0md3v/wappalyzer-next.git
cd wappalyzer-next

1. Build and run with Docker Compose:

docker compose up -d

1. To scan URLs using the Docker container:

2. Scan a single URL:

docker compose run --rm wappalyzer -i https://example.com

• Scan Multiple URLs from a file:

docker compose run --rm wappalyzer -i https://example.com -oJ output.json

For Users

Some common usage examples are given below, refer to list of all options for more information.

• Scan a single URL: wappalyzer -i https://example.com
• Scan multiple URLs from a file: wappalyzer -i urls.txt -t 10
• Scan with authentication: wappalyzer -i https://example.com -c "sessionid=abc123; token=xyz789"
• Export results to JSON: wappalyzer -i https://example.com -oJ results.json

Options

Note: For accuracy use 'full' scan type (default). 'fast' and 'balanced' do not use browser emulation.

• -i: Input URL or file containing URLs (one per line)
• --scan-type: Scan type (default: 'full')
• fast: Quick HTTP-based scan (sends 1 request)
• balanced: HTTP-based scan with more requests
• full: Complete scan using wappalyzer extension
• -t, --threads: Number of concurrent threads (default: 5)
• -oJ: JSON output file path
• -oC: CSV output file path
• -oH: HTML output file path
• -c, --cookie: Cookie header string for authenticated scans

For Developers

The python library is a available on pypi as wappalyzer and can be imported with the same name.

Using the Library

The main function you'll interact with is analyze():

from wappalyzer import analyze

# Basic usage
results = analyze('https://example.com')

# With options
results = analyze(
    url='https://example.com',
    scan_type='full',  # 'fast', 'balanced', or 'full'
    threads=3,
    cookie='sessionid=abc123'
)

analyze() Function Parameters

• url (str): The URL to analyze
• scan_type (str, optional): Type of scan to perform
• 'fast': Quick HTTP-based scan
• 'balanced': HTTP-based scan with more requests
• 'full': Complete scan including JavaScript execution (default)
• threads (int, optional): Number of threads for parallel processing (default: 3)
• cookie (str, optional): Cookie header string for authenticated scans

Return Value

Returns a dictionary with the URL as key and detected technologies as value:

{
  "https://github.com": {
    "Amazon S3": {"version": "", "confidence": 100, "categories": ["CDN"], "groups": ["Servers"]},
    "lit-html": {"version": "1.1.2", "confidence": 100, "categories": ["JavaScript libraries"], "groups": ["Web development"]},
    "React Router": {"version": "6", "confidence": 100, "categories": ["JavaScript frameworks"], "groups": ["Web development"]},
  "https://google.com" : {},
  "https://example.com" : {},
}}

FAQ

Why use Firefox instead of Chrome?

Firefox extensions are .xpi files which are essentially zip files. This makes it easier to extract data and slightly modify the extension to make this tool work.

What is the difference between 'fast', 'balanced', and 'full' scan types?

• fast: Sends a single HTTP request to the URL. Doesn't use the extension.
• balanced: Sends additional HTTP requests to .js files, /robots.txt annd does DNS queries. Doesn't use the extension.
• full: Uses the official Wappalyzer extension to scan the URL in a headless browser.

KitPloit - PenTest Tools!

Telegram-Checker - A Python Tool For Checking Telegram Accounts Via Phone Numbers Or Usernames

Enhanced version of bellingcat's Telegram Phone Checker!

A Python script to check Telegram accounts using phone numbers or username.

✨ Features

• 🔍 Check single or multiple phone numbers and usernames
• 📁 Import numbers from text file
• 📸 Auto-download profile pictures
• 💾 Save results as JSON
• 🔐 Secure credential storage
• 📊 Detailed user information

🚀 Installation

1. Clone the repository:

git clone https://github.com/unnohwn/telegram-checker.git
cd telegram-checker

1. Install required packages:

pip install -r requirements.txt

📦 Requirements

Contents of requirements.txt:

telethon
rich
click
python-dotenv

Or install packages individually:

pip install telethon rich click python-dotenv

⚙️ Configuration

First time running the script, you'll need: - Telegram API credentials (get from https://my.telegram.org/apps) - Your Telegram phone number including countrycode + - Verification code (sent to your Telegram)

💻 Usage

Run the script:

python telegram_checker.py

Choose from options: 1. Check phone numbers from input 2. Check phone numbers from file 3. Check usernames from input 4. Check usernames from file 5. Clear saved credentials 6. Exit

📂 Output

Results are saved in: - results/ - JSON files with detailed information - profile_photos/ - Downloaded profile pictures

⚠️ Note

This tool is for educational purposes only. Please respect Telegram's terms of service and user privacy.

📄 License

MIT License

KitPloit - PenTest Tools!

Torward - An Improved Version Based On The Torghost-Gn And Darktor Scripts, Designed To Enhance Anonymity On The Internet

Torward is an improved version based on the torghost-gn and darktor scripts, designed to enhance anonymity on the Internet. The tool prevents data leaks and forces all traffic from our computer to be routed exclusively through the Tor network, providing a high level of privacy in our connections.

Installation

   git clone https://github.com/chundefined/Torward.git

   cd Torward

   chmod +x install.sh

   ./install.sh

Security Enhancements

This version includes several key security improvements to protect your identity and ensure better network configuration:

1. IPv6 Leak Prevention
   IPv6 is now disabled to prevent any potential IP leaks. All traffic is forced through the Tor network by modifying system IPv6 settings in network_config.py.

2. Enhanced iptables Rules
   Strict iptables rules are implemented to ensure only Tor traffic is allowed. Non-Tor traffic is blocked, DNS queries are routed through Tor, and only essential connections to Tor ports are permitted. Additionally, IPv6 traffic is blocked to prevent leaks.

3. Tor Configuration Adjustments
   The torward file has been updated to enforce that all traffic, including DNS queries, is routed through Tor, improving anonymity.

TODO

• Get the IP from the last Tor exit node: Currently, the script does not display the IP of the last Tor exit node in the console. This can be achieved by using Tor's API to get the public IP of the exit node.
• Better error handling: Ensure that the tool properly handles errors, such as Tor disconnection or network issues.

KitPloit - PenTest Tools!

Instagram-Brute-Force-2024 - Instagram Brute Force 2024 Compatible With Python 3.13 / X64 Bit / Only Chrome Browser

Instagram Brute Force CPU/GPU Supported 2024

(Use option 2 while running the script.)

(Option 1 is on development)

(Chrome should be downloaded in device.)

Compatible and Tested (GUI Supported Operating Systems Only)

Python 3.13 x64 bit Unix / Linux / Mac / Windows 8.1 and higher

Install Requirements

pip install -r requirements.txt

How to run

python3 instagram_brute_force.py [instagram_username_without_hashtag]
python3 instagram_brute_force.py mrx161

KitPloit - PenTest Tools!

QuickResponseC2 - A Command & Control Server That Leverages QR Codes To Send Commands And Receive Results From Remote Systems

QuickResponseC2 is a stealthy Command and Control (C2) framework that enables indirect and covert communication between the attacker and victim machines via an intermediate HTTP/S server. All network activity is limited to uploading and downloading images, making it an fully undetectable by IPS/IDS Systems and an ideal tool for security research and penetration testing.

Capabilities:

• Command Execution via QR Codes:
  Users can send custom commands to the victim machine, encoded as QR codes.
  Victims scan the QR code, which triggers the execution of the command on their system.
  The command can be anything from simple queries to complex operations based on the test scenario.

• Result Retrieval:
  Results of the executed command are returned from the victim system and encoded into a QR code.
  The server decodes the result and provides feedback to the attacker for further analysis or follow-up actions.

• Built-in HTTP Server:
  The tool includes a lightweight HTTP server that facilitates the victim machine's retrieval of command QR codes.
  Results are sent back to the server as QR code images, and they are automatically saved with unique filenames for easy management.
  The attacker's machine handles multiple requests, with HTTP logs organized and saved separately.

• Stealthy Communication:
  QuickResponseC2 operates under the radar, with minimal traces, providing a covert way to interact with the victim machine without alerting security defenses.
  Ideal for security assessments or testing command-and-control methodologies without being detected.

• File Handling:
  The tool automatically saves all QR codes (command and result) to the server_files directory, using sequential filenames like command0.png, command1.png, etc.
  Decoding and processing of result files are handled seamlessly.

• User-Friendly Interface:
  The tool is operated via a simple command-line interface, allowing users to set up a C2 server, send commands, and receive results with ease.
  No additional complex configurations or dependencies are needed.

Usage

1. First, install the Dependencies - pip3 install -r requirements.txt
2. Then, run the main.py python3 main.py
3. Choose between the options:

1 - Run the C2 Server

2 - Build the Victim Implant

1. Enjoy!

Demonstration

https://github.com/user-attachments/assets/382e9350-d650-44e5-b8ef-b43ec90b315d

Workflow Overview

1. Initialization of the C2 Server

• The attacker launches QuickResponseC2, which creates a lightweight HTTP server (default port: 8080).
• This server serves as the intermediary between the attacker and victim, eliminating any direct connection between them.

2. Command Delivery via QR Codes

• The attacker encodes a command into a QR code and saves it as commandX.png on the HTTP server.
• The victim machine periodically polls the server (e.g., every 1 second) to check for the presence of a new command file.

3. Victim Command Execution

• Once the victim detects a new QR code file (commandX.png), it downloads and decodes the image to retrieve the command.
• The decoded command is executed on the victim's system.

4. Result Encoding and Uploading

• The victim encodes the output of the executed command into a QR code and saves it locally as resultX.png.
• The result file is then uploaded to the HTTP server.

5. Result Retrieval by the Attacker

• The attacker periodically checks the server for new result files (resultX.png).
• Once found, the result file is downloaded and decoded to retrieve the output of the executed command.

TODO & Contribution

• [x] Generate a Template for the Implant
• [ ] Compile the implant as an .exe automatically
• [x] Save the generated QR Code as bytes in a variable instead of a file - VICTIM Side
• [ ] Add an obfuscation on the commands decoded from the QR Codes automatically

Feel free to fork and contribute! Pull requests are welcome.

KitPloit - PenTest Tools!

Telegram-Scraper - A Powerful Python Script That Allows You To Scrape Messages And Media From Telegram Channels Using The Telethon Library

A powerful Python script that allows you to scrape messages and media from Telegram channels using the Telethon library. Features include real-time continuous scraping, media downloading, and data export capabilities.

___________________  _________
\__    ___/  _____/ /   _____/
  |    | /   \  ___ \_____  \ 
  |    | \    \_\  \/        \
  |____|  \______  /_______  /
                 \/        \/

Features 🚀

• Scrape messages from multiple Telegram channels
• Download media files (photos, documents)
• Real-time continuous scraping
• Export data to JSON and CSV formats
• SQLite database storage
• Resume capability (saves progress)
• Media reprocessing for failed downloads
• Progress tracking
• Interactive menu interface

Prerequisites 📋

Before running the script, you'll need:

• Python 3.7 or higher
• Telegram account
• API credentials from Telegram

Required Python packages

pip install -r requirements.txt

Contents of requirements.txt:

telethon
aiohttp
asyncio

Getting Telegram API Credentials 🔑

1. Visit https://my.telegram.org/auth
2. Log in with your phone number
3. Click on "API development tools"
4. Fill in the form:
5. App title: Your app name
6. Short name: Your app short name
7. Platform: Can be left as "Desktop"
8. Description: Brief description of your app
9. Click "Create application"
10. You'll receive:
11. api_id: A number
12. api_hash: A string of letters and numbers

Keep these credentials safe, you'll need them to run the script!

Setup and Running 🔧

1. Clone the repository:

git clone https://github.com/unnohwn/telegram-scraper.git
cd telegram-scraper

1. Install requirements:

pip install -r requirements.txt

1. Run the script:

python telegram-scraper.py

1. On first run, you'll be prompted to enter:
2. Your API ID
3. Your API Hash
4. Your phone number (with country code)
5. Your phone number (with country code) or bot, but use the phone number option when prompted second time.
6. Verification code (sent to your Telegram)

Initial Scraping Behavior 🕒

When scraping a channel for the first time, please note:

• The script will attempt to retrieve the entire channel history, starting from the oldest messages
• Initial scraping can take several minutes or even hours, depending on:
• The total number of messages in the channel
• Whether media downloading is enabled
• The size and number of media files
• Your internet connection speed
• Telegram's rate limiting
• The script uses pagination and maintains state, so if interrupted, it can resume from where it left off
• Progress percentage is displayed in real-time to track the scraping status
• Messages are stored in the database as they are scraped, so you can start analyzing available data even before the scraping is complete

Usage 📝

The script provides an interactive menu with the following options:

• [A] Add new channel
• Enter the channel ID or channelname
• [R] Remove channel
• Remove a channel from scraping list
• [S] Scrape all channels
• One-time scraping of all configured channels
• [M] Toggle media scraping
• Enable/disable downloading of media files
• [C] Continuous scraping
• Real-time monitoring of channels for new messages
• [E] Export data
• Export to JSON and CSV formats
• [V] View saved channels
• List all saved channels
• [L] List account channels
• List all channels with ID:s for account
• [Q] Quit

Channel IDs 📢

You can use either: - Channel username (e.g., channelname) - Channel ID (e.g., -1001234567890)

Data Storage 💾

Database Structure

Data is stored in SQLite databases, one per channel: - Location: ./channelname/channelname.db - Table: messages - id: Primary key - message_id: Telegram message ID - date: Message timestamp - sender_id: Sender's Telegram ID - first_name: Sender's first name - last_name: Sender's last name - username: Sender's username - message: Message text - media_type: Type of media (if any) - media_path: Local path to downloaded media - reply_to: ID of replied message (if any)

Media Storage 📁

Media files are stored in: - Location: ./channelname/media/ - Files are named using message ID or original filename

Exported Data 📊

Data can be exported in two formats: 1. CSV: ./channelname/channelname.csv - Human-readable spreadsheet format - Easy to import into Excel/Google Sheets

1. JSON: ./channelname/channelname.json
2. Structured data format
3. Ideal for programmatic processing

Features in Detail 🔍

Continuous Scraping

The continuous scraping feature ([C] option) allows you to: - Monitor channels in real-time - Automatically download new messages - Download media as it's posted - Run indefinitely until interrupted (Ctrl+C) - Maintains state between runs

Media Handling

The script can download: - Photos - Documents - Other media types supported by Telegram - Automatically retries failed downloads - Skips existing files to avoid duplicates

Error Handling 🛠️

The script includes: - Automatic retry mechanism for failed media downloads - State preservation in case of interruption - Flood control compliance - Error logging for failed operations

Limitations ⚠️

• Respects Telegram's rate limits
• Can only access public channels or channels you're a member of
• Media download size limits apply as per Telegram's restrictions

Contributing 🤝

Contributions are welcome! Please feel free to submit a Pull Request.

License 📄

This project is licensed under the MIT License - see the LICENSE file for details.

Disclaimer ⚖️

This tool is for educational purposes only. Make sure to: - Respect Telegram's Terms of Service - Obtain necessary permissions before scraping - Use responsibly and ethically - Comply with data protection regulations

KitPloit - PenTest Tools!

Lobo Guará - Cyber Threat Intelligence Platform

Lobo Guará is a platform aimed at cybersecurity professionals, with various features focused on Cyber Threat Intelligence (CTI). It offers tools that make it easier to identify threats, monitor data leaks, analyze suspicious domains and URLs, and much more.

Features

1. SSL Certificate Search

Allows identifying domains and subdomains that may pose a threat to organizations. SSL certificates issued by trusted authorities are indexed in real-time, and users can search using keywords of 4 or more characters.

Note: The current database contains certificates issued from September 5, 2024.

2. SSL Certificate Discovery

Allows the insertion of keywords for monitoring. When a certificate is issued and the common name contains the keyword (minimum of 5 characters), it will be displayed to the user.

3. Tracking Link

Generates a link to capture device information from attackers. Useful when the security professional can contact the attacker in some way.

4. Domain Scan

Performs a scan on a domain, displaying whois information and subdomains associated with that domain.

5. Web Path Scan

Allows performing a scan on a URL to identify URIs (web paths) related to that URL.

6. URL Scan

Performs a scan on a URL, generating a screenshot and a mirror of the page. The result can be made public to assist in taking down malicious websites.

7. URL Monitoring

Monitors a URL with no active application until it returns an HTTP 200 code. At that moment, it automatically initiates a URL scan, providing evidence for actions against malicious sites.

8. Data Leak

• Data Leak Alerts: Monitors and presents almost real-time data leaks posted in hacker forums and websites.
• URL+User+Password: Allows searching by URL, username, or password, helping identify leaked data from clients or employees.

9. Threat Intelligence Feeds

Centralizes intelligence news from various channels, keeping users updated on the latest threats.

Installation

The application installation has been approved on Ubuntu 24.04 Server and Red Hat 9.4 distributions, the links for which are below:

Lobo Guará Implementation on Ubuntu 24.04

Lobo Guará Implementation on Red Hat 9.4

There is a Dockerfile and a docker-compose version of Lobo Guará too. Just clone the repo and do:

docker compose up

Then, go to your web browser at localhost:7405.

Dependencies

Before proceeding with the installation, ensure the following dependencies are installed:

• PostgreSQL
• Python 3.12
• ChromeDriver and Google Chrome (version 129.0.6668.89)
• FFUF (version 2.0.0)
• Subfinder (version 2.6.6)

Installation Instructions

1. Clone the repository:

git clone https://github.com/olivsec/loboguara.git

1. Enter the project directory:

cd loboguara/

1. Edit the configuration file:

nano server/app/config.py

Fill in the required parameters in the config.py file:

class Config:
    SECRET_KEY = 'YOUR_SECRET_KEY_HERE'
    SQLALCHEMY_DATABASE_URI = 'postgresql://guarauser:YOUR_PASSWORD_HERE@localhost/guaradb?sslmode=disable'
    SQLALCHEMY_TRACK_MODIFICATIONS = False

    MAIL_SERVER = 'smtp.example.com'
    MAIL_PORT = 587
    MAIL_USE_TLS = True
    MAIL_USERNAME = 'no-reply@example.com'
    MAIL_PASSWORD = 'YOUR_SMTP_PASSWORD_HERE'
    MAIL_DEFAULT_SENDER = 'no-reply@example.com'

    ALLOWED_DOMAINS = ['yourdomain1.my.id', 'yourdomain2.com', 'yourdomain3.net']

    API_ACCESS_TOKEN = 'YOUR_LOBOGUARA_API_TOKEN_HERE'
    API_URL = 'https://loboguara.olivsec.com.br/api'

    CHROME_DRIVER_PATH = '/opt/loboguara/bin/chromedriver'
    GOOGLE_CHROME_PATH = '/opt/loboguara/bin/google-chrome'
    FFUF_PATH = '/opt/loboguara/bin/ffuf'
    SUBFINDER_PATH = '/opt/loboguara/bin/subfinder'

    LOG_LEVEL = 'ERROR'
    LOG_FILE = '/opt/loboguara/logs/loboguara.log'

1. Make the installation script executable and run it:

sudo chmod +x ./install.sh
sudo ./install.sh

1. Start the service after installation:

sudo -u loboguara /opt/loboguara/start.sh

Access the URL below to register the Lobo Guará Super Admin

http://your_address:7405/admin

Online Platform

Access the Lobo Guará platform online: https://loboguara.olivsec.com.br/

KitPloit - PenTest Tools!

Telegram-Story-Scraper - A Python Script That Allows You To Automatically Scrape And Download Stories From Your Telegram Friends

A Python script that allows you to automatically scrape and download stories from your Telegram friends using the Telethon library. The script continuously monitors and saves both photos and videos from stories, along with their metadata.

Important Note About Story Access ⚠️

Due to Telegram API restrictions, this script can only access stories from: - Users you have added to your friend list - Users whose privacy settings allow you to view their stories

This is a limitation of Telegram's API and cannot be bypassed.

Features 🚀

• Automatically scrapes all available stories from your Telegram friends
• Downloads both photos and videos from stories
• Stores metadata in SQLite database
• Exports data to Excel spreadsheet
• Real-time monitoring with customizable intervals
• Timestamp is set to (UTC+2)
• Maintains record of previously downloaded stories
• Resume capability
• Automatic retry mechanism

Prerequisites 📋

Before running the script, you'll need:

• Python 3.7 or higher
• Telegram account
• API credentials from Telegram
• Friends on Telegram whose stories you want to track

Required Python packages

pip install -r requirements.txt

Contents of requirements.txt:

telethon
openpyxl
schedule

Getting Telegram API Credentials 🔑

1. Visit https://my.telegram.org/auth
2. Log in with your phone number
3. Click on "API development tools"
4. Fill in the form:
5. App title: Your app name
6. Short name: Your app short name
7. Platform: Can be left as "Desktop"
8. Description: Brief description of your app
9. Click "Create application"
10. You'll receive:
11. api_id: A number
12. api_hash: A string of letters and numbers

Keep these credentials safe, you'll need them to run the script!

Setup and Running 🔧

1. Clone the repository:

git clone https://github.com/unnohwn/telegram-story-scraper.git
cd telegram-story-scraper

1. Install requirements:

pip install -r requirements.txt

1. Run the script:

python TGSS.py

1. On first run, you'll be prompted to enter:
2. Your API ID
3. Your API Hash
4. Your phone number (with country code)
5. Verification code (sent to your Telegram)
6. Checking interval in seconds (default is 60)

How It Works 🔄

The script: 1. Connects to your Telegram account 2. Periodically checks for new stories from your friends 3. Downloads any new stories (photos/videos) 4. Stores metadata in a SQLite database 5. Exports information to an Excel file 6. Runs continuously until interrupted (Ctrl+C)

Data Storage 💾

Database Structure (stories.db)

SQLite database containing: - user_id: Telegram user ID of the story creator - story_id: Unique story identifier - timestamp: When the story was posted (UTC+2) - filename: Local filename of the downloaded media

CSV and Excel Export (stories_export.csv/xlsx)

Export file containing the same information as the database, useful for: - Easy viewing of story metadata - Filtering and sorting - Data analysis - Sharing data with others

Media Storage 📁

• Photos are saved as: {user_id}_{story_id}.jpg
• Videos are saved with their original extension: {user_id}_{story_id}.{extension}
• All media files are saved in the script's directory

Features in Detail 🔍

Continuous Monitoring

• Customizable checking interval (default: 60 seconds)
• Runs continuously until manually stopped
• Maintains state between runs
• Avoids duplicate downloads

Media Handling

• Supports both photos and videos
• Automatically detects media type
• Preserves original quality
• Generates unique filenames

Error Handling 🛠️

The script includes: - Automatic retry mechanism for failed downloads - Error logging for failed operations - Connection error handling - State preservation in case of interruption

Limitations ⚠️

• Subject to Telegram's rate limits
• Stories must be currently active (not expired)
• Media download size limits apply as per Telegram's restrictions

Contributing 🤝

Contributions are welcome! Please feel free to submit a Pull Request.

License 📄

This project is licensed under the MIT License - see the LICENSE file for details.

Disclaimer ⚖️

This tool is for educational purposes only. Make sure to: - Respect Telegram's Terms of Service - Obtain necessary permissions before scraping - Use responsibly and ethically - Comply with data protection regulations - Respect user privacy

KitPloit - PenTest Tools!

Snoop - OSINT Tool For Research Social Media Accounts By Username

OSINT Tool for research social media accounts by username

Install Requests

```Install Requests pip install requests

#### Install BeautifulSoup
```Install BeautifulSoup
pip install beautifulsoup4

Execute the program

Execute Snoop python3 snoop.py

KitPloit - PenTest Tools!

Lazywarden - Automatic Bitwarden Backup

Secure, Automated, and Multi-Cloud Bitwarden Backup and Import System

Lazywarden is a Python automation tool designed to Backup and Restore data from your vault, including Bitwarden attachments. It allows you to upload backups to multiple cloud storage services and receive notifications across multiple platforms. It also offers AES encrypted backups and uses key derivation with Argon2, ensuring maximum security for your data.

Features

• 🔒 Maximum Security: Data protection with AES-256 encryption and Argon2 key derivation.
• 🔄 Automated Backups and Imports: Keep your Bitwarden vault up to date and secure.
• ✅ Integrity Verification: SHA-256 hash to ensure data integrity on every backup.
• ☁️ Multi-Cloud Support: Store backups to services such as Dropbox, Google Drive, pCloud, MEGA, NextCloud, Seafile, Storj, Cloudflare R2, Backblaze B2, Filebase (IPFS) and via SMTP.
• 🖥️ Local Storage: Save backups to a local path for greater control.
• 🔔 Real-Time Alerts: Instant notifications on Discord, Telegram, Ntfy and Slack.
• 🗓️ Schedule Management: Integration with CalDAV, Todoist and Vikunja to manage your schedule.
• 🐳 Easy Deployment: Quick setup with Docker Compose.
• 🤖 Full Automation and Custom Scheduling: Automatic backups with flexible scheduling options (daily, weekly, monthly, yearly). Integration with CalDAV, Todoist and Vikunja for complete tracking and email notifications.
• 🔑 Bitwarden Export to KeePass: Export Bitwarden items to a KeePass database (kdbx), including TOTP-seeded logins, URI, custom fields, card, identity attachments and secure notes.

Platform Compatibility  

KitPloit - PenTest Tools!

Docf-Sec-Check - DockF-Sec-Check Helps To Make Your Dockerfile Commands More Secure

DockF-Sec-Check helps to make your Dockerfile commands more secure.

Done

• [x] First-level security notification in the Dockerfile

TODO List

• [ ] Correctly detect the Dockerfile.
• [ ] Second-level security notification in the Dockerfile.
• [ ] Security notification in Docker images.
• [ ] ***** (Private Repository)

Installation

From Source Code

You can use virtualenv for package dependencies before installation.

git clone https://github.com/OsmanKandemir/docf-sec-check.git
cd docf-sec-check
python setup.py build
python setup.py install

From Pypi

The application is available on PyPI. To install with pip:

pip install docfseccheck

From Dockerfile

You can run this application on a container after build a Dockerfile. You need to specify a path (YOUR-LOCAL-PATH) to scan the Dockerfile in your local.

docker build -t docfseccheck .
docker run -v <YOUR-LOCAL-PATH>/Dockerfile:/docf-sec-check/Dockerfile docfseccheck -f /docf-sec-check/Dockerfile

From DockerHub

docker pull osmankandemir/docfseccheck:v1.0
docker run -v <YOUR-LOCAL-PATH>/Dockerfile:/docf-sec-check/Dockerfile osmankandemir/docfseccheck:v1.0 -f /docf-sec-check/Dockerfile

Usage

-f DOCKERFILE [DOCKERFILE], --file DOCKERFILE [DOCKERFILE] Dockerfile path. --file Dockerfile

Function Usage

from docfchecker import DocFChecker

#Dockerfile is your file PATH.

DocFChecker(["Dockerfile"]) 

Development and Contribution

See; CONTRIBUTING.md

License

Copyright (c) 2024 Osman Kandemir \ Licensed under the GPL-3.0 License.

Donations

If you like DocF-Sec-Check and would like to show support, you can use Buy A Coffee or Github Sponsors feature for the developer using the button below.

Or

Sponsor me : https://github.com/sponsors/OsmanKandemir 😊

Your support will be much appreciated😊

KitPloit - PenTest Tools!

SafeLine - Serve As A Reverse Proxy To Protect Your Web Services From Attacks And Exploits

SafeLine is a self-hosted WAF(Web Application Firewall) to protect your web apps from attacks and exploits.

A web application firewall helps protect web apps by filtering and monitoring HTTP traffic between a web application and the Internet. It typically protects web apps from attacks such as SQL injection, XSS, code injection, os command injection, CRLF injection, ldap injection, xpath injection, RCE, XXE, SSRF, path traversal, backdoor, bruteforce, http-flood, bot abused, among others.

How It Works

By deploying a WAF in front of a web application, a shield is placed between the web application and the Internet. While a proxy server protects a client machine's identity by using an intermediary, a WAF is a type of reverse-proxy, protecting the server from exposure by having clients pass through the WAF before reaching the server.

A WAF protects your web apps by filtering, monitoring, and blocking any malicious HTTP/S traffic traveling to the web application, and prevents any unauthorized data from leaving the app. It does this by adhering to a set of policies that help determine what traffic is malicious and what traffic is safe. Just as a proxy server acts as an intermediary to protect the identity of a client, a WAF operates in similar fashion but acting as an reverse proxy intermediary that protects the web app server from a potentially malicious client.

its core capabilities include:

• Defenses for web attacks
• Proactive bot abused defense
• HTML & JS code encryption
• IP-based rate limiting
• Web Access Control List

Screenshots

Get Live Demo

FEATURES

List of the main features as follows:

• Block Web Attacks
• It defenses for all of web attacks, such as SQL injection, XSS, code injection, os command injection, CRLF injection, XXE, SSRF, path traversal and so on.
• Rate Limiting
• Defend your web apps against DoS attacks, bruteforce attempts, traffic surges, and other types of abuse by throttling traffic that exceeds defined limits.
• Anti-Bot Challenge
• Anti-Bot challenges to protect your website from bot attacks, humen users will be allowed, crawlers and bots will be blocked.
• Authentication Challenge
• When authentication challenge turned on, visitors need to enter the password, otherwise they will be blocked.
• Dynamic Protection
• When dynamic protection turned on, html and js codes in your web server will be dynamically encrypted by each time you visit.

KitPloit - PenTest Tools!

PolyDrop - A BYOSI (Bring-Your-Own-Script-Interpreter) Rapid Payload Deployment Toolkit

BYOSI

- Bring-Your-Own-Script-Interpreter

- Leveraging the abuse of trusted applications, one is able to deliver a compatible script interpreter for a Windows, Mac, or Linux system as well as malicious source code in the form of the specific script interpreter of choice. Once both the malicious source code and the trusted script interpeter are safely written to the target system, one could simply execute said source code via the trusted script interpreter.

PolyDrop

- Leverages thirteen scripting languages to perform the above attack.

The following langues are wholly ignored by AV vendors including MS-Defender: - tcl - php - crystal - julia - golang - dart - dlang - vlang - nodejs - bun - python - fsharp - deno

All of these languages were allowed to completely execute, and establish a reverse shell by MS-Defender. We assume the list is even longer, given that languages such as PHP are considered "dead" languages.

- Currently undetectable by most mainstream Endpoint-Detection & Response vendors.

The total number of vendors that are unable to scan or process just PHP file types is 14, and they are listed below:

• Alibaba
• Avast-Mobile
• BitDefenderFalx
• Cylance
• DeepInstinct
• Elastic
• McAfee Scanner
• Palo Alto Networks
• SecureAge
• SentinelOne (Static ML)
• Symantec Mobile Insight
• Trapmine
• Trustlook
• Webroot

And the total number of vendors that are unable to accurately identify malicious PHP scripts is 54, and they are listed below:

• Acronis (Static ML)
• AhnLab-V3
• ALYac
• Antiy-AVL
• Arcabit
• Avira (no cloud)
• Baidu
• BitDefender
• BitDefenderTheta
• ClamAV
• CMC
• CrowdStrike Falcon
• Cybereason
• Cynet
• DrWeb
• Emsisoft
• eScan
• ESET-NOD32
• Fortinet
• GData
• Gridinsoft (no cloud)
• Jiangmin
• K7AntiVirus
• K7GW
• Kaspersky
• Lionic
• Malwarebytes
• MAX
• MaxSecure
• NANO-Antivirus
• Panda
• QuickHeal
• Sangfor Engine Zero
• Skyhigh (SWG)
• Sophos
• SUPERAntiSpyware
• Symantec
• TACHYON
• TEHTRIS
• Tencent
• Trellix (ENS)
• Trellix (HX)
• TrendMicro
• TrendMicro-HouseCall
• Varist
• VBA32
• VIPRE
• VirIT
• ViRobot
• WithSecure
• Xcitium
• Yandex
• Zillya
• ZoneAlarm by Check Point
• Zoner

With this in mind, and the absolute shortcomings on identifying PHP based malware we came up with the theory that the 13 identified languages are also an oversight by these vendors, including CrowdStrike, Sentinel1, Palo Alto, Fortinet, etc. We have been able to identify that at the very least Defender considers these obviously malicious payloads as plaintext.

Disclaimer

We as the maintainers, are in no way responsible for the misuse or abuse of this product. This was published for legitimate penetration testing/red teaming purposes, and for educational value. Know the applicable laws in your country of residence before using this script, and do not break the law whilst using this. Thank you and have a nice day.

EDIT

In case you are seeing all of the default declarations, and wondering wtf guys. There is a reason; this was built to be more moduler for later versions. For now, enjoy the tool and feel free to post issues. They'll be addressed as quickly as possible.

KitPloit - PenTest Tools!

Secator - The Pentester'S Swiss Knife

secator is a task and workflow runner used for security assessments. It supports dozens of well-known security tools and it is designed to improve productivity for pentesters and security researchers.

Features

• Curated list of commands

• Unified input options

• Unified output schema

• CLI and library usage

• Distributed options with Celery

• Complexity from simple tasks to complex workflows

• Customizable

Supported tools

secator integrates the following tools:

Feel free to request new tools to be added by opening an issue, but please check that the tool complies with our selection criterias before doing so. If it doesn't but you still want to integrate it into secator, you can plug it in (see the dev guide).

Installation

Installing secator

pipx install secator

pip install secator

wget -O - https://raw.githubusercontent.com/freelabz/secator/main/scripts/install.sh | sh

docker run -it --rm --net=host -v ~/.secator:/root/.secator freelabz/secator --help

alias secator="docker run -it --rm --net=host -v ~/.secator:/root/.secator freelabz/secator"

secator --help

git clone https://github.com/freelabz/secator
cd secator
docker-compose up -d
docker-compose exec secator secator --help

Note: If you chose the Bash, Docker or Docker Compose installation methods, you can skip the next sections and go straight to Usage.

Installing languages

secator uses external tools, so you might need to install languages used by those tools assuming they are not already installed on your system.

We provide utilities to install required languages if you don't manage them externally:

secator install langs go

secator install langs ruby

Installing tools

secator does not install any of the external tools it supports by default.

We provide utilities to install or update each supported tool which should work on all systems supporting apt:

secator install tools

secator install tools <TOOL_NAME>

secator install tools httpx

Please make sure you are using the latest available versions for each tool before you run secator or you might run into parsing / formatting issues.

Installing addons

secator comes installed with the minimum amount of dependencies.

There are several addons available for secator:

secator install addons worker

secator install addons google

secator install addons mongodb

secator install addons redis

secator install addons dev

secator install addons trace

secator install addons build

Install CVEs

secator makes remote API calls to https://cve.circl.lu/ to get in-depth information about the CVEs it encounters. We provide a subcommand to download all known CVEs locally so that future lookups are made from disk instead:

secator install cves

Checking installation health

To figure out which languages or tools are installed on your system (along with their version):

secator health

Usage

secator --help

Usage examples

Run a fuzzing task (ffuf):

secator x ffuf http://testphp.vulnweb.com/FUZZ

Run a url crawl workflow:

secator w url_crawl http://testphp.vulnweb.com

Run a host scan:

secator s host mydomain.com

and more... to list all tasks / workflows / scans that you can use:

secator x --help
secator w --help
secator s --help

Learn more

To go deeper with secator, check out: * Our complete documentation * Our getting started tutorial video * Our Medium post * Follow us on social media: @freelabz on Twitter and @FreeLabz on YouTube

KitPloit - PenTest Tools!

Damn-Vulnerable-Drone - An Intentionally Vulnerable Drone Hacking Simulator Based On The Popular ArduPilot/MAVLink Architecture, Providing A Realistic Environment For Hands-On Drone Hacking

The Damn Vulnerable Drone is an intentionally vulnerable drone hacking simulator based on the popular ArduPilot/MAVLink architecture, providing a realistic environment for hands-on drone hacking.

About the Damn Vulnerable Drone

What is the Damn Vulnerable Drone?

The Damn Vulnerable Drone is a virtually simulated environment designed for offensive security professionals to safely learn and practice drone hacking techniques. It simulates real-world ArduPilot & MAVLink drone architectures and vulnerabilities, offering a hands-on experience in exploiting drone systems.

Why was it built?

The Damn Vulnerable Drone aims to enhance offensive security skills within a controlled environment, making it an invaluable tool for intermediate-level security professionals, pentesters, and hacking enthusiasts.

Similar to how pilots utilize flight simulators for training, we can use the Damn Vulnerable Drone simulator to gain in-depth knowledge of real-world drone systems, understand their vulnerabilities, and learn effective methods to exploit them.

The Damn Vulnerable Drone platform is open-source and available at no cost and was specifically designed to address the substantial expenses often linked with drone hardware, hacking tools, and maintenance. Its cost-free nature allows users to immerse themselves in drone hacking without financial concerns. This accessibility makes the Damn Vulnerable Drone a crucial resource for those in the fields of information security and penetration testing, promoting the development of offensive cybersecurity skills in a safe environment.

How does it work?

The Damn Vulnerable Drone platform operates on the principle of Software-in-the-Loop (SITL), a simulation technique that allows users to run drone software as if it were executing on an actual drone, thereby replicating authentic drone behaviors and responses.

ArduPilot's SITL allows for the execution of the drone's firmware within a virtual environment, mimicking the behavior of a real drone without the need for physical hardware. This simulation is further enhanced with Gazebo, a dynamic 3D robotics simulator, which provides a realistic environment and physics engine for the drone to interact with. Together, ArduPilot's SITL and Gazebo lay the foundation for a sophisticated and authentic drone simulation experience.

While the current Damn Vulnerable Drone setup doesn't mirror every drone architecture or configuration, the integrated tactics, techniques and scenarios are broadly applicable across various drone systems, models and communication protocols.

Features

• Docker-based Environment: Runs in a completely virtualized docker-based setup, making it accessible and safe for drone hacking experimentation.
• Simulated Wireless Networking: Simulated Wifi (802.11) interfaces to practice wireless drone attacks.
• Onboard Camera Streaming & Gimbal: Simulated RTSP drone onboard camera stream with gimbal and companion computer integration.
• Companion Computer Web Interface: Companion Computer configuration management via web interface and simulated serial connection to Flight Controller.
• QGroundControl/MAVProxy Integration: One-click QGroundControl UI launching (only supported on x86 architecture) with MAVProxy GCS integration.
• MAVLink Router Integration: Telemetry forwarding via MAVLink Router on the Companion Computer Web Interface.
• Dynamic Flight Logging: Fully dynamic Ardupilot flight bin logs stored on a simulated SD Card.
• Management Web Console: Simple to use simulator management web console used to trigger scenarios and drone flight states.
• Comprehensive Hacking Scenarios: Ideal for practicing a wide range of drone hacking techniques, from basic reconnaissance to advanced exploitation.
• Detailed Walkthroughs: If you need help hacking against a particular scenario you can leverage the detailed walkthrough documentation as a spoiler.

KitPloit - PenTest Tools!

File-Unpumper - Tool That Can Be Used To Trim Useless Things From A PE File Such As The Things A File Pumper Would Add

file-unpumper is a powerful command-line utility designed to clean and analyze Portable Executable (PE) files. It provides a range of features to help developers and security professionals work with PE files more effectively.

Features

• PE Header Fixing: file-unpumper can fix and align the PE headers of a given executable file. This is particularly useful for resolving issues caused by packers or obfuscators that modify the headers.

• Resource Extraction: The tool can extract embedded resources from a PE file, such as icons, bitmaps, or other data resources. This can be helpful for reverse engineering or analyzing the contents of an executable.

• Metadata Analysis: file-unpumper provides a comprehensive analysis of the PE file's metadata, including information about the machine architecture, number of sections, timestamp, subsystem, image base, and section details.

• File Cleaning: The core functionality of file-unpumper is to remove any "pumped" or padded data from a PE file, resulting in a cleaned version of the executable. This can aid in malware analysis, reverse engineering, or simply reducing the file size.

• Parallel Processing: To ensure efficient performance, file-unpumper leverages the power of parallel processing using the rayon crate, allowing it to handle large files with ease.

• Progress Tracking: During the file cleaning process, a progress bar is displayed, providing a visual indication of the operation's progress and estimated time remaining.

Installation

file-unpumper is written in Rust and can be easily installed using the Cargo package manager:

cargo install file-unpumper

Usage

• <INPUT>: The path to the input PE file.

Options

• --fix-headers: Fix and align the PE headers of the input file.
• --extract-resources: Extract embedded resources from the input file.
• --analyze-metadata: Analyze and display the PE file's metadata.
• -h, --help: Print help information.
• -V, --version: Print version information.

Examples

1. Clean a PE file and remove any "pumped" data:

bash file-unpumper path/to/input.exe

1. Fix the PE headers and analyze the metadata of a file:

bash file-unpumper --fix-headers --analyze-metadata path/to/input.exe

1. Extract resources from a PE file:

bash file-unpumper --extract-resources path/to/input.exe

1. Perform all available operations on a file:

bash file-unpumper --fix-headers --extract-resources --analyze-metadata path/to/input.exe

Contributing

Contributions to file-unpumper are welcome! If you encounter any issues or have suggestions for improvements, please open an issue or submit a pull request on the GitHub repository.

Changelog

The latest changelogs can be found in CHANGELOG.md

License

file-unpumper is released under the MIT License.

KitPloit - PenTest Tools!

Imperius - Make An Linux Kernel Rootkit Visible Again

A make an LKM rootkit visible again.

This tool is part of research on LKM rootkits that will be launched.

It involves getting the memory address of a rootkit's "show_module" function, for example, and using that to call it, adding it back to lsmod, making it possible to remove an LKM rootkit.

We can obtain the function address in very simple kernels using /sys/kernel/tracing/available_filter_functions_addrs, however, it is only available from kernel 6.5x onwards.

An alternative to this is to scan the kernel memory, and later add it to lsmod again, so it can be removed.

So in summary, this LKM abuses the function of lkm rootkits that have the functionality to become visible again.

OBS: There is another trick of removing/defusing a LKM rootkit, but it will be in the research that will be launched.

KitPloit - PenTest Tools!

BYOSI - Evade EDR's The Simple Way, By Not Touching Any Of The API's They Hook

Evade EDR's the simple way, by not touching any of the API's they hook.

Theory

I've noticed that most EDRs fail to scan scripting files, treating them merely as text files. While this might be unfortunate for them, it's an opportunity for us to profit.

Flashy methods like residing in memory or thread injection are heavily monitored. Without a binary signed by a valid Certificate Authority, execution is nearly impossible.

Enter BYOSI (Bring Your Own Scripting Interpreter). Every scripting interpreter is signed by its creator, with each certificate being valid. Testing in a live environment revealed surprising results: a highly signatured PHP script from this repository not only ran on systems monitored by CrowdStrike and Trellix but also established an external connection without triggering any EDR detections. EDRs typically overlook script files, focusing instead on binaries for implant delivery. They're configured to detect high entropy or suspicious sections in binaries, not simple scripts.

This attack method capitalizes on that oversight for significant profit. The PowerShell script's steps mirror what a developer might do when first entering an environment. Remarkably, just four lines of PowerShell code completely evade EDR detection, with Defender/AMSI also blind to it. Adding to the effectiveness, GitHub serves as a trusted deployer.

What this script does

The PowerShell script achieves EDR/AV evasion through four simple steps (technically 3):

1.) It fetches the PHP archive for Windows and extracts it into a new directory named 'php' within 'C:\Temp'.
2.) The script then proceeds to acquire the implant PHP script or shell, saving it in the same 'C:\Temp\php' directory.
3.) Following this, it executes the implant or shell, utilizing the whitelisted PHP binary (which exempts the binary from most restrictions in place that would prevent the binary from running to begin with.)

With these actions completed, congratulations: you now have an active shell on a Crowdstrike-monitored system. What's particularly amusing is that, if my memory serves me correctly, Sentinel One is unable to scan PHP file types. So, feel free to let your imagination run wild.

Disclaimer.

I am in no way responsible for the misuse of this. This issue is a major blind spot in EDR protection, i am only bringing it to everyones attention.

Thanks Section

A big thanks to @im4x5yn74x for affectionately giving it the name BYOSI, and helping with the env to test in bringing this attack method to life.

Edit

It appears as though MS Defender is now flagging the PHP script as malicious, but still fully allowing the Powershell script full execution. so, modify the PHP script.

Edit

hello sentinel one :) might want to make sure that you are making links not embed.

KitPloit - PenTest Tools!

Psobf - PowerShell Obfuscator

Tool for obfuscating PowerShell scripts written in Go. The main objective of this program is to obfuscate PowerShell code to make its analysis and detection more difficult. The script offers 5 levels of obfuscation, from basic obfuscation to script fragmentation. This allows users to tailor the obfuscation level to their specific needs.

./psobf -h

    ██████╗ ███████╗ ██████╗ ██████╗ ███████╗
    ██╔══██╗██╔════╝██╔═══██╗██╔══██╗██╔════╝
    ██████╔╝███████╗██║   ██║██████╔╝█████╗
    ██╔═══╝ ╚════██║██║   ██║██╔══██╗██╔══╝
    ██║     ███████║╚██████╔╝██████╔╝██║
    ╚═╝     ╚══════╝ ╚═════╝ ╚═════╝ ╚═╝
    @TaurusOmar
    v.1.0                                               

Usage: ./obfuscator -i <inputFile> -o <outputFile> -level <1|2|3|4|5>
Options:
  -i string
        Name of the PowerShell script file.
  -level int
        Obfuscation level (1 to 5). (default 1)
  -o string
        Name of the output file for the obfuscated script. (default "obfuscated.ps1")

Obfuscation levels:
  1: Basic obfuscation by splitting the script into individual characters.
  2: Base64 encoding of the script.
  3: Alternative Base64 encoding with a different PowerShell decoding method.
  4: Compression and Base64 encoding of the script will be decoded and decompressed at runtime.
  5: Fragmentation of the script into multiple parts and reconstruction at runtime.

Features:

• Obfuscation Levels: Four levels of obfuscation, each more complex than the previous one.
  • Level 1 obfuscation by splitting the script into individual characters.
  • Level 2 Base64 encoding of the script.
  • Level 3 Alternative Base64 encoding with a different PowerShell decoding method.
  • Level 4 Compression and Base64 encoding of the script will be decoded and decompressed at runtime.
  • Level 5 Fragmentation of the script into multiple parts and reconstruction at runtime.
• Compression and Encoding: Level 4 includes script compression before encoding it in base64.
• Variable Obfuscation: A function was added to obfuscate the names of variables in the PowerShell script.
• Random String Generation: Random strings are generated for variable name obfuscation.

Install

go install github.com/TaurusOmar/psobf@latest

Example of Obfuscation Levels

The obfuscation levels are divided into 5 options. First, you need to have a PowerShell file that you want to obfuscate. Let's assume you have a file named script.ps1 with the following content:

Write-Host "Hello, World!"