Reading view
An AI security auditor that red-teams PRs to find exploits, not just patterns (open-source + Ollama support)
Hey everyone,
Iโve been working on an experiment in AI-driven application security called SentinAI. Iโm a backend engineer in fintech, and I spent part of my recent leave trying to explore a simple question:
Most SAST tools are basically metal detectors:
theyโre great at catching obvious patterns like unsafe functions or missing headers.
But they struggle with the stuff that actually matters in real systems:
- IDORs
- authorization drift
- multi-tenant isolation issues
- broken middleware assumptions
- cross-file logic flaws
Attackers donโt think in patterns.
They think in systems.
So I built something experimental to explore that gap.
๐ง The Architecture (3-Agent Loop)
Instead of a single LLM prompt (which tends to hallucinate easily), SentinAI uses a structured multi-agent flow:
1. The Architect
Maps the system:
- routes
- auth boundaries
- data flows
- trust assumptions
2. The Adversary ๐ฅท
Tries to break it:
- generates exploit paths
- builds step-by-step attack chains
- simulates real-world abuse scenarios
3. The Guardian ๐ก๏ธ
Validates everything:
- checks exploits against actual code context
- verifies whether attacks are truly possible
- filters hallucinated or low-confidence outputs
Anything below a confidence threshold (~40%) is dropped.
The goal is not to โfind everything.โ
Itโs to only surface things that are actually exploitable.
๐ก What surprised me
A few things stood out while building this:
- Most real vulnerabilities only appear at interaction points between files, not within a single file
- LLMs are surprisingly good at generating attack paths, but unreliable without a validation layer
- The hardest problem wasnโt detection โ it was noise control
- Without a โGuardianโ layer, the system becomes mostly hallucinated security reports very quickly
๐ Privacy / Local-first design
Coming from fintech, sending proprietary code to external APIs is not acceptable.
So SentinAI is built to run:
- fully local via Ollama
- or inside a private VPC
- with no code leaving the environment
๐ Web3 expansion (experimental)
I expanded it beyond Web2 into smart contract security:
- Solana: missing signer checks, PDA misuse
- EVM: reentrancy, tx.origin issues
- Move: resource lifecycle bugs
Total coverage: ~45 vulnerability patterns.
๐ง Open questions (honest part)
Iโm still actively figuring out:
- how to reduce hallucinated exploit paths at scale
- whether multi-agent reasoning actually holds up on large, messy codebases
- where the boundary is between โuseful security reasoningโ and โLLM storytellingโ
- whether this can realistically outperform hybrid static analysis + human review
One thing Iโve already noticed:
Thatโs still an open problem.
๐งช Why Iโm sharing this
This started as a โleave experimentโ and somehow got ~200+ organic npm installs without any promotion.
I cleaned it up and open-sourced it mainly to:
- get feedback from people deeper in security engineering
- understand where this approach fails in real-world systems
- see if โAI attacker reasoningโ is actually useful in practice
๐ If you want to poke at it
Curious to hear honest thoughts from people here:
- Where would this completely break in real codebases?
- Is multi-agent security reasoning actually useful, or just a fancy abstraction over static + LLM prompts?
- Has anyone tried something similar in production security pipelines?
[link] [comments]