PentAGI Dissected: 13 Agent Roles, a Monitoring Stack, and a Knowledge Graph in One Pentest Platform

1. Verdict: Is It Worth Your Time

Rating: ★★★★☆ (4 / 5)

PentAGI is the heavyweight on the autonomous-pentesting spectrum. When I covered Cairn last issue, the pitch was a research-flavored engine — zero roles, zero workflows, let the agent search its own path through the state space. PentAGI sits at the opposite extreme: 13 agent roles, Docker sandboxing, a monitoring stack, a knowledge graph, and a GraphQL API, all packed into one Docker Compose. Cairn is the lab; PentAGI is the factory.

The missing star comes down to three things. The README is 4,100 lines, most of it provider configuration — meaning the real barrier isn’t getting it running, it’s getting it tuned. With execution monitoring and intelligent planning both enabled, token cost runs 2-3x a bare run (and with smaller models, quality collapses without those two switches). And there are 71 open issues — fast iteration, but still visibly mid-polish.

Key Data (as of 2026-09-21)

Who It’s For

Forking It

Allowed? MIT — the most permissive of the four issues so far (remember Shannon’s AGPL?). Modify, embed, sell; no obligation to open-source your changes. The EULA still requires authorized-targets-only testing, and that clause stands regardless of use case.

How hard? Three tiers:

• Swap model / tweak config: ★☆☆☆☆ (one key in .env; even custom OpenAI-compatible endpoints are supported)
• Add tools / strip the monitoring stack: ★★☆☆☆ (docker-compose is layered by Graphiti / Langfuse / observability — don’t want a layer, don’t start it)
• Modify the agent core: ★★★★☆ (Go backend, 13 cooperating roles — first digest the five-tier memory and chain-summarization context management)

2. What It Is, and What It Isn’t

It is not “nmap with an LLM wrapper”. The full name is Penetration testing Artificial General Intelligence — ambitious naming, pragmatic execution: a self-hosted autonomous pentesting platform. Everything runs inside Docker sandboxes, specialized agents split the labor, and every command and output lands in PostgreSQL (with the pgvector extension) — replayable and auditable at any time.

What it isn’t, the authors spell out themselves in four refreshingly honest lines:

• Not a CALDERA-style BAS (Breach & Attack Simulation) product — no predefined attack campaigns
• Agent-authored attack scripts are “conceptual or future work”, not a shipped feature
• Flow report export covers web / clipboard / Markdown / PDF — no JSON
• If you came for adversary emulation, buy a commercial BAS; this project runs real authorized pentests

In one line: it does actual testing, not drills.

3. The Shape of the Platform: One Compose, Five Layers

1
2
3
4
5

┌─ Core ──────── React UI + Go API (REST/GraphQL) + PostgreSQL/pgvector + async task queue + multi-agent system
├─ Knowledge graph (optional) ─ Graphiti + Neo4j — agent actions and results settle into a flow-scoped knowledge base
├─ Monitoring ── OpenTelemetry + Grafana + VictoriaMetrics + Jaeger + Loki
├─ Analytics ─── Langfuse (LLM observability) + ClickHouse + Redis + MinIO
└─ Security ──── isolated scraper browser + 20+ pentest tools, all sandboxed

A few points that deserve their own paragraph:

Memory comes in five tiers. Long-term memory, working memory, episodic memory, knowledge base, and context management — with chain summarization handling the ever-growing context window instead of naive truncation. Unusually serious engineering for an autonomous-agent project.

Search is a first-class citizen. Eight external search integrations — Tavily, Firecrawl, Perplexity, Sploitus, Searxng, Google CSE, DuckDuckGo, Traversaal — plus an isolated browser scraper. An autonomous pentest agent lives or dies on current CVEs, exploits, and PoCs, and PentAGI treats that as infrastructure.

Every command is on the record. Everything lands in PostgreSQL + pgvector with Grafana on top. For enterprise security teams, “what did the agent actually do” being frame-by-frame replayable is the line between a demo and production.

4. Thirteen Roles, and the Four Gates That Supervise Them

Roles come in two tiers:

• General agents (tool-call cap 100): Assistant, Primary Agent, Pentester, Coder, Installer
• Limited agents (cap 20): Searcher, Enricher, Memorist, Generator, Reporter, Adviser, Reflector, Planner

Division of labor alone isn’t enough — the thing autonomous agents fear most is spinning in a loop burning tokens. PentAGI’s answer is four gates, from always-on to beta:

1. Tool Call Limits (always on): hard caps of 100 (general) and 20 (limited); Reflector guides a graceful finish as the limit approaches;
2. Reflector (always on): if the LLM fails to produce a tool call three times in a row, it steps in, analyzes the failure pattern, and nudges the agent toward correct tool usage or barrier tools (done / ask);
3. Execution Monitoring (Beta, off by default): a Mentor agent watches execution — identical tool repeated 5 times or 10 total calls triggers intervention, detects loops, recommends alternative strategies. Cost: 2-3x execution time and tokens; payoff: 2x result quality (measured by the authors on Qwen3.5-27B-FP8);
4. Intelligent Task Planning (Beta, off by default): before anything runs, the Planner decomposes the goal into 3-7 concrete steps, with the enricher analyzing context — scope-creep prevention by construction.

The design position is unambiguous: when the model is small, buy quality with supervision. The authors’ own words — for models under 32B, both beta switches are “essential”, not optional optimizations. Planning an air-gapped deployment on open-source models? Study those two switches first.

5. Two Postures: Automation and Assistant

Automation: give it target, scope, and expected output; it runs end-to-end and produces a report in web / clipboard / Markdown / PDF.

Assistant: interactive — watch it work, interject, redirect. You can also toggle Use Agents so it delegates subtasks to specialists — the main thread keeps chatting while Searcher chases CVEs and Coder writes verification scripts.

Every flow ships a Files tab: uploads (yours), resources (from your library), and container snapshots — 300MB per file. Uploaded files are auto-mirrored into the running container at /work/uploads/, readable by the agent with plain shell tools. The details are thought through.

The API is complete: REST + GraphQL, Bearer-token auth, client generation. Wiring it into CI for continuous security testing is an officially supported posture, and the repo ships a base_web_pentest.md starter template — repeatable assessments can be saved as reusable Templates.

6. Model Freedom: The Full Domestic Lineup, Plus an Air-Gap Guide

10+ providers: OpenAI, Anthropic, Gemini, AWS Bedrock, Ollama, DeepSeek, GLM, Kimi, Qwen, MiniMax, plus aggregators (OpenRouter, DeepInfra, Atlas Cloud) and any OpenAI-compatible custom endpoint.

What’s rare: the authors maintain an official vLLM + Qwen3.5-27B-FP8 guide — single card, closed network, fully offline autonomous pentesting. Chinese models aren’t “compatible after tinkering”; they’re first-class citizens, each with its own README section and .provider.yml example. For anyone outside the OpenAI bubble, that alone beats most competitors.

7. Boundaries and Risks

1. The token bill: monitoring + planning means 2-3x consumption; budget a long Automation run on a frontier model accordingly;
2. Configuration surface: 4,100-line README, 16KB .env.example — “running” and “tuned” are different verbs, especially with smaller models;
3. Beta is beta: monitoring and planning are labeled Beta, default-off, and results vary heavily by model;
4. No compliance drills: need BAS, see CALDERA — there are no attack campaigns here;
5. Data grows: everything lands in the database; long-term use needs a storage plan. The authors are candid — deleting a flow does not yet clean its on-disk data directory. Operators close that loop themselves.

8. Getting Started

Three commands with the official installer:

1
2
3

mkdir pentagi && cd pentagi
curl -fsSL https://raw.githubusercontent.com/vxcontrol/pentagi/main/build/installer/pentagi-installer.tar.gz -o pentagi-installer.tar.gz
tar -xzf pentagi-installer.tar.gz && ./pentagi-installer.sh

Configure at least one LLM provider in .env, then docker compose up. Log into the web UI → Flows → New Flow → pick mode, model, and describe target and scope in natural language (four things to state: target, assessment type, scope limits, expected output).

Advice for the first run: pick a narrow target with a single clear objective — the official docs recommend exactly this; the output is easier to review and the prompts easier to iterate before you scale up.

The usual disclaimer applies, and the EULA says it too: only systems you own or are explicitly authorized to test.

9. One-Sentence Verdict

Cairn answers “how smart can an agent get”; PentAGI answers “how complete can a platform get”. One is a research specimen of zero roles and zero workflows — the other takes the opposite pole and fills it completely: 13 roles, a five-layer platform, four supervision gates. Which end you pick depends on whether you want to study the search itself, or ship a client-ready report this weekend.