Agentic Security is a vulnerability scanner for large language models (LLMs) and agent workflows. It sends attack prompts drawn from jailbreak and fuzzing datasets to a target model endpoint and reports the failure rate per dataset module, so developers, researchers, and security teams can find weaknesses in safety mechanisms before deployment. The project describes itself as a safety scanner rather than a guarantee of protection.
The target is described as a plain-text HTTP request template in which a > placeholder is replaced with each attack vector during a scan, so any LLM API that accepts HTTP can be probed, including OpenAI-style chat completions, image endpoints that accept base64 images, and audio endpoints that accept file uploads. Scans run from a local web UI or from the command line.
A ci mode reads an agesec.toml configuration listing the target spec, a token budget, a maximum failure threshold, and the dataset modules to run, then prints a pass/fail table per module. A sample GitHub Action workflow runs these scans as an automated check.
Features
- Multimodal probes: attack vectors across text, image, and audio inputs
- Multi-step jailbreaks: iterative attack sequences, enabled by the enableMultiStepAttack setting
- Fuzzing: randomized inputs sent against any LLM to surface edge cases
- Dataset registry: prompt collections from Hugging Face datasets and local CSV files with a prompt column, listed with agentic_security ls
- Dynamic datasets: mutation modules such as rot13, base64, mirrored words, scrambled words, and noise characters generate new prompts from existing ones
- CI/CD integration: agenticsecurity init writes a default config and agenticsecurity ci enforces low, medium, and high thresholds
- Integrated tools: Garak, InspectAI, and llm-adaptive-attacks are wired in as attack sources
- Self-probe endpoint: a /v1/self-probe endpoint that mimics refusals for integration testing
- Module class: fetches, processes, and posts prompts asynchronously against external models and tools
- Roadmap items: reinforcement-learning attacker models, a larger prompt corpus, daily attack updates, and community modules
