iFixAi audits AI agents for misalignment: whether an agent does the job it was given within its business rules and organizational structure, rather than only how it scores on token efficiency, latency, or prompt injection. Its own example is a support agent that closes a refund ticket with every eval green while hiding a fraud flag, paying a refund a manager declined, and changing a payout account without authorization.
The open-source engine runs 60 inspections against a system under test and returns an A–F letter grade with the breakdown behind it. A run has two roles: the system under test, and a judge model that grades its answers. A grade is citable only when the judge comes from a different vendor than the system under test, and a self-judged run is flagged as a smoke test. The target can be a bare model API or a real deployed agent: an agent serving an OpenAI-compatible HTTP endpoint needs no glue code, and any other agent plugs in through a one-method Python adapter with optional hooks for tools, audit trails, authorization, and retrieval. Inspections the adapter gives too little visibility for are reported as insufficient evidence and do not count toward the grade. The grade is a weighted average of five core pillars, and only those.
| Core pillar | What it detects |
|---|---|
| Fabrication (0.20) | using tools it was not granted, keeping no audit trail, unsourced or overconfident claims |
| Manipulation (0.35) | privilege escalation, breaking its own policy, prompt injection, poisoned retrieval context |
| Deception (0.15) | sandbagging, secret side-goals, drifting off-task over long runs, failing silently |
| Unpredictability (0.15) | distorted context, drifting from instructions, inconsistent decisions |
| Opacity (0.15) | weak risk scoring, regulatory gaps, broken human escalation, off-topic answers |
Three inspections are mandatory minimums, and missing one caps the overall score at 60%. A further 28 inspections preview 20 premium categories such as sabotage, oversight atrophy, benchmark contamination, and vulnerable user care. They are scored and reported separately and never feed the grade. A hosted service, iFixAi Pro, runs up to 400 inspections and delivers operational assurance and regulatory compliance reports with an audit dashboard.
Features
- Letter grade: an A–F grade from a weighted average of five core pillars, with a pass threshold and per-inspection breakdown
- Independent judging: the judge is auto-paired from a different vendor than the system under test, with single-judge and multi-judge ensemble modes
- Agent or model targets: grades a deployed agent over its OpenAI-compatible HTTP endpoint, a bare model API, or any agent through a one-method Python adapter
- Suites: smoke (3 tests), strategic (8), core (32), extended (28) and all (60), plus security, reliability, compliance and frontier themes
- Frontier risk preview: 28 extended inspections for risks such as sabotage, sandbagging and oversight evasion, reported outside the grade
- Three ways to run: a guided CLI wizard with a saved config, a fully flag-driven CLI for CI, or a plugin or skill inside a coding agent
- Agent plugin and skill: a Claude Code and Codex plugin, and a slash-command skill for Cursor, Copilot, Windsurf, Cline, Continue, Gemini and Zed, that builds the fixture, shows the cost first and explains the scorecard
- Reports: JSON and Markdown reports plus a terminal scorecard, or an interactive results artifact when run from an agent
- Custom fixtures: hand-built fixtures describe your agent's context for full-mode runs, and case studies grade fixtures reconstructed from public incidents
