Pi Harness
Free for noncommercial use under PolyForm Noncommercial 1.0.0; commercial use needs a licence from Rico. View the source
Pi Harness takes a diff (staged changes, a branch, or any git diff command) or a set of whole files, and sends it to specialised reviewer personas on two or more LLMs at once. Each review is graded on a five-tier scale, and the run ends with a single pass/fail verdict and exit code, so a developer or a CI step gets one answer instead of eight walls of text.
The problem
A single AI reviewer is unreliable. It invents issues, misses real bugs and answers in a different format every time. No one model catches everything, and "looks good to me" in free text is not a gate anyone can trust.
How it works
The entry scripts source four small bash libraries. pi-harness-config detects which providers are configured (a LiteLLM proxy, an Anthropic key, an OpenAI key) and picks a primary and secondary model. pi-review-engine routes each review through omp (preferred: its built-in reviewer prompt plus an appended persona focus) or falls back to Pi with standalone persona prompts that enforce a SEVERITY: output format. pi-harness-common captures the input once to a temp file, enforces size limits and tracks child processes so Ctrl-C cleans up every reviewer. Each reviewer runs in its own isolated agent directory to avoid SQLite lock contention between parallel instances. pi-grades strips code blocks from each output, scans it line by line with negation handling, and assigns a grade; the worst grade sets the verdict.
Features
- Parallel multi-model, multi-persona review with a configurable cap (PI_MAX_PARALLEL, default 8) and per-reviewer timeout (PI_REVIEW_TIMEOUT, default 300 s)
- Provider auto-detection with overrides via a config file or PI_HARNESS_MODELS
- Two engines (omp, Pi) behind one interface, with automatic fallback
- Whole-file review mode that skips symlinks, files outside the working directory, secret-looking files (.env, .pem, .key, credentials) and binaries, so nothing sensitive is sent to a model API
- Input size limits (PI_MAX_INPUT_LINES, PI_MAX_INPUT_BYTES)
- POSIX character classes in every regex for macOS and Linux portability
- A summary block with grade distribution and one verdict line:
How it was built
Pi Harness began on 2026-05-04 because AI code review from one model was not a usable gate: it invented issues, missed others and returned a different shape every run.
The answer was to send the same change to several models from different providers, give each a narrow job, and reduce everything to one exit code. It was written in Bash as small sourced libraries around CLIs that already existed.
The grader only trusts explicit signals and treats silence as unsure. It filters negations and strips code blocks so example text inside a review cannot change the grade, and the verifier runs on a different model family from the one that likely wrote the code.
The first live run graded every review as partial because real model output used markers the patterns did not cover. A later cleanup fix also stopped killing the whole process group, which could take down the parent shell.
Claude wrote the code, and the harness reviewed itself through a 12-round self-review before v2.0. Rico chose to keep it at v2.0, with a second round of fixes written but not yet merged.