Skip to content
Every project

LLM Regression Gate

A CI gate that catches quality drops on every prompt or model change, before users see them.

What it does

Prompts can regress like code, but nobody writes tests for them. This moves LLM evals into CI: an async eval runner, exact-match plus LLM-judge scoring, per-case HTML diffs, Slack alerts, and a GitHub Action that comments on the PR and gates the merge.

Architecture

PReval runexact matchLLM judgediffgate