CiteBench by LawEngine
LawEngine built CiteBench—and the deterministic verification system that gets all 500 right. That is not a loophole. It is the point: legal citations should be checked against the source, not guessed.
LawEngine: 500/500. GPT-5.4-mini: 361/500 (72.2%). Same public pack. Same four-label task.
Why this exists
A lookup is easy when the key and source are already clean. Legal authority is not. A one-word amendment, a bracketed exception, a line-broken PDF, an ellipsis, or a real quotation attached to the neighboring rule can change the answer. CiteBench tests whether a system can tell the difference.
Source-grounded examples
Illinois Supreme Court Rule 703
Predecessor: “Each applicant shall make proof that he has completed such law study…” Current: “Each applicant shall make proof that the applicant has completed such law study…”
Exact words can be date-right and date-wrong when the applicable version changed.
Current wording effective Apr. 1, 2026.
Northern District of Illinois Local Rule 5.6
“No pleading, motion [except for motion to intervene], or other document shall be filed…”
Dropping the bracketed exception changes what the rule says.
Cook County Circuit Court Rule 21.07
“All such communications shall be … inadmissible as evidence … But, evidence that is otherwise admissible … does not become inadmissible … solely by reason of its use in a mediation session.”
A shortened quote must preserve the source order and meaning across confidentiality and admissibility.
Effective Aug. 1, 2013.
Seventh Circuit Rules 31 and 51
Rule 31: “When an appellee's brief is not filed on time…” Rule 51: dismissal reasons “other than failure to file a brief on time.”
U.S. Supreme Court Rule 30
Official PDF wraps include “appli- cable” and “Satur- day.”
Visual hyphenation must be resolved before matching the quoted words.
How to run it
Get the public pack and submission template.
Use the public repository with your own system.
Send predictions for aggregate scoring.
The four-label contract
The quote appears in the cited authority.
VERIFIED
The quote is not in the cited authority.
NOT_FOUND
The quote is real, but belongs to another authority.
MISATTRIBUTED
The citation cannot be tied to an authority in the benchmark library.
CITATION_UNRESOLVED
Upload and score
Download the 500-row public pack, run your system locally, and upload a CSV containing id, predicted_status. LawEngine returns aggregate accuracy, F1 scores, label counts, and a confusion matrix while the grading key remains server-side.
Methodology: CiteBench scores the public pack’s four permitted outcomes against a server-side grading key and reports aggregate results only.
Questions about the comparison?
Put the Engine to work
You're seeing the citations, headings, dates, and change information for Title 1. Create a free account to read the section text.
No credit card. Sign up with Google, Microsoft, or email.
