All articles
engineering notes · Prototype/demo

Contract AI Works Best as First-Pass Triage, Not Legal Advice

By SyntaxLab · 2 min read

A useful contract assistant can organise clauses, quotes, dates, and review priorities—while leaving legal judgment to qualified people.

Contract AI Works Best as First-Pass Triage, Not Legal Advice

Contract review is full of detail that deserves human judgment: commercial context, negotiation leverage, local law, sector rules, and the relationship between clauses. An AI tool should not blur that boundary.

It can still be useful as first-pass triage.

Ask for evidence, not just a verdict

SyntaxLab's contract demo asks a model to review a fixed set of clause categories from a named perspective and return a short exact quote and section reference for each. It also extracts key dates and obligations, then calculates a fixed-weight health score in code.

The useful output is therefore review material: “Here is the clause, here is the quoted language, and here is why this category needs attention.” It is not a legal conclusion.

Make the score explainable

Scores can be useful for sorting documents, but only when people can see what produced them. The demo's deductions are based on visible clause categories and fixed weights. A reviewer can inspect the relevant quote rather than treating a single number as an oracle.

This pattern applies broadly: use models to extract and organise evidence; use clear policy to rank review priority; preserve the human decision for negotiation, legal advice, and acceptance of risk.

Defend against hostile document content

Documents are untrusted input. A contract might contain text intended to manipulate the model rather than describe a clause. The system prompt should explicitly treat document text as data, scope tools tightly, and avoid allowing the model to take external actions based on document instructions. OWASP identifies prompt injection as a core risk for LLM applications. OWASP Top 10 for LLM Applications

Build a review workflow around uncertainty

Useful failure states include: unreadable document, missing clause, conflicting language, low confidence, and “requires legal review.” Keep original files, extracted quotes, reviewer notes, and a versioned record of policy changes. A tool that helps a legal team find work is more credible than one that claims to replace it.

Project evidence

syntaxlab-backend/src/contracts.ts implements clause extraction, quoted evidence, perspectives, and an explainable fixed-weight score. The site labels the feature as a demo and “not legal advice” in syntaxlab/src/routes/demos.contracts.tsx. This draft makes no claim of legal accuracy, compliance, or replacement of professional counsel.