I built an AI contract reviewer that has to cite its sources. Here's the architecture.
AI contract summaries share one flaw: you can't tell when they're wrong. The model says "this lease has an aggressive auto-renewal," and you either trust it or read the whole contract anyway, which defeats the point. So I built ClauseShift around one rule: every risk it flags must quote the exact clause it came from, verbatim. If the model can't point to the wording, it doesn't get to make the claim. The interesting part is the premium pass: two models (Gemini 3.1 Pro and DeepSeek V4) review the
AI contract summaries share one flaw: you can't tell when they're wrong. The model says "this lease has an aggressive auto-renewal," and you either trust it or read the whole contract anyway, which defeats the point.
So I built ClauseShift around one rule: every risk it flags must quote the exact clause it came from, verbatim. If the model can't point to the wording, it doesn't get to make the claim.
The cross-check architecture
The interesting part is the premium pass: two models (Gemini 3.1 Pro and DeepSeek V4) review the same contract in parallel, then a consolidation step merges their findings and computes a combined risk score. Where they disagree, the report says so instead of hiding it. Disagreement between models turns out to be signal, not noise: if only one model flags a clause, that's a "read this yourself" marker.
A few hard-won lessons:
1. Structured output needs an escape hatch. Both models return JSON memos (findings, risk enums, key dates). The JSON keys and risk levels stay English even when the report language is German or Japanese; only the prose is localized. Mixing schema and language in one instruction was the #1 source of malformed output.
2. Fallback chains fail silently. My last-resort fallback (direct Gemini Flash) was broken in production for weeks and I only found out by deliberately breaking my primary API key in a test. The bug: Gemini 2.5+ are thinking models, and the thinking tokens consumed the entire output budget, so responses came back empty. One thinkingBudget cap fixed it. Test your fallbacks by breaking things on purpose.
3. Fairness beats margins. If one of the two premium models fails mid-run, the user gets the surviving model's report, a notice that it fell back, and their credit is not charged. That policy cost me an hour to build and is the kind of thing that makes people trust a paid product.
Stack
- Web: Next.js (App Router) on Railway, one app serving the marketing site and the product split by hostname in middleware
- Mobile: Flutter, one codebase for Android (Play, Amazon, Huawei) with per-store billing behind build flags
- Data: Supabase (auth + entitlements), reports stored on-device by default with opt-in sync
- Payments: Paystack on web, native billing in each store
Privacy design
Contracts are the documents people least want in a chatbot log. So: no training on user contracts, uploaded files are not retained after the review, reports live on the device unless you opt into sync, and account deletion actually deletes.
The free tier is 3 reviews a month (no card). If you review contracts regularly, I'd genuinely like to know where it falls short: the two-model cross-check either earns your trust or it's just doubled inference cost, and I want to find out which.
