Skip to main content
The Deliverome ProjectSurfaceomev1.0

147 genes, five databases, three models.

SurfaceBench is a 147-protein benchmark assembled from cases where the five public surface-membership databases disagree on surface status. Each protein was manually reviewed and classified as surface, contextually surface, or non-surface. Sonnet 4.6 and Opus 4.8 — with no external tools, working from training-time knowledge plus a short NCBI context block — outperform every gold-standard database on this set. The table below shows every model × prompt-variant call so you can audit the reasoning per gene.

Models: Haiku 4.5 · Sonnet 4.6 · Opus 4.8·Click a gene to compare all 4 prompt variants; click a verdict cell to read the agent's reasoning.

147 genesTSV ships verdicts + reason codes + telemetry. Free-text reasoning per cell is on GET /v1/benchmark/matrix.

UniProt
GO
SURFY
CSPA
HPA

Reason · the curator's single ground-truth reason behind each verdict, drawn from the same closed TriageReason vocabulary the triage agent must pick from — so each model's reason is directly comparable to the truth reason. Hover a truncated label for the full text.

DB columns · UniProt · GO · SURFY · CSPA · HPA — the five gating databases that drive M1 universe membership (same set the homepage catalog shows). Each cell is one source's vote on whether the protein reaches the plasma membrane: filled dot = surface, empty ring = not on the surface in that source.

LLM columns · Three cells per row — one per model on the headline NCBI prompt variant (Haiku / Sonnet / Opus). Each is a verdict pill coloured by call; pills outlined in maroon disagree with truth (collapsing yes ≡ contextual when scoring correctness). Hover for the full (model → verdict + reason) tooltip. Click + on the row to expand the full 3 model × 4 variant grid (Opus has runs on 2 of 4 variants only); each cell in that grid carries its own + to reveal the agent's free-text reasoning for that specific call.