Open Health AI
The benchmark for health AI
Browse 329 open health LLMs and 266 datasets. Run reproducible evals. Compare clinical reasoning, medical knowledge, and RAG reliability — all at temperature 0.
- 329
- Health LLMs
- 266
- Datasets
- 7
- Reproducible evals
- 0°
- Temperature — always
The problem
Why this exists
Health AI needs transparency
Models are being deployed in clinical tools, triage systems, and patient-facing apps. Without open, reproducible benchmarks it is impossible to know which models are actually safe to use with health data.
General benchmarks miss the point
A model that scores well on MMLU can still fail at clinical reasoning, misidentify diseases, or hallucinate drug interactions. We test specifically for healthcare: facts, reasoning, evidence, and structured output.
Open data, open scores
Every eval uses a publicly licensed dataset. Every score comes from a results file checked into this repo. No proprietary data, no closed judging — just runs anyone can reproduce and verify.
RAG changes the picture
Most health deployments use retrieval-augmented generation (RAG) to ground answers in clinical documents. We test JSON reliability and claim verification — the two capabilities that break first in real RAG pipelines.
Stay in the loop
Get notified of updates
New health models, datasets, and leaderboard scores — delivered to your inbox weekly. No spam.
Get involved
Add your model in < 1 hour
Fork on GitHubAny open model that runs in Ollama can be evaluated and listed. No API keys, no cloud costs.