improvement over the same foundation model without fine-tuning
valid structured output, directly usable in FHIR and mCODE pipelines
to train on a single GPU: small enough to run inside a hospital's firewall
An estimated 80 percent of the clinical information that matters in oncology lives in narrative text: physician notes, pathology reports, treatment summaries. Today, certified registrars and abstractors read those notes by hand and re-enter the findings into cancer registries. The work is slow, expensive, and does not scale with cancer incidence. The pandemic made it worse, leaving longer, more fragmented patient histories just as trained abstractors became scarce nationally.
General-purpose AI foundation models, used out of the box, have not closed that gap reliably enough for clinical use. Acellus Health took a different path.
01Fine-tuning, not bigger models
Rather than using a large general-purpose model as delivered, the team spent a year building expert-adjudicated training data from real clinical notes and fine-tuning a compact, open source, four-billion-parameter model specifically for oncology. That model is the engine behind the ACE Engine™. The training corpus was deliberately balanced with negative examples, so the model learns not only what to extract but when there is nothing to extract.
02Beating the benchmark
The fine-tuned model scored 0.78 F1 across 3,548 held-out extraction tasks covering fifteen essential oncology data elements, scored against ground truth adjudicated by clinical experts. That is roughly 2.5x the same foundation model without fine-tuning, and it matched a fine-tuned model more than twice its size.
Results
- →0.78 F1 (95% CI 0.74 to 0.79) across 3,548 held-out extraction tasks, ~2.5x the same model without fine-tuning
- →Precision 0.77, recall 0.80: the model finds four of every five clinically relevant facts while keeping false positives low
- →0.85 F1 on coded clinical classifications, including cancer clinical status, procedure status, and history of metastatic disease, the elements central to registry abstraction
- →99.9% valid structured output, directly usable in FHIR and mCODE data pipelines
- →0.1% false abstention rate: the model almost never silently returns nothing when information is present
- →Trained in ~4.5 hours on a single GPU; the 4B-parameter model matched a fine-tuned model more than twice its size
03Built for hospitals, not just labs
Because the model is small, it runs inside a hospital's own firewall on modest hardware. Patient records never leave the institution, and a community cancer program does not need a national vendor contract to benefit. Retraining as clinical standards evolve is inexpensive, so the investment keeps compounding.
04What it means for cancer care
Hospitals and health systems do not have to wait for ever-larger general models to make their data usable. A focused investment in clinical fine-tuning can make cancer records computable today: for registries, clinical trial matching, quality measurement, and care coordination.