info@acellushealth.com
Research Story

How Acellus Health taught a small AI model to read cancer records

A year-long effort, funded through Delaware's American Rescue Plan investment with Tech Impact's Data Innovation Lab, showed that clinically supervised fine-tuning beats out-of-the-box foundation models for making EHR data usable in cancer care. The ACE Engine™, Acellus's purpose-trained extraction engine, outperformed its benchmark by two and a half times.

2.5x

improvement over the same foundation model without fine-tuning

99.9%

valid structured output, directly usable in FHIR and mCODE pipelines

4.5 hrs

to train on a single GPU: small enough to run inside a hospital's firewall

An estimated 80 percent of the clinical information that matters in oncology lives in narrative text: physician notes, pathology reports, treatment summaries. Today, certified registrars and abstractors read those notes by hand and re-enter the findings into cancer registries. The work is slow, expensive, and does not scale with cancer incidence. The pandemic made it worse, leaving longer, more fragmented patient histories just as trained abstractors became scarce nationally.

General-purpose AI foundation models, used out of the box, have not closed that gap reliably enough for clinical use. Acellus Health took a different path.

01Fine-tuning, not bigger models

Rather than using a large general-purpose model as delivered, the team spent a year building expert-adjudicated training data from real clinical notes and fine-tuning a compact, open source, four-billion-parameter model specifically for oncology. That model is the engine behind the ACE Engine™. The training corpus was deliberately balanced with negative examples, so the model learns not only what to extract but when there is nothing to extract.

02Beating the benchmark

The fine-tuned model scored 0.78 F1 across 3,548 held-out extraction tasks covering fifteen essential oncology data elements, scored against ground truth adjudicated by clinical experts. That is roughly 2.5x the same foundation model without fine-tuning, and it matched a fine-tuned model more than twice its size.

Results

  • 0.78 F1 (95% CI 0.74 to 0.79) across 3,548 held-out extraction tasks, ~2.5x the same model without fine-tuning
  • Precision 0.77, recall 0.80: the model finds four of every five clinically relevant facts while keeping false positives low
  • 0.85 F1 on coded clinical classifications, including cancer clinical status, procedure status, and history of metastatic disease, the elements central to registry abstraction
  • 99.9% valid structured output, directly usable in FHIR and mCODE data pipelines
  • 0.1% false abstention rate: the model almost never silently returns nothing when information is present
  • Trained in ~4.5 hours on a single GPU; the 4B-parameter model matched a fine-tuned model more than twice its size

03Built for hospitals, not just labs

Because the model is small, it runs inside a hospital's own firewall on modest hardware. Patient records never leave the institution, and a community cancer program does not need a national vendor contract to benefit. Retraining as clinical standards evolve is inexpensive, so the investment keeps compounding.

04What it means for cancer care

Hospitals and health systems do not have to wait for ever-larger general models to make their data usable. A focused investment in clinical fine-tuning can make cancer records computable today: for registries, clinical trial matching, quality measurement, and care coordination.

Known limits. The ACE Engine™ is not a replacement for certified abstractors and makes no medical decisions; it carries the mechanical portion of the work so clinical judgment stays with people. Residual error concentrates in known areas, numeric tumor measurements and long medication lists, which are the subject of continued work. A peer-reviewed publication of the methods is in preparation.

Work with us

Acellus Health partners with hospital systems to pilot fine-tuned clinical AI inside their own infrastructure.

Get in touch