Rad AI Study Finds Radiology-Specific Models Preferred Over General LLMs

Rad AI announced in a press release the publication of peer-reviewed research in npj Digital Medicine showing that radiology-specific AI models outperform general-purpose large language models in generating radiology impressions. The study, conducted with researchers at Moffitt Cancer Center, is the first comprehensive evaluation comparing domain-specific and general models for this task.

The analysis included 200 oncologic CT reports and compared impressions written by radiologists with those generated by a radiology-focused AI trained on institutional data and a general-purpose LLM. The domain-specific system aligned more closely with human radiologists across measures such as completeness, correctness, and conciseness. Radiologists consistently favored the tailored AI model, citing clearer communication and better fit with clinical workflows.

The study also found that the domain-specific model produced high-quality impressions more quickly while maintaining concise, high-signal summaries important for patient care. General models were deprioritized due to lower usability and clarity, with differences in preference ranging from about 28% to nearly 50%. Risk-of-harm scores remained low for all outputs.

Researchers noted differences between radiologists and oncologists in evaluating the same AI-generated impressions, underscoring the need for adaptable systems that can support varied clinical preferences. The results highlight that effective AI in healthcare depends not only on accuracy but on alignment with real clinical practice.