LLMs Could Automate Quality Control Processes, Saving Radiology Departments Significant Time

A new study published in the Journal of the American College of Radiology suggests that large language models (LLMs) can automate quality control processes in radiology departments. The research found that LLM-based systems can streamline the processing of radiology report checks, with potential applications beyond breast imaging alone.

The study focused on identifying variability in breast ultrasound reports and used a dataset of 735 patients from 60 hospitals in China. Researchers compared the performance of an LLM to manual reviewers who converted free text reports into standardized BI-RADS-based structured reports. The results showed that the LLM outperformed human reviewers in evaluating several key characteristics, including margins and echo patterns.

The model’s accuracy persisted even when dealing with complex reports involving multiple lesions, and it improved as the level of suspicion increased. In fact, the LLM demonstrated particularly strong performance in reports containing more suspicious findings. This suggests that LLMs could be a valuable tool for automating quality control processes in radiology departments.

One key advantage of using an LLM is its ability to process complex clinical narratives with accuracy comparable to manual reviewers. The model’s extensive pretraining on medical texts and strong contextual understanding enable it to identify errors, inconsistencies, and omissions in reports. Additionally, the LLM does not suffer from fatigue like human reviewers do, ensuring stable and consistent performance.

The study also highlighted a significant workflow benefit of using an LLM. In comparison to manual reviewers who took an average of 213 minutes to complete quality control for 50 reports, the model completed the same task in just 13 minutes. This substantial time savings could be particularly valuable for radiology departments looking to streamline their workflows.

While the study’s findings are promising, researchers acknowledge that there are concerns related to data privacy and the need for frequent updates with new information. They suggest that future studies should evaluate user acceptance, infrastructure requirements, and long-term model stability in real-world settings.