Radiologists Outperform AI in Identifying Lung Diseases on Chest X-Rays

News

Radiologists Outperform AI in Identifying Lung Diseases on Chest X-Rays

Radiologists vs. Machines: Radiologists Still Outperform Artificial Intelligence in Identifying Lung Diseases on Chest X-Rays.

A recent study published in the journal “Radiology” has found that radiologists outperform artificial intelligence (AI) tools in identifying or ruling out three common lung diseases from over 2,000 chest X-rays.

According to a study published in “Radiology,” radiologists excel in accurately detecting three common lung diseases from chest X-rays compared to artificial intelligence.

While AI tools have shown sensitivity, they have generated more false positives, making them less reliable for autonomous diagnosis but valuable for second opinions.

A study published in the Radiological Society of North America (RSNA) journal “Radiology” on September 26th revealed that in a study involving over 2,000 chest X-rays, radiologists performed better than AI in accurately identifying the presence or absence of three common lung diseases.

The Role of Radiology

Dr. Louis L. Plesner, MD, PhD, Chief Researcher, Resident Radiologist, and PhD Fellow at the Herlev and Gentofte Hospital, Copenhagen, Denmark, stated, “Chest X-rays are a common diagnostic tool, but interpreting the results correctly requires extensive training and experience.”

While FDA-approved AI tools are available to assist radiologists, the clinical application of AI based on deep learning for radiology diagnosis is still in its early stages. Dr. Plesner remarked, “While more AI tools are being approved for radiology, the need for further testing of these tools in real clinical settings has yet to be satisfied. AI tools can assist radiologists in interpreting chest X-rays, but their actual diagnostic accuracy remains unclear.”

Radiologists Outperform AI in Identifying Lung Diseases on Chest X-Rays

A) Posteroanterior chest radiograph of a 71-year-old male patient who was referred for progressive dyspnea showing bilateral fibrosis (arrow)

B) Posteroanterior chest radiograph of a 31-year-old female patient who was referred for radiographic examination because of a month-old cough. Radiograph shows subtle air gap obscurity at the right heart border (arrow).

(C) Anterior chest radiograph of a 78-year-old male patient referred after central venous catheter placement shows a right skin fold (arrow).

(D) Posteroanterior chest radiograph of a 78-year-old male patient referred for exclusion of pneumothorax shows a very subtle pneumothorax (arrow) on the right apex.

(E) Posterior anteroposterior chest X-ray showing chronic rounding of the costophrenic angle (arrow) in a 72-year-old male patient who was referred for radiology for no specific reason.

(F) Anterior chest radiograph of a 76-year-old female patient referred with suspicion of congestion and/or pneumonia shows a very small left pleural effusion (arrow).

All three pleural effusions on the anterior chest radiograph can be analyzed. Liquid’s artificial intelligence tools are all missing diagnoses.

Source: Radiological Society of North America

Research Findings

Dr. Plesner and his research team compared the performance of four commercially available AI tools and 72 radiologists in interpreting 2,040 adult chest X-rays taken over two years at four hospitals in Denmark in 2020. The median age of the patient population was 72 years. Among the chest X-ray samples, 669 (32.8%) had at least one target finding.

The chest X-rays were assessed for three common findings: airway diseases (changes in chest X-ray morphology caused by conditions like pneumonia or pulmonary edema), pneumothorax (collapsed lung), and pleural effusion (fluid around the lungs).

The AI tools showed sensitivities ranging from 72% to 91% for airway diseases, 63% to 90% for pneumothorax, and 62% to 95% for pleural effusion. Dr. Plesner said, “AI tools exhibited moderately high sensitivity in detecting airway diseases, pneumothorax, and pleural effusion on chest X-rays compared to radiologists. However, they produced more false positives (predicting disease in the absence of disease) compared to radiologists, especially in cases with multiple findings and smaller targets.”

Comparison of Positive Predictive Values

For pneumothorax, the positive predictive values of AI systems – the probability that patients with positive screening actually have the disease – ranged from 56% to 86%, whereas radiologists had a predictive value of 96%.

“AI performed worst in identifying pneumothorax, with positive predictive values between 40% and 50%,” Dr. Plesner said. “In this challenging sample of elderly patients, AI predicted the presence of pneumothorax that didn’t exist in 5 to 6 out of 10 cases. You can’t rely on an AI system to work independently at this speed.”

Radiologists aim to strike a balance between their ability to detect and rule out diseases, avoiding both overlooking significant diseases and overdiagnosing. “AI systems appear to excel in detecting diseases but fall short of radiologists in determining the absence of diseases, especially in complex chest X-rays,” he said. “Excessive false-positive diagnoses can lead to unnecessary imaging, radiation exposure, and increased costs.”

Most studies tend to evaluate the ability of AI to determine the presence or absence of a single disease, which is far easier than the reality where patients often have multiple conditions. In many previous studies claiming AI superiority over radiologists, radiologists only looked at images without access to patients’ clinical histories and prior imaging studies. Researchers speculate that if the next generation of AI tools can also integrate these aspects, their performance might become stronger, but such systems do not currently exist.

“Our study suggests that in real-life scenarios with diverse patient populations, radiologists generally outperform AI,” he said. “While AI systems can effectively identify normal chest X-rays, they should not be used for autonomous diagnosis.”

Dr. Plesner noted that these AI tools could enhance radiologists’ confidence in their diagnoses through a second review of chest X-rays.