Supplementary Material for: Computer-Aided Diagnosis Systems in Diagnosing Malignant Thyroid Nodules on Ultrasonography: A Systematic Review and Meta-Analysis

Background: Computer-aided diagnosis (CAD) systems are being applied to the ultrasonographic diagnosis of malignant thyroid nodules, but it remains controversial whether the systems add any accuracy for radiologists. Objective: To determine the accuracy of CAD systems in diagnosing malignant thyroid nodules. Methods: PubMed, EMBASE, and the Cochrane Library were searched for studies on the diagnostic performance of CAD systems. The diagnostic performance was assessed by pooled sensitivity and specificity, and their accuracy was compared with that of radiologists. The present systematic review was registered in PROSPERO (CRD42019134460). Results: Nineteen studies with 4,781 thyroid nodules were included. Both the classic machine learning- and the deep learning-based CAD system had good performance in diagnosing malignant thyroid nodules (classic machine learning: sensitivity 0.86 [95% CI 0.79–0.92], specificity 0.85 [95% CI 0.77–0.91], diagnostic odds ratio (DOR) 37.41 [95% CI 24.91–56.20]; deep learning: sensitivity 0.89 [95% CI 0.81–0.93], specificity 0.84 [95% CI 0.75–0.90], DOR 40.87 [95% CI 18.13–92.13]). The diagnostic performance of the deep learning-based CAD system was comparable to that of the radiologists (sensitivity 0.87 [95% CI 0.78–0.93] vs. 0.87 [95% CI 0.85–0.89], specificity 0.85 [95% CI 0.76–0.91] vs. 0.87 [95% CI 0.81–0.91], DOR 40.12 [95% CI 15.58–103.33] vs. DOR 44.88 [95% CI 30.71–65.57]). Conclusions: The CAD systems demonstrated good performance in diagnosing malignant thyroid nodules. However, experienced radiologists may still have an advantage over CAD systems during real-time diagnosis.