Purpose: A deep learning-based CAD system was developed for use in the CT diagnosis of cervical LN metastasis in patients with thyroid cancer and freely available on a website (http://cdss.co.kr/). This study aimed to validate the CAD system in a large population and to evaluate its role as a training tool to help trainees.
Materials ands method: A total of 3838 axial CT images (benign: n = 3606 and malignant: n = 232) were collected from 698 patients with thyroid cancer. We validated the model's diagnostic performance using the DenseNet121 algorithm. We compared the diagnostic performance of the model with those of two trainees on test set (n=241) and evaluated the changes in the level of confidence on a scale of 1-5 in the interpretation before and after the CAD information was provided.
Results: The sensitivity, specificity, and AUROC of the CAD system were 83.6%, 81.4%, and 0.874, respectively. On test set, the sensitivity of the CAD system was not significantly different from those of the two trainees (P =0.500 and P =0.500); however, the specificity and accuracy were higher than those of the two trainees (all P <0.001). When the CAD system was used to assist the trainees, the sensitivities did not changed (97.4% vs. 97.4% and 100.0% vs. 97.4%); but both the specificity and accuracy increased (58.3% vs. 63.7%, 62.7% vs. 67.6%) (64.5% vs. 69.0%, 68.6% vs. 72.3%). Confidence level was higher with the CAD system (from 3.0 ± 0.9 to 3.8 ± 1.1 for trainee 1, from 4.3 ± 0.8 to 4.6 ± 0.7 for trainee 2).
Conclusion: A deep learning-based CAD system could accurately classify cervical LN metastasis in patients with thyroid cancer with an AUROC of 0.874. This approach may have clinical utility as a training tool to help trainees to gain confidence in diagnoses.