Summary Background Deep learning offers considerable promise for medical diagnostics. We aimed to evaluate the diagnostic accuracy of deep learning algorithms versus health-care professionals in classifying diseases using medical imaging. Methods In this systematic review and meta-analysis, we searched Ovid-MEDLINE, Embase, Science Citation Index, and Conference Proceedings Citation Index for studies published from Jan 1, 2012, to June 6, 2019. Studies comparing the diagnostic performance of deep learning models and health-care professionals based on medical imaging, for any disease, were included. We excluded studies that used medical waveform data graphics material or investigated the accuracy of image segmentation rather than disease classification. We extracted binary diagnostic accuracy data and constructed contingency tables to derive the outcomes of interest: sensitivity and specificity. Studies undertaking an out-of-sample external validation were included in a meta-analysis, using a unified hierarchical model. This study is registered with PROSPERO, CRD42018091176. Findings Our search identified 31 587 studies, of which 82 (describing 147 patient cohorts) were included. 69 studies provided enough data to construct contingency tables, enabling calculation of test accuracy, with sensitivity ranging from 9·7% to 100·0% (mean 79·1%, SD 0·2) and specificity ranging from 38·9% to 100·0% (mean 88·3%, SD 0·1). An out-of-sample external validation was done in 25 studies, of which 14 made the comparison between deep learning models and health-care professionals in the same sample. Comparison of the performance between health-care professionals in these 14 studies, when restricting the analysis to the contingency table for each study reporting the highest accuracy, found a pooled sensitivity of 87·0% (95% CI 83·0–90·2) for deep learning models and 86·4% (79·9–91·0) for health-care professionals, and a pooled specificity of 92·5% (95% CI 85·1–96·4) for deep learning models and 90·5% (80·6–95·7) for health-care professionals. Interpretation Our review found the diagnostic performance of deep learning models to be equivalent to that of health-care professionals. However, a major finding of the review is that few studies presented externally validated results or compared the performance of deep learning models and health-care professionals using the same sample. Additionally, poor reporting is prevalent in deep learning studies, which limits reliable interpretation of the reported diagnostic accuracy. New reporting standards that address specific challenges of deep learning could improve future studies, enabling greater confidence in the results of future evaluations of this promising technology. Funding None.

A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis

Developments, application, and performance of artificial intelligence in dentistry - A systematic review.

Aim To verify the diagnostic performance of an artificial intelligence system based on the deep convolutional neural network method to detect periapical pathosis on cone-beam computed tomography (CBCT) images. Methodology images of 153 periapical lesions obtained from 109 patients were included. The specific area of the jaw and teeth associated with the periapical lesions were then determined by a human observer. Lesion volumes were calculated using the manual segmentation methods using Fujifilm-Synapse 3D software (Fujifilm Medical Systems, Tokyo, Japan). The neural network was then used to determine (i) whether the lesion could be detected; (ii) if the lesion was detected, where it was localized (maxilla, mandible or specific tooth); and (iii) lesion volume. Manual segmentation and artificial intelligence (AI) (Diagnocat Inc., San Francisco, CA, USA) methods were compared using Wilcoxon signed rank test and Bland-Altman analysis. Results The deep convolutional neural network system was successful in detecting teeth and numbering specific teeth. Only one tooth was incorrectly identified. The AI system was able to detect 142 of a total of 153 periapical lesions. The reliability of correctly detecting a periapical lesion was 92.8%. The deep convolutional neural network volumetric measurements of the lesions were similar to those with manual segmentation. There was no significant difference between the two measurement methods (P > 0.05). Conclusions Volume measurements performed by humans and by AI systems were comparable to each other. AI systems based on deep learning methods can be useful for detecting periapical pathosis on CBCT images for clinical application.

Evaluation of artificial intelligence for detecting periapical pathosis on cone‐beam computed tomography scans

The aim of this study was to evaluate the use of a convolutional neural network (CNN) system for detecting vertical root fracture (VRF) on panoramic radiography. Three hundred panoramic images containing a total of 330 VRF teeth with clearly visible fracture lines were selected from our hospital imaging database. Confirmation of VRF lines was performed by two radiologists and one endodontist. Eighty percent (240 images) of the 300 images were assigned to a training set and 20% (60 images) to a test set. A CNN-based deep learning model for the detection of VRFs was built using DetectNet with DIGITS version 5.0. To defend test data selection bias and increase reliability, fivefold cross-validation was performed. Diagnostic performance was evaluated using recall, precision, and F measure. Of the 330 VRFs, 267 were detected. Twenty teeth without fractures were falsely detected. Recall was 0.75, precision 0.93, and F measure 0.83. The CNN learning model has shown promise as a tool to detect VRFs on panoramic images and to function as a CAD tool.

Evaluation of an artificial intelligence system for detecting vertical root fracture on panoramic radiography

In this study, we used panoramic X-ray images to classify and clarify the accuracy of different dental implant brands via deep convolutional neural networks (CNNs) with transfer-learning strategies. For objective labeling, 8859 implant images of 11 implant systems were used from digital panoramic radiographs obtained from patients who underwent dental implant treatment at Kagawa Prefectural Central Hospital, Japan, between 2005 and 2019. Five deep CNN models (specifically, a basic CNN with three convolutional layers, VGG16 and VGG19 transfer-learning models, and finely tuned VGG16 and VGG19) were evaluated for implant classification. Among the five models, the finely tuned VGG16 model exhibited the highest implant classification performance. The finely tuned VGG19 was second best, followed by the normal transfer-learning VGG16. We confirmed that the finely tuned VGG16 and VGG19 CNNs could accurately classify dental implant systems from 11 types of panoramic X-ray images.

/pdf/deep-neural-networks-for-dental-implant-system-2790v2tpwn.pdf

Deep Neural Networks for Dental Implant System Classification.

Objective Although the deep learning system has been applied to interpretation of medical images, its application to the diagnosis of cervical lymph nodes in patients with oral cancer has not yet been reported. The purpose of this study was to evaluate the performance of deep learning image classification for diagnosis of lymph node metastasis. Study Design The imaging data used for evaluation consisted of computed tomography (CT) images of 127 histologically proven positive cervical lymph nodes and 314 histologically proven negative lymph nodes from 45 patients with oral squamous cell carcinoma. The performance of a deep learning image classification system for the diagnosis of lymph node metastasis on CT images was compared with the diagnostic interpretations of 2 experienced radiologists by using the Mann-Whitney U test and χ2 analysis. Results The performance of the deep learning image classification system resulted in accuracy of 78.2%, sensitivity of 75.4%, specificity of 81.0%, positive predictive value of 79.9%, negative predictive value of 77.1%, and area under the receiver operating characteristic curve of 0.80. These values were not significantly different from those found by the radiologists. Conclusions The deep learning system yielded diagnostic results similar to those of the radiologists, which suggests that this system may be valuable for diagnostic support.

Contrast-enhanced computed tomography image assessment of cervical lymph node metastasis in patients with oral cancer by using a deep learning system of artificial intelligence.

Objective The aim of this study was to investigate whether a deep learning object detection technique can automatically detect and classify radiolucent lesions in the mandible on panoramic radiographs. Study Design Panoramic radiographs of patients with mandibular radiolucent lesions of 10 mm or greater, including ameloblastomas, odontogenic keratocysts, dentigerous cysts, radicular cysts, and simple bone cysts, were included. Lesion labels, including region of interest coordinates, were created in text format. In total, 210 training images and labels were imported into the deep learning GPU training system (DIGITS). A learning model was created using the deep neural network DetectNet. Two testing data sets (testing 1 and 2) were applied to the learning model. Similarities and differences between the prediction and ground-truth images were evaluated using Intersection over Union (IoU). Sensitivity and false-positive rate per image were calculated using an IoU threshold of 0.6. The detection performance for each disease was assessed using multiclass learning. Results Sensitivity was 0.88 for both testing 1 and 2. The false-positive rate per image was 0.00 for testing 1 and 0.04 for testing 2. The best combination of detection and classification sensitivity occurred with dentigerous cysts. Conclusions Radiolucent lesions of the mandible can be detected with high sensitivity using deep learning.

Automatic detection and classification of radiolucent lesions in the mandible on panoramic radiographs using a deep learning object detection technique.

PROBLEM TO BE SOLVED: To provide an image analysis system that makes it possible to compare, as a means of identity confirmation using dental information, a dentistry panorama X-ray image of, for example, a discovered corpse with dentistry-related information about a subject whose identity is to be confirmed and thereby easily obtain information for identity confirmation, the image analysis system further having high reliability pertaining to tooth-number information for the improved accuracy of an image analysis process.SOLUTION: An image analysis system makes it possible to acquire tooth-number information by deep learning of a dentistry panorama X-ray image and easily obtain information for identity confirmation of a subject whose identity is to be confirmed, and further cut out a single tooth image from the dentistry panorama X-ray image by deep learning and, after obtaining tooth-number information by deep learning, manually correct an error in the tooth-number information.SELECTED DRAWING: Figure 1

Yudai Yanashita

Papers

Evaluation of an artificial intelligence system for detecting vertical root fracture on panoramic radiography

Contrast-enhanced computed tomography image assessment of cervical lymph node metastasis in patients with oral cancer by using a deep learning system of artificial intelligence.

Automatic detection and classification of radiolucent lesions in the mandible on panoramic radiographs using a deep learning object detection technique.

Image analysis system