Skip to content

Nonparametric comparison of convolutional neural networks and transformers to classify COVID-19

Notifications You must be signed in to change notification settings

Wb-az/COVID-19-classification

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

64 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Comparison of convolutional neural networks and transformers for the classification of Images of COVID-19, Pneumonia and Healthy individuals as Observed with Computed Tomography

This work has been published on the Journal of Imaging. If you use our code and/or our manuscript please cite us as:

Ascencio-Cabral, A.; Reyes-Aldasoro, C.C. Comparison of Convolutional Neural Networks and Transformers for the Classification of Images of COVID-19, Pneumonia and Healthy Individuals as Observed with Computed Tomography. J. Imaging 2022, 8, 237. https://doi.org/10.3390/jimaging8090237

Introduction

Medical classification has been widely benefited with recent developments in computer vision, especially the use of deep artificial neural networks. In this work we evaluated the performance of five deep learning architectures to classify COVID-19 in a multi-class setup.

We built an experimental setup with the following deep neural network architectures:

  • CaiT-24-XXS-224 (Class Attention in image Transformers)
  • DenseNet-121
  • MobileNet-v3-large
  • ResNet-50
  • ResNet-50r (ResNet-50 with resized kernel in the first convolution layer Conv1 from 7 X 7 to 5 x 5)

Experimental setup

Our experimental setup consisted of the combination of the five architectures, two loss functions and two optimizers. We minimize the Cross entropy (CE) and weighted Cross Entropy (wCE) loss functions with Adam and AdamW optimizers.

Exp Architecture Loss Optimzer
1 CaiT CE Adam
2 CaiT CE AdamW
3 CaiT wCE Adam
4 CaiT wCE AdamW
5 DenseNet-121 CE Adam
6 DenseNet-121 CE AdamW
7 DenseNet-121 wCE Adam
8 DenseNet-121 wCE AdamW
9 MobileNet-v3-l CE Adam
10 MobileNet-v3-l CE AdamW
11 MobileNet-v3-l wCE Adam
12 MobileNet-v3-l wCE AdamW
13 ResNet-50 CE Adam
14 ResNet-50 CE AdamW
15 ResNet-50 wCE Adam
16 ResNet-50 wCE AdamW
17 ResNet-50r CE Adam
18 ResNet-50r CE AdamW
19 ResNet-50r wCE Adam
20 ResNet-50r wCE AdamW

Datasets

We sourced our CT images from two public sources:

Methods

We trained and validated the models for 8 epochs and recorded their accuracy and loss during all process.

Evaluation

We evaluated the performance of each of the experiments by using Accuracy, Balanced Accuracy (BA), F1, F2, Mathew's correlation coefficient (MCC), Sensitivity and Specificity metrics on the test dataset.

Non parametric comparison

We bootstrapped the results, compute the confidence intervals, ranked the bootstrapped results and compared the performance of the models with the Friedman-Nemenyi test.

Architecture Accuracy BA F1 F2 MCC Sen Spec
Cait 5.00 5.00 5.00 5.00 5.00 5.00 5.00
DenseNet-121 3.20 2.98 2.82 3.18 3.05 2.92 3.30
MobileNet-v3-l 3.63 3.62 3.58 3.50 3.72 3.54 3.56
ResNet-50 1.40 1.38 1.92 1.84 1.53 1.75 1.51
ResNet-50r 1.77 2.02 1.68 1.47 1.71 1.80 1.63

Results

Our results showed that models based ResNet-50 are not only more accurate but learn faster than the other networks. Our best models Exp-18 and Exp-20 achieved a MCC of 98.66% and specificity of 99.53% and 99.50% respectively.

  • By network

  • By experiment

Installation

  • Setup a Python 3.8 environment or higher
  • Install Pytorch and Torchvision
  • Install torchmetrics
  • Install timm

Note:

  • When running the files replace the paths/directories with your paths to files
  • The csv file contains results for the comparison
  • The classification of COVID-19 can be run directly from the jupyter notebook main_class.ipynb or on the terminal by using the command line python3 /your/path/to/main_class.py
  • The comparison of the models performance can be run on the jupyter notebook main_stats.ipynb or on the terminal by using the command line python3 /your/path/to/main_stats.py