Covid Detection using Deep Learning Methods

2

3

Detection of Covid-19 using an Infrared Fever Screening System (IFSS) based on Deep Learning Technology

V Muthu1 and S Kavitha2

1Department of Computing Technologies, SRM Institute of Science and Technology, Kattankulathur, Tamil Nadu, India

2Department of Computing Technologies, SRM Institute of Science and Technology, Kattankulathur, Tamil Nadu, India

1 [email protected], [email protected]

* V Muthu and S Kavitha

Abstract. Treatment for the new Coronavirus is both expensive and challenging, preventing infections may have a significant impact on healthcare costs and quality of life. This research presents a novel paradigm for identifying people displaying symptoms of covid-19 infection in public settings, one that makes use of Convolution Neural Network (CNN) and deep machine learning methods. Infection with the COVID-19 virus is characterized by a high temperature, or fever. Accordingly, the goal of this research is to develop a prototype for an automated system that can detect and isolate covid-19 carriers in public settings. The study's suggested prototype is novel and cutting-edge in many ways: Our system consists of three components: 1.An Infrared Fever Screening System that automatically detects thermal signals and measures temperature to check whether an individual has a fever or not; 2.HD visual/facial auto-calibration system that provides accurate and presides facial landmarks that help track people and measure their temperature at various regions; 3.A real-time sensor fusion of visual and thermal camera data that forms a single model with multiple lidars, racial recognition. The suggested prototype improves upon existing methods by allowing for more precise identification of patients with elevated temperatures even when they are wearing masks.

Keywords: Convolutional neural network, covid-19, deep learning, face detection, fever screening, ımage processing, ır sensor, thermal imaging.

1 Introduction

Since the outbreak of COVID-19 in 2019 and its spreading into a pandemic, different governments in different countries have been putting countermeasures in place, such as quarantine to curb the transmission of this disease. Since fever is a key sign and symptom of this disease, many temperature sensors and detection systems have been developed to help identify people with this disease and isolate them from the uninfected population. Most devices today use non-contact temperature sensors to detect and measure human temperature to reduce the risk of transmitting Coronavirus through contact. Nonetheless, technology is also advancing rapidly, and new and more advanced devices and systems are developed daily. These devices play a critical role in detecting and controlling the spread of the current COVID-19 pandemic.

In this article, we present an Infrared fever screening system (IFSS) that screen people with fever in real-time and places as they go about their daily business. Instead of an individual having to stop and place their hand near a non-contact temperature detector, the proposed prototype uses motion and facial detection technology to automatically track each individual who enters an airport and screen them for fever (Ghassemi et al., 2018). This allows free movement of people in an airport and reduces overcrowding at the checkpoint. It makes it easier to screen travelers and reduces the risk of transmitting the Coronavirus among travelers (Ghassemi et al., 2018). The system uses infrared thermography and thermal image processing to capture human facial images and measure their temperature using the infrared radiation emitted from different facial regions such as the forehead and the inner canthus of the eyes. Fig. 1 below shows a checkpoint where people need to manually check their temperature and another checkpoint where people can move freely in an airport checkpoint as their temperature is measured automatically.

Fig. 1. Manual temperature detection (Left) VS Automated infrared temperature detection (Right)

The proposed automated infrared fever screening system will allow staff at the airport checkpoint to measure the temperature of their client from a safe distance and eliminate the need for human intervention. At airport checkpoints, staff deal with multiple people, bringing a major concern for COVID-19 since the disease can easily spread from one person to another when in close contact (Pool, 2022). IFSS eliminates this problem by taking advantage of the security camera at the airport checkpoint to perform sensor fusion of thermal and visual data in real-time and in a free movement of travelers in and out of the airport. The system is also connected to cloud servers that collect thermal data, analytics processes, and future references. IFSS is an advanced system based on deep machine learning technology. It offers different benefits and advantages in the process of curbing the transmission of Coronavirus through air travel by:

1. Provides a novel screening solution that uses simultaneous techniques to measure the temperature of multiple travelers in real-time from a distance without human intervention.

2. Providing a novel sensor fusion technique fuses thermal and visual frames/images to detect and measure travelers' temperature at various distances, and even then, travelers have hats, sunglasses, and masks.

3. Providing a novel screening solution that is safer for travelers and airport staff at the airport checkpoint reduces the need to come into contact during the temperature screening process.

2. Background and challenges

Infrared Fever Screening System (IFSS) is one of the most advanced systems in the market today, which allows huge organizations with a high traffic flow of people in and out of the premises to measure temperature and warn when an individual shows abnormal temperature. The system is designed to identify and locate an individual with a temperature above a predetermined threshold. According to the Center for Disease Control and Prevention, an individual with a body temperature above 380 C or 100.40 F is considered to have a fever. The normal skin temperature for a human being is about 330 c or 910 F.

Although the body temperature is not equally distributed, it is possible to measure the average body temperature from different facial landmarks such as the eyes using visual and thermal imaging and processing technology. Measuring the human body temperature through facial screening thermography is one of the most researched topics today. Most research studies reveal that it is an accurate and effective way to detect human fever and diseases. Fig. 2 show shows how an individual can use this technology to detect and measure human body temperature from different facial landmarks.

Fig.2. A Thermal image with different temperatures in different facial landmarks

Nonetheless, there are several challenges that IFSS is still yet to address to achieve its capability to detect and measure human body temperature automatically and in real-time. First, the reliability of measuring the core temperature of many people simultaneously can be limited since people in a crowded place can occlude each other (Tan et al., 2004). Secondly, people can have to obstruct objects on their faces, such as masks, glasses, and caps, which can interrupt the input of thermal and visual images used to measure human body temperature.

2.1 Related work and research gap

Due to the unprecedented COVID-19 pandemic, many researchers and scientists have become very interested in finding ways to help the government, organizations, and individuals adopt and mitigate the risks of getting infected or spreading the virus. Many studies show that most of the challenges and problems related to COVID-19 are still not well addressed, and this disease is a problem that will continue to impact business and human activities now and in the future (Brzezinski et al., 2021; Grewe et al., 2021). The transportation industry, such as the airport, is most affected by these outbreaks and continuing pandemics. Most airports use temperature detectors that need human intervention to screen travelers for potential COVI-19 infection. The approach allows staff at the checkpoint to accurately detect and identify people with abnormal body temperature or fever and isolate them from other travelers to reduce the risk of them spreading COVID-19 if infected.

However, this approach has also been proven impractical due to the high traffic situation at most airports' checkpoints. Due to this problem, many researchers have proposed and recommended using contactless infrared fever screening systems integrated with visual and thermal cameras. These systems are also integrated with data collection applications that produce a dataset from which a machine can learn to detect and identify people with fever and signs of COVID-19 infection (Brzezinski et al., 2021). Traditionally, thermal-only and optical-thermal detectors can use infrared images to detect facial features such as eye canthus and measure human body temperature. Machine learning technology is also critical in providing facial algorithms that help machines learn and use statistical models to analyze and draw inferences from infrared images. Through this technology, machines are becoming more accurate in analyzing facial infrared images and predicting input data output. Katte et al. (2022) reveal that through open-source datasets, individuals can use machine learning models to train machines to detect visual images such as facial landmarks and automatically screen people for COVID-19 in public places.

Although machine learning technology has paved the way for using infrared imaging and fever screening, making it easier to identify people with COVID-19 in places with high traffic flow, its application has some major limitations. For instance, an IFSS model based on machine learning has poor transfer learning ability and limited reusability of the modules. The system relies entirely on human input to learn, so its performance cannot be guaranteed for consistent accuracy and precession, especially in a dynamic environment. In general, IFSS based on machine learning technology and requires a lot of human input and supervision to learn, which makes it impractical for real-time detection of people infected with COVID-19 in an environment with a fast traffic flow of people, such as an airport checkpoint. In contrast, the deep learning model uses artificial neural networks with multiple layers of processing data. It allows the machine to automatically adopt and develop new features for analyzing infrared images to identify individuals infected with COVID-19 by measuring human body temperature.

Kumar et al. (2022) suggest that deep learning technology is good for developing practical and affordable modalities and techniques for COVID-19 diagnosis. He also suggests that through artificial intelligence, an individual enhances image character and improves the accuracy and capability of the machine in anticipating the COVID-19 virus. He studies how different deep learning models are. He reveals that Deep learning techniques that use conventional neural networks to train and detect COVID-19 outperform traditional techniques that use other methods such as machine learning. Diaz-Escobar et al. (2021) also suggest that deep learning techniques can assist computers in analyzing lungs ultrasound imagery and enable screening tools to diagnose people for COVID-19 virus. The researchers in this study reveal that a screening system based on deep learning can detect the COVID-19 virus using ultrasound data with a significantly high level of accuracy and consistency. Akter et al. (2021) proposed an automated deep learning-based classification model based on Convolutional Neural Network (CNN) and conducted a study to test its efficiency. The result of the study shows that the proposed model produced the highest accuracy in classification and identification of the symptoms of COVID-19 from X-ray images than other existing models. All these studies show a screening system based on deep learning technology. CNN has a high potential to accurately classify and identify a symptom of COVID-19 from X-ray and CTR images/pictures. However, none of this study focuses on how a screening system based on deep learning technology and CNN can use facial thermal images to identify symptoms and detect COVID-19.

2.2 Overview of the proposed IFSS based on the Deep learning model

The proposed Infrared fever screening system in this article is designed to operate in a free-flow manner where people are not required to stop or pause as they enter the airport checkpoint for their temperature to be checked. It also does not require the staff at the check port to contact the travelers to ensure they measure their temperature. Instead, it has a motion sensor that detects humans moving in and out of the checkpoint section and a thermal sensor that automatically measures the temperature of each person at the checkpoint and provides their temperature on a screen. The major aspect of this system is that it is countless and hence increases the safety of the staff at the checkpoint. It also works in real-time and can measure the temperature of many individuals simultaneously, allowing an airport to save time at the checkpoint.

Figure 3 shows a setup of the deployment of the IFSS system at an airport checkpoint. The image on the left shows how the IFSS system can be integrated into the CCTY security system to form a 2in1 system that can measure the temperature of travelers automatically while recording the camera security footage. The camera is located at a designated point where it can detect their motion, capture their facial image and measure their temperature automatically when people enter the entrance. It also has wide fields of view, allowing the system to capture many people simultaneously, even in motion (Tan et al., 2004). After capturing and processing the visual and thermal images, the system displays the data on the screen, where an operator can evaluate and monitor each individual. Suppose the system detects an individual among the crowd has a fever or abnormal temperature. In that case, it identifies that individual through a facial recognition sensor which helps the staff monitor the screen to isolate them from the crowd for further assessment.

The proposed IFSS work similarly to the CCTV system, where people suspected of having COVID-19 are requested to step aside where the final assessment is done by a healthcare professional with proper personal protective equipment. This makes it easier to control and maintain physical contact between the checkpoint staff and travelers since the screening process is done automatically and with minimum need for close contact (Tan et al., 2004). In addition, the entire process can be controlled from a central room where the operator at the checkpoint can request an individual to step aside from the crowd using audio speakers without coming in contact with the travelers.

Fig. 3. IFSS Deployment

3. IFSS Convolution Neural Network (CNN) design

The proposed IFSS system in this study is based on CNN, which forms the entire model's primary architecture components. The system consists of multiple layers with different functions that classify the visual and thermal images as input data and then process it to provide the screening data for each individual as output (Muller et al., 2021).

3.1 Convolutional

The system extracts feature from the input images through a convolution process that involves filtering and feature mapping to learn and produce output data. It has a kernel that uses the matrix number processes over the input images to transform images into a feature map (Muller et al., 2021).

The subsequent feature maps are computed through O[m,n]=(1*F)[m,n]=∑∑F[i,j]I[m+i, n+j) as the general formula in this formula I denotes the Input Image. F denotes the kernel while the indexes (m,n) are the rows and column, respectively, and (m*n) denote the dimension or pixel of the filters (Peddinti et al., 2021). The convolution process also uses a nonlinear operation to eliminate negative pixel values based on the ReLu activation function.

Fig. 4. Schematic diagram of background subtraction

3.2 Pooling layers

Generally, a pooling layer is a subsequent layer after the convolutional layer that reduces the feature maps' dimensions, allowing the machine to automatically learn different computations in the network. The proposed IFSS system in this study uses this layer after the nonlinear ReLu activation function to summarize the features present in the region of the feature map generated by the convolution layer (Zulkifley et al., 2020). There are three types of pooling layers: max pooling, average pooling, and global pooling. Max pooling is a pooling layer and an operation in CNN that selects the maximum element from the region of the feature map in a filter. This operation allows a CNN-based system to produce a feature map that contains prominent features of the feature map from the convolutional layer. Unlike the max pooling operation, the average pooling operation provides the average feature present in a respective patch. The global pooling operation, on the other hand, reduces each operation in a CNN-based system to a single value.

3.3 Fully connected layer

The output from the pooling layer becomes the input for the full connected layer, which forms a feed-forward network and the last pooling layer of the convolutional layer. IT is a three-dimensional matrix based on an artificial neural network that performs the same mathematical operations. In the proposed IFSS system, the artificial neural network computes the g(Wx+b) calculation in each layer. In this calculation, X is the input vector with dimension [p_1,1], and W is the weight matrix with dimensions [p_l, n_l], where p_l is the value of neurons in the previous layer and n_l is the value of neuron in the current layer (Huang et al., 2019). The initial b is the bias vector with dimension [p_l, 1], and g is the activation function base on the ReLu activation function. This calculation helps the system to predict the classification of the visual and thermal images and produce based on the interconnected neurons of the feature map (Huang et al., 2019). The system also uses extensive thermal data images, which form the pre-trained network that the system learns from, and mask the features in the input patch to detect human temperature based on the thermal video frames. In general, the above-stated work provides a significant result which depicts that the proposed system in this study has a chance to provide reliable and valid results. Fig. 5 shows a schematic diagram that outlines CNN's building block.

Fig. 5. Schematic Diagram of CNN

4. Experimental setup, methodology, and result

The proposed system uses visual and thermal data as the input frames fed to the CNN for feature mapping, extraction, and poling process processes. The study also introduces a traditional handheld contact thermometer with manual measurement. It then collects the result of each system to evaluate the effectiveness of the proposed system compared to the existing system in most airports today.

The researcher in this study assumes that since the proposed system uses advanced technology, it will be more accurate and highly efficient in detecting travelers with COVID-19 (Grewe et al., 2021). For experimental purposes, the study has two datasets, PRODUCTION A and PRODUCTION B, as shown in Table 1 below. All the two datasets are in the same lab and the same environment. The same entrance is used for the screening process, and at the same time way two service desks are provided for the screening process.

Table 1: Dataset used in the experiment

Dataset	Number of travelers screened with negative results	Number of travelers Screened with Positive results	Total	Time Take for the screening process
Production A	270	20	290	12 hours
Production B	520	50	578	12 hours

One desk uses the traditional handheld contact thermometer, and the other uses the Infrared fever screening and contactless system. The total number of people screened, both those with a fever and those without a fever, is recorded as shown in Table one below. The traditional handheld contact thermometer uses the PRODUCTION A dataset, while the Infrared Fever Screening and contactless system use the PRODUCTION B dataset. Both systems use a thermal sensor for the screening process, and we assume the result of each system is accurate.

Fig. 6. Efficiency of the traditional-hand-held contact thermometer VS Infrared Screening contactless screening system

5. Discussion and Conclusion

Many research studies have been carried out and support the perspective that the proposed model can be implemented in places with a high traffic flow of people, like airport checkpoints. Previous studies' work makes significant progress, making it possible to successfully capture human facial landmarks from a dynamically changing background of the infrared image. The result of the deployed experimental system reveals that implementing an automatic infrared fever screening system in an airport checkpoint can reduce the risk of spreading COVID-19 by detecting infected travelers and isolating them from the no-infected travelers (Brzezinski et al., 2021). It also can provide a safer system for the airport checkpoint to screen travelers for COVID-19. The proposed system can detect COVID-19 using a thermal sensor more effectively than the traditional handheld thermometer with a thermal sensor.

The CNN was based on multiclass layers with subsequent feature maps and mathematical algorithms achieved through visual and thermal sensors to automatically measure human temperature and detect people infected with Coronavirus. The trained network automatically screens fever in real-time by displaying the human body temperature based on the predetermined threshold and alerting the staff at the checkpoint of individuals suspected of being infected with the Coronavirus. It combines the security system with the thermal camera, which better uses the resources at the airport checkpoint. The experiments also show that one individual can carry out both security and screening roles simultaneously using the same screen as the infrared fever screening system.

References

1] Akter, S., Shamrat, F. J. M., Chakraborty, S., Karim, A., & Azam, S. (2021). COVID-19 detection using deep learning algorithm on chest X-ray images.  Biology,  10(11), 1174.

2] Brzezinski, R. Y., Rabin, N., Lewis, N., Peled, R., Kerpel, A., Tsur, A. M., ... & Hoffer, O. (2021). Automated processing of thermal imaging to detect COVID-19.  Scientific Reports,  11(1), 1-10.

3] Diaz-Escobar, J., Ordóñez-Guillén, N. E., Villarreal-Reyes, S., Galaviz-Mosqueda, A., Kober, V., Rivera-Rodriguez, R., & Rizk, J. E. L. (2021). Deep-learning based detection of COVID-19 using lung ultrasound imagery.  Plos one,  16(8), e0255886.

4] Ghassemi, P., Pfefer, T. J., Casamento, J. P., Simpson, R., & Wang, Q. (2018). Best practices for standardized performance testing of infrared thermographs intended for fever screening.  PloS one,  13(9), e0203302.

5] Grewe, L., Choudhary, S., Gallegos, E., Jain, D. P., & Aguilera, P. (2021, May). Low-resolution infrared temperature analysis for disease situation awareness via machine learning on a mobile platform. Signal Processing, Sensor/Information Fusion, and Target Recognition XXX (Vol. 11756, pp. 287-299). SPIE.

6] Huang, C., Fan, J., Li, W., Chen, X., & Zhu, Q. (2019). Reach Reachability analysis of neural-network controlled systems. ACM Transactions on Embedded Computing Systems (TECS),  18(5s), 1-22.

7] Katte, P., Kakileti, S. T., Madhu, H. J., & Manjunath, G. (2022). Automated Thermal Screening for COVID-19 using Machine Learning.  arXiv preprint arXiv:2203.14128.

8] Kumar, M., Atalla, S., Almuraqab, N., & Moonesar, I. A. (2022). Detection of COVID-19 Using Deep Learning Techniques and Cost Effectiveness Evaluation: A Survey.  Frontiers in Artificial Intelligence,  5.

9] Müller, D., Ehlen, A., & Valeske, B. (2021). Convolutional neural networks for semantic segmentation as a tool for multiclass face analysis in thermal infrared.  Journal of nondestructive evaluation,  40(1), 1-10.

10] Peddinti, B., Shaikh, A., Bhavya, K. R., & KC, N. K. (2021). Framework for Real-Time Detection and Identification of possible patients of COVID-19 in public places. Biomedical Signal Processing and Control,  68, 102605.

11] Pool, R. (2022). Flying in the COVID-19 Era: Science-based Risk Assessments and Mitigation Strategies on the Ground and Air: Proceedings of a Workshop.

12] Tan, Y. H., Teo, C. W., Ong, E., Tan, L. B., & Soo, M. J. (2004, April). Development and deployment of infrared fever screening systems. In  Thermosense XXVI (Vol. 5405, pp. 68-78). SPIE.

13] Zulkifley, M. A., Abdani, S. R., & Zulkifley, N. H. (2020). COVID-19 screening using a lightweight convolutional neural network with generative adversarial network data augmentation.  Symmetry,  12(9), 1530.

Infected Travellers Traditional hand held contact thermometer Infrared fever screening contactless system 20 50 non-infected travellers Traditional hand hel d contact thermometer Infrared fever screening contactless system 270 520 Total number of travellers screened Traditional hand held contact thermometer Infrared fever screening contactless system 290 590

image3.jpeg

image4.jpeg

image5.jpeg

image1.jpeg

image2.jpeg