Universidad de Jaén

Description

MINERVAS proposal will carry out research in the field of signal processing techniques and AI for monitoring information from the analysis of biomedical signals in different problems. In this research project, biomedical signals are considered to be ECG and sound signals emitted by the heart and lungs, that is, heart and lung sound signals. In addition, coughing and respiratory sounds (inhalation/exhalation of air throug h the mouth) are also considered pulmonary signals. We propose to work with signals in two different scenarios: i) research and development of techniques for the estimation of cardiac and respiratory vital signs (as well as other useful biomedical parameters to improve the reliability of the estimation of vital signs) and additional biomedical information from the analysis of ECG and sound signals captured in the auscultation process; ii) research and development of techniques applied to the early detection, by means of alerts, of cardiac and pulmonary disorders based on the analysis of cough and respiratory sounds (inhalation/exhalation of air through the mouth) emitted by the subject and captured by acoustic sensors. The proposal is characterized by organizing activities in order to develop monitoring applications for the elderly and nursling by analyzing cardiopulmonary sound signals on a mobile device (embedded systems and handheld computers) in real time. Note that the proposed team has previous experience in the implementation of applications on mobile devices in real time. The applications considered in this project will be based on the use and development of innovative AI and signal processing algorithms based on time-frequency signal models, non-negative matrix factorization (NMF), non-negative matrix co-factorization (NMPCF), extraction of characteristics and intelligent classifiers, modeling using neural networks, interpretable decision systems, knowledge acquisition systems, eco-efficiency and neutral computation, etc. The use of these algorithms and techniques in practice demands highly efficient computational resources and makes the development of high performance computing (HPC) methods relevant within this project. The consortium, thanks to the group from the University of Oviedo (UOVI), will facilitate these computationally efficient solutions. In order to achieve this ambitious challenge, the collaboration of groups with different but complementary skills becomes essential. The consortium is made up of a group (UJA) with experience in signal processing and AI algorithms; while UOVI is an expert in AI, optimization, quantum computing and HPC (having worked for a long time in the development of parallel computing algorithms for signal processing applications). In addition, as part of the research and collaborating team, clinical staff specialized in cardiac and pulmonary diseases are included to advise on the research to be carried out, as well as a researcher from the Openlab division at CERN with extensive experience in artificial intelligence, HPC and quantum computing. All these skills and competencies are complementary and justify collaboration on a proposal, since these ambitious goals cannot be achieved by a single group. Therefore, the expected impact of the project will only be achieved with this collaboration.

COORDINATION MECHANISIM

Two research teams participate in this consortium in a wide variety of activities and tasks. Therefore, the proper management of such objectives and tasks becomes crucial to maximize the positive results of this project. Thus, the following coordination plan has been designed to ensure successful collaboration:

  • The project coordinator will organize meetings every two months. The meeting agenda will be agreed between the principal investigators (PI) of all subprojects. The agenda will include, at least, the following items: Approval of the minutes of the last meeting, Analysis of progress, Discussion of current issues, Planning of future actions and Updating of the expected risks.
  • The execution and development of the project will be carried out by periodically writing technical reports, which will be distributed to all researchers.
  • Face-to-face meetings are expected to take place twice a year (provided safe sanitary conditions are met), once organized and held by the members of a subproject and vice versa. All project members will be informed about the progress made.
  • In recent years, the research groups of MINERVAS project have worked together and have successfully achieved results in the context of the aforementioned projects, with the following notable results: ● UJA-UOVI: Development of efficient musical accompaniment systems for mobile devices with the publications ([Alo17a] [Alo17b] [Cab18] [Cab19] [Muñ19] [Muñ20a]), Parallel source separation system for heart and lung sounds ([Muñ20b]) and a software license (ReMAS).

Background

Heart and lung diseases are one of the main causes of death in the world and, currently, the latter have increased exponentially due to the respiratory complications that the COVID-19 virus is generating. According to the World Health Organization (WHO):

  • An estimated 17.5 million deaths from cardiovascular diseases in 2012 (30% of the total deaths registered in the world) and 23.6 million deaths from the same type of diseases have been predicted by 2030 [OMSa].
  • It has been estimated that hundreds of millions of people suffer the consequences of chronic respiratory diseases (CKD) every day, of which approximately 235 million suffer from asthma, 64 million suffer from chronic obstructive pulmonary disease (COPD) and the rest from other CKD that are often undiagnosed. Specifically, almost 400,000 deaths from asthma were registered in 2004, especially in the elderly and child population as they are the most vulnerable sectors, mainly in countries with weak health systems [OMSb]. Approximately, 7% of the world population is suffering chronic respiratory diseases from 1990 to 2017 according to data from The Lancet [GRDC] as shown in Figure 1.
Figure 1. All-age chronic respiratory disease prevalence by country in 1990 and 2017 [GRDC].

In addition to the fact that the suffering of these heart and respiratory diseases reduces the well-being and life-quality of citizens, the medical treatment that must be applied is increasing the costs of the health systems due to the multiple visits to medical centers and the large number of citizens who suffer from them. Consequently, it is interesting to investigate new techniques that allow monitoring and obtaining fast, reliable and individualized biomedical information to establish personalized medicine as a strategy for sustainability and efficiency in health systems. It is at this point where the use of the eHealth concept becomes interesting and necessary, trying to apply new techniques and tools derived from Information and Communication Technologies (ICT). The objective is to be able to attend, healthily and economically, an increasing fraction of the population using reliable, lower-cost and non-invasive techniques so that monitoring tasks are transferred step by step to the subject, which will significantly lower the costs. Thus, eHealth could be useful in the following scenarios:

● Retrieval of biomedical information related to the appearance of cardiac or respiratory risk factors [Cat17] [Ses18]:

i) estimation of heart rate (HR);

ii) detection of heart murmurs (HM);

iii) estimation of heart rate variability (HRV);

iv) estimation of respiratory rate (RR);

v) estimation of QT interval (QT, time from the start of the Q wave to the end of the T wave). addressed in a non-invasive way from the analysis of ECGs and cardiopulmonary sound signals to be applied to the continuous monitoring of the cardiac, respiratory and nervous activity of the subject. The development of strategies that combine the joint use of this information (HR, HM, HRV, RR and QT) could improve the reliability of diagnoses and minimize medical decision times.

● Retrieval of biomedical information related to the presence of alert situations applying robust, cheap, non-invasive and accessible techniques to citizens so that they can be performed individually from the analysis of lung sounds, such as coughing, wheezing, respiratory sounds and from the analysis of the limb I-lead ECG.

In the field of HR estimation, most of the published works are based on ECG signals. In fact, ECG signal is the reference method for HR monitoring [Dia19]. However, even when using a wearable to get the limb I-lead ECG, it should be carried intentionally, avoiding the possibility of monitoring the ECG continuously in real time (the use of a holter-ecg is not real time). The cardiac auscultation process, phonocardiography (PCG), performed using a stethoscope or microphone normally placed on the subject's chest, allows listening to the sounds emitted by the heart where each cardiac cycle consists of two main sounds: S1 (systole) followed by S2 (diastole) [Chen15]. In that case, the HR estimate can be measured as the distance between two S1 sounds in a similar way as occurs in the succession of two R peaks in an ECG signal [Dia19]. In general, HR estimation methods can be divided into two broad categories [Kumar06]: methods that use a reference ECG signal and methods that do not use an ECG signal but sound signals [Kumar06] [Ari08] [Papadaniil14] [Dia19]. This research project is focused on the latter methods. Thus, Kumar and others [Kumar06] used wavelet decomposition; Ari et al [Ari08] presented a low pass filtering method where the filtered energy peaks were classified; in [Papadaniil14], Ensemble Empirical Mode Decomposition (EEMD) was applied; and recently, Dia and others [Dia19] have proposed a system based on NMF from cardiac sound signals applying the source-filter model with promising results.

When using a wearable to get the limb I-lead ECG, the signal is strongly related, among others, to sudden deaths as a consequence of fatal ventricular arrhythmias due to aberrant QT intervals [El-sherif20]. Improper use of medicaments used against COVID-19 is a recent and clear example [Jankelson20]. Regarding the measurement of the QT interval, there are also prior studies like [Haixia19] that are based on the previous estimation of the QRS complex [Rahul20] and the T wave from ECG. However, given that the detection of the T wave and the location of its end represents some difficulty as it can be confused with the isoelectric line or when there are alterations in morphology [Immanuel16], its extraction requires a particular analysis and treatment. In addition, it is also necessary to estimate the corrected QT interval in order to refer to the duration of the QT interval at 60 ppm. This project considers new AI-based techniques for classification and knowledge acquisition systems for the QT interval estimation from the limb I-lead ECGs, which is the lead that can be obtained from a typical wearable device.

Although auscultation is still the most widely used method applied to a subject who goes to a health center for the first time to assess the state of his heart, this method is highly subjective because it is conditioned to the skill, experience and training of each doctor in the type of sounds he/she is hearing. The appearance of HM must be detected, these murmurs are indicators of cardiac anomalies (mainly of the heart valves) and act as interference for a correct estimation of HRV by being temporarily separated from normal heart sounds in an interval less than 30 ms, which makes it difficult to detect through sound analysis. Many works published in this area have been based on ECG signals. However, in this project we will focus on the detection of HM using sound signal analysis without using ECG information for its detection (only for evaluation if necessary) because it could be implemented in mobile devices. In the existing literature in this line of research, most systems for detecting HM are usually composed of three stages [Yuenyong11]: segmentation of heart sounds S1 and S2, extraction of characteristics and classification [Wei19]. A support vector machine (SVM)-based classifier was used to discriminate normal and abnormal heart sounds using subspace analysis [Ghaderi11]. Jusak et al [Jusak16] demonstrated that EEMD provides fewer modes to generate the extracted SCs compared to CEEMD (complete EEMD). Other authors addressed the elimination of ambient noise mixed acoustically with cardiac sound signals [Naseri13]. Other NMF-based methods have been used in biomedical signal separation tasks with promising results [Muñ20b]. Thus, Shah and others [Shah14] investigated the potential of NMF in the separation of HM. Recently, Wei and others [Wei19] developed a cardiac sound segmentation system using the standard deviation of the Short Time Fourier Transform (STFT) as a characteristic, together with an algorithm for generating HM characteristics based on clustering of systolic frames

HRV is a parameter that informs about the state of the activity of the autonomic nervous system and is used as an indicator in cardiovascular risk factors [Cat17] [Ses18]. In fact, HRV is increasingly being used to measure both the physical and mental health status of subjects: stress level, presence of chronic heart disease, speed of health recovery in critical illness, etc. Although most of the published work on HRV is based on ECG signals, lately the scientific community has focused its efforts on methods based on the analysis of sound signals [Li13] [ Kamran13] [Shah14] [Sharma14] [Yil14] [Chen15] [Sharma19] [Dia19]. Li et al [Li13] applied a cyclostationary envelope with peak detection based on energy thresholding. In [Sharma14], HRV was estimated with heart sounds applying binary search and Shannon energy where the medium frequency components were emphasized and the low intensity components were attenuated. In [Shah14], a hierarchical NMF decomposition was applied whereby HRV was determined from acoustically separated cardiac sound signals. Sharma et al. [Sharma19] proposed the Hilbert-Huang transform (HHT) to construct an instantaneous energy envelope applied to cardiac cycle segmentation using the K-means algorithm.

With respect to the RR estimation, it has been defined as one of the key parameters hours before a patient enters a critical condition, such as readmission to the intensive care unit (ICU), chronic heart failure or pneumonia as the body tries to maintain oxygen delivery to the tissues. Specifically, a RR below 6 breaths/min and a RR above 24 breaths/min more reliably predicts the mortality of a patient when compared with the information provided by HR or blood pressure [Liu19]. In fact, an RR higher than 27 breaths/minute was the most determining indicator in predicting the onset of cardiac arrest in hospital wards [Nam2016]. Although there is an accepted definition for a normal HR (60-100 beats/minute) in adults, a normal RR in an adult is set at 12-20 breaths/min [RCP15] [RCP17]. However, despite the medical importance of the RR parameter, nowadays it is still not possible to obtain it in a reliable, cheap and individualized way. According to the literature in this field, the most common methods for monitoring RR are the following [Nam2016a] [Liu19]: i) manual counting of chest wall movements (easy, non-invasive, imprecise and slow); ii) Spirometry (accurate, interference with natural breathing and difficulty in maintaining continuous monitoring); iii) Capnography (precise, easy, allows continuous monitoring, uncomfortable due to device contact); iv) Auscultation of breath sounds with a stethoscope (easy, non-invasive, imprecise). Based on research, in the last decade it has been shown that these counting methods tend to be unreliable [Hillman05] [Kellet11], with a multitude of acoustic and non-acoustic methods [Muthusamy19] appearing to estimate RR. In this project, we will focus on those methods based on the analysis of respiratory sound signals captured by microphones or stethoscopes, among which those based on the detection of the inspiration/expiration phases [Chuah04] [Kulkas09] [Palaniappan15], relative changes of the energy distribution in frequency [Nakano04], Shannon entropy [Yadollahi06] [Gomes18], adaptive filtering [Golabbakhsh05], bioacoustic analysis [Hult00] and spectral dynamic characteristics of the nasal air flow in the inspiration/expiration stages [Nam2016a].

Several studies have shown the relation between cough, dyspnea, wheezing and vital signs and the exacerbation of several chronic diseases such as asthma, Chronic Obstructive Pulmonary Disease (COPD), bronchiectasis, etc [Niimi2011] [Roversi2018] [Sumner2013] [Smith2006] [Frappé2012] [Burgel2013] [Doenges2020]. Asthma is the most common cause of cough, wheezes, and dyspnea in children and adults [Weinberger2007]. However, cough can occur with pertussis, cystic fibrosis, primary ciliary dyskinesia, airway abnormalities such as tracheomalacia and bronchomalacia, chronic purulent or suppurative bronchitis in young children, and habit-cough syndrome [Weinberger2007]. Asthma is a disease in which the airways narrow excessively in response to various stimuli in the presence of airway hyper-responsiveness and eosinophilic airway inflammation. In “classic asthma” (CA) variable airflow obstruction typically leads to symptoms such as monophonic or polyphonic wheezes [Ulukaya19], dyspnea and cough [Niimi2011]. In ‘variant asthma’ usually named ‘cough variant asthma’ (CVA) cough can be the sole presenting symptom. CVA remains one of the most typical causes of chronic cough worldwide [Niimi2011]. On the other hand, COPD is characterized by airway inflammation and progressive airflow obstruction, most commonly caused by cigarette smoking and indoor air pollution. Cough, shortness of breath and sputum production are the major symptoms of which patients complain, although the relative importance depends on the stage of disease [Smith2006]. Although prevalence rates vary across published literature, it is estimated that about 17–35% of patients with heart failure are affected by COPD [Roversi2018]. In the era of TICs applied to health (eHealth), some studies [Roversi2018][Sumner2013] highlight the importance of continuously measuring and monitoring patients’ symptoms. Grading shortness of breath may point physicians in the right direction when evaluating stable, smoking patients with heart failure, i.e. suspecting the presence of lung impairment and help uncovering COPD [Roversi2018]. In this sense, a careful medical assessment of symptoms is the first approach for making a correct diagnosis (although additional investigations such as spirometry are eventually required). Stable COPD patients cough much more by day than night and most of them seem to be sensitive to the time spent coughing, as in large surveys the presence of the symptom is very commonly reported. Objective cough counts provide new insights, and ultimately automated cough counting [You17] will provide new objective outcome measures in COPD [Smith2006]. In [Sumner2013], ambulatory objective monitoring provides novel insights into the determinants of cough in COPD, suggesting sputum production, smoking, and airway inflammation may be more important than sensitivity of the cough reflex. As chronic respiratory symptoms and the presence of expiratory flow limitation (EFL) are commonly reported in the elderly, chronic cough and the composite outcome "chronic cough or sputum" were significantly associated with the presence of EFL [Frappé2012], concluding that monitoring could be advocated in elderly subjects presenting with respiratory symptoms. Finally, some works encourage continuing monitoring symptoms to evaluate the exacerbations of asthma and COPD [Frappé2012] [Burgel2013] [Infante17] [Doenges2020]. In [Burgel2013], it is stated that objective cough frequency monitoring of COPD patients could represent an interesting novel approach for complementing current methods of cough measurement that include subjective assessment of cough via questionnaires, symptoms diaries, visual analog scale, and self-counting of cough via the push-button method. Infante et al. [Infante17] reported that the analysis of cough sounds seemed to perform just as reliable as wheeze or crackle analysis for diagnostic prediction, concluding that cough sounds have value as a rapid and simple pulmonary analysis tool. In [Doenges2020], it is stated that patients with nocturnal symptoms like wheezing and cough often suffer from lower sleep quality and impaired daytime performance. The lack of an appropriate method for standardized and objective monitoring of respiratory symptoms leads to difficulties in asthma management. They proposed a new method for automated wheeze and cough detection during sleep and to assess the actual level of asthma control by the Asthma Control Test (ACT), classifying asthma as “controlled”, “partially controlled” or “uncontrolled” within the last 4 weeks.

The main methods for detecting cough are based on Neural Networks (NN) [Barry06] [Bales20], Empirical Mode Decomposition (EMD) [Ferdousi15], Hidden Markov Models (HMM) [Matos07], Principal Component Analysis (PCA) [Abaza09a], characteristics and classifiers [Martinek08] and NMF [You17]. You and others [You17] addressed detection using NMF for pattern modeling from the properties of the spectral structure of cough.

Classification of cough sounds is helpful in identifying the underlying cause or pathology so that appropriate treatment can be offered to patients. Most of the published work in this area performs manual detection of cough sounds before automatic classification can be performed. The main methods for classifying cough are based on energy thresholding [Chatrzarrin11], characteristics and classifiers [Infante17] [Amrulloh18] and wavelet [Kosasih15]. Chatrzarrin and others [Chatrzarrin11] found that the second phase of the dry cough has less energy compared to the wet cough. They also observed that, during this phase, most of the signal energy is contained between 0-750 Hz in the case of wet cough and 1500-2250 Hz in the case of dry cough. In [Kosasih15], they developed an algorithm for the automatic diagnosis of childhood pneumonia by evaluating the sounds of coughs and crackles. In the work [Amrulloh18], sound characteristics (MFCC, non-Gaussianity index, LPC and pitch) were extracted as input to a genetic algorithm to select the optimal characteristics and NN to classify wet and dry cough. Highlight the work [Cohen20] in which a database (NoCoCoDa) made up of cough sounds from subjects confirmed with COVID-19 is presented, including the annotation of the cough phases.

Some clinical works have reported that abnormal respiratory patterns could provide detailed clues for clinical treatments related to specific respiratory diseases [Boulding16], including novel analysis of the cadence associated to respiratory patterns of the breathing [Wang20]. On the other hand, wheezing sounds have been deeply investigated by the research team of this project [Torre18] [Torre19] [Torre20a] [Torre20b] [Torre20c] reaching competitive results in sound enhancement, detection, location and classification of this specific type of adventitious sounds. All commented works show the great interest, in recent years, of the scientific community in signal processing and AI in the biomedical field, including ECGs and biomedical sounds, and their possible applications in new areas of eHealth. One example of this effort is the detection of the presence of COVID-19. However, this is not the only application born under the eHealth paradigm. In the case of asthma disease, recently the following applications have been released for its individual management: i) Wheezo ( https://respiri.co/products/ ), an innovative eHealth platform to assist asthma patients with the management of their condition, especially children. Specifically, the device and eHealth app records and analyses breath sounds for the presence of wheezes in order to assist asthma monitoring; ii) Leo-sound ( https://hul.de/en/produkt/leosound-2/ ), a special developed new respiratory diagnostic system, that is able to record and analyze cough, breath and lung sounds over a long period; iii) WheezeScan ( https://www.omron-healthcare.co.uk/nebulisers/wheezescan.html ), a tool designed to detect wheeze in children with asthma symptoms.

In recent years, quantum machine learning (QML) has been proposed as a way of hybridizing techniques from classical machine learning and quantum computing. Although scalable and fault-tolerant quantum hardware is not yet available, some researchers have shown that quantum annealers like the ones offered by D-Wave can be used to obtain advantages over some classical methods in problems relevant to MINERVAS, like NMF factorization [O’Malley18][Golden20]. Also, some experiments with QML methods suggest that these techniques can obtain good classification results with less data than comparable classical machine learning methods [Mott17][Terashi20]. Moreover, in the case of the strong classifier constructed in [Mott17], the training on the quantum annealer is conducted off-line and the resulting model can be used afterwards on classical computers. It also leads to the possibility of interpreting the results since it implicitly selects the characteristics that are more relevant for the classification. This is helpful in reducing the number of features that need to be used. It is also important to notice that the quantum classifiers proposed, for instance, in [Hav19] and [Terashi20] use a reduced number of qubits so they can be efficiently simulated with classical computers and also lead to the development of quantum-inspired algorithms that can be run on classical hardware.

The consortium presented in the MINERVAS project has already begun to research in these fields, and they have a solid base in biomedical signal processing, time-frequency modeling, detection-classification, NMF techniques, machine learning, HPC techniques, quantum computing and software optimization in real-time on mobile devices, which guarantees the achievement of the objectives. This background has been acquired in previous projects, such as “Smart Sound Processing for the Digital Living” (TEC2015-67387) detailed above, in which both groups worked together.

Objectives