Using acoustic features and features fusion in infant cry diagnostic system

Abstract

Newborns communicate their needs and discomforts through crying. Throughout the years, researchers discovered that the cry emanates opulent information about the newborn’s health, needs, and emotional state. However, this information is not evident to the human ear and there is an inevitable need for the development of precise systems capable of perceiving the information embodied in the cry signal. The abstruseness of the cry signal reveals the newborns to many complications since they cannot divulge their needs to their caregivers. This may be one of the reasons behind the high newborn mortality rates worldwide. In fact, the newborns face the highest risks among all the young adolescent age groups. Therefore, the development and introduction of an automated tool that is susceptible of translating the underlying information in different levels of the cry signal could be beneficial to saving thousands of lives. The cry signal was discerned to hold peculiar characteristics that could be altered in the presence of a pathology or under the impression of an emotional state such as fear. The differences across the patterns of healthy and pathologic cry signals promoted the emerge of Newborn Cry Diagnostic Systems (NCDS) that facilitate diagnosis and distinguishing the pathologies only based on the cry signals of the newborns. Later on, it was discovered that the cries during the neonatal phase are merely due to intrinsic and independent biological rhythms and sensorimotor maturation, which means that the neonate has no control over the cry generation. This discovery led to recognition of the cry signals as powerful biomarkers in identifying pathologic newborns. This thesis aimed to propose a comprehensive NCDS that would benefit from simple yet effective methods and algorithms to yield a desirable performance. This objective was realized from two perspectives: firstly, sepsis as a leading newborn mortality root was targeted which is unprecedented in NCDS designs; and secondly, the NCDS was improved across all stages of its design by the proper utilization of novel features, classifiers, fusion, and optimization methods. The feature extraction stage was improved with the apropos combination of speech-based and music-based features that represented different levels of information. These features included low-level features of spectral centroid and crest, mid-level features of MFCC, GFCC, and BFCC, and finally, high-level features of harmonic ratio and entropy for the cepstral analysis. Subsequently, the feature space consisting of various combinations of these features was pruned against redundancy and high dimensionality with fuzzy entropy and neighborhood component analysis methods of feature selection. In order to consolidate different feature sets into one uniform feature space, the canonical correlation analysis was employed as a fusion method at feature level. The next stage of the NCDS comprises classification and fine-tuning the classifiers for each of the experiments. In this study, we employed support vector machine, K-nearest neighborhood, multilayer perceptron, and long short-term memory classification schemes to classify the cry signals based on their corresponding classes. Each of these classifiers were tuned with different hyperparameter optimization methods such as random search, grid search, and Bayesian to fit each experiment. The final stage of our proposed NCDS introduces the decision template fusion method for the fusion of decisions made by different classifiers that were trained by diverse features that capacitates the employment of features from different modalities and origins without the need for any extra measures to combine them. The performance of the proposed NCDS was assessed through different evaluation measures such as accuracy, area under curve of receiver operator characteristic (AUC-ROC), precision, recall and F-score. The main target of this study was the development of a comprehensive NCDS while revolving around the unexplored pathology of sepsis as a focal point. Accordingly, in addition to identifying septic newborns from the healthy, the NCDS was designed to distinguish between two closely entangled pathologies for the first time. Succeeding the former accomplishments, the NCDS was taken one step former to detect septic newborns from an ensemble of 32 other pathologies. Finally, a comprehensive non-intrusive and unsophisticated design was attained that can be used as an alert system in marking the newborns encountering a higher risk of being diagnosed with a critical pathology group such as sepsis.

Date	19 Dec 2023
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Chakib Tadj (Supervisor)