Currently, an abundance of data and information are available in the clinical domain. Grasping this opportunity, clinicians have been successfully combining the informative and structured data, which includes laboratory test results, medical imaging, and wearable sensor data, with novel data analytic algorithms to provide pervasive and personalized healthcare. However, considerable constraints are imposed by clinical narrative sources, which are short notes on patients originally written by doctors and physicians. Although the notes are continuously provided and plentifully stored in clinical data warehouses, they are underutilized in practice. The limitation mainly comes from their unstructured or semi-structured format. Fortunately, the deployment of machine learning algorithms in recent years helps to effectively capture the hidden representation of clinical narratives because of its high computational capacity. In particular, the improvement of machine learning performance on clinical notes is continually reinforced by employing natural language processing (NLP) techniques as a data preprocessing step in advance. NLP becomes a necessary approach to overcome the existing challenges of unstructured clinical text notes because it effectively maps the words in unstructured data into a continuously-valued lower dimensional space.
Fortunately, a large data source of clinical notes is currently stored in our Research Data Warehouse at CHU Sainte-Justine (CHUSJ) hospital. There are 7 caregiver notes/patient/day from 1386 patients (containing a dataset of more than 2.5x107 words). These notes are scribed extensively from admission notes, evaluation notes and summary notes. Admission notes outline reasons for admission to intensive care units, historical progress of disease, medication, surgery and additional baseline status of the patient. Daily ailments and laboratory test results are described in evaluation notes, from which patient condition is evaluated and diagnosed later by doctors. All these details from admission to discharge of a patient are outlined in summary notes. However, these information sources are being used as clinical documentation for reporting and billing instead of prior clinical knowledge for predicting disease condition. To prevent the loss of scientific information from these beneficial data points, a machine-learning-powered NLP method is developed to predict patient condition by using clinical notes stored at the Research Data Warehouse at CHUSJ hospital. The proposed algorithm can effectively learn a latent representation of clinical notes to draw a conclusion about a patient’s cardiac failure condition which cannot be depicted by traditional approaches.
First, our study provides important insights into using machine learning models in limited datasets. Specifically, we found that smaller and simpler models can work better in such contexts. To this end, our framework combines TF-IDF and MLP-NN, and we demonstrate that feature selection from the learning representation vector space can further improve performance. Our proposed algorithm effectively learns a latent representation of clinical notes to conclude a patient’s cardiac failure condition, which traditional approaches cannot depict. We achieved an overall classification performance with 89% accuracy, 88% recall, and 89% precision. Furthermore, we found that encoding decimal points as a string "DOT" helps retain the information from numerical values in clinical notes, which can improve model performance.
Furthermore, the thesis highlights that a critical factor for improving the performance of machine learning classifiers in clinical natural language processing is the appropriate treatment of the representation space feature. Specifically, the study demonstrates that incorporating an autoencoder (AE) during training can effectively compress the feature space of the term frequency-inverse document frequency (TF-IDF) model, making it an effective mechanism for interpretability and transparency in the CDSS system. The second step involves using an MLPNN to predict the health status based on the compressed feature space. The efficient ensemble model achieves 92% accuracy, 91% recall, 91% precision, and 91% f1-score, outperforming all alternative approaches.
Finally, while Transformer has been widely recognized as the state-of-the-art approach in natural language processing, it still faces limitations when applied to short and limited clinical NLP. We propose a simplified Switch Transformer framework that we train from scratch on a small French clinical text classification dataset at CHU Sainte-Justine hospital. Our results show that the simplified small-scale Transformer models perform better than pre-trained BERT-based models, such as DistillBERT, CamemBERT, FlauBERT, and FrALBERT. The proposed framework achieves an accuracy of 87%, precision at 87%, and recall at 85%, which outperforms the third-best pre-trained BERT-based model, FlauBERT, which achieved an accuracy of 84%, precision at 84%, and recall at 84%. However, Switch Transformers have some limitations, such as a generalization gap and sharp minima. To address these limitations, we compare it with a multi-layer perceptron neural network for small French clinical narratives classification and show that the latter outperforms all other models.
Overall, the study demonstrates the effectiveness of the proposed framework and provides valuable insights for developing NLP techniques in clinical settings. It improves the time-consuming and costly disease treatment process, health interventions, and prevention management at the Pediatric Critical Care Unit of CHUSJ hospital.
| Date | 12 Sept 2023 |
|---|
| Original language | American English |
|---|
| Awarding Institution | - École de technologie supérieure
|
|---|
| Supervisor | Rita Noumeir (Supervisor) & Philippe Jouvet (Co-supervisor) |
|---|