Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset

Keyphrases

• BERT Model 16%
• Camembert 33%
• Clinical Narratives 16%
• Clinical Notes Classification 100%
• Clinical Setting 16%
• Clinical Text 33%
• Clinical Text Classification 16%
• Computational Demand 16%
• Computational Resources 50%
• Distil-BERT 16%
• Dual Challenge 16%
• Effective Training 16%
• F1 Score 16%
• French Clinical Notes 100%
• Generalization Gap 16%
• Gustave Flaubert 16%
• Hospital Use 16%
• Hospital-based 16%
• Limited Datasets 100%
• Low-resource 16%
• Mixture-of-experts 100%
• Natural Language Processing 16%
• Pre-trained BERT 16%
• Sharp Minima 16%
• Training Model 16%
• Transformer 50%
• Transformer Model 66%
• Transformer Performance 100%

Computer Science

• Bidirectional Encoder Representations From Transformers 33%
• Clinical Setting 33%
• Computational Resource 100%
• Dual Challenge 33%
• Natural Language Processing 33%
• Pre-Trained Model 33%
• Text Classification 33%
• Transformer Model 100%

Engineering

• Computational Resource 100%
• Effective Training 33%
• Limitations 33%
• Natural Language Processing 33%