Smart optical networks enabled by performance monitoring and machine learning

Abstract

Reliable optical network operation depends on effective quality of transmission (QoT) management across the operational lifetime of each lightpath. Hard failures cause immediate service disruption, while soft failures produce gradual degradation that may go undetected until service is impacted. Both ultimately manifest as QoT violations. QoT management encompasses the functions or tasks required to ensure that each lightpath meets its quality-ofservice requirements. These functions include assessing the feasibility of a lightpath based on its estimated QoT before establishment, detecting abnormal QoT degradations during operation, and forecasting the evolution of QoT to enable proactive intervention before service quality is compromised. Optical network failure management (ONFM) addresses network failures with six identified use cases in the literature: QoT estimation, failure detection, failure prediction, root-cause identification, failure localization, and failure magnitude estimation. Although machine learning (ML) has proven effective for individual ONFM use cases, existing solutions have addressed them separately. Moreover, the relationships among the QoT management tasks, the selection of appropriate ML approaches for each task, and the ways in which these tasks can reinforce one another through cross-task integration, remain underexplored. This thesis addresses three QoT management tasks corresponding to three ONFM use cases: lightpath feasibility assessment through QoT estimation, anomaly detection for failure detection, and QoT forecasting for failure prediction. For lightpath feasibility assessment, performed prior to lightpath establishment, a lightweight and interpretable supervised learning model is developed for QoT estimation of unestablished lightpaths. To address the difference between synthetic or simulated training data and real deployment environments with scarce datasets, domain adaptation techniques integrated with feature selection based on explainable artificial intelligence (XAI) are implemented. This enables robust performance with limited target domain samples while maintaining the computational efficiency and interpretability required for large-scale optical network planning operations. Random Forest (RF) classifiers with Boruta-Shapley additive explanation (Boruta-SHAP) automated feature selection and transfer learning achieve 98.64%-99.65% accuracy for QoT estimation across four datasets while providing superior computational efficiency (reductions of training time by 52.78%-70.68%, inference times of 1.1-2.4 μs per lightpath) and SHAP-based interpretability. Feature-selection-guided domain adaptation achieves 86% accuracy with only 50 target domain instances. For anomaly detection during network operations, complementary unsupervised learning methods are applied to identify diverse anomalies within multivariate optical performance monitoring (OPM) datasets without relying on labeled failure data. A comparison of unsupervised methods such as density based spatial clustering applications with noise (DBSCAN), one-class support vector machine (OCSVM), Isolation Forest (IF), on 4 years of production network data, demonstrates that DBSCAN achieves superior anomaly detection performance (precision 0.54, silhouette score 0.82) while detecting subtle degradations traditional threshold-based monitoring fails to observe. For QoT forecasting, a framework that integrates change-point detection (CPD) and collective anomaly detection (CAD) is introduced using multivariate sequence-to-sequence (seq2seq) deep learning models for QoT forecasting. By integrating change point and collective anomaly indicators into time-series prediction, the approach enables the computation of actionable reaction windows that support proactive network management. A change point and collective anomaly-aware long short-term memory (LSTM)-based forecasting framework achieves 28.21% root mean square error (RMSE) reduction and 42.36% mean absolute error (MAE) reduction compared to baseline forecasting, providing consistent reaction windows of 1-10 days before failure events. A systematic validation across diverse additional datasets, forecasting architectures (temporal convolutional network (TCN) and LSTM), unsupervised CAD techniques, and input sequence lengths confirm the robustness of the approach. The results show consistent forecasting improvements (averaged reductions of RMSE and MAE of roughly 10 25%), along with 100% failure coverage with a stable positive lead time across all detection sensitivity. These results demonstrate that selecting ML models according to the specific context of each task yields robust solutions for QoT management. Moreover, they show that integrating CPDCAD indicators into the QoT forecasting framework significantly improves both prediction accuracy and early-warning capability in production optical networks.

Date	15 Jul 2026
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Christine Tremblay (Supervisor) & Lena Wosinka (Co-supervisor)