A proposed framework for the design and analysis of metaheuristics

Abstract

This doctoral thesis critically evaluates and seeks to improve methodological practices in metaheuristics research. These black-box stochastic optimization algorithms are widely applied to NP-complete problems, including vehicle routing, neural network training, and aerospace structure design. Despite their popularity, the literature reveals significant shortcomings in methodological rigor, transparency, and reproducibility, which undermine the scientific validity of reported results. To address these issues, the research is structured around three complementary studies: (1) a critical analysis of the Bat Algorithm, revealing its structural similarities to earlier methods and the lack of robust evidence supporting its claimed superiority; (2) an investigation into the use of chaotic maps, showing that reported performance gains often stem from sequence effects rather than intrinsic properties; (3) a quantitative assessment of statistical practices in 70 recent articles, highlighting systematic deficiencies in hypothesis testing, effect size reporting, and confidence interval usage. Building on these findings, the thesis introduces the Metaheuristics Design and Analysis Framework (MDAF) as a structured methodological framework integrating guidelines for experimental design, statistical analysis, and transparent reporting. Adoption of the MDAF aims to enhance the quality, reproducibility, and practical relevance of metaheuristics research, ultimately contributing to more robust and scientifically credible optimization solutions.

Date	3 Mar 2026
Original language	American English
Awarding Institution	École de technologie supérieure
Supervisor	Alain April (Supervisor) & Alain Abran (Co-supervisor)