A Pattern-Driven and LLM-Assisted Approach for Decomposing Monolithic ML-Based Systems into Microservices

The evolution of software systems has witnessed a marked shift from monolithic architectures to microservices. This migration is driven by the need to improve the scalability and maintainability of monolithic software systems. However, this shift is most noticeable in Machine Learning (ML)-based systems, where adding learning components brings extra layers of complexity. As ML becomes increasingly embedded in diverse application domains, the challenges of evolving, scaling, and maintaining these systems demand novel architectural solutions. While microservices have proven effective in addressing such challenges in traditional systems, a principled and systematic decomposition strategy tailored specifically to ML-based monoliths remains underexplored. In this paper, we introduce an automated approach for decomposing ML-based monolithic systems into microservices. Leveraging ML-specific architectural patterns, our method employs Large Language Models (LLMs) to detect ML layers, transformer embeddings to capture semantic similarities, and clustering to form coherent microservice candidates. We validate our approach on three monolithic ML-based systems and compare our decomposition results with two baseline approaches from the literature. The results demonstrate the effectiveness of our method in producing modular and ML-aware decompositions, with a precision of 84% and a recall of 65%, outperforming the baseline approaches.