TY - GEN
T1 - Emerging Results on Automated Support for Searching and Selecting Evidence for Systematic Literature Review Updates
AU - Napoleão, Bianca Minetto
AU - Sarkar, Ritika
AU - Hallé, Sylvain
AU - Petrillo, Fabio
AU - Kalinowski, Marcos
N1 - Publisher Copyright:
© 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
PY - 2024/8/9
Y1 - 2024/8/9
N2 - Context: The constant growth of primary evidence and Systematic Literature Reviews (SLRs) publications in the Software Engineering (SE) field leads to the need for SLR Updates. However, searching and selecting evidence for SLR updates demands significant effort from SE researchers. Objective: We present emerging results on an automated approach to support searching and selecting studies for SLR updates in SE. Method: We developed an automated tool prototype to perform the snowballing search technique and to support the selection of relevant studies for SLR updates using Machine Learning (ML) algorithms. We evaluated our automation proposition through a small-scale evaluation with a reliable dataset from an SLR replication and its update. Results: Effectively automating snowballing-based search strategies showed feasibility with minor losses, specifically related to papers without Digital Object Identifier (DOI). The ML algorithm giving the highest performance to select studies for SLR updates was Linear Support Vector Machine with approximately 74% recall and 15% precision. The use of such algorithms with conservative thresholds to minimize the risk of missing papers can already significantly reduce evidence selection efforts. Conclusion: The preliminary results of our evaluation point in promising directions, indicating the potential of automating snowballing search efforts and of reducing the number of papers to be manually analyzed by about 2.5 times when selecting evidence for updating SLRs in SE.
AB - Context: The constant growth of primary evidence and Systematic Literature Reviews (SLRs) publications in the Software Engineering (SE) field leads to the need for SLR Updates. However, searching and selecting evidence for SLR updates demands significant effort from SE researchers. Objective: We present emerging results on an automated approach to support searching and selecting studies for SLR updates in SE. Method: We developed an automated tool prototype to perform the snowballing search technique and to support the selection of relevant studies for SLR updates using Machine Learning (ML) algorithms. We evaluated our automation proposition through a small-scale evaluation with a reliable dataset from an SLR replication and its update. Results: Effectively automating snowballing-based search strategies showed feasibility with minor losses, specifically related to papers without Digital Object Identifier (DOI). The ML algorithm giving the highest performance to select studies for SLR updates was Linear Support Vector Machine with approximately 74% recall and 15% precision. The use of such algorithms with conservative thresholds to minimize the risk of missing papers can already significantly reduce evidence selection efforts. Conclusion: The preliminary results of our evaluation point in promising directions, indicating the potential of automating snowballing search efforts and of reducing the number of papers to be manually analyzed by about 2.5 times when selecting evidence for updating SLRs in SE.
KW - SLR update
KW - searching for evidence
KW - selecting evidence
KW - systematic review update
UR - https://www.scopus.com/pages/publications/85203107984
U2 - 10.1145/3643664.3648202
DO - 10.1145/3643664.3648202
M3 - Contribution to conference proceedings
AN - SCOPUS:85203107984
T3 - Proceedings - 2024 IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering, WSESE 2024
SP - 34
EP - 41
BT - Proceedings - 2024 IEEE/ACM International Workshop on Methodological Issues with Empirical Studies in Software Engineering, WSESE 2024
PB - Association for Computing Machinery, Inc
T2 - 1st International Workshop on Methodological Issues with Empirical Studies in Software Engineering, WSESE 2024
Y2 - 16 April 2024 through 16 April 2024
ER -