Source code summarization generates summarized information on the purpose, usage, and implementation of methods and classes to facilitate comprehension of these code entities. These summaries contain valuable information that can assist and guide software developers in tasks such as developing, maintaining, and refactoring source code.
Most often, developers rely on official documentation to understand code entities and API methods used in a project. However, researchers have identified insufficient or inadequate documentation examples and flaws with a method’s complex structure as barriers to learning an API. Therefore, to understand an API, developers may refer to other resources such as Stack Overflow, GitHub, etc. According to recent research findings, unofficial documentation is a valuable source of knowledge for generating code summaries. However, collecting data from these sources is time-consuming and requires extra effort on the developers’ side to find and extract the needed information, and sometimes, the data is inaccurate and may contain some flaws.
Our research proposes a novel approach for summarizing code entities by leveraging deep learning algorithms and informal documentation, i.e., Stack Overflow, to produce high-quality summaries for API methods. Our approach takes the API method’s name as input, generates a natural language summary by utilizing Stack Overflow discussions, and provides information about frequent problems and potential solutions related to the API.
In the first section of our research, we used an unsupervised machine learning algorithm, i.e., TextRank, to generate extractive summaries for API methods. To assess the quality of our generated summaries, we surveyed sixteen professional developers to evaluate the automatically generated summaries and compare them with the official Android documentation. Our findings indicate that although developers generally use official documentation, our generated summaries are valuable sources of information, especially when providing implementation details. Moreover, developers agreed that our summaries could be complementary to official documentation, assisting software developers in their software development and maintenance tasks.
In the second research section, we improved the summaries’ quality by applying a state-of-the-art deep learning algorithm (BART) to generate abstractive summaries. Furthermore, we created an oracle of human-generated summaries to evaluate our automatically generated summaries using ROUGE and BLEU metrics, which are frequently used in code summarization. Finally, we assessed the quality of our code summaries compared to summaries produced during the first section of our research. Our results indicate that deep learning algorithms enhance the quality of the generated code summaries. The improvements brought by our approach in the second section of this research significantly increased the accuracy of our summarization approach, reaching an average of 57 percent for Precision, 66 percent for Recall, and 61 percent for F-measure. Additionally, the new approach performs better in terms of execution time, running 4.4 times faster than the first one.
In the final part of our study, we applied the BERTopic algorithm to determine key topics from discussions about Android APIs on Stack Overflow, which serves as an essential informal documentation source. Additionally, we employed the BERT algorithm to pinpoint frequent issues and their possible solutions, creating brief yet informative summaries for these topics. To evaluate the effectiveness of our findings, we had thirty Android developers review these summaries, focusing on their performance, coherence, and interoperability. This provided us with crucial insights into the real-world applicability of our method.
| Date | 8 Jun 2024 |
|---|
| Original language | American English |
|---|
| Awarding Institution | - École de technologie supérieure
|
|---|
| Supervisor | Sylvie Ratté (Supervisor) |
|---|