Explore how IntelliGame, a gamification plugin for IntelliJ, impacts JavaScript unit testing in a controlled experiment with 152 participants.
Authors: Philipp Straubinger, University of Passau, Passau, Germany and this author contributed equally to this research; Tommaso Fulcini, Politecnico di Torino, Torino, Italy and this author contributed equally to this research; Gordon Fraser, University of Passau, Passau, Germany; Marco Torchiano, Politecnico di Torino, Torino, Italy.
Table of Links Abstract and Introduction Background and Related Work Implementation Experiment Results Conclusions, Acknowledgement, and References 5 RESULTS While the analysis of results is ongoing, we provide a preliminary overview by addressing the challenges encountered, our problem-solving approaches, and insights from the exit survey completed by students regarding their perceived user experience. 5.1 Survey Answers Based on the responses to the exit survey , the chosen target class was well-received by all groups. All groups unanimously agreed that the class was easy to comprehend and implement. However, it is noteworthy that just under half of the participants in the treatment group and fewer than 40% in the control group indicated they had sufficient time to complete the implementation. Consequently, it is not surprising that only approximately 25% in both groups felt they had enough time to conduct thorough testing of their implementations. Interestingly, around two-thirds of all participants actively tested their code. Notably, a smaller percentage of participants wrote tests during the development phase, with 52% in the treatment group compared to 44% in the control group. Roughly 50% of participants in both groups were confident in the correctness of their implementations. Conversely, only 21% in the treatment group and 14% in the control group were certain about the quality of their test suites. No significant differences were observed between the two groups. Concerning the responses of the treatment group regarding IntelliGame, they are predominantly positive or, at worst, undecided . Participants demonstrated a clear understanding of the tool’s descriptions and how to make progress in the presented achievements. They also appreciated the frequency of the notifications. About 40% of the participants reported that the achievements positively influenced their testing behavior, and an equivalent percentage mentioned being motivated by both the notifications and the plugin itself. Encouragingly, 42% of participants expressed a desire to use IntelliGame in their own projects. 5.2 Problems Faced and Lessons Learned In the initial adaptation phase, we faced challenges while transitioning the original TypeScript project to JavaScript. Although automatic transpilation facilitated obtaining the correct JavaScript code, the hurdle lies in converting the TS configuration to a JScompatible one. Since the original project used another testing framework, i.e., Karma, we had to modify the configuration to fit Jest. Another challenge arose when configuring the main.js file for manual function assessment. Unlike Java, where a Main class defines the starting point, JavaScript, especially in the Node framework, lacks that. So, in order to reproduce the same setting as the original validation experiment , we had to create a custom file for this purpose. Throughout the experiment, additional issues surfaced and required adaptation of the plugin. In the first session, participants could only choose between running the project via tests or main execution due to the provided configuration. Despite being shown how to switch between configurations, most participants stuck with one method for the entire session. Upon reflection, we observed that the number of tasks overwhelmed participants, leading them to feel demotivated despite knowing that their performance would not be evaluated. While they found the functions appropriately challenging, the abundance of tasks influenced their survey responses negatively . We attribute the superior performance of students in the pilot study to their greater experience and a less pressured environment. To address this, future experiments will involve students with comparable experience levels to the overall sample. We hypothesize that incorporating gamification elements aimed at promoting TDD approach to the plugin may yield to even more encouraging results. Minor issues included students incorrectly selecting a parent directory when opening the project, interrupting the script that committed their progress to the repository. We promptly identified and resolved these problems. General challenges with the committing process stemmed from variations in participants’ laptop configurations, which we sometimes deemed unsolvable, opting to collect the final project state at the designated time. This paper is under CC BY-SA 4.0 DEED license. available on arxiv
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Action-packed Film Showcasing Director's Skillset and Rising Action StarA film directed by Patel showcases his skillset as a director and introduces him as a rising action star. The film follows a mysterious young man on a revenge mission who infiltrates a sex club to uncover the murderer of a loved one. The action sequences are intense and filled with violence, filmed in a way that gives the audience a front row seat to the action. The film's South Korean influence is evident in its thrilling and messy nature. Patel's direction cleverly balances the intensity with comedy.
Read more »
Eesee's CEO Vova Sadkov on Gamifying Digital Asset TradingMr. Sadkov, the company's CEO, recently sat down for an interview to discuss the potential of gamified trading and the challenges it presents.
Read more »
IntelliGame in Action: Gamifying JavaScript Unit TestsExplore how IntelliGame, a gamification plugin for IntelliJ, impacts JavaScript unit testing in a controlled experiment with 152 participants.
Read more »
IntelliGame in Action: Gamifying JavaScript Unit Tests - Background and Related WorkExplore how IntelliGame, a gamification plugin for IntelliJ, impacts JavaScript unit testing in a controlled experiment with 152 participants.
Read more »
IntelliGame in Action: Gamifying JavaScript Unit TestsExplore how IntelliGame, a gamification plugin for IntelliJ, impacts JavaScript unit testing in a controlled experiment with 152 participants.
Read more »
IntelliGame in Action: Gamifying JavaScript Unit TestsExplore how IntelliGame, a gamification plugin for IntelliJ, impacts JavaScript unit testing in a controlled experiment with 152 participants.
Read more »