What happened to my application? Helping end users comprehend evolution through variation management

Abstract Context: Millions of end users are creating software applications. These end users typically do not have clear requirements in mind; instead, they debug their programs into existence and reuse their own or other persons’ code. These behaviors often result in the creation of numerous variants of programs. Current end-user programming environments do not provide support for managing such variants. Objective: We wish to understand the variant creation behavior of end user programmers. Based on this understanding we wish to develop an automated system to help end user programmers efficiently manage variants. Method: We conducted an on-line survey to understand when and how end-user programmers create program variants and how they manage them. Our 124 survey respondents were recruited via email from among non-computer science majors who had taken at least one course in the computer science department at our university; the respondents were involved in the Engineering, Sciences, Arts, and Management fields. Based on the results of this survey we identified a set of design requirements for providing variation management support for end users. We implemented variation management support in App Inventor – a drag and drop programming environment for creating mobile applications. Our support, AppInventorHelper, is meant to help end-user programmers visualize the provenance of and relationships among variants. We conducted a think-aloud study with 10 participants to evaluate the usability of AppInventorHelper. The participants were selected on a first-come, first-served basis from those who responded to our recruitment email sent to list-servers. They were all end users majoring in electrical engineering, mechanical engineering, or physics. None had formal training in software engineering methods, but all had some experience with visual programming languages. Results: Our (user study) results indicate that AppInventorHelper can help end users navigate through variants and find variants that could be utilized cost-effectively as examples or actual code upon which to build new applications. For example, in one of our empirical studies end users explored variants of a paint application in order to find a variant that could easily be extended to incorporate a new feature. Conclusions: Our survey results show that end users do indeed reuse program variants and suggest that understanding the differences between variants is important. Further, end users prefer running code and looking at outputs, accessing source code and meta information such as filenames, referring to the creation and update dates of programs, and having information on the authors of code. When selecting variants users prefer to look at their major features such as correctness, similarity and authorship information. End users rely primarily on memory to track changes. They seldom make use of online or configuration management tools. Hence, integrated domain-specific variation management tools like AppInventorHelper can significantly help improve users’ interactions with the system. A key contribution of our work is a set of design requirements for end-user programming environments that facilitate the management and understanding of the provenance of program variants.