Won’t Somebody Please Think of the Tests? A Grounded Theory Approach to Industry Challenges in Continuous Practices

Continuous integration and delivery are well-established paradigms in the software development community. With these continuous practices come many challenges; while some of these challenges are immediate and well documented in literature, others may be revealed only after sustained application of these practices in large-scale and complex contexts. Based on researcher observations and interviews with 22 senior professionals from four companies, all with significant but varied experiences of continuous practices, we present a set of underreported challenges with continuous practices observed in multiple industry settings. Through a grounded theory approach we construct the Tapco model — Test Automation Progression in Continuous Practices — identifying two distinct ways in which companies progress towards continuous delivery, onto which the studied cases are mapped. This model is then validated by presenting it to the four studied companies, operating in disparate industry segments, and to three additional industry cases, letting them evaluate its relevancy, accuracy and novelty. We find that the model provides industry professionals with essential guidance on how to avoid common pitfalls, as well as an understanding of their causes and possible remediation.