Crowdtesting Practices and Models: An Empirical Approach

Crowdsourced software testing (CST) has received significant attention. After these years, CST has made new progress and changes.While current literature lists many CST challenges, this paper analyzes industrial CST practices, finds that many challenges already have practical solutions, summarizes their commonalities, and comes up with new CST models and processes.We look for well-known CST websites to participate in and take a secret and unobtrusive approach where customers, platform managers, and fellow workers do not know that we are mainly interested in CST research. We then register at selected CST websites, collect any public documents such as whitepapers, open rules, and public training materials, and join as many test tasks as possible.We analyze the confrontation and collaboration among clients, platforms, and workers in the CST sessions. Clients want to get as much bug information as possible for a small amount of pay, but workers want to get paid as much as possible for a small amount of bug information. We also study the process and method of selecting suitable CST workers. Based on these, this paper proposes three future research directions.Data security and privacy at CST are paramount. If this problem can be overcome, CST will have wider applications. Additionally, the integration of workers, internal workers, software automation, and artificial intelligence will be major drivers for CST. It is also critical to develop a standardized CST structure and processes, and this will push the field to grow significantly.