Supporting an Iterative Conversation Design Process.

Conversation design is an essential step in building a chatbot. Much like visual user interface design, conversation design benefits from prototyping and user testing to allow for conversation exploration and improvement. However, it can be overwhelming to quickly iterate on the conversation design as the iterative process requires not only designing a conversation but also building and testing a working chatbot equipped with the conversation. We developed ProtoChat, a prototype system that supports an iterative conversation design by allowing designers to (1) prototype conversations, (2) test the conversations with the crowd, and (3) review and analyze the crowdsourced conversation data. Results of an exploratory study with four conversation designers show that the designers successfully iterated on their conversation design by reviewing how the crowd followed the conversation, which provided insights into concrete action items for improving their conversation design.