Real-time Delphi

Real-time Delphi (RTD) is an advanced form of the Delphi method. The advanced method “is a consultative process that uses computer technology” to increase efficiency of the Delphi process. Real-time Delphi (RTD) is an advanced form of the Delphi method. The advanced method “is a consultative process that uses computer technology” to increase efficiency of the Delphi process. Gordon and Pease define the advanced approach as an innovative way to conduct Delphi studies that do not involve sequential “rounds” and consequently lead to a higher degree of efficiency with regard to the time frame needed to perform such studies. Friedewald, von Oertzen, and Cuhls underline that aspect by writing, in “a Real-Time-Delphi, the participants do not only judge twice but can change their opinion as often as they like when they see the aggregated results of the other participants”. So, here it becomes clear that the Real-Time Delphi approach requires real-time calculation and provision of group responses. Friedewald et al. further state that the Real-Time Delphi method has beneath its explorative and predictive elements also normative and communicative elements. These latter are investigated by Bolognini, who explores the potential of computer-based Delphi as a communication technique for electronic democracy. Comparative studies of von der Gracht and colleagues have revealed that Real-time Delphi studies are comparable to the outcome of conventional round-based Delphi surveys. The basic idea of a real-time, therefore computer-based (usually web-based), Delphi approach originates in a paper published by Turoff back in 1972 about an online Delphi conference conducted in the United States. The conference was characterized by remote locations of participants, an online tool to access and give judgments, anonymity of the participants, continuous operations and analysis of results (i.e. participants were able to see given answers of the other participants in real-time), as well as asynchronous participation (i.e. participants could independently login and logout how often and when they desired). The stated aspects are some of the key characteristics of Real-Time Delphi studies, which shows that the original idea of conducting such studies can be traced back to the respective year. Today, nevertheless, technological innovations and advanced computer aided design possibilities (e.g. high-speed internet connections, high definition graphic, and advanced processor performance) facilitate more sophisticated studies in this context.The general idea to develop a faster advanced form of Delphi studies by using ideas and basic concepts of Turoff, was initiated by the U.S. Defense Advanced Research Projects Agency (DARPA), which awarded a grant in 2004 to develop an approach to improve “speed and efficiency of collecting judgments in tactical situations”. A small software company named Articulate Software in San Francisco was awarded an innovation research grant to develop what DARPA was asking for. Adam Pease, principal consultant and CEO of Articulate Software, published the findings and methodology together with Theodore Gordon in 2006. Based on the findings in this seminal paper, Heiko von der Gracht developed a second Real-time Delphi platform until 2008. In subsequent years, von der Gracht and colleagues used the platform extensively for prospective research studies. In 2011, von der Gracht and Gordon presented a joint paper at WFS Annual Summit 2011, which summarizes the lessons learned of 40 RTD studies across the two platforms. The question arises how a Real-Time Delphi study differs from a Conventional Delphi study. The basic framework is to think of a Delphi study which is conducted in form of an online questionnaire. However, a Conventional round-based Delphi study conducted via the internet is called “Internet Delphi”. The basic difference to Internet Delphi is that the process of a Real-Time Delphi is not characterised by single iterated rounds. In fact, real-time calculation and provision of responses are the key characteristics of Real-Time Delphis. Various other labels for Real-Time Delphi can be found in literature and many authors are not completely aware of the differences: “Electronic Delphi”, “Computer Delphi”, “Computer-aided Delphi”, and “Technology Delphi”. However, it is important to truly understand the design and process a researcher has chosen to find out whether real-time calculations and provisions have been applied or not. The typical Real-Time Delphi process can be described in the way that participants get access to an online questionnaire portal for a certain time frame, within which they are allowed to log in and log out as often as they want. Whenever they login, they will see all their quantitative and qualitative answers of previous sessions and they can change all answers as desired within the given period of time. Besides their own answers they will see the ongoing – hence, real-time – responses of other participants, and with regard to metric assessments the group as a whole will be visualised in terms of median, average, and interquartile range (IQR). It has to be pointed out that the numerical visualisations as well as the qualitative inputs change in the course of other participants changing their responses. Consequently, a participant can find out to what extent his own responses from an earlier point of time are still within the group opinion (i.e. IQR). The core innovation, then, of Real-Time Delphi studies is the real-time calculation and provision of results. The core methodological innovation of Real-Time Delphi studies are the absence of iterated rounds and the real-time calculation and provision of group responses. Whereas Conventional Delphi studies are characterised by repeating sequential rounds, the Real-Time Delphi approach is characterised by a continuous round-less procedure leading to a reduced time frame needed to conduct such studies. Consequently, conducting large-scale studies of huge complexity in a relatively short period of time becomes possible. Another core methodological innovation is the fact that experts may not only judge once or twice, depending on the number of rounds, as it has been usual in a Conventional Delphi study. During a Real-Time Delphi, experts can independently reassess their responses as often as they want. Hartman and Baldwin discuss further advantages of the Real-Time Delphi approach: First, the number of experts participating in the real-time study can be increased due to a higher degree of automation during and improved possibilities for analysis after the study. Additionally, the Internet provides the possibility to invite a worldwide expert panel to participate in the study. Second, the degree of interaction among the experts can be increased due to the fact that they can immediately react on others’ comments. Additionally, the time frame between giving own answers and getting insights into others’ responses is very short, which encourages stronger cognitive examination with the respective issue in question. Hartman and Baldwin argue that with the help of this procedure the validity of results is maximised.