Winograd Schema Challenge

The Winograd Schema Challenge (WSC) is a test of machine intelligence proposed by Hector Levesque, a computer scientist at the University of Toronto. Designed to be an improvement on the Turing test, it is a multiple-choice test that employs questions of a very specific structure: they are instances of what are called Winograd Schemas, named after Terry Winograd, a professor of computer science at Stanford University.The city councilmen refused the demonstrators a permit because they violence.The city councilmen refused the demonstrators a permit because they feared violence.The women stopped taking pills because they were . Which individuals were ? The Winograd Schema Challenge (WSC) is a test of machine intelligence proposed by Hector Levesque, a computer scientist at the University of Toronto. Designed to be an improvement on the Turing test, it is a multiple-choice test that employs questions of a very specific structure: they are instances of what are called Winograd Schemas, named after Terry Winograd, a professor of computer science at Stanford University. On the surface, Winograd Schema questions simply require the resolution of anaphora: the machine must identify the antecedent of an ambiguous pronoun in a statement. This makes it a task of natural language processing, but Levesque argues that for Winograd Schemas, the task requires the use of knowledge and commonsense reasoning. Nuance Communications announced in July 2014 that it would sponsor an annual WSC competition, with a prize of $25,000 for the best system that could match human performance. However, the prize is no longer offered. The Winograd Schema Challenge was proposed in the spirit of the Turing Test. Proposed by Alan Turing in 1950, the Turing Test plays a central role in the philosophy of artificial intelligence. Turing proposed that instead of debating what intelligence is, the science of AI should be concerned with demonstrating intelligent behavior, which can be tested. But the exact nature of the test Turing proposed has come under scrutiny, especially since an AI chat bot named Eugene was claimed to pass it in 2014. The Winograd Schema Challenge was proposed in part to ameliorate the problems that came to light with the nature of the programs that performed well on the test. Turing's original proposal was what he called the Imitation Game, which involves free-flowing, unrestricted conversations in English between human judges and computer programs over a text-only channel (such as teletype). In general, the machine passes the test if interrogators are not able to tell the difference between it and a human in a five-minute conversation. On June 7, 2014, a computer program named Eugene Goostman was declared to be the first AI to have passed the Turing Test in a competition held by the University of Reading in England. In the competition Eugene was able to convince 33% of judges that they were talking with a 13-year-old Ukrainian boy. The supposed victory of a machine that thinks aroused controversies about the Turing Test. Critics claimed that Eugene passed the test simply by fooling the judge and taking advantages of its purported identity. For example, it could easily skip some key questions by joking around and changing subjects. However, the judge would forgive its mistakes because Eugene identified as a teenager who spoke English as his second language. The performance of Eugene Goostman exhibited some of the Turing Test's problems. Levesque identifies several major issues, summarized as follows: The key factor in the WSC is the special format of its questions, which are derived from Winograd Schemas. Questions of this form may be tailored to require knowledge and commonsense reasoning in a variety of domains. They must also be carefully written not to betray their answers by selectional restrictions or statistical information about the words in the sentence. The first cited example of a Winograd Schema (and the reason for their namesake) is due to Terry Winograd: