System accident

A system accident (or normal accident) is an 'unanticipated interaction of multiple failures' in a complex system. This complexity can either be of technology or human organization, and frequently has major aspects of both. A system accident can be very easy to see in hindsight, but extremely difficult in foresight because there are often too many action pathways to seriously consider all of them. William Langewiesche writes, 'the control and operation of some of the riskiest technologies require organizations so complex that serious failures are virtually guaranteed to occur.'' . . . It was found that the accident was not the result of a chance malfunction in a statistical sense, but rather resulted from an unusual combination of mistakes, coupled with a somewhat deficient and unforgiving design . . .'g. In reviewing these procedures before the flight, officials of NASA, ER, and Beech did not recognize the possibility of damage due to overheating. Many of these officials were not aware of the extended heater operation. In any event, adequate thermostatic switches might have been expected to protect the tank.'From the article 'A New Accident Model for Engineering Safer Systems,' by Nancy Leveson, in Safety Science, April 2004:'However, instructions and written procedures are almost never followed exactly as operators strive to become more efficient and productive and to deal with time pressures. . . . . even in such highly constrained and high-risk environments as nuclear power plants, modification of instructions is repeatedly found and the violation of rules appears to be quite rational, given the actual workload and timing constraints under which the operators must do their job. In these situations, a basic conflict exists between error as seen as a deviation from the normative procedure and error as seen as a deviation from the rational and normally used effective procedure (Rasmussen and Pejtersen, 1994).' A system accident (or normal accident) is an 'unanticipated interaction of multiple failures' in a complex system. This complexity can either be of technology or human organization, and frequently has major aspects of both. A system accident can be very easy to see in hindsight, but extremely difficult in foresight because there are often too many action pathways to seriously consider all of them. William Langewiesche writes, 'the control and operation of some of the riskiest technologies require organizations so complex that serious failures are virtually guaranteed to occur.' Safety systems themselves are sometimes the added complexity which leads to this type of accident. Once an enterprise passes a certain point in size, with many employees, specialization, backup systems, double-checking, detailed manuals, and formal communication, employees can all too easily recourse to protocol, habit, and 'being right.' Rather like attempting to watch a complicated movie in a language one is unfamiliar with, the narrative thread of what is going on can be lost. And since real world accidents almost always have multiple causes, other phenomena such as groupthink can also be occurring at the same time. In particular, it is a mark of a dysfunctional organization to simply blame the last person who touched something. In 2012 Charles Perrow wrote, 'A normal accident is where everyone tries very hard to play safe, but unexpected interaction of two or more failures (because of interactive complexity), causes a cascade of failures (because of tight coupling).'Charles Perrow uses the term normal accident to emphasize that, given the current level of technology, such accidents are highly likely over a number of years or decades. James T. Reason extended this approach with human reliability and the Swiss cheese model, now widely accepted in aviation safety and healthcare. There is an aspect of an animal devouring its own tail, in that more formality and effort to get it exactly right can actually make the situation worse. For example, the more organizational riga-ma-role involved in adjusting to changing conditions, the more employees will likely delay reporting such changes, 'problems,' and unexpected conditions. These accidents often resemble Rube Goldberg devices in the way that small errors of judgment, flaws in technology, and insignificant damages combine to form an emergent disaster. William Langewiesche writes about, 'an entire pretend reality that includes unworkable chains of command, unlearnable training programs, unreadable manuals, and the fiction of regulations, checks, and controls.' An opposing idea is that of the high reliability organization.