Invisible Crack

Microscopic defects propagate silently until catastrophic failure.
In this way a tiny unseen crack may start from any hole or notch or irregularity in a stressed metal and may spread across the material, which is not, as a whole, changed in any obvious way. Sooner or later, such a ‘fatigue crack’ will reach the critical length for an ordinary common or garden crack. When this happens, the crack will immediately speed up and run right across the material, often with very serious consequences.
Structures

Structures

“Brittle fracture,” in which a large crack runs spontaneously through a structure near the speed of sound, severing steel with a report that signals the breakup of ships, the bursting of pressure vessels, or the collapse of bridges, has been a chronic problem for centuries. There is almost always a period of “gestation”—the slow lengthening and sharpening of cracks through the process of fatigue that precedes the catastrophe.
To Engineer Is Human

To Engineer Is Human

A potentially weakened engine component made its way all along the supply chain until it was placed into the engine, and there to await its inevitable breakage—and the equally inevitable destruction of the entire engine. It should have failed inspection, but it didn’t. It just failed in real life.
The Perfectionists

The Perfectionists

In each case the minor error triggers, or combines with, other unforeseen consequences in the system, also minor. But because of the tight interdependence of parts, minor glitches in the right improbable sequence cascade until the trouble becomes an unstoppable wave and reaches catastrophic proportions. ... Perrow did an exhaustive minute-by-minute study of 50 large-scale technological accidents (such as Three Mile Island, the Bhopal disaster, Apollo 13, Exxon Valdez, Y2K, etc.) and concluded, “We have produced designs so complicated that we cannot anticipate all the possible interactions of the inevitable failures; we add safety devices that are deceived or avoided or defeated by hidden paths in the systems.”
What Technology Wants

What Technology Wants

Complexity catastrophes help explain why bureaucracy seems to grow with the tenacity of weeds. ... Rather, network growth creates interdependencies, interdependencies create conflicting constraints, and conflicting constraints create slow decision making and, ultimately, bureaucratic gridlock. ... The reason is that IBM fell prey to a complexity catastrophe.
The Origin of Wealth

The Origin of Wealth

Politicians use the term “creeping normalcy” to refer to such slow trends concealed within noisy fluctuations. If the economy, schools, traffic congestion, or anything else is deteriorating only slowly, it’s difficult to recognize that each successive year is on the average slightly worse than the year before, so one’s baseline standard for what constitutes “normalcy” shifts gradually and imperceptibly. It may take a few decades of a long sequence of such slight year-to-year changes before people realize, with a jolt, that conditions used to be much better several decades ago, and that what is accepted as normalcy has crept downwards.
Collapse

Collapse

Some systems not only resist policy and stay in a normal bad state, they keep getting worse. One name for this archetype is “drift to low performance.” ... It is also called the “boiled frog syndrome,” from the old story (I don’t know whether it is true) that a frog put suddenly in hot water will jump right out, but if it is put into cold water that is gradually heated up, the frog will stay there happily until it boils.
Thinking in Systems

Thinking in Systems

The first is the persuasive power of the incremental. ... What incrementalism does is put the potential resister on the defensive, making the savagery seem like an issue of rationality rather than of morality. ... The descent into savagery can be so incremental as to come with nothing but arbitrary boundaries, and our descent becomes like the proverbial frog cooked alive without noticing.
Behave

Behave

When a group’s fundamental tenets are at stake, those who demonstrate the most steadfast commitment—who continue to chant the loudest or clench their eyes the tightest in the face of conflicting evidence—earn the most trust from their fellow group members. The employee who drinks the company Kool-Aid, however epistemically noxious, will tend to win favor from colleagues, especially in management, and move faster up the chain. In fact, we often measure loyalty in our relationships by the degree to which a belief is irrational or unwarranted by the evidence.
The Elephant in the Brain

The Elephant in the Brain

The surprising thing is that this pathological mistrust within the ranks leads not to dissension but to strict conformity. Knowing themselves continually watched, the faithful strive to escape suspicion by adhering zealously to prescribed behavior and opinion. Strict orthodoxy is as much the result of mutual suspicion as of ardent faith.
The True Believer

The True Believer