Ignoring compliments should be easy, but it’s not. We so desperately want to hear them that we are often tricked into registering them as positive data points instead of vacuous fibs. ... Rule of thumb: Compliments are the fool’s gold of customer learning: shiny, distracting, and entirely worthless.
Proxy Trap
Metrics become mirages when optimized directly.
← Back to trails
The perverse incentive is that looking busy is far more important to your career than adding value.
The final outcome of the Chicago study is further testament to the power of incentives: the following year, cheating by teachers fell more than 30 percent.
If you define the goal of a society as GNP, that society will do its best to produce GNP. It will not produce welfare, equity, justice, or efficiency unless you define a goal and regularly measure and report the state of welfare, equity, justice, or efficiency.
But here’s the problem: The idea that an improving score guarantees that you’re approaching the objective is wrong. ... This situation, when the objective function is a false compass, is called deception, which is a fundamental problem in search.
A rational person would act accordingly in the selection of strategies, and set his emotions in accordance with his results. Yet I have experienced leaps of joy over results that I knew were mere noise, and bouts of unhappiness over results that did not carry the slightest degree of statistical significance.
They provide, under a wide range of unknowable circumstances, future and past, a rational and economic guide to minimum economic loss from both mistakes. The control chart sends statistical signals, which detect existence of a special cause (usually specific to some worker or group or to some special fleeting circumstance), or tell us that the observed variation should be ascribed to common causes, chance variation attributable to the system.
In light of this, often the best way to get others to believe something is to make it a reality. ... Similarly, often the best way to convince others that we believe something is to actually believe it. ... Faced with the translucency of our own minds, then, self-deception is often the most robust way to mislead others.
I emphasize the exponential-versus-linear perspective because it’s the most important failure that prognosticators make in considering future trends. ... Indeed, almost everyone I meet has a linear view of the future. That’s why people tend to overestimate what can be achieved in the short term (because we tend to leave out necessary details) but underestimate what can be achieved in the long term (because exponential growth is ignored).
Surely if the AI is superintelligent, it must understand that when we asked it to make us happy, we didn’t mean that it should reduce us to a perpetually repeating recording of a drugged-out digitized mental episode!”—The AI may indeed understand that this is not what we meant. However, its final goal is to make us happy, not to do what the programmers meant when they wrote the code that represents this goal.