Surely if the AI is superintelligent, it must understand that when we asked it to make us happy, we didn’t mean that it should reduce us to a perpetually repeating recording of a drugged-out digitized mental episode!”—The AI may indeed understand that this is not what we meant. However, its final goal is to make us happy, not to do what the programmers meant when they wrote the code that represents this goal.
Proxy Trap
Metrics become mirages when optimized directly.
False Positives
Rare signals drown in false positives.
Tempo Gradient
Decision-making speed determines conflict outcomes.