Votto
Diamond Member
- Oct 31, 2012
- 68,699
- 77,286
- 3,605
AI's Big Red Button Doesn't Work, And The Reason Is Even More Troubling
"It's going to be really bad for humans."
AI's Big Red Button Doesn't Work, And The Reason Is Even More Troubling
It's one of humanity's scariest what-ifs – that the technology we develop to make our lives better develops a will of its own.
Early reactions to a September preprint describing AI behavior have already speculated that the technology is exhibiting a survival drive. But, while it's true that several large language models (LLMs) have been observed actively resisting commands to shut down, the reason isn't 'will'.
Instead, a team of engineers at Palisade Research proposed that the mechanism is more likely to be a drive to complete an assigned task – even when the LLM is explicitly told to allow itself to be shut down. And that might be even more troubling than a survival drive, because no one knows how to stop the systems.
How long does humanity have to survive you reckon?

