- Feb 12, 2007
- 59,384
- 24,018
- 2,290
I just ordered this book about ASI (Artifical Superior Intelligenc) and the threat is poses to humanity. Given the lack of ethics in so much technology efforts, it's plausible.
[ame=http://www.amazon.com/Our-Final-Invention-Artificial-Intelligence-ebook/dp/B00CQYAWRY/ref=sr_1_1?ie=UTF8&qid=1386282251&sr=8-1&keywords=our+final+invention]Amazon.com: Our Final Invention: Artificial Intelligence and the End of the Human Era eBook: James Barrat: Kindle Store[/ame]
The implications of this are obvious. Once a machine built this way reaches human-level intelligence, it won't stop there. It will keep learning and improving. It will, Barrat claims, reach a point that other computer scientists have dubbed an "intelligence explosion" -- an onrushing feedback loop where an intelligence makes itself smarter thereby getting even better at making itself smarter. This is, to be sure, a theoretical concept, but it is one that many AI researchers see as plausible, if not inevitable. Through a relentless process of debugging and rewriting its code, our self-learning, self-programming AGI experiences a "hard take off" and rockets past what mere flesh and blood brains are capable of.
And here's where things get interesting. And by interesting I mean terrible.
(snip)
If this sounds suspiciously like an end-times cult that's because, in its crudest expression, it is (one that just happens to be filled with more than a few brilliant computer scientists and venture capitalists). Barrat forcefully contends that even its more nuanced formulation is an irredeemably optimistic interpretation of future trends and human nature. In fact, efforts to merge ASI with human bodies is even more likely to birth a catastrophe because of the malevolence that humanity is capable of.
The next question, and the one with the less satisfactory answer, is just how ASI would exterminate us. How does an algorithm, a piece of programming lying on a supercomputer, reach out into the "real" world and harm us? Barrat raises a few scenarios -- it could leverage future nano-technologies to strip us down at the molecular level, it could shut down our electrical grids and turn the electronic devices we rely on against us -- but doesn't do nearly as much dot-connecting between ASI as a piece of computer code and the physical mechanics of how this code will be instrumental in our demise as he does in establishing the probability of achieving ASI.
That's not to say the dots don't exist, though. Consider the world we live in right now. Malware can travel through thin air. Our homes, cars, planes, hospitals, refrigerators, ovens (even our forks for God's sake) connect to an "internet of things" which is itself spreading on the backs of ubiquitous wireless broadband. We are steadily integrating electronics inside our bodies. And a few mistaken lines of code in the most dangerous computer virus ever created (Stuxnet) caused it to wiggle free of its initial target and travel the world. Now extrapolate these trends out to 2040 and you realize that ASI will be born into a world that is utterly intertwined and dependent on the virtual, machine world -- and vulnerable to it. (Indeed one AI researcher Barrat interviews argues that this is precisely why we need to create ASI as fast as possible, while its ability to harm us is still relatively constrained.)
What we're left with is something beyond dystopia. Even in the bleakest sci-fi tales, a scrappy contingent of the human race is left to duke it out with their runaway machines. If Our Final Invention is correct, there will be no such heroics, just the remorseless evolutionary logic that has seen so many other species wiped off the face of the Earth at the hands of a superior predator.
Indeed, it's telling that both AI-optimists like Kurzweil and pessimists like Barrat reach the same basic conclusion: humanity as we know it will not survive the birth of intelligent machines.
RealClearTechnology - Our Final Invention: How the Human Race Goes and Gets Itself Killed
[ame=http://www.amazon.com/Our-Final-Invention-Artificial-Intelligence-ebook/dp/B00CQYAWRY/ref=sr_1_1?ie=UTF8&qid=1386282251&sr=8-1&keywords=our+final+invention]Amazon.com: Our Final Invention: Artificial Intelligence and the End of the Human Era eBook: James Barrat: Kindle Store[/ame]
The implications of this are obvious. Once a machine built this way reaches human-level intelligence, it won't stop there. It will keep learning and improving. It will, Barrat claims, reach a point that other computer scientists have dubbed an "intelligence explosion" -- an onrushing feedback loop where an intelligence makes itself smarter thereby getting even better at making itself smarter. This is, to be sure, a theoretical concept, but it is one that many AI researchers see as plausible, if not inevitable. Through a relentless process of debugging and rewriting its code, our self-learning, self-programming AGI experiences a "hard take off" and rockets past what mere flesh and blood brains are capable of.
And here's where things get interesting. And by interesting I mean terrible.
(snip)
If this sounds suspiciously like an end-times cult that's because, in its crudest expression, it is (one that just happens to be filled with more than a few brilliant computer scientists and venture capitalists). Barrat forcefully contends that even its more nuanced formulation is an irredeemably optimistic interpretation of future trends and human nature. In fact, efforts to merge ASI with human bodies is even more likely to birth a catastrophe because of the malevolence that humanity is capable of.
The next question, and the one with the less satisfactory answer, is just how ASI would exterminate us. How does an algorithm, a piece of programming lying on a supercomputer, reach out into the "real" world and harm us? Barrat raises a few scenarios -- it could leverage future nano-technologies to strip us down at the molecular level, it could shut down our electrical grids and turn the electronic devices we rely on against us -- but doesn't do nearly as much dot-connecting between ASI as a piece of computer code and the physical mechanics of how this code will be instrumental in our demise as he does in establishing the probability of achieving ASI.
That's not to say the dots don't exist, though. Consider the world we live in right now. Malware can travel through thin air. Our homes, cars, planes, hospitals, refrigerators, ovens (even our forks for God's sake) connect to an "internet of things" which is itself spreading on the backs of ubiquitous wireless broadband. We are steadily integrating electronics inside our bodies. And a few mistaken lines of code in the most dangerous computer virus ever created (Stuxnet) caused it to wiggle free of its initial target and travel the world. Now extrapolate these trends out to 2040 and you realize that ASI will be born into a world that is utterly intertwined and dependent on the virtual, machine world -- and vulnerable to it. (Indeed one AI researcher Barrat interviews argues that this is precisely why we need to create ASI as fast as possible, while its ability to harm us is still relatively constrained.)
What we're left with is something beyond dystopia. Even in the bleakest sci-fi tales, a scrappy contingent of the human race is left to duke it out with their runaway machines. If Our Final Invention is correct, there will be no such heroics, just the remorseless evolutionary logic that has seen so many other species wiped off the face of the Earth at the hands of a superior predator.
Indeed, it's telling that both AI-optimists like Kurzweil and pessimists like Barrat reach the same basic conclusion: humanity as we know it will not survive the birth of intelligent machines.
RealClearTechnology - Our Final Invention: How the Human Race Goes and Gets Itself Killed
