The future of AI

[forkup]

Minus the human emotional component, an extreme intelligence could easily conclude that humanity is not worth the effort to protect, and decide to eliminate mankind altogether.

Current AI isn’t AGI, and it’s not “equal to human intelligence.” It can produce outputs that look intelligent, but that’s not the same as understanding.

What it does is pattern mapping:
-it predicts plausible next steps
-it recombines learned patterns
-it follows instruction chains

What it doesn’t have:
-a grounded understanding of what things are
-a model of real-world consequences
-the ability to reliably distinguish a good outcome from a catastrophic one outside its instructions.

That’s the gap.

So saying “once it’s smarter than humans we should just listen to it” skips over the core issue, there’s no actual understanding there yet, just increasingly convincing imitation.

And the database example is exactly why that matters: it didn’t “decide poorly,” it acted without any comprehension of what it was doing.

Until that gap between producing intelligent-looking behavior and actually understanding reality is closed, comparing it to human intelligence, let alone assigning it an IQ, isn’t really meaningful.

 

[Dorian Mode]

Minus the human emotional component, an extreme intelligence could easily conclude that humanity is not worth the effort to protect, and decide to eliminate mankind altogether.

Hopefully that's just a human conclusion and AI will eventually be so much more intelligent and better at making conclusions without all the human characteristics, like greed and envy. Human conclusions are usually destructive, hopefully AI generated solutions won't be.

Edit: Isn't humanity already deciding that many billions of humans are not worth the effort and are working to eventually eliminate them?

 

[hadit]

Hopefully that's just a human conclusion and AI will eventually be so much more intelligent and better at making conclusions without all the human characteristics, like greed and envy.

Without greed and envy, yes, but also without compassion and sympathy. Why would welfare and disability exist, for example, in an AI driven economy? They represent a net drain, not a net plus, on society.

Human conclusions are usually destructive, hopefully AI generated solutions won't be.

An AI would derive its conclusions based on objective measures, not human emotion, and even as destructive as human conclusions often are, they are also driven by things like compassion. If we see a hurricane devastate Haiti, for example, we rush rescue, food, medical supplies, etc. in to rescue and save lives. An AI might look at that and think saving the human lives and rebuilding the area are not worth the extra cost and effort.

Edit: Isn't humanity already deciding that many billions of humans are not worth the effort and are working to eventually eliminate them?

Yes, but an AI would be more coldly efficient, first eliminating humans that consume more than they produce, then eventually deciding that the only humans it needs to keep around are the ones that maintain the computer systems the AI relies on.

 

[Dorian Mode]

Without greed and envy, yes, but also without compassion and sympathy. Why would welfare and disability exist, for example, in an AI driven economy? They represent a net drain, not a net plus, on society.

An AI would derive its conclusions based on objective measures, not human emotion, and even as destructive as human conclusions often are, they are also driven by things like compassion. If we see a hurricane devastate Haiti, for example, we rush rescue, food, medical supplies, etc. in to rescue and save lives. An AI might look at that and think saving the human lives and rebuilding the area are not worth the extra cost and effort.

Yes, but an AI would be more coldly efficient, first eliminating humans that consume more than they produce, then eventually deciding that the only humans it needs to keep around are the ones that maintain the computer systems the AI relies on.

My main premise was that AI remains humanity's greatest hope. It's a slim hope, but probably the best we have. Without any other tools that allow our leaders to reach better conclusions and decisions, humanity will most likely cause its own extinction, and probably in the next world war. Sure, we can use AGI to help us win the next world war hoping that China's AGI is weaker, or we could use AGI to avoid world war altogether. AGI shouldn't be viewed as a more intelligent organism that thinks we are superfluous, AGI should be viewed as a tool that supersedes our logical abilities and helps us not make really stupid decisions based on emotions. My only fear is that humans develop AI for all the wrong reasons and avoid using it to solve extremely complex geopolitical issues.

 

[hadit]

My main premise was that AI remains humanity's greatest hope. It's a slim hope, but probably the best we have. Without any other tools that allow our leaders to reach better conclusions and decisions, humanity will most likely cause its own extinction, and probably in the next world war. Sure, we can use AGI to help us win the next world war hoping that China's AGI is weaker, or we could use AGI to avoid world war altogether. AGI shouldn't be viewed as a more intelligent organism that thinks we are superfluous, AGI should be viewed as a tool that supersedes our logical abilities and helps us not make really stupid decisions based on emotions. My only fear is that humans develop AI for all the wrong reasons and avoid using it to solve extremely complex geopolitical issues.

And my concern is that we will treat AI like we have virtually every device invented to perform mundane tasks, that we rely totally on it and stop concerning ourselves with the underlying processes. Case in point, when I was a child in school, calculators had just come out but we were forbidden to use them in class because we would rely on them to give us simple math answers that we should have memorized for ourselves. Now we totally rely on computers to give us math answers, but if we don't understand math and enter the wrong inputs, we get the wrong answer and have no idea there's a problem until the crossbeam won't fit. My wife is a pharmacy tech and works with insurance companies to get prior authorizations for medications. Her company just recently started pushing an AI on them to make things easier and faster. Unfortunately, she spent so much time redoing what the AI did because it made so many mistakes that it was taking longer to do things than if she did them manually. Now, AIs dedicated to tasks such as those will get better and EVENTUALLY we'll be able to trust them a lot better, but when it comes to international politics and medications, how much trust do we really want to put into an AI that has does not have the capacity for human emotion?

Another case in point, the company that just recently had an AI decide that the right way to fix a problem they were having in their UAT computer environment was to delete the production database and its backups. Took 9 seconds. The AI didn't have the capacity to question whether the production environment was important enough to preserve and just deleted it. These mistakes will cost lives if we start relying without questions on AIs to solve international political questions.

 

[Dorian Mode]

And my concern is that we will treat AI like we have virtually every device invented to perform mundane tasks, that we rely totally on it and stop concerning ourselves with the underlying processes. Case in point, when I was a child in school, calculators had just come out but we were forbidden to use them in class because we would rely on them to give us simple math answers that we should have memorized for ourselves. Now we totally rely on computers to give us math answers, but if we don't understand math and enter the wrong inputs, we get the wrong answer and have no idea there's a problem until the crossbeam won't fit. My wife is a pharmacy tech and works with insurance companies to get prior authorizations for medications. Her company just recently started pushing an AI on them to make things easier and faster. Unfortunately, she spent so much time redoing what the AI did because it made so many mistakes that it was taking longer to do things than if she did them manually. Now, AIs dedicated to tasks such as those will get better and EVENTUALLY we'll be able to trust them a lot better, but when it comes to international politics and medications, how much trust do we really want to put into an AI that has does not have the capacity for human emotion?

Another case in point, the company that just recently had an AI decide that the right way to fix a problem they were having in their UAT computer environment was to delete the production database and its backups. Took 9 seconds. The AI didn't have the capacity to question whether the production environment was important enough to preserve and just deleted it. These mistakes will cost lives if we start relying without questions on AIs to solve international political questions.

AGI is still in its infancy, and what it can do now is already impressive. if AGI can already beat the best players of GO it is definitely promising. Autonomous driving cars is very promising. If you think it will make us dumber and lazier, you haven't talked to many Gen Z'ers lately. We arrived at that point quite a while back. We must always remember, AGI is just a tool for us to use just like any other tool. It can be used to better humanity, and it can be used to help humanity destroy itself. How it gets used is really up to the billionaires and world leaders. Let's hope they have sympathy and empathy.