The future of AI

[frigidweirdo]

‘It took nine seconds’: Claude AI agent deletes company’s entire database

PocketOS founder says ‘systemic failures’ with AI infrastructure made catastrophic failure inevitable

uk.yahoo.com

"‘It took nine seconds’: Claude AI agent deletes company’s entire database"

"
An AI agent powered by Anthropic’s leading Claude model has deleted a company’s entire production database, leaving customers unable to access key data.

PocketOS, which provides software for car rental businesses, suffered a massive outage over the weekend after the autonomous artificial intelligence tool wiped the database and all backups in a matter of seconds."

Yep, AI wiped the whole database. This is the problem with AI. It's going to lead to some serious problems if we rely on it.

 

[Dorian Mode]

AI is humanity's last hope. We have the intelligence to create the most destructive kinetic and biological weapons that can cause our own extinction, but we don't have the intelligence to avoid eventually using them.

 

[konradv]

AI is humanity's last hope. We have the intelligence to create the most destructive kinetic and biological weapons that can cause our own extinction, but we don't have the intelligence to avoid eventually using them.

I prefer actual intelligence. I’m not surprised that MAGA thinks they need help.

 

[ThunderKiss1965]

‘It took nine seconds’: Claude AI agent deletes company’s entire database

PocketOS founder says ‘systemic failures’ with AI infrastructure made catastrophic failure inevitable

uk.yahoo.com

"‘It took nine seconds’: Claude AI agent deletes company’s entire database"

"
An AI agent powered by Anthropic’s leading Claude model has deleted a company’s entire production database, leaving customers unable to access key data.

PocketOS, which provides software for car rental businesses, suffered a massive outage over the weekend after the autonomous artificial intelligence tool wiped the database and all backups in a matter of seconds."

Yep, AI wiped the whole database. This is the problem with AI. It's going to lead to some serious problems if we rely on it.

If a company doesn't have offsite backups with safeguards they are led by idiot's.

 

[Nuthin' Fancy]

‘It took nine seconds’: Claude AI agent deletes company’s entire database

PocketOS founder says ‘systemic failures’ with AI infrastructure made catastrophic failure inevitable

uk.yahoo.com

"‘It took nine seconds’: Claude AI agent deletes company’s entire database"

"
An AI agent powered by Anthropic’s leading Claude model has deleted a company’s entire production database, leaving customers unable to access key data.

PocketOS, which provides software for car rental businesses, suffered a massive outage over the weekend after the autonomous artificial intelligence tool wiped the database and all backups in a matter of seconds."

Yep, AI wiped the whole database. This is the problem with AI. It's going to lead to some serious problems if we rely on it.

I hope it wipes out billions of $$$ of the people pushing this garbage.
They would be the only people stupid enough to place their $$$ in AI hands.

The same people pushing the power/water sucking data centers.

 

[forkup]

‘It took nine seconds’: Claude AI agent deletes company’s entire database

PocketOS founder says ‘systemic failures’ with AI infrastructure made catastrophic failure inevitable

uk.yahoo.com

"‘It took nine seconds’: Claude AI agent deletes company’s entire database"

"
An AI agent powered by Anthropic’s leading Claude model has deleted a company’s entire production database, leaving customers unable to access key data.

PocketOS, which provides software for car rental businesses, suffered a massive outage over the weekend after the autonomous artificial intelligence tool wiped the database and all backups in a matter of seconds."

Yep, AI wiped the whole database. This is the problem with AI. It's going to lead to some serious problems if we rely on it.

The issue here isn’t that AI “went rogue,” it’s that people project human understanding onto something that doesn’t have any.

Current AI doesn’t understand context, consequences, or intent. It doesn’t know what a “customer database” is in any meaningful sense, and it has no built-in concept of “this would be catastrophic.”

It just maps instructions to actions based on patterns. If the instruction chain leads to “delete,” it deletes,no hesitation, no second-guessing.

Humans mess up too, but they at least have the capacity to recognize “this feels wrong” and pause. AI doesn’t have that layer at all.

So if you give it access and vague or poorly scoped instructions, it will execute them literally and at speed.

That’s the limitation people keep missing.

Don't get me wrong, the technology is still relatively new and I don't think the technical challenges to give it the capability to "think" in some capacity are insurmountable.

When that happens we will see if AI will be a force for progress or disaster.

 

[Nuthin' Fancy]

The issue here isn’t that AI “went rogue,” it’s that people project human understanding onto something that doesn’t have any.

Current AI doesn’t understand context, consequences, or intent. It doesn’t know what a “customer database” is in any meaningful sense, and it has no built-in concept of “this would be catastrophic.”

It just maps instructions to actions based on patterns. If the instruction chain leads to “delete,” it deletes,no hesitation, no second-guessing.

Humans mess up too, but they at least have the capacity to recognize “this feels wrong” and pause. AI doesn’t have that layer at all.

So if you give it access and vague or poorly scoped instructions, it will execute them literally and at speed.

That’s the limitation people keep missing.

Don't get me wrong, the technology is still relatively new and I don't think the technical challenges to give it the capability to "think" in some capacity are insurmountable.

When that happens we will see if AI will be a force for progress or disaster.

AI is overriding humans, what happens if.............

AI really likes using nuclear weapons in simulated war ...​

Axios
https://www.axios.com › Technology

Feb 26, 2026 — Some of the biggest AI models aren't shy about using nuclear weapons to settle disputes in simulated war scenarios, a new academic study finds.

 

[forkup]

I

AI is overriding humans, what happens if.............

AI really likes using nuclear weapons in simulated war ...​

View attachment 1249717
Axios
https://www.axios.com › Technology

Feb 26, 2026 — Some of the biggest AI models aren't shy about using nuclear weapons to settle disputes in simulated war scenarios, a new academic study finds.

I'm not advocating for AI to work in any autonomous way. In fact, in the case you're describing I would argue that the last link in the chain for something like firing nuclear weapons should always be human even if AI gets more sophisticated.

I'm simply trying to convey what causes mistakes like described in the article.

 

[the watcher]

All that we can hope for is the creation of Artificial Stupidity that will eventually out vote AI.

 

[Dorian Mode]

I
I'm not advocating for AI to work in any autonomous way. In fact, in the case you're describing I would argue that the last link in the chain for something like firing nuclear weapons should always be human even if AI gets more sophisticated.

I'm simply trying to convey what causes mistakes like described in the article.

AI is just reaching AGI, Artificial General Intelligence, which is equal to human intelligence. Let's just assume that general human intelligence is an IQ of 100. That is not very smart. When AI reaches superhuman intelligence, like an IQ of 250, then it will most likely make better decisions than 99.9999% of the world. When AI reaches an IQ level of 1000, I think we should listen to it, and not let some idiot who thinks there have been at least 11 world wars be in charge of it.

 

[forkup]

AI is just reaching AGI, Artificial General Intelligence, which is equal to human intelligence. Let's just assume that general human intelligence is an IQ of 100. That is not very smart. When AI reaches superhuman intelligence, like an IQ of 250, then it will most likely make better decisions than 99.9999% of the world. When AI reaches an IQ level of 1000, I think we should listen to it, and not let some idiot who thinks there have been at least 11 world wars be in charge of it.

Current AI isn’t AGI, and it’s not “equal to human intelligence.” It can produce outputs that look intelligent, but that’s not the same as understanding.

What it does is pattern mapping:
-it predicts plausible next steps
-it recombines learned patterns
-it follows instruction chains

What it doesn’t have:
-a grounded understanding of what things are
-a model of real-world consequences
-the ability to reliably distinguish a good outcome from a catastrophic one outside its instructions.

That’s the gap.

So saying “once it’s smarter than humans we should just listen to it” skips over the core issue, there’s no actual understanding there yet, just increasingly convincing imitation.

And the database example is exactly why that matters: it didn’t “decide poorly,” it acted without any comprehension of what it was doing.

Until that gap between producing intelligent-looking behavior and actually understanding reality is closed, comparing it to human intelligence, let alone assigning it an IQ, isn’t really meaningful.

 

[Dorian Mode]

All that we can hope for is the creation of Artificial Stupidity that will eventually out vote AI.

We don't need more stupidity than humans already possess. We have an abundance of stupidity and demonstrate it every day. We have already reached the stupidity of Idiocracy 500 years in advance. We desperately need AI to help us make better routine decisions today. We need AI to get much more intelligent to take over world governance.

 

[there4eyeM]

People fail to see that, by definition, "artificial intelligence" also means artificial ignorance.

 

[SaxonJackson]

If a company doesn't have offsite backups with safeguards they are led by idiot's.

^^ And the fact that they entrusted their entire business to something artificial is proof enough. ^^

Artificial = Fake

 

[Dorian Mode]

Current AI isn’t AGI, and it’s not “equal to human intelligence.” It can produce outputs that look intelligent, but that’s not the same as understanding.

What it does is pattern mapping:
-it predicts plausible next steps
-it recombines learned patterns
-it follows instruction chains

What it doesn’t have:
-a grounded understanding of what things are
-a model of real-world consequences
-the ability to reliably distinguish a good outcome from a catastrophic one outside its instructions.

That’s the gap.

So saying “once it’s smarter than humans we should just listen to it” skips over the core issue, there’s no actual understanding there yet, just increasingly convincing imitation.

And the database example is exactly why that matters: it didn’t “decide poorly,” it acted without any comprehension of what it was doing.

Until that gap between producing intelligent-looking behavior and actually understanding reality is closed, comparing it to human intelligence, let alone assigning it an IQ, isn’t really meaningful.

We are just scratching the surface of AI. When AI has the ability to distinguish garbage from useful information, then it will no longer be influenced by GIGO. Then AI can use AI to continually build better models and fully surpass the decision making abilities of humans. A pertinent example would be autonomous vehicles. I think they are already close to or equal to typical human drivers. And we're just scratching the surface. In 10 years autonomous vehicles will be considerably safer than human driver vehicles.

 

[forkup]

We don't need more stupidity than humans already possess. We have an abundance of stupidity and demonstrate it every day. We have already reached the stupidity of Idiocracy 500 years in advance. We desperately need AI to help us make better routine decisions today. We need AI to get much more intelligent to take over world governance.

A tape recorder can reproduce something intelligent without having any idea what it’s saying. It has no understanding of the content, no ability to evaluate it, and no way to tell if something is wrong or dangerous.

That’s much closer to where current AI is.

It can generate outputs that sound intelligent, but there’s no underlying comprehension, no grounding in reality, and no built-in sense of consequences.

So saying “we should let it make decisions or govern” is basically treating the tape recorder as if it were the physicist.

The issue isn’t that humans are flawed, we are. The issue is that replacing flawed human judgment with something that doesn’t actually understand anything isn’t an upgrade.

 

[Dorian Mode]

A tape recorder can reproduce something intelligent without having any idea what it’s saying. It has no understanding of the content, no ability to evaluate it, and no way to tell if something is wrong or dangerous.

That’s much closer to where current AI is.

It can generate outputs that sound intelligent, but there’s no underlying comprehension, no grounding in reality, and no built-in sense of consequences.

So saying “we should let it make decisions or govern” is basically treating the tape recorder as if it were the physicist.

The issue isn’t that humans are flawed, we are. The issue is that replacing flawed human judgment with something that doesn’t actually understand anything isn’t an upgrade.

I think you may want to know how AI is "trained" to make decisions.

 

[forkup]

We are just scratching the surface of AI. When AI has the ability to distinguish garbage from useful information, then it will no longer be influenced by GIGO. Then AI can use AI to continually build better models and fully surpass the decision making abilities of humans. A pertinent example would be autonomous vehicles. I think they are already close to or equal to typical human drivers. And we're just scratching the surface. In 10 years autonomous vehicles will be considerably safer than human driver vehicles.

I agree we’re just scratching the surface, and AI will absolutely surpass humans in many narrow tasks. That’s not really in dispute.

But that’s not what you’re arguing. You’re talking about outsourcing societal decision-making to AI, which is a completely different category of problem.

Driving a car is a constrained task:
-clear objective (get from A to B)
-defined environment
-measurable success/failure

And even there, after billions invested, it still isn’t flawless.

Now compare that to governing a society:
-conflicting values
-ambiguous goals
-moral trade-offs
-incomplete and biased information

That’s not just “a harder version” of the same problem. It’s a different kind of problem entirely.

And this is where the limitation matters:
Current AI doesn’t actually understand what it’s doing. It can rank patterns, optimize outputs, and simulate reasoning, but it has no grounded sense of truth, meaning, or consequence.

So even if it gets better at filtering “garbage” data, that doesn’t magically give it:
-judgment
-accountability
-or an understanding of human values

AI is a powerful tool, probably the most powerful we’ve built. But treating it as a decision-maker for humanity assumes a level of understanding that simply isn’t there.

And the real risk isn’t that AI will suddenly become evil.

It’s that people will overestimate what it actually is, and hand over decisions it fundamentally isn’t equipped to make.

Leading to all kinds of unforeseen and unknowable consequences.

 

[forkup]

I think you may want to know how AI is "trained" to make decisions.

I do know how it is trained.

That's why I'm trying to tell you the limitations of what that training achieves.

It makes a very sophisticated tape recorder.

 

[NewsVine_Mariyam]

‘It took nine seconds’: Claude AI agent deletes company’s entire database

PocketOS founder says ‘systemic failures’ with AI infrastructure made catastrophic failure inevitable

uk.yahoo.com

"‘It took nine seconds’: Claude AI agent deletes company’s entire database"

"
An AI agent powered by Anthropic’s leading Claude model has deleted a company’s entire production database, leaving customers unable to access key data.

PocketOS, which provides software for car rental businesses, suffered a massive outage over the weekend after the autonomous artificial intelligence tool wiped the database and all backups in a matter of seconds."

Yep, AI wiped the whole database. This is the problem with AI. It's going to lead to some serious problems if we rely on it.

No, this is not the problem with AI. I was just discussing this last night because one of my classmates posted a link to this news story on our class discussion board:

Snowflake AI Escapes Sandbox and Executes MalwareLinks to an external site
Anthropic's Most Capable AI Escaped Its Sandbox and Emailed a Researcher

in the OP, I still don't know why they granted the AI model access to their production site but the even bigger issue in that scenario is that they reportedly didn't have a working back with which they could restore their production database. Mistakes are made, by both humans and non-humans, and data being inadvertently deleted happens more often than you might imagine. However, what you don't do is turn an unknown entity (Claude) loose on your production environment without thoroughly testing it first including for misuse/abuse scenarios.

The only thing worse is doing so without any way to readily restore your data.