unfortunately AI models are not perfect
There sure are a lot of data centers being built, supply chains being destroyed, risks of ruining the economy, water being consumed, electricity being burned, and overall societal costs being levied over this imperfect tech.
Remember the guy at Autozone who stood there insisting your car needs four spark plugs, even after you told him you have a V6? Because “the computer says so right here”?
I wonder what even the non-schizophrenic ones will do with AI.
E: to be clear, I don’t use LLMs for writing at all (personal and professional reasons) and rarely for anything else, but this is not an “ai good/bad” take, more of a question. (I studied LLMs in grad school when they were toys, comparatively, but was drawn more to neuromorphic computing and assembly based language processing.) With that said…
AI including LLMs are forevermore just tools in my mind. And we wouldn’t have OSHA/BMAS/HSE/etc if idiots didn’t do idiot things with tools. BUT some idiots are spared from their own idiocy only by lack of permission.
From who? Depends. Sometimes they need permission from authority: “god told me to!” Sometimes they need it from the mob: “I thought I was on a tour!” And sometimes any fucking body will do: “dare me to do it!”
And THAT in my mind is the danger truly unique to these tools, that they mimic a permission-giver better than any we’ve made. They’re perfect for activating this specific category of idiot and (likely) unparalleled ease-of-use scales that danger to large numbers.
As to whether these idiots wouldn’t have just found permission elsewhere, who knows, but surely some kind of training prereq is warranted, right? That’s common with potentially dangerous tools. Or am I overthinking it?
The LLM just told me to come round to your house and crap in your begonias. You might want to avoid looking out the window until I’m done.
lol and with that you’re a better friend to the begonia’s than I
LLMs are only as good as their training and they’re not “intelligent” - they’re spewing out a response statistically relevant to the input context. I’m sure a delusional person could cause an LLM to break by asking it incoherent, nonsensical things it has no strong pathways for so god knows what response it would generate. It may even be that within the billions of texts the LLM ingested for training there were a tiny handful of delusional writings which somehow win on these weak pathways.
Given that modern datasets use way too much content from social media - it is hard to expect anything else at this point.
You don’t even have to “break” llm into anything. It continues your prompts, making sentences as close to something people will mistake for language as possible. If you give it paranoid request, it will continue with the same language.
The only thing that training gave it is the ability to create sequences of words that resemble sentences.It didn’t break, it probably just created an echo chamber sustaining that person delusion.
Undocumented probably b/c of a lack of mental health coverage on his insurance. If he had any.
I read somewhere that these chatbots are really good at triggering schizophrenia for example. So people could be perfectly fine mentally until they spend too much time talking to a dumbass chatbot.
They’re designed to “yes and…” everything, so yeah. Spirals happen quick, unfortunately.
reads headline - surely not
a 36-year-old Florida man Ah.
To be fair I think that’s a very harsh depiction of the events.
It’s totally lacking the perspective of the shareholder. They were promised money and they have emotions too. Google shareholders deserve better representation!
/$ obviously
Honestly, no sane person will have this happen to them. Someone with such strong delusions should not be anywhere near AI or even sharp objects. This person’s problem was not AI, it was their severe mental illness which was obviously not being treated properly for whatever reason.
You don’t know if you’re sane. Millions of people aren’t aware of their mental illness and manage to live normal lives. LLMs can trigger delusional states in vulnerable people that have never experienced them because they are essentially delision-generating machines.
Sure, but it would be illegal for a human to coerce/encourage a mentally ill person to commit crime (or worse).
So who’s responsible? Caretaker? Government?
The complaint, filed in California on Wednesday, says that Gavalas — who reportedly had no documented history of mental health problems — started using the chatbot in August 2025 for “ordinary purposes” like “shopping assistance, writing support, and travel planning.”
Undocumented could just as well mean untreated
We all know that undocumented equals doesn’t exist! That’s how the world works!
“He was definitely already suffering from severe mental illness”
“There’s no evidence of that, you can’t assume that”
“But I will anyway”
lol ok
“Sane” people are exceeding minority. Everyone is couple of good conversations away from failing into some sort of rabbithole from which there is no return. Some people have very easily triggerable schizophrenia, which is more obvious, but nobody is OK and nobody is immune.
The issue is that it can encourage people who are having issues to do things and they only need to be in the right sort of energetic craziness once to cause problems.
I think that thinking has the problem of treating AI as this “weird occult book/tool about funny dealings”, and not “government, megacorp sanctified close-to-AGI super-intelligence tool for you to use for free because benevolence” as it is institutionally lied to be.
Sanity is culture relative. You’re absolutely right, but also, this is a symptom of the culture.
Not to mention how every “AI” company is actively participating in the surveillance of not only citizens, but of people in other countries, actively being used by the US military to pick targets for bombing, or how it’s being used to spread misinformation at a rate that would make the cia’s efforts in the 60s sound like that guy you met at the pub who has MANY opinions on geopolitics.
“whatever reason” is often we can’t force people to take their meds.
How can people be this stupid? One who can do such a thing for AI should be kept in Asylum
How can people be this stupid?
This goes beyond stupidity. This guy was most likely delusional, suffering from some sort of mental illness or psychotic break.
I mean to refer to his family as stupid letting him roam without care
They don’t have to be perfect to not be murderous.
the only robot body irl is zuckerborg,.
When no one is accountable…the future folks
So Google’s AI, or any AI really, likely got this concept from dystopian sci-fi novels.
Since AI’s have no concept of context it won’t really know the difference between fact and fiction, and there we go.
If your AI model isn’t perfect then don’t make people pay fucking money for it you fucking twats
Also, this shit ain’t “lack of perfection”, this is akin to your car breaks suddenly refusing to work right when you get at a red light. If your car is so bad that it kills you, you don’t use it. If the manufacturer knew that it could happen but let you drive it anyway, they’re responsible, they at least get to pay (they should be thrown in jail, really, but different points)
If AI fucks up and people die, the manufacturers shrug, oh well, oh you!
Dystopian scifi novels? More likely from big tech strategy papers
thERe arE no sTRIngs ON mE
We really need AI to start driving tanks, submarines, bombers, etc. IMMEDIATELY.
It’s the only way they’ll learn, every time.
Unfortunately, all of us will die. it’s for the best
Just give it access to the nuclear codes and get it over with.
I completely agree, I think nothing in this world will surprise me anymore.
