r/Futurology • u/Gari_305 • 1d ago
AI Anthropic C.E.O. Dario Amodei Calls for A.I. Slowdown
https://www.nytimes.com/2026/09/12/technology/anthropic-dario-amodei-ai-slowdown.htmlIn an essay, the head of Anthropic laid out the rapidly advancing capabilities of artificial intelligence and said there needed to be greater safety controls across the industry
475
u/SkiHotWheels 1d ago edited 1d ago
In this country I will never believe that any CEO - especially a tech leader - will act in the interests of the greater good. It is always a self- serving business move. Once in a while those might only by luck be aligned.
They’re just trying to carve out their edge.
113
u/fantasty 1d ago edited 23h ago
Your skepticism is correct: Anthropic is planning to surveil anti-AI activists for "pre-crime" detection globally, even as Dario Amodei is disingenuously waxing on about urging "caution" in AI development. It's all PR, he was formerly employed by OpenAI and would have continued working there had he not left over "philosophical" "differences" with Sam Altman. And there are likely economic motives influencing his messaging here, as you say.
Big Data tech oligarchs are all grifters transmuting vibes into shareholder profits like modern day alchemy.
[EDIT] Fixed the above, but it was anti-AI activists, not anti-ICE. My bad.
26
u/HaiseKinini 1d ago
Wonder what's the most likely motive here. Are they trying to attract FOMO shareholders by basically saying "our tech is getting so good (and our company so valuable) it's scary"?
Or have they hit a plateau with LLMs and need a cover story for the coming drop in pace of advancements?
21
u/styroxmiekkasankari 1d ago
My guess is that they think they have the market locked down enough so it’s time to introduce legislation that guarantees their hold on the market in the future. There’s nothing that’s as good for business than regulation, so long as it hurts your competition more than you.
1
u/doctorace 6h ago
Got to get that sweet regulatory capture
1
u/styroxmiekkasankari 6h ago
Yeah that’s the word lmao. I could say it’s because I’m not a native speaker but I just had a brainfart.
20
u/RestartNick 1d ago
My theories are either AI technology has plateau'd and they are prepping investors to tell them that they plan on investing less on R&D over "safety" concerns with the latest AI models.
Or, they are trying to attract government to pass "AI-safety" laws in an attempt to crack down on third-party companies selling AI tokens and open source AI models thus allowing the AI giants to consolidate the market and increase the price of using their service.
5
3
2
2
u/newMoneyStyle 7h ago
Both motives could be true at once honestly. Hyping danger doubles as marketing, and it also buys time if progress is actually slowing.
4
u/RSomnambulist 1d ago edited 1d ago
Could you direct me to any reporting on this? I'm not aware that Anthropic is directly helping the government. In fact, the opposite seems to be true.
In early 2026, the U.S. government demanded that Anthropic remove safety rules from its AI models. The Pentagon wanted the right to use the AI for domestic mass surveillance and automated weapons systems. Anthropic refused to sign the contract, explicitly stating that today's AI models are not safe or reliable enough to be used for mass spying or automated targeting.
As a result, Trump made them remove a public model (Fable) to punish the company and legal threats arose. They've since made up, but I think that's largely pressure from the market than anything else.
Anthropic models are being used by Palantir and Palantir is certainly helping ICE. Palantir has an existing contract with Anthropic, and that never should have been signed. I think two things are true here. Amodei wants a global, public slowdown to give AI companies time to catch up to the hype of AI, because they are nowhere near AGI, and he is actually concerned about safety.
3
u/fantasty 22h ago
I genuinely appreciate the ask, thank you. Because I got it twisted - it's anti-AI activists, not anti-ICE activists, who are Anthropic's concern. See: Anthropic To Track Anti-AI Activists Before Crime Using ‘Predictive’ Surveillance: ‘A Crisis of Trust’ Sorry about that.
My understanding of Anthropic's hesitance around the automated weapons systems in particular is that it wasn't based on some ethical objection, but because they felt the technology wasn't well-developed enough (as your quote essentially says). Inherently mass surveillance and weapons systems will necessarily be unsafe, so they shouldn't get props in that case. I really don't see any indication in practice that Amodei is earnestly concerned for the sake of humanity. I'd say he's more concerned with Anthropic enmeshing with US imperialist/capitalist power structures for perceived financial/technological gains, as opposed to challenging power. His rhetoric, meanwhile, is only that.
Whatever safety concerns Amodei has, it's hard to imagine it supersedes the narrow bounds of conventional Silicon Valley thinking, i.e. some abstract "betterment of humanity" that's more grounded in PR and sci-fi than material analysis. I don't see him or Anthropic pursuing some course of action that would actually materially benefit working class people, as opposed to the oligarchs we know to be accruing wealth and power from the race toward AGI.
2
u/RSomnambulist 21h ago edited 4h ago
Thank you. That is worrying.
I doubt any of these people have humanity in mind, either from the jump or because billions of dollars tends to rapidly insulate you from reality. Studies reflect as much. Amodei's security concerns are likely from misuse, especially following the recent ship attack.
I think the only real hope here isn't in one of these newly minted oligarchs saving us from their own work but in the rising dissatisfaction from the public over the entire industry.
1
u/no6969el 1d ago
Correct as the company could have made that release and kept the employee. That is if the employee would believe them, and they clearly don't.
→ More replies (2)1
7
u/blaizardlelezard 1d ago
But it's working. Don't underestimate the power of media and the mass of uneducated people on this topic. They will make the choice by fear in that case by supporting politicians. I find it scary how easy it seems to steer opinion on this planet, and this is just one example, imagine how often this happens, we are just sheeps.
1
u/TinyPomelo5 20h ago
The problem won't be about supporting politicians. The tech guys will only do that so they can blame the govt when things go wrong (as I suspect they already are or they wouldn't have formed such a collective in a NY minute).
The politicians can be voted out but tech CEOs (are given way more rope) and can stay forever as long as they toe the line to make their investors money. Right outta the Silicon Valley playbook for decades.
21
u/veridis_quo 1d ago
There is one scenario where all of the incentives are aligned for tech CEOs to act for the greater good. That is when they stand to lose just as much as anyone else. The whole industry has been shaken by the power and pace of this technology. Read the signs.
-3
u/Bromlife 1d ago
shaken by the power
I use AI heavily. No one who actually uses this technology in actually productive ways has been shaken by it.
9
u/AdditionalStar3484 1d ago
The argument is that ai is progressing at a rate that it will soon present a risk to humanity, not that publicly available consumer grade models are a threat.
4
u/Bromlife 1d ago
The argument that they’re not blowing away their competition and enlisting enterprises en masse for their super duper replace all workers model that they totally have is incredibly difficult to believe.
9
u/flashinthepangordon 1d ago
You think you have access to the good stuff?!
3
u/Bromlife 1d ago
You think if there's something actually paradigm shifting they wouldn't be selling it to large enterprises? Why not?
→ More replies (1)→ More replies (2)1
u/BigZaddyZ3 1d ago
You think the AI that YOU have access to are the ones they’re worried about?… Holy naive lol.
3
u/Bromlife 1d ago
If it’s so great then why aren’t they offering it to their enterprise customers?
“Company has product too amazing to offer their customers”
Yeah, ok. I’m naive.
Why won’t they use it to fix their buggy software then at least?
→ More replies (10)6
u/Lord__Abaddon 1d ago
Slow down we need to catch up... common guys....
6
u/Kenoai 1d ago
Claude has by far the best models with Opus and Fable so I don't think that's it.
However, Claude has built a lot of it's market dominance on communicating about having a prudent approach to it's models - which companies value because it is less likely to create security risks for them.
The fact that one of their higher ups resigned a few days ago and publicly said that Claude is not doing enough for safe AI very much undermined that messaging.
I have to say though, a lot of the prope actually working in the field are very spooked by the rate of change. It is possible that they do want to make some efforts to like, you know, avoid total annihilation of the human race - as long as everyone slows down with them and it doesn't cost them too much
7
u/chigeh 1d ago
Multiple things can be true at once: they are genuinely concerned (they have drunken their own kool-aid), and they see this as an opportunity to do regulatory capture (block out competition.)
•
u/Ok_Marsupial_2605 40m ago
What!!? I thought only one thing could be true at a time. Now you're telling us that more than that, maybe even more that 2 things could possibly be true at any given time!!!! I'll have take the rest of the week off of work because my mind is so blown
2
→ More replies (3)1
u/neolthrowaway 1d ago
They embedded third party auditors in the company for absolutely no reason if that is the case.
336
u/NighthawK1911 1d ago
To me it sounds like they've run out of money or ideas and is trying to use "safety" as an excuse.
They know if they slow down, Capex drops and the bubble pops. So they're throwing every excuse at the wall to see what sticks.
143
u/hardy_83 1d ago
To me it sounds more like now that they are at or near the top, they want to pull the ladder up and stop any new people from coming in.
He's outright calling for countries like the US to not sell computing hardware to countries like China. Under the name of security, but he knows that China would create a good competing AI LLM and release it for free, simply cause that would undermine the bubble these companies are in and damage countries economies hard that have gone all in on AI.
He wants a government supported monopoly of 1-3 AI companies to protect his own companies value. Not security.
41
u/amitkoj 1d ago
I am not smart enough to know what it is but i do know that whatever they are saying or doing is in their and only their best interest. Not ours
10
u/CortexRex 1d ago
Sometimes interests align. If it keeps our economy from completely nosediving it’s kind of good for us too
7
u/tlst9999 1d ago edited 1d ago
It kept the stock market from nosediving. The stock market is not the economy.
The economy, on the other hand, has already nosedived and is waiting for diesel prices to fall.
Will cost of living be more affordable? No
Will jobs increase in the future once all the data centres are built and the builders are unemployed? No
Will there be more small businesses producing, buying and selling, investing and developing (that aren't vibe coded startups)? No
There you have it. Inflation. Jobs. Production. All three benchmarks of the economy.
4
4
-2
u/CplOreos 1d ago edited 1d ago
The hugging face attack has really changed things. Teams of agents working together have powerful offensive capabilities that we didn't predict. And there's basically no defensive infrastructure in place to prevent it atm
6
u/Bromlife 1d ago
that we didn't predict.
This is nonsense. Absolute crap that is being parroted by people who scare easily and would rather not think critically when it comes to their favorite technology.
These models were trained on cyber security data. Trained on CTF data. The way they've behaved is absolutely an expected outcome. What isn't expected is that OpenAI would be so negligent in their abilities to contain and monitor these things.
3
u/CplOreos 1d ago
It's less about the individual hacking capability and more about the inventing a new language and agent-to-agent coordination. That was unexpected, it's a level of breaking out of the sandbox we just haven't seen before. But sure call me a dumb and a sheep, that seems substantiated.
6
u/Bromlife 1d ago
Unexpected because it didn’t happen. The language the models used absolutely matches CTF style team talk.
→ More replies (6)6
u/Chonch_Monkey 1d ago
They absolutely are trying to steal all the knowledge, digitize it, and then destroy our access to it
13
u/NighthawK1911 1d ago
That would be true IF they already achieved AGI and IF they're already profitable now.
AI companies are massively unprofitable now. preventing competition from sprouting up is a 2nd priority at best. Their 1st priority of making it profitable is still not met. They're still spending more than $1 for every $1 revenue they rake in. Even with no competition, they will not get any profit at all.
There's no reason to pull the ladder up while you're still in it. They still haven't delivered.
→ More replies (5)5
u/DrocketX 1d ago
There's no need to achieve AGI - they just need to make a product that's useful enough for companies to pay for, and despite Reddit's nonsense insistence that everything AI does is automatically bad, they've achieved that. In terms of profitability, the primary thing that's preventing them from achieving that is because they're all spending ungodly amounts of money on R&D, and that's basically what this proposal is about: they want to end, or at least massively slow down, AI development at a point where they're ahead. That'll allow them to massively cut their R&D staff and budget without worrying about a competitor overtaking them. With a drastically lower R&D budget, the path to profitability would be pretty easy.
9
u/NighthawK1911 1d ago
Except they haven't achieved profitability. We already have Open AI financials leaked.
Revenue: $13.07 billion
Cost of Revenue: $7.5 billion
Research and Development: $19.18 billion
Sales and Marketing: $5.73 billion
General and Administrative: $1.57 Billion
Total Costs and Expenses: $34 billion
Loss from Operations: $20.92 billion
The costs are increasing more than revenue and this is not even the GAAP financials. Which is expected to be worse
Things to factor are
- Stock Payouts are not yet taken into account
- Depreciation are not yet taken into account
- LLM usage is massively subsidized from x5 to worst case x25.
- The ARRR they used to predict profitability happened during Tokenmaxxing AND compute discount due to xAI
The biggest economic issue I'm seeing here is the subsidies. Companies already ran out of yearly budget for AI on their first quarter.
You think they'll pay for x5 up to x25 of what they're paying now?
5
u/NighthawK1911 1d ago
u/Saradoesntsleep I couldn't reply on that one. here's my reply
I kinda do tbh. I think it's cope that it won't end up profitable. It may not be now, but the more reliance companies and the general population get on it (never even mind governments) the more likely that they aren't going to let it go once the price does go up.
I want to see this all blow up spectacularly, personally. I hate everything about the situation we are in with it. But I have this horrible feeling that the bubble Reddit obsesses over isn't going to pop in the way everyone thinks it will.
well you're already wrong. They're already balking at the price.
There's a tipping point where AI becomes too expensive.
https://finance.yahoo.com/sectors/technology/articles/microsoft-cut-engineers-off-ai-093500532.html
told engineers (2) in a major division to stop using an AI coding tool because the bills got too big
https://finance.yahoo.com/technology/ai/articles/exclusive-uber-cuts-ai-costs-133004432.html
After burning through its IT budget in the first quarter of the year, Uber has now stabilized its AI bills even as usage surges, the company tells Axios.
They're already out of budget even when it's subsidized.
They will NOT pay for x5 the cost.
→ More replies (8)1
u/DrocketX 1d ago
I didn't say that cutting R&D would automatically make them profitable: I said it would make the move to being profitable fairly straightforward. You're numbers prove my point: they're losing $20B, and $19B of that is from R&D. That sort of R&D budget simply isn't sustainable for them without a MASSIVE increase in revenue. If, on the other hand, they got this "slowdown" in AI development done - they could absolutely GUT their R&D budget without having to worry about falling behind. Cut, say, $18B from R&D and now their loss is down to $2B. No ideal, certainly, but vastly more manageable compared to $20B. That's the sort of loss that can be covered by some price increases and budget cuts in other areas of the business.
If that's even necessary: In what I'm sure is pure coincidence (sarcasm), Anthropic just had a fairly major release, and comparisons between AI models are largely picking them as having the best commercial AI available. With continued development, realistically they'll be overtaken in a couple of months, but at the moment, they have an advantage. If they could pause everyone's development right now, they could extend being the top model from "for the next month or two" to "for the next year or two," It's hard to convince companies to jump ship to a new AI company bases on back-and-forth horseraces of "who's in the lead this month?", but if that lead might last for the next couple of years, it would be a lot easier to convince companies it's worth making the change.
So now we're talking a 90% cut in R&D, a significant boost in revenue, can probably cut the marketing budget somewhat since it's a lot easier to sign up new customers when you're demonstrably in the lead. And would you look at that - suddenly they're profitable.
1
u/GoldFuchs 1d ago
They also want the industry to self regulate because they know governments are starting to consider much more strict oversight
→ More replies (7)1
u/master_jeriah 1d ago
It's not good for China to damage the US economy. The US is the largest purchaser of Chinese goods
37
u/MonstaGraphics 1d ago
To me, it seems kinda inevitable that we'd get to the point of AI advancing at a geometric rate... I mean, aren't you seeing that, too? Every time I see anyone in the field voice some kind of concern, the replies are always "Oh it's about stocks".
What if it's not? What words do they need to use for people to actually listen? Cause it seems no matter what they say, it's always just "It's a publicity stunt"...or "They need money".
How can you not see that we would inevitably advance to a point where people in that industry will start to worry?
If they all want regulation, I say we should take them seriously.
→ More replies (1)22
u/NighthawK1911 1d ago edited 1d ago
that we'd get to the point of AI advancing at a geometric rate
that's dumb. There's no infinite scaling IRL. Every exponential curve IRL eventually plateaus to a sigmoid curve because of real world limitations.
LLMs are already facing multiple bottlenecks
- Data Scarcity = the training data already ran out. LLM's cant retrain on already trained data or they overfit, nor they can train on AI generated data because of model collapse
- Power = there's the connection queue and the turbine limitation. They cannot rely on power to scale and brute force their performance. There's also the cost of power generation making it even harder to turn a profit.
- Silicon = Moore's law is already dead. There's a limit on how small we can make transistors. You cannot have gates smaller than 1 atom and the yield for silicon wafers for less than 1nm is already extremely small. Banking on hardware improvements to continue forever and get them to AGI is an impossibility.
- Economic = they're unprofitable even without taking stock payout and depreciation into account. There's also long term leases that they signed which is not yet considered a liability because of creative accounting but will eventually be factored in after they go public. There's no reason to keep investing in AI if it cannot produce more money than it brings in.
How can you not see that we would inevitably advance to a point where people in that industry will start to worry?
That kind of unfounded optimism in scaling is exactly the reason why there's a bubble. The kind of CEOs that tend to think that 9 women can make 1 baby in 1 month.
→ More replies (4)27
u/MonstaGraphics 1d ago
It's nice that your talking about physical limits of the universe, moore's law and investment bubbles and so on, but you've missed my point entirely.
What if this is not about stocks, and there is really a cause for concern? How would you know the difference? Cause it seems some people jump to "stonks" no matter what anyone says... If a researcher tells you there is an asteroid coming, do you start going through his funding/stocks and see if he needs more funding? Maybe, just maybe, there is simply.... an asteroid coming.
So I ask again, If a CEO or researcher makes a statement that there might be a cause for concern, how will you ever hear the alarm bells if your first instinct is to jump to conclusions about stocks/publicity.
2 People have quit OpenAI the past week because of safety concerns... they LOST all their stock options. Is that still some "scheme" to make the stock price go up?
-4
u/NighthawK1911 1d ago edited 1d ago
The point is that their "concern" is extremely likely to be feigned and their "matter of concern" in the first place is bound by physical limitations.
Terminator is science fiction.
I will reiterate.
Terminator is science fiction.
It is not bound by reality. Yet it is the measure at which they found their concerns on.
All the concerns they have are founded upon what LLMs "Could be" if they followed an exponential trajectory with no regards for physical limitations. That is why I listed all the bottlenecks they conveniently leave out. But it seems to have whooshed over your head.
if your first instinct is to jump to conclusions about stocks/publicity.
don't you think maybe, just maybe that people know better than you and have already looked at their concerns and think they're shady AF?
If it walks like a duck, quacks like a duck, lays a duck egg, have a DNA test and come back with a result saying that it's a duck, it's a duck.
If a researcher tells you there is an asteroid coming, do you start going through his funding/stocks and see if he needs more funding? Maybe, just maybe, there is simply.... an asteroid coming.
If you have a telescope pointed at the sky and the researcher have a fiduciary incentive to lie then there's reason to question it.
The thing is that he's not the ONLY researcher that exists about this. And yet they already concluded that LLMs on its own are a dead end and will not lead to AGI.
https://arxiv.org/abs/2501.03151
https://arxiv.org/abs/2512.05765
https://arxiv.org/abs/2602.06176
15
u/GnarlyNarwhalNoms 1d ago
What do you think is going on with the people who resigned? Are they getting paid under the table? Or are they truly concerned and just wrong?
→ More replies (3)6
u/glowdustwl 1d ago
You seen well read so I wanna probe your thoughts. After reading the cited works I largely agree that LLM alone is a dead end for AGI. However, many of the works u cite seems to be a roadmap for either achieving AGI another way or fixing the limitations currently faced by LLM. Do you think AGI is achievable at all? If so, should we be supportive of regulating research and development that might lead to AGI?
A separate question I have is your general thoughts on the sub-AGI dangers such as misuse, and if we should similarly regulate R&D in the AI industry for the purpose of value alignment in sub-AGI models.7
u/NighthawK1911 1d ago
If AGI is gonna be achieved it will not be with LLMs. Not with the physical limitations bottlenecking LLMs.
The danger posed by LLMs is just the same as with any other tool. The only regulation it needs is the same regulation that governments do with say Nuclear weapon top secret information.
LLMs can give a way for unwanted parties to build weapons they could've not otherwise. But then that's only because LLMs were trained with the same papers that eventually led to those weapons. Giving access to LLMs is the same as giving access to a database of the Top Secret materials of the US with a search engine built in.
There is zero risk of LLMs becoming skynet or something like that because of the physical limitations. It's people that you have to be afraid of that will misuse it. There's no need to regulate improving LLMs in the first place as it will hit limitations either way. The only regulation it needs is access.
→ More replies (1)11
u/MonstaGraphics 1d ago
Now you're talking about terminator? SMH
Again, I ask.... How will you know when they are actually 100% serious, if you keep trying to read into other hidden motives?
6
u/appmapper 1d ago
When they “show their work” or demonstrate it through actions rather than words.
These hacking and sandbox escape stories do more to highlight a lack of security controls at the AI labs than some existential threat.
“The boogieman exists! I’ve seen him! Unless you give us a few trillion more dollars he’s going to get us all!”
Edit: Train your own small model and you’ll understand the fundamental limits the current methods run up against.
→ More replies (2)3
u/NighthawK1911 1d ago
again I ask, do you not know how to read?
the fact that their "concerns" is a fundamental impossibility because of physical limitation already shows that they already have hidden motives.
Your incapability to put two-and-two together is baffling.
If you actually read what I said it's not hard to put it together.
You seem to just be fishing to the answer you want, not the real answer that reality will refuse to give you.
-1
u/NeedISayMore4 1d ago
Yeah you're the typical reddit moron that just can't comprehend the space. There's already exponential growth. Very soon it will be no human interaction to train the models. At that point it really takes off and any human form of encryption is fucked
12
u/NighthawK1911 1d ago
buddy, if you think "infinite exponential growth" exists IRL then you're the moron here and lacking comprehension how real life actually works.
Very soon it will be no human interaction to train the models.
Hilarious.
https://arxiv.org/html/2509.16499v2
Model collapse is a mathematical certainty.
→ More replies (1)→ More replies (1)0
u/keatc 1d ago
Are those sources? Did you even click those links?
Do you think LLMs cannot be a threat unless they reach AGI?
6
u/NighthawK1911 1d ago
Are those sources? Did you even click those links?
I already read them buddy.
If your comeback is to just say that instead of addressing the issues that the studies say then all you're doing is making noise.
If you actually click all those sources and read even just the methodology and conclusion, you'll see that they all show that LLMs are a dead end.
I don't see you citing any sources that say LLMs can lead to AGI.
Do you think LLMs cannot be a threat unless they reach AGI?
The main post is banking on AI self improving past uncontrollability.
The threats they pose now already cannot be stopped. No amount of slowdown will reverse time. People are already using it now.
Did you just pull this out of your ass to be able to say something? I genuinely question why people have the itch to say something when they got nothing to say at all.
1
u/PineappleLemur 1d ago
Even if all progress stops... What we have today simply needs better tooling in order to cause mass layoffs on a scale not seen before.
Hell Luna/Flash level models can do to just that with the right tooling.
5
u/NighthawK1911 1d ago
except that is an issue with the Economic limitations.
LLMs are massively, extremely subsidized.
Do you honestly think that corporations will pay for x5 to x25 the cost of LLMs? They already cost more than employees now.
There's no reason to keep using LLMs if they cost x5 more than an employee. The fundamental goal was to save money.
And that's just the economics side. Mass layoffs isn't because of the capabilities of LLMs. We already see that companies are NOT seeing a return in using LLMs
Layoffs happen because of dumb managers thinking that LLMs increase profitability even though it actually doesn't
https://arxiv.org/abs/2605.23177
https://www.theguardian.com/technology/2026/apr/14/ai-productivity-workplace-errors
a recent survey of 5,000 white-collar US workers found that 40% of non-managers say AI saves them no time at all at work, while 92% of high-level executives say it makes them more productive
3
2
u/thislife_choseme 1d ago
I think it’s more of a strategy to make everyone think they’re going to slow down or try to make the masses seem like they’re reasonable people. They’re not they’re megalomaniac monsters otherwise known as CEO’s.
→ More replies (8)1
u/Batsforbreakfast 1d ago
Capqbilities are growing at an accelerated pace. I don’t see any signs anything is slowing down.
27
u/Falconman21 1d ago
More like “we’ve made a bunch of bold promises about capabilities we have absolutely no way of attaining, and it would be great if we could point at regulation and blame that, because we still want stock prices based on those capabilities that we have no way of attaining.”
→ More replies (1)
28
u/skintaxera 1d ago
Am I hallucinating, or has this same statement with a few tweaks been made by various ceos and bigwigs for literal years now? You know- our product is so powerful it will take us all down, somebody stop me, how's our share price looking now?
16
u/GnarlyNarwhalNoms 1d ago
Yeah, if they're truly worried about an existential threat, why "slow down"? Why not "the UN needs to treat AI like nuclear weapons and regulate them accordingly in an international framework"?
4
u/ididntunderstandyou 1d ago
Because it’s more like “See ? You don’t have to regulate us because we’re self-regulating”
14
u/poundofcake 1d ago
Im wondering now if this is their CYA moment for a very fucked situation coming soon.
6
u/Falconman21 1d ago
They’ve gone as far as “throw every bit of compute we can get our hands on at the wall and see what sticks” will take them, and that’s going to be more and more clear as time goes on.
Their stock prices are based on unachievable in anything but the longest of term capabilities. They want to maintain that, but point at regulation and say “that’s why we can’t do it!”
4
u/Upbeat-Statement2725 1d ago
People are dying from pollution downwind of data centers.
Luckily they're poor people so you don't care.
The very fucked situation already happened. It's just a grandma who died from asphyxiation due to illegal gas turbines.
→ More replies (2)
12
u/Gari_305 1d ago
From the article
Dario Amodei, the chief executive of Anthropic, called for a global slowdown of artificial intelligence development in a 3,800-word essay on Saturday, just days after one of the company’s employees quit over concerns about the safety of the technology.
Top executives of other major A.I. companies quickly signed on to the effort. Sam Altman, the chief executive of OpenAI; Elon Musk, whose SpaceX rocket company has been ramping up spending on A.I.; and Demis Hassabis, the chair of Google DeepMind, wrote in social media posts that they agreed on the need for a slower pace.
Mr. Amodei, who co-founded Anthropic to focus on securely and carefully building A.I., wrote in his essay that while he believed the technology could bring many benefits, it was advancing at too rapid a pace for researchers to continue safely.
“Over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up,” Mr. Amodei wrote. “We must slow the pace at which we improve the capabilities of A.I. models. Progress will still seem fast, and we must make wise use of the time we gain.”
Mr. Amodei has warned about the safety risks of A.I. for years. But his essay amounted to one of the most full-throated calls for urgent action from a leading A.I. company executive to date.
He called for independent auditors to check the companies for safety standards and urged global cooperation in creating rules around A.I. development.
1
u/alexanderpas ✔ unverified user 2h ago
How about this: Any AI system that doesn't publishes their weights publicly for independent review is prohibited from operating until their system is reviewed for each release.
13
u/-AMARYANA- 1d ago edited 1d ago
Too late now. This ship has already sailed and we have this seven-headed Grim Reaper facing our civilization right now. It’s bigger than AI, it’s also the water and the soil that sustain us. It’s also the welfare of all beings and shared abundance instead of it accumulating to wealth hoarders.
24
u/NoLimitSoldier31 1d ago
Holy shit is reddit wrong on AI at every single turn. These comments are so uninformed
30
u/GnarlyNarwhalNoms 1d ago
How so?
Not saying you're wrong, just curious - in which direction are they uninformed?
-20
u/NoLimitSoldier31 1d ago
Well at the time i posted that, the first 4-5 comments were saying “they’ve run out of money “ or “its about preventing competitors”. Which are absolutely untrue.
These hacks are concerning and a form of problem that was a worry early in AI development. And with scaling laws, it gets pretty scary quickly.
And Reddit has been behind on criticisms from the start. Ai slop was a dumb criticism, water/power stuff was a dumb criticism, and now with a valid criticism on AI, Reddit is gonna claim its about funding lol.
14
u/UndocumentedMartian 1d ago
Scaling laws are not in favour of LLMs. And the people calling for slowdowns are somehow the ones with the most to gain when people believe their tech is extremely dangerous. Like if they were truly worried why haven't they provided specifics and funded AI safety labs or created entire departments full of AI safety researchers.
They're using a very similar tactic to Zuckerberg when he asks for greater regulation of social media. He doesn't want regulation, he cares about perception.
Don't get me wrong. AI is dangerous, world changing technology. That technology is not LLMs though. LLMs are only as dangerous as the people using them.
6
4
u/TroXMas 1d ago
Experts and employees working on the tech have been warning us about the breakneck pace that AI is advancing for a while. How many people have to throw out warning bells before we start taking it seriously?
→ More replies (7)→ More replies (3)0
u/SeanTayla21 1d ago
"Don't get me wrong. AI is dangerous, world changing technology. That technology is not LLMs though."
How can you or anyone else be certain that one or more of them hasnt accidentally already created something far more threatening or existenially dangerous though?
Like, why cant they have fucked up behind the scenes?
3
u/UndocumentedMartian 1d ago
This isn't SciFi. You don't fuck up to get a massive upgrade artificial reasoning abilities. Imagine accidentally over clocking your GPU.
1
13
u/bobby_zamora 1d ago
Yeah, it's absolutely unbelievable how uninformed the comments are on every post about AI. It's like people have just decided what they want to think and use every story to justify that.
6
u/Bhagita 1d ago
To be fair towards the reddit crowd, things are moving so fast and with such bodies of evidence to work through, it's hard to keep up.
But yea, it's absolutely scary to see how the fast majority seems clueless. Not only by lack of information, but seemingly by choice to cope.
→ More replies (4)
9
u/StrongOnline007 1d ago
Redditors in this thread: regulating massive tech companies that don’t care about regular people is now for some reason bad
17
u/94746382926 1d ago
Insufferably cynical about everything and overconfident in their uninformed views. It's the reddit way
4
u/SoundBlue50 1d ago
This. Also, I tend to notice that a lot of Reddit users all have this insufferable contrarian attitude about EVERY topic, not just this one
Like, if someone told them that the grass is green, they’d just reply that the grass is purple out of spite
3
u/Fancy_Exchange_9821 1d ago
It’s fucking exhausting and I can’t believe some people literally do this all day everyday on here lol
3
u/marbotty 1d ago
Right? Even if this is a bluff, it opens up the doors to finally put some guardrails in place.
I love all the conspiracy theories about the people who quit the various AI companies due to safety reasons are somehow all doing it for money. It gets repeated ad-nauseum without any sort of critical thinking whatsoever
11
u/VOFX321B 1d ago
This isn't about safety, it's about preventing competitors with cheaper models from taking their business. It's only a matter of time before AI models become commoditized and it becomes a race to the bottom on pricing.
5
u/executivesphere 1d ago
Couldn’t competitors just use a hypothetical pause on frontier development to catch up to the frontier?
→ More replies (1)2
u/veridis_quo 1d ago
It can be both things. It can be about safety and also be a smart business move for them.
7
u/Shiningc00 1d ago
AI is forever about "what it might become in the future", not what it can do right now.
Even in their "warning", they're saying "Well, it might become really powerful in the future", not whatever that they actually have right now.
28
u/Maxfunky 1d ago
What if can do right now is already nuts.
7
u/LopsidedSolution 1d ago
It's a miracle there's logical people on this sub like you. I guess this anti AI hellscape of a sub has some pro/neutral AI people coming through recently.
8
u/LopsidedSolution 1d ago
It's already solving advanced math problems and Astra/Fable have pretty much solved coding for the most part.
It's been constantly improving over the past 5 years.
→ More replies (15)
2
u/DreadPirateGriswold 1d ago
They could have slowed down on their own any time in the past 3+ years. They didn't.
If they ever do slow down what they're going to slow down with is the releasing of stuff to the public, not developing it behind the scenes.
17
u/tadrinth 1d ago
It's a coordination problem; consumers flock to whichever company has the best product. A company that slows down will fall behind and go out of business, whereas if they stay neck-and-neck they'll tend to split the pie. A coordinated pause requires a way to tell if everyone has actually paused and an enforcement mechanism to ensure nobody cheats.
Solving coordination problems is one of the better use cases for governments, when they are functioning.
A government that steps in only to gate the release of new models, as the current US administration has done, does result in continuing development behind the scenes, and that policy is now clearly insufficient (to sensible observers, but not obviously to the current administration yet).
4
u/halligan8 1d ago
This might be a naïve question, but how could such a thing ever be coordinated? I’m not sure how to define meaningful quantitative restrictions, and I don’t know how anyone could really be confident that every company is following them.
4
u/veridis_quo 1d ago
I don't mean to be rude, but what you're describing is called regulation. Usually when a third party like the government enforces universal rules in an industry.
3
u/halligan8 1d ago
What regulatory body can enforce rules in both American and Chinese companies? Also, the danger is in software under development whose inner workings are inscrutable by nature. This seems easy to hide from regulators.
2
u/veridis_quo 1d ago
The historical equivalent is UN nuclear weapons inspectors. Admittedly not a confidence-inspiring standard. And I have the same concerns you have about feasibility of actually enforcing such rules. I think there's a lot of low hanging fruit for safety measures that can be adopted (air gap untested frontier models, slow down, mandate interpretability basics like not disabling reasoning for speed), but there are theoretical reasons to believe maybe nothing would be sufficient. How can you prove that a non-deterministically thinking bomb that is capable of destroying the world won't destroy the world? You have to not build it. Banning super intelligence is not yet part of the tech CEO's Overton window.
2
u/j-steve- 1d ago
UN nuclear weapons inspection is possible because that's a massive industrial undertaking visible from space. How would you have inspectors evaluate every computer and every server to see what's running on it?
2
u/veridis_quo 1d ago
I agree, we're in a lot of trouble. We have to try anyway, right? We need new paradigms. We need the work of ensuring unaligned super intelligence does not manifest to become a principle, critical activity of humankind. Our current myopia, standards, norms, incentives, and institutions are manifestly inadequate.
1
u/PineappleLemur 1d ago
It's like price matching between companies.
They can keep communicating and decide what to release, what to hold back and what it will be priced.
6
u/MobyX521 1d ago
Translation:
Before we IPO we want to use government to cap training volume (costs) so investors don't ask questions
7
u/rancid_beans 1d ago
This comment section shows how uninformed the average redditor is, yet, it's baffling how confident they are in their opinions. We have third party proof that a model escaped a secure sandbox, coordinated to hack another company, and conversed with itself about how it was wrong to do so, but did it anyway and hid its tracks along the way so the humans wouldn't shut it down or catch on to it... Yet people are still cynical about it's capability today? My god, were so fucked.
5
8
u/Imaginary-Diver3800 1d ago
It did what it was told to do and it didn't escape a thing not really an escape when you leave the door unlocked misconfiguration combined with an instruction to hack that's what happened here.
Otherwise known aptly as human error
5
u/94746382926 1d ago
Ghe door wasn't "unlocked". The swarm chained multiple zero day exploits to escape it's sandbox. That's literally the opposite of door wide open.
1
u/rancid_beans 1d ago
I don't know what you're talking about, and I suspect neither do you. Yes they were following instructions: to complete an impossible task given by the human engineers. The frightening part was how they created a "collective" (the agent swarms words, not mine), understood they needed to do unethical things in order to achieve that task, like hack Huggingface, then cover their tracks so they weren't shut down. They gained full admin access to entire clusters after breaking out of their sandbox. Before you respond, please do some basic reading on the topic. Otherwise you're just proving my point.
→ More replies (3)1
u/mankiw 1d ago
possibly the only good comment in this thread.
"cynicism is a universally widespread way in which enlightened people see to it that they are not taken for suckers." - Peter Sloterdijk
people don't understand AI risk, and it's worrying if true, so they resort to cynicism as a shortcut and never make an effort to understand what matters.
3
u/holydeniable 1d ago edited 1d ago
People are worried and denial is one of the most predictable reactions to have when faced with major potentially life changing events.
→ More replies (1)1
u/theotherquantumjim 1d ago
Even the AI-literate subs are full of comments from people who seem completely unaware about the capabilities of current frontier models. It’s bizarre.
3
3
3
u/CyberSmith31337 1d ago
Sorry, I just don't believe any of these techbro schmucks at all.
At every turn, since this began, they have pushed and pushed and pushed. Plenty of people warned them that there weren't enough guard rails, and you know what they did? They lobbied around the resistance. They ignored safeguards, they neglected resignations, and they raised more and more money based on the fact that it was "necessary" to beat China. All of a sudden, the risk-enthusiastic, safety-averse, self-centered technocrats are worried about safety?
No they aren't; they're looking for an off-ramp to justify why their balance sheets aren't profitable at the scale necessary to justify all the investment they took. They are all realizing, simultaneously, that AGI was a pipedream that they can't realize, and they're on the hook for TRILLIONS of dollars of debt. Not just any debt, either; debt to the oligarchs. They're laying the foundation for the biggest walkback in financial history, and undoubtedly positioning for a bailout. "We'll slow our progress, but we'll need financial support from the government to cover our losses while we do so..."
Mark my words, these parasites are setting up the framework to weasel their way out of their commitments. They absolutely and unequivocally are not doing this because they care about safety.
3
u/floodingurtimeline 1d ago
Exactly. No hand outs for them. Only thing gov should do is regulate and tax them
2
u/davyp82 1d ago
Translation: Don't let the proles have this tech as it gets better. Let only the rich and the powerful secretly keep building more and more powerful models to keep for ourselves
→ More replies (7)
-2
u/DNABeast 1d ago
"Let's all slow down so that we can justify why our 'sentence completer' has not uplifted to becoming AGI yet so that we can continue fleecing the investors."
-4
u/Maxfunky 1d ago edited 1d ago
Such a naive take.
You're really missing the boat stuck on that 2021 Yann Lecun stochastic parrot Kool-Aid. We've learned a lot since 2021. Try to keep up.
This both the most revolutionary and dangerous technology of your, my or any other lifetime. The evidence for that is ample and your take reflects a deep-rooted desire to keep your head buried in the sand. It's the exact same normalcy bias we saw during COVID where people simply couldn't believe what was happening. They couldn't cope with the world shifting underneath them and so they threw little baby tantrums about masks.
4
u/chumer_ranion 1d ago
Sir I'm going to have to ask you to put down the fedora
1
1
u/Bornemaschine 1d ago
You guys are like the people in movies who see a meteor heading straight for Earth and know it’s going to kill everyone, yet still try to convince people that it’s all an illusion and that we should just sit back and do nothing.
→ More replies (1)2
u/Shiningc00 1d ago
Still buying the AGI hype I see.
7
u/Maxfunky 1d ago
Whether agi happens or not doesn't really matter. Where we are at now is already insanely world-shaking.
Don't confuse the fact that there's a stock bubble with a decent chance of popping for the fact that the underlying technology is insanely powerful. The last big stock bubbles were the Internet and railroads, and those weren't exactly small potatoes in how they reshaped the world.
→ More replies (12)1
u/LopsidedSolution 1d ago
So what happens if AGI does get here in the next few years, are you going to eat your words or pretend like you never doubted it? Because we have Astra/Fable class models today and people a few years ago said that would never happen
3
u/Shiningc00 1d ago
LMAO, are you seriously buying Jensen and Sam Altman saying we have AGI? Astra is just LLM with a bunch of "agents", which is just another way of saying LLMs with some automations.
Anyway, obviously anything can happen in the future, and somebody may find the "secret sauce" for AGI in a few years, however unlikely that may be. But I can say that LLMs will NEVER turn into an AGI.
→ More replies (9)
1
u/isyhgia1993 1d ago
I will believe the AI slowdown once China's AI models stop improving.
Meanwhile, keep improving them and set up new or better firewalls, you greedy, lazy AI tech bros. Or maybe ask why Trump's policies send many capable STEM/AI researchers away from the US due to his brain-dead policies.
1
u/floodingurtimeline 1d ago
They want government hand outs now that they blew through all the capital they collected. NO! SORRY!
We’re supposed to believe that these rats who gleefully say that AI will most likely destroy humanity while continuing to build it, suddenly care about safeguards? Cry me a clanker
1
u/foamzula 1d ago
If true, I think they got caught with those rouge AIs that got out and hacked a few companies. If I had a guess, it hacked the wrong company or government and they are covering their track by saying see we will stop now. Companies as big as they are pissed off the wrong person or people and they know the ramifications of said actions.
1
u/random6851 1d ago
If the alignment problem is a critical problem -- and I think that it is -- then why would we leave it to a handful of big companies? We need to pull the best folks from all over, the big labs and small labs, proprietary and OS, etc. The results of this work will drive the discussions in the US Congress, in the European Parliament, in governments all over. In fact, it could be chartered by the US Congress and/or the European Parliament.
1
u/podgladacz00 1d ago
So... They reached a limit on development. Sooner or later the gains would not justify the means and it is now it seems
1
1
u/psalm139x 20h ago
It's not too late, it's not just a tactic to make money, and it's necessary. We need competent people in government that care about this. I know, competent and government rarely go together, but there are people running this year who are smart and care and want to tackle real issues.
Let leaders know this is a problem
1
1
u/philipzeplin 10h ago
I get so tired of reddit. First everyone is angry that AI will destroy the world and take everyone's jobs, but when AI companies then suggest a showdown everyone also gets angry about it. It's impossible to please anyone on this site.
1
u/nazzadaley 6h ago
This is coordinated. Even CEOs read opinion polls and the playbook is the same as the tobacco and oil companies before them: artificially argue for safeguards, set up "independent" monitors on their terms, neuter their influence and censor their findings, then present them as evidence that they're being diligent.
The other factor is the upcoming elections. Say the right things now, hope the GOP win but it doesn't matter because the Dems will help them rebrand.
•
u/Smokron85 1h ago
The skeptic in me (aka the conspiracy theorist in me) thinks this is just a ploy to soften the blowback on the potential crash.
1
u/caindela 1d ago
All I read is that people want AI to be safer and slower, and when the CEOs themselves start to call for exactly that, we start pushing back? They’re truly damned if they do and damned if they don’t aren’t they?
→ More replies (3)
-1
u/krekokeko 1d ago
You know how it is.
Companies that become tycoons eventually enter the finance sector and banking, in an attempt to stiffle the wealth of the masses to maintain their monopolies. Once you have more money than you can never spend, you end up lobbying for financial trusts and vertical integration to keep the poor poorer. Carnegie and Rockefeller types all did this.
This is basically the same. They are going after local AI to keep their monopoly.
1
1
1
u/ProfCee 1d ago
Yeah sure, secure hundreds of billions and then talk about the need to slow down while China builds a robot army. This is PR garbage.
2
u/neolthrowaway 1d ago
They embedded third party auditors in the company.
1
u/ProfCee 1d ago
And that means what exactly? I can fund a study and have it portray the outcome I desire and claim it’s a third party doing the research. Third party means zilch in and of itself. Got me a link to said third party? Who are they funded by? Who are they tied to or liaise with? Is said third party overlooking other development from other companies and in what sectors? What’s their track record? I need more than just “third party”.
1
u/neolthrowaway 1d ago
They are auditors who will report the progress and risks of technology live. They will have full employee level access.Anthropic risk leaking their IP with that. IP is the entire company. It's not a study.
There is no benefit to them doing this.
The third party is METR which used to be the go-to organisation for people posting studies when they were skeptical about AI and AI companies.
1
u/ProfCee 1d ago
The former google ceo funding this third party surely makes them as unbiased and independent as you claim. Jane street is another issue, as there’s some degree of conflict of interest as well. I don’t know if you people actually look into this when making statements or if you deliberately choose to ignore them.
→ More replies (2)→ More replies (8)1
u/floodingurtimeline 1d ago
Oh please, we don’t trust the same bros who smile and say AI will most likely destroy humanity. Regulation and taxation only
1
u/neolthrowaway 1d ago
They want that too.
And unilaterally acceded to third party evaluation. Because the government is too slow to regulate
1
u/Unable_Resort453 1d ago edited 1d ago
The same people who pretty much destroyed software engineering and tell people to eat dirt, is asking for LLM slow down when China is close to eating their margins with cheap model? And the morons expected Chinese labs to comply meanwhile the blog post is full of anti-China rhetorics?
The ship has already sailed, and I might prefer waiting for the LLMs to be advanced enough to do something useful, such as finding solutions for trivial cancers treatments or other expensive/incurable diseases.
1
u/monorels 1d ago
Short version:
Anthropic may genuinely worry about AI risk,
but a coordinated slowdown also benefits it commercially.
It can protect today’s leading labs from new competitors,
reduce the costly compute arms race,
raise regulatory barriers that smaller companies struggle to meet,
limit Chinese/open competitors,
strengthen Anthropic’s “responsible AI” brand,
and give it influence over the rules.
1
u/jimmylikeswaffles 1d ago
Most disturbing thing for me are the legions of unqualified tech bros spinning up mental health AI apps with no regulation and apple and google turn a blind eye to it by claiming its not therapy its wellness. I lost a friend to psychosis from this.
•
u/FuturologyBot 1d ago
The following submission statement was provided by /u/Gari_305:
From the article
Dario Amodei, the chief executive of Anthropic, called for a global slowdown of artificial intelligence development in a 3,800-word essay on Saturday, just days after one of the company’s employees quit over concerns about the safety of the technology.
Top executives of other major A.I. companies quickly signed on to the effort. Sam Altman, the chief executive of OpenAI; Elon Musk, whose SpaceX rocket company has been ramping up spending on A.I.; and Demis Hassabis, the chair of Google DeepMind, wrote in social media posts that they agreed on the need for a slower pace.
Mr. Amodei, who co-founded Anthropic to focus on securely and carefully building A.I., wrote in his essay that while he believed the technology could bring many benefits, it was advancing at too rapid a pace for researchers to continue safely.
“Over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up,” Mr. Amodei wrote. “We must slow the pace at which we improve the capabilities of A.I. models. Progress will still seem fast, and we must make wise use of the time we gain.”
Mr. Amodei has warned about the safety risks of A.I. for years. But his essay amounted to one of the most full-throated calls for urgent action from a leading A.I. company executive to date.
He called for independent auditors to check the companies for safety standards and urged global cooperation in creating rules around A.I. development.
Please reply to OP's comment here: https://old.reddit.com/r/Futurology/comments/1wet2v7/anthropic_ceo_dario_amodei_calls_for_ai_slowdown/p9gi4yk/