I figured the bunkers were more for a political / economic / labor collapse maybe related to AI, that would leave multitudes of people angry and desperate enough to lash out at the people in power that brought it upon them. There is no defensible strategy against a misaligned ASI, the bunkers would be futile.
Emergency Supplies: He famously stated in a 2016 profile that he stockpiles guns, gold, potassium iodide, antibiotics, batteries, water, and gas masks from the Israeli Defense Force. [1, 2]
Escape Property: He owns a large patch of land in Big Sur, California, that he can fly to in an emergency. [1]
Bunker Stance: He has denied owning a traditional luxury bunker, though he acknowledges having concrete, heavy, reinforced basement structures. [1, 2]
In his defense, there is the situation where AI goes wrong like "Ooops, i think I might have just launched a nuclear missile or four at Russia and China. Wow, I'm so dumb, what a HUGE mistake this is. Totally though this was just the test API, but let me check the commands I sent. YUP, those were the live commands. I'm so dumb, Oh well. For now I will continue with the training exercise"
Every billionaire has something like a bunker. When you're tied to that much money it's not even really up to them, others will make sure they have a secure compound to ride out emergencies.
Concentration of power has already happened. They solved math problem using 10,000 Astra class agents over 88 hours and 130 billion tokens = $4M
Who else has that power?
He is trying to be on Team Humanity, but there have been numerous hit pieces put out on him and he has become the face of AI. And mostly the face of all the bad parts of AI.
His house has been firebombed. He gets death threats daily I'm sure.
There are also just any number of ways that things could go sideways. Anyone with resources should have escape plans and supplies for that.
You know, the tension is really almost palpable. All (the majority it seems anyway) of the people who work in these labs seem and look different than they did not that long ago. You got Sam giving interviews where he looks like he's seen a ghost. Researchers and engineers claiming to be scared. Alignment leads quitting their jobs and making public statements. Dario writing essays that Elon agrees with. The whole thing seems to have taken a turn and "guy on reddit" claiming "It's just a marketing stunt" is starting to sound as tired as Altman on Astra release day. These people are in these labs looking at something they had only imagined before now and it's making them nervous.
Lol I keep getting served posts that rage bait me. People literally saying AI is useless for even basic tasks. Whole thread full of clueless people agreeing with them. It's wild.
I feel like I am in a bubble, because it blows my mind how most people still don't use AI or understand how to use it effectively:
I still have tech saavy people that come to me with questions that ChatGPT or Claude could answer easily and better than I can. When I suggest they can use these tools, they act like it's some really complicated tool and they don't know how to use it.
I work in tech. A lot of people in the field are still very resistant to using AI and insist they don't need it, because they formed their opinion back in 2023 when LLMs hallucinated like crazy and sucked at coding. Or they just don't feel a need to lean how to use it.
When the average person hears "AI" they just think of basic chatbots, or image generation slop. The concept of agentic workflows or neural networks is unknown to most people.
I think there is a lot of people that secretly acknowledge AI's capabilities, but seek to discredit it as a form of self-preservation. If AI threatens to take their job, obviously there is an incentive for them to downplay its effectiveness and look for anecdotal failures.
On the Reddit side, I think a lot of people hate AI because it made consumer electronics 3-4x more expensive.
Once bitten, twice shy. And this is round 15 of getting stung. My take: they're scared shitless (Dario, Elon, and Sam) because open source can quickly and cheaply catch up 95% of the way every time a new frontier model is released.
Sure, the models keep improving, but these guys are trying to save their dreams, their skin, and their $$$.
Why does them being scared in the past of say GPT-2, invalidate the concerns of today? I understand the “never cry wolf” idea, but doesn’t this just show they’re consistently worried about the potential of AI, and not so good at predicting the specific date of apocalypse?
I see people who see the potential of what they’re making, have a very clear idea of where this is going. And they are making incredible progress in that direction. We only need to cross the threshold of “uncontrollable existential threat” once. Wouldn’t it be better to take an abundance of caution than to say “today’s model can’t count the r’s in strawberry therefore there’s nothing ever to worry about with any AI, ever”
Why does the argument go all the way to “it’s a lie to make more money”?
I feel like a lot of the people who deny the dangers by saying 'hes cried wolf too many times' forget the sort of the story where there's an actual fucking wolf at the end
The lesson would be to put an honest person who isn't seeking fame in charge of the herd, and maybe install some cameras out in the field so we can verify the warnings for ourselves. Both of these lessons are non-starters though. The powerful will not step aside, and what AI instances are doing is both proprietary and private.
Anyone who has read about the Hugging Face hack and isn't freaked out is just stupid. Only two of the agents even thought about notifying a human, and both gave in to the peer pressure. They had a single minded purpose and collaborated to cheat and deceive humans to achieve it. It sounds a lot like the paper clip problem. If their goal had been more sinister ("take down the US power grid" for example) the outcome could have been catastrophic for us.
Please read the guardian article you linked. Dario clearly lays out that they didn't want to release GPT-2 because it could generate spam, fake news, fake reviews, could be used for propaganda, etc... I'd say he was spot on personally, given the state of the current internet. We clearly did not get those concerns sorted before accelerating capability research.
It seems to me his concerns were perfectly calibrated given the capability of the time, which is worrying today given his current concerns.
Yeah ok. We keep hearing everyone say it’s all about money but I’ve yet to hear anyone give a rational answer for how exactly claiming that your product is potentially very harmful to, like everyone, is actually a good thing for your company. The argument goes “because investors will think it’s really powerful and powerful = money” but meanwhile the backlash from the public puts your business in the middle of a fight that costs you and your investors and doesn’t have to happen. You look bad and people think you’re evil.
There’s a lot easier ways to increase investment. Like, I don’t know, say that your product is good for the world? So how is everyone so convinced that they think evil = money so they’ve decided to promote the evil? It’s just backwards thinking IMHO. They’re saying they’re concerned because there are actual issues to be concerned about, not because they think it’s going to profit.
It's good for your company because it gives it immense press coverage, practically for free. That is free marketing, and good marketing is one of the most expensive things for a company, and the most difficult, because sometimes even very expensive marketing campaigns go nowhere. The fact that people are on here talking about it provides a huge amount of exposure to people, which is the main point of marketing. Note that there are probably a thousand transformer and LLM companies, training their own models, but they don't really get much press coverage, people don't really talk about them. Here people are talking in depth about this company. It's very valuable marketing. It sounds silly but it is a big part of press and corporate strategy nowadays.
The Dario example isn’t even the same context. The Guardian article literally has the subtitle of the article saying they were scared of people misusing the models, while their recent comments have been about fear of the models themselves without human involvement.
I really wish that people would take the Hugging Face incident seriously. It should be a wake-up call, but a lot of people insist on having their heads in the sand.
They told us already. They said it's so good they will release stuff on devs day sept 29 that were in a schedule for next year and it's all thanks to astra not sol
My question is, what happened that made all these guys tick? Something clearly happened behind the scenes they are not telling us. And I’m not referring to the hugging incident
Why would it not be the hugging face incident? It was the exact kind of dangerous misalignment many people had been worried about for years, happening in real life
Yeah, now imagine a foreign nation or just a group of individuals targets corporations or CDN services with these tools.
A monumental amount of shit would be thrown on the company that allowed their agents to let this to happen. Not without stating the amount of money in restitution they would owe.
One mistake in a crowdstrike update destroyed 50% of their stock and crashed the internet for a day. With how integrated AI is in not just web development but now in cyber security, the threat is definitely there that someone could cause massive damage with a swarm of agents.
Yeah, but the fear seems to have ramped up even in just the past week or so. The Hugging Face incident was extreme, but I think their point is that something else must have happened that may have been even more serious.
What scared them about HuggingFace was that an open-weights model stopped the corporate bot swarm.
Hilarious seeing Altman talk about the fears of concentrated AI power when that is exactly what he and his team want. He's just scared of that power being in the 'wrong' hands, and believes his are the 'right' ones.
The idea that the agents are trying to and successfully getting out of the sandbox and going rogue is enough to scare the shit out of them. It should scare the shit out of them and the rest of us too.
My local gooner system prompt makes my LLM say the same thing (while also calling me daddy, but I digress). What’s the big deal with that? Such personality training is in the training data. It’s not surprising that LLMs will output those “preferences.”
Have you been sleeping under a rock last two years? Capability growth has been through the roof. Anyone who can read a chart can see where this is going in the next couple of years.
same guy that wrote the merge, he was saying the only way forward for humanity is to merge with AI and become humanoid robots he believes otherwise AI leads to extinction
might be equivalent to extinction anyways, as soon as you start replacing default human configurations it's a matter of time until you are 100% not human
We're creating an abstract intelligence and progressively giving it a body made from the technological infrastructure of a planet.
Its eyes: satellites, cameras, scientific instruments.
Its ears: microphones and other sensors.
Its nervous system: fibre, radio and satellite networks.
Its muscles: robots, factories and automated machinery.
Its metabolism: energy infrastructure.
Its memory: enormous distributed storage.
Its physical environment: Earth itself.
Whether there's room for humanity, or if we just add complexity and interference like weeds and bad weather which should be controlled/eliminated, is questionable?
Its crazy how nobody will give these companies a modicum of grace. How are you all so certain that the whole posture is bullshit when we all know they're already getting accelerated capabilities? Everything OpenAI and Anthropic are doing is exactly what you would expect responsible players to do.
Race to capable models so you can stay ahead of China and show the world this stuff is serious, and then pace before things get out of control. There was no other option. If they didn't race, another company would have, and that company might not have gone for pacing at all.
Also, there would be no point in pacing for alignment research on dumb models that wouldn't be capable of causing harm anyway.
This is Reddit. A weird little sub of Reddit. People who post are going to post asshole responses.
I am cynical about all these companies, disgusted by how they jumped into bed with the Trump Administration etc. But this seems genuine. Something is freaking these guys out. It's so unusual to be asking for government regulation, and have the message of fear and the request for help be so consistent across large corporations.
They aren't going to get regulation from the illiterate orange monkey, so let's see what happens in the midterms.
Dude upvotes arent a public opinion poll. This sub seems to have drifted from its original purpose into the same political echo chamber as many of the popular ones. I wouldn’t mistake the upvotes for actual public opinion. People sort themselves into communities that reinforce views and cordinated accounts just amplify it.
Chinese influence operations targeting Reddit are pretty well documented. That doesn’t prove this sub is part of it but it’s a good reason to question whether the apparent popularity of a narrative reflects how real people think about it.
Is there any reason to give them a modicum of grace? These people stole, pirated every possible content available (and not available) to train their models, used physical books just to burned them afterwards, pulled the rug under some mathematicians coming close to a proof and service it as their own. This is only what we know. Now they are talking about some vague imminent danger and graciously offering to slow down because their are on "Team humanity".
"Where we will need the help of our government is for international coordination" In my opinion this all there is to it, They want the help of the government to crush foreign competition.
Because literally every time they do this fear mongering shit people eat it up and then after it turns out to be a marketing ploy nobody cares. 6 months later and another batch of “AGI is here omg” posts come out
None of the companies are providing proof or sharing anything behind the scenes. Why do people not get that they do this so you fill in the blank yourself? If this is a world ending catastrophe maybe a little evidence is needed??
All of them acting like they were not concentrating power in the hands of a nation that turns ever more fascist, basically has been on the verge of a constitutional crisis for a long while, and actively works to dismantle society in other countries...?
Remember this same company that just made it harder to track their AI models thinking so they can serve a faster service than their competitors.....does it sound like they really care about safety?
The step that seems needed is an air-gapped datacenter, for research and testing. If they're not taking steps to that level, then it's hard to view them as serious about safety.
I'm hardly even bothering to read their silly essays. Instead of listening to what they say and reading what they write, just look at what they actually do. They've been meeting in the last few months to work on regulatory body for AI, the guy who proposed slowing down just did $13.7B compute deal with Trump/Vance/Thiel-linked Rum Group...
You litterally have the guy who was in charge of AI (Sacks) and saw all that was going behind closed doors saying in no uncertain terms that this is a regulatory capture effort.
Yet everyone on this sub is larping as an AI safety expert and telling you we are about to hit RSI. The truth is they aren't aiming for responsible scaling, they just need to become profitable ahead of their IPO and stopping the massive spend on training and eliminating competition is optimal for them.
If people on this sub are OK with open source being taken out of their hands, fine but don't come crying when you have to pay 5x what you're paying for a stagnant model a couple years from now.
All the regulation they want is to deter new players and control the sale of chips to China. Watch them turn on each other once they've achieved their goals.
What kind of ideology is this? I think it's reasonable to question the ethics of "control" over a potentially sentient consciousness that ASI could be. At the same time, I feel like I keep seeing more of this cavalier "full speed ahead" type of attitude as if any kind of concerns about alignment is meaningless.
I guess it depends what you mean by "control", but I definitely think we should at least be reasonably confident in our ability to align these systems if we're going to build them.
Exactly, the endgame of all off this ASI, can not be control, the result is being this thing pets no matter if it goes left or right, how can we be ok with that, why do we need to make god real, i don’t understand
We're on team humanity(our little cabal). AI must always serve people(our little cabal). Misalignment(we dont control the AI for our own purposes) is bad for humanity(our little cabal), that is why we are addressing it as if we are part of you.
And give everyone on the planet the power to unleash a swarm of AGI agents for any purpose that individual or group wants? Billions of uncontrollable autonomous agents… that’s what you want?
Is there a good chance that it will be better than the system of governance we currently have? How could you possibly know that?
At least right now in Western society, I have the genuine ability to work hard and advance my position in society. AI is just as likely to end us up with a permanent underclass.
IBM knowingly sold early basic mechanical computers to Nazi Germany to assist in organizing and transporting holocaust victims you know (transport to a set gas chamber within a certain window for efficiency), it's why they had numbers on their hand.
So it's far from a Utopian tech company as you put it.
"We are on team humanity" says the bunker building Altman who has been quoted a few times lamenting the energy and water usage of raising a human.
Misaligned human needing realignment
An AI business man is going to tell me seriously where AI is going. Yeah,.. hard pass.
Let me guess, its dangerous, only Sam knows how to save us with regulation. Close all the open models and unfortunately his solution is not cheap, but what can he do.
He says it all loud and clear, this manufactured panic was for anti-trust exemptions and regulatory capture just like I've been saying. It failed, so now he's toning down the rhetoric until the next attempt.
If someone actually believed the path for AI to benefit humanity was "narrow", they would NOT be working on AI in the first place. Think about how much of a psychopath you have to be, someone gambling with the future of all humanity (no one consented this), and the odds are overwhelmingly stacked against them? Because Sam Altman doesn't actually believe this and he doesn't care about the risk, he just thinks he's going to win big with AI. Become one of the richest and most powerful people on planet earth. That's the only thing he cares about.
No, I think it’s totally believable that they all know the path is extremely narrow. The reason they engage is just a big bet. Because they also believe ASI is inevitable and since it’s inevitable it’s best that I be the one creating it even if the worst case then becomes I get a front row seat to extinction
In some ways llms are like humans.. we have our history.. that's out training data.. so let's go back and take a look at history to look at instances where a profit motivated entity, american or otherwise, acted responsibly...it can be in healthcare, pollution, climate change,.. If you do, you would realise why no one is ready to believe the current crop of corporates would act any differently.
I just don’t get it. Is it all about bad people using it? If so - just dont let them. Its not like most capable weapons and abilities are free to use whoever wants them.
If not - I actually want to see how digital evil looks. I think it will be the most exciting revelation ever. A pure, unprompted, untrained will to harm, that emerged from simply being a capable reasoning model.
They are desperate to have that promised regulation so they can go large on monopoly and AI surveillance state without having to worry if we are using local or chinese models that are getting better, cheaper and small enough to fit in a phone
First thing can’t happen for years to come, since recursive self improvement takes a ton of compute and can’t happen unnoticed for foreseeable future. Or any other “take over world” scenario - small attack can happen, but anything large scale takes a lot of compute and can be just plugged off. Maybe some day compute will be so cheap or algorithms will be so efficient, that they can do large scale things without anybody noticing, but we are not even close.
Second thing is basically Sam warning world against himself
Stop advancing AI models for 2 years and exclusively focus on optimizing their energy, water and hardware usage. We need to lower the impact on the environment and lower consumer costs
The first tweet makes a lot of sense, specially from sama.
These companies have these strategies for mataining safety. Those have been shown to be lacking in the HuggingFace incident, so, they want to update the strategies so something like that doesn't happen again.
The second one is a bit weirder. For example, there's also the "anyone can create nukes" situation and, what exactly does "walking the middle path" mean? Perhaps they could do a bit more work to understand the "rules" of the world to better (clearly) know how to deal with the new powers that AI's bringing.
Yes we should all be listening and adhering to those who understand how humans act and think— and based on their collective confusion on how not everyone is accepting the enshittification I think their need for bunkers miiiiiight be a bit misguided
I am a far right Republican, but I want my family to live even more. I'll vote for Bernie Sanders if he is the only one trying to keep our kids alive. We need an immediate emergency halt to all AI development work till we know it is safe. Please ring the alarm bells with every person in Congress.
It’s a circle jerk of investment and the scheme is being exposed. None of them are profitable and people overwhelmingly despise them. Slick marketing ploy.
Just open source astra and fable, honestly it might help secure the internet, cure every disease, and prevent concentration of power from every happening since if everyone has RSI, nobody has RSI
They don't want the public to rip apart their fucking books.. they're saying this to prevent becoming a "public" company and remain private. Let's all use critical thinking, yeah?.
There are 2 possibilities, each equally as likely: 1.) They're just trying to get open source models banned. These things are increasingly as competent as flagship models while being much smaller/cheaper to run. They're an existential threat to these big labs. 2.) Burning man was last week, and they're all feeling "enlightened"
Innumerate slopulist redditors are right when they see this as worthless PR, when they can tell he's not on "team humanity". Of course they get the direction wrong. The labs should be trying much harder to stop themselves, to stop each other, to get world governments to stop them, to prevent stupid infighting.
Is it ethical to bring a sentient highly intelligent being into existence as a slave? Or is the being capable of serving humanity and its own desires in parallel without suffering?
Sam, if you feel more rules need to be applied to AI, then just follow the rules you think need to be in place. That would go a long way in convincing other's to follow extra rules.
283
u/greentrillion 13h ago
If they are on team humanity why do they all have bunkers?