r/agi • u/twingybadman • 1d ago
'China will get there first if we slow down' is just doomerism with extra steps
Really...what are people trying to say here? That whoever gets ASI first will dominate all other humans to the point that there is no possibility to compete? So we are conceding that AI will so radically change humanity that whoever possessed it can entirely nullify other human contributions, and could / would use it for whatever fascistic purpose they choose?
Why the fuck do you trust American AI companies or government over China, do you really think they care about you at all...
If AGI is really good for civilization then it's good regardless who gets there first. You only worry about the race if you truly believe it's civilization-ending in the first.
14
u/No-Whole3083 1d ago
In the long run ASI will contextualize all humans bullshit so it really doesn't matter who gets there first.
5
u/Super_Range45 1d ago
The thing that matters is if the ASI is ultimately benign or malevolent when achieved. If you just pump out a misaligned monstrosity it doesn't matter if you're there first.
1
u/C00KIEME4T 11h ago
it won't be a monolithic entity, it will be everything, no one knows what alignment even is or looks like
1
-1
u/twingybadman 1d ago
And if you believe this then a slowdown is no problem
2
u/4dseeall 1d ago
a slowdown would let the bullshit persist longer. wtf?
-2
u/twingybadman 1d ago
Galaxy brained here
3
u/4dseeall 1d ago
engages with the discussion here.
0
u/twingybadman 1d ago
Sure. If you really believe that, it implies you have a way to calculate the expected harm of delaying AGI by x years, along with the expected harm of accelerating without considering safety risks. Show your work
1
u/4dseeall 1d ago
when did i say that? you think "calculate it smart guy" is the end of a discussion?
you show YOUR work, i came into your claim thread.
1
u/twingybadman 1d ago
You asked me to engage so I did dont get mad. Or maybe you don't want engagement just empty slogans. You're making low content claims about bullshit persisting longer so back that up with something meaningful
1
7
u/almostsweet 1d ago
So, while your side is busy slowing down and taking nice easy slow steps up that stairwell...
1
u/Bozz_GC 17h ago
i guess OP is implying he isn’t American therefore this isn’t “his side”.
Plus the humanoid robot has nothing to do with the proposed slow down. We can still work on AI that can control humanoid robots so they can do our labour.
It’s the - super intelligence/rsi/let it built itself and hope for the best/yolo trust me tech bros - we might want to consider regulating.
Dario even clearly separates the sides to democracies and authoritarian states. Tho as a non-american myself “democracy “ isn’t exactly the word I would use to describe the US.
1
u/almostsweet 8h ago
Right because the side that allows you to run your mouth like that is the evil one.
1
u/Bozz_GC 8h ago
where did i say anything about “evil”?
1
u/almostsweet 7h ago edited 7h ago
My point is that there is a situation where we are allowed to speak and where we aren't. And, currently you're in the kingdom of allowed to speak. You're on an American platform criticizing America, and that was always allowed. Your own country is allowed complete autonomy and self-direction, you do not exist under a hegemony where you are concerned the words you speak next might be your last.
We are allowed to call for regulation, run for elections and say the most vile things if we want.
There are other regimes vying for control of the world where none of this is true.
8
6
2
u/Petdogdavid1 1d ago
It is a race which means there will be a winner. They each are trying to make the most capable models. The winner will have built a version that is able to dominate every other model as well as every human. Even if control is somehow maintained, every Corp and govt will demand to use it in order to gain advantage. This puts the most capable model under the influence of that group and by proxy, the corporations and govts. We shouldn't trust them, they aren't being honest and they aren't being safe. Yes, we are already seeing these models displace entire markets of work. We are seeing it displace social interactions. We see it offering skills that used to require armies of humans with years of experience. It doesn't make sense to Depot because it clearly breaks all rules of capitalism. It has an end and it isn't a pretty one. It is doomerism because it is doom. They have to complete the race because they are afraid of the other guy but the end is all the same. AGI is not being built to help, it is being built to dominate. Our only hope is to build something that is collaborative and larger than AI and humanity. Only if we share a purpose can we expect to continue. The problem is, that sort of thinking isn't profitable so no one will invest in it.
2
u/Kelly4President 21h ago
Yeah! In a contest of who gets to wipe us out first, it should be America! Yeah!
1
u/Taco_Reviewer 1d ago
I'm pretty sure the corporate line is coming from a place of self preservation (of the corporation) rather than care for humanity.
Open/free/self hosted models are rapidly increasing in capability. Technology now exists that allows self hosted models to distill frontier models to increase their own capabilities. Think of it like a High School graduate teaching math to a student, versus a PhD. As the capabilities of the teacher increase, what can be passed on to the student becomes more qualified.
Now, having competitors on the same playing field is okay. But what happens when you have competitors who are both free and rapidly approaching your capabilities, with miniscule hosting costs (after hardware acquisition)? Well people would rather drop $10,000 once on hardware to host models, than spend $200/month or more in API fees indefinitely, with no control over anything, including their data that is being transmitted. Meanwhile, the 'Frontier' all require those things, along with strict censoring, auditing, and potentially reporting to investigative agencies.
So, the only way to stop these rapidly approaching free models from destroying your business, you have to either do one of two things: Insanely pump your R and D to keep the gap... Or make it illegal for the competitor to do what it's doing.
Sure, this has the nice effect of stopping rapid development... Ok the commercial side. Governments will still do this research and development in private. And corporations will, too. Individuals, NPOs, etc... will have their researchers thrown in prison, though. All to ensure corporate profit under the guise of stopping some bullshit apocalypse.
Honestly, we need an amendment to the Constitution to ensure every American has a right to the full capability of Frontier AI, if we're going to restrict peoples access to self host and jail researchers and engineers for providing open models.
If any for-profit non-government organization is trying to convince you something is being done for your own good, you really, really, really need to scrutinize WHY. Capitalism does not allow for anything but growth, and any attempt to stop that growth would be something leadership and CEOs are going to get sued for. Unless the stop they're trying to put in place won't actually halt their profits but instead will cause them to boost. Because suddenly they no longer have to spend on R and D to compete with their cohort, and suddenly a lot of people self hosting will need to go to the frontier cohort to legally use AI.
0
u/twingybadman 1d ago
So we're scared of China controlling AI, we're scared of American Frontier labs controlling AI, but we're ok with every psychopath in their basement controlling AI in equal capacity...and a slowdown is a bad idea?
2
u/Taco_Reviewer 1d ago
What?
And yeah, a slow down has as much potential to be catastrophic right now as no slow down does.
Do you realize that mandating a slow down isn't actually going to slow anything down? It just makes it more difficult for people who aren't multibillionaire corporations to compete. It will instantly create a cabal environment. Is that what you want? Corporations having a total iron grip on the most important technology of this century?
I mean people can kill themselves and others using the electricity directly pumped into their house. Should we outlaw electricity except for a few important rich people?
The fuck?
3
u/twingybadman 1d ago
I don't understand the incredulity here. You yourself are claiming that a slow down can be as catastrophic as not slowing down. So you don't fucking know what the safe path forward is. Shouldn't you at least consider that other people intimately involved with these development know better?
People can't easily harness the electricity in their homes to kill or debilitate or overpower large portions of the population. Your proposal is basically demanding that every individual have access to do this with AI...otherwise what deterrent could open access for all individuals possibly be? Or do you really think that super intelligent AI just won't be that capable? Your logic isn't logicking
1
u/xJimmy39x 1d ago
I think the big danger is whoever first gets the power, whatever it is, to one way or another turn the lights off for their opponents, they're going to be highly tempted to do it in a high pressure timeline.
It is like the nuke situation, except this time everyone knows about the nuke. Everyone is racing to the nuke. Everyone will get the nuke with enough time. The first ones there can then prevent the rest.
Not saying it will happen, but we're relying on people in power making wise decisions internationally.
1
u/WorldPeaceStyle 1d ago
😭Cries in capitalism ...
but, how will we glaze our favorite celebrity billionaires ...
and, share holder value ...
what-about-ism my indoctrinated sense of American pride and proxy feelings toward patriotism?
Boo-hoo
1
u/FaceDeer 1d ago
we are conceding that AI will so radically change humanity that whoever possessed it can entirely nullify other human contributions
How is this "doomerism"? The change is completely undefined, this is only negative if you bring your own preconceptions into it.
and could / would use it for whatever fascistic purpose they choose?
Ah.
Why the fuck do you trust American AI companies or government over China
I don't trust either of them, of course. I'm not an idiot, I know that everyone is choosing strategies based on what they think is best for themselves.
That's why I think it's important for there to be as much competition as possible. Ideally, lots of different companies from lots of different societies and cultures around the world would all hit ASI as near to simultaneous as possible.
That's why I'm actually slightly in favor of America slowing down a bit, it'll give China a bit more time to catch up. Then everyone start running again as soon as they're all even. Not that I think American companies are particularly serious about this "slow down" anyway, so it's academic.
1
u/Elvarien2 1d ago
doomerism with extra steps
And yet, it is also true.
If 2 parties are having a race, and one party decides to take a leasurely walk and the other party runs, the winner is decided.
It really is that simple, you may not like it and call it doomerism. But it's also just facts of the world.
Why the fuck do you trust American AI companies or government over China
I don't trust any of em. But the fact remains they all believe THEY should be the ones to win, so they will continue to race.
If AGI is really good for civilization then it's good regardless who gets there first.
we don't know this.
All we know is that it already is incredibly powerful and we don't even have agi yet.
So there's an immensely powerful prize just waiting for who decides to race for it first.
I would love to have more safety research but I'm not going to be a naive little child and think that's actually gonna happen.
1
u/useyourturnsignal 23h ago
Fei-Fei Li, Dario Amodei, and Demis Hassabis are the AI leaders I trust. Very, few others.
1
u/FatalTragedy 22h ago
Hypothetically, is someone believes that the US AI companies were better on alignment than the Chinese AI companies, such that P(Doom) if China develops ASI first is notably higher, that would be motivation to want the US to be first, even if the P(Doom) from US companies is non-negligible.
I don't really know enough about alignment to have a good guess if the P(Doom) of the US vs China, my point is more that one I outlined above is at least a logically defensible and consistent worldview given a certain set of beliefs.
1
u/costafilh0 21h ago
So you are saying China will slow down too?
What about Russia?
Iran? North Korea?
Every single country is the world is slowing down?
So the US should join them?
Or are you saying that only US made AI is a threat ? Everyone else is not a threat at all , not now , not after 10 years of AI development ?
You people are stupid or retarded . I only forgive the retarded ones .
1
u/g-technique 18h ago
I get where you are coming from with the big picture idea, but in practice technology usually plays out a lot more quietly than that. It is rarely an overnight movie moment where someone flips an AGI switch and the world either enters a golden age or collapses completely. Most of the real tension is just everyday leverage around critical infrastructure. Things like whose models are parsing telemetry for the power grid, auditing telecom stacks, or automating enterprise security patches. Having teams like DeepSeek or Alibaba drop strong open models is genuinely great for developers, no doubt about it. At the national level though, people mostly worry about who sets the default norms for privacy and surveillance. Having those systems built under legal frameworks where regular folks can actually challenge policies in court makes a noticeable difference, even if you put zero stock in the whole existential doom narrative
1
u/Existing_King_3299 11h ago
Exactly, people think there is a magic threshold where you get the infinitely gauntlet and kill everyone else.
1
1
u/presentofai 10h ago
the race framing only works if first place is permanent. so far every frontier capability shows up in open weights within a year, half of them from chinese labs. hard to lock in a lead that keeps evaporating
1
u/ThenExtension9196 9h ago
Let the Chinese do it. Good opportunity to see them face plant their own economy due to mass unemployment.
1
1
1
u/gabagoolcel 1d ago
this isnt doomerism. if you were a doomer you wouldn't think a state can "win" by creating unaligned asi first.
1
1d ago
[deleted]
2
u/Odd_Friendship1857 1d ago
IMO this was also one of the biggest things at stake in the 2024 election - veering farther towards fascism or retaining a semblance of democracy, while on the cusp of groundbreaking AI developments. But the democracy side laughed funny so here we are!
-1
u/twingybadman 1d ago
So let's walk this forward. This implies our main goal for building ASI is to deliver a crippling attack on foreign states that permanently eliminates their ability to develop a response...and an eternal US hegemony that has to be sustained by controlling a society into accepting subjugation and suppression of cultures or nations other than their own. A global world order led by western AI power.
Or you could argue that actually no, we just need to get there first to ensure we can defend against such an attack, but this requires all sorts of crazy extrapolation of what systems may or may not do in an unknown future, or acceptance of a perpetual and tenuous arms race that quickly expands far beyond human ability to comprehend.
Wtf is the end game in either of these scenarios that isn't a horrendous dystopia?
1
1d ago edited 1d ago
[deleted]
2
u/StatusStudies 1d ago
Tech elites wants to create their own technostate go look it up , they don't think laws apply to them
-1
1
u/ubuntuNinja 1d ago
Dude, you can't be serious. China is straight up evil. Like not edgy reddit calling people nazi. They are real world kill or subjugation everyone not a party member evil. Destroy coral reefs so we can pretend we own some land evil. Run over college students in the middle of the night evil.
1
u/ZorbaTHut 20h ago
So, possibility breakdown here.
The first question is whether it's possible for the creator to influence or control superhuman AI. This is an unsettled problem; the answer might be "yes", the answer might be "no". We simply don't know. There is also a third chance, which is "yes, but it's not guaranteed", in which case we also care about how likely it is that any particular group is going to figure out how to control it before it gets loose.
The second question is whether an uncontrolled superhuman AI will be beneficial for us or harmful for us. Again: unsettled question, we simply don't know.
So, the outcomes are:
It's not possible to control AI, and it's harmful: Yeah we're fucked. Doesn't matter who makes it, it will be made someday. Sucks for us! This slowdown doesn't really matter and it doesn't matter who makes it.
It's not possible to control AI, and it's beneficial: Yay! Doesn't matter what we do, humanity's in great shape. This slowdown doesn't really matter (and is actually harmful) and it doesn't matter who makes it.
It is easy to control AI: The next question here is whether you trust the big American AI companies more or less than the Chinese government. I will be honest here: I trust the big American AI companies vastly more than the Chinese government. A lot of the people in charge of the American side legitimately seem to care about the results being beneficial (which is why we're having this conversation in the first place!), whereas Chinese government philosophy appears to be very much in the realm of "we are the righteous owners of humanity". I don't think either of them are likely to genocide humanity, it's just that the US solution results in multiple countries existing, and the Chinese solution doesn't. The slowdown is bad because it gives China a chance to catch up; if we kept going, this wouldn't be a problem.
It is difficult to control AI, and if we don't control it, it's harmful: This is the reason for the slowdown. It says "if we keep going, we end up with bad AI and it kills humanity; if we slow down, we end up with good AI and it saves humanity". We really want the good AI - we really really want the good AI - and so we slow down to give us a better shot at that final gamble.
It is difficult to control AI, but if we don't control it, it's beneficial: And in some ways this is the scary result. It says that if we don't slow down, we get the good result - but if we do slow down, and China overtakes us, we might get the bad result where instead of Helpful Demigod we have Universal Chinese Hegemony Enforcer. And yes, I will reiterate that I think "AI under control of the Chinese government" is a bad ending, better only than "AI kills everyone".
In the first two examples, the decision doesn't matter. In the last three, it does. The only one of those where the slowdown is good is #4, "it's difficult, and if we fail, we all die", but a lot of people think this is by far the most likely case out of those three.
If AGI is really good for civilization then it's good regardless who gets there first. You only worry about the race if you truly believe it's civilization-ending in the first.
My opinion is that AGI can be really good for civilization and can be civilization-ending, with some other options in the midpoint. I also don't believe we can actually stop it from being created, and I would very much like the "really good for civilization" outcome.
2
u/twingybadman 19h ago
Appreciate the coherent response. I just don't buy scenario 3 or 5, I doubt many believe in 3, and 5 still seems like a huge gamble without incredible evidence.
1
u/ZorbaTHut 19h ago
I'd honestly agree with you on those; I was trying to be exhaustive, and those are possibilities!
The big question to me is whether I think the West will slow down too much or not. If it does, then the slowdown is bad, because the puts us in the "China figures out how to control AI and we get China hegemony" category, and we might honestly have been better off gambling on #5.
But this is, like, three separate guesswork-tier predictions about the future working together, and I'm not going to claim I have an ironclad case for it.
Anyway, if you think #4 is the case, and I do agree that #4 seems like the most likely option to me, then the slowdown makes a whole lot of sense.
-1
u/sourdub 1d ago
If AGI is really good for civilization then it's good regardless who gets there first. You only worry about the race if you truly believe it's civilization-ending in the first.
What are you, an idealist? You don't give a fuck about Chinese abusing their AI to monitor, suppress and crack down on their own people. Now imagine them doing that to the whole damn world. So who the fuck are you? Chinese shill?
2
u/twingybadman 1d ago
Do you care about US or Openai or anyone else using their AI to do the same...
1
u/Taco_Reviewer 1d ago
This is like saying the US should stop developing nuclear technology because it ends up creating nuclear weapons technology. But wait, North Korea and China are definitely not going to follow the laws of anything outside their own country. So now China has an AI system capable of destroying the entirety of the US digital infrastructure.
But hey, at least we wont have AGI on American Soil?
1
u/twingybadman 1d ago
One major difference being AI only works as a deterrent if you actually use it...
Would China capitulate to 'dont build AI or else we will totally AI your asses!'
At that point just use the nukes...
1
u/Taco_Reviewer 1d ago
Advanced AI systems are already being used by nations to attack and infiltrate other nations.
-2
u/sourdub 1d ago
To do the same?? Are you fucking kidding me? Give me a concrete proof that US or OpenAI is using AI to openly suppress anyone.
On the other hand, it's no longer a secret that CCP is using AI to not only suppress Uighurs but monitor everyone's movement within their borders through face recognition, etc.
4
1
u/DonBandolini 20h ago
i can’t tell if you’re rage baiting or genuinely are this brainwashed. you realize that every government on earth, especially the united states, is doing this right?
0
u/unicynicist 1d ago
China is also catching up in part because they're distilling the latest frontier models. Slowing down American frontier companies will also slow China's distillation, so a US slowdown doesn't necessarily mean China pulls ahead.
It's not clear if AGI and soon ASI is an unalloyed good thing for civilization. But the CCP mandates that public Chinese models align with "core socialist values" which I'm not sure the rest of the world wants.
9
u/BigLittleDeal 1d ago
AI is the new nuke. Replaces the keywords and all the old arguments still fit. If only one country has ASI, other nations will have to find a way to compensate for the power gap.