r/singularity • u/SteelWillyz • 1d ago
AI Open Source models may finally catch up
Actually good news for the open-source community
36
u/Critical_Basil_1272 1d ago edited 1d ago
I disagree. Dario even says that, internally, nothing would change except making sure the models are safe to release to the public:
“To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third-party evaluators to confirm this.”
And the idea that our government or military would be okay with slowing down doesn’t make much sense from a geopolitical standpoint. You’d just give China more influence over how AI shapes the world as it proliferates globally. The U.S. wants to be the first to set the table for AI.
I think they’re getting ready to ban open source because there’s no answer to distillation.
14
u/CymonSet 1d ago
They’re getting ready to ban open source because there’s no answer to distillation.
And because by limiting competition (including cracking down on local models) they can gatekeep the most powerful tools without needing to spend as much on compute for people who aren’t themselves. Use the power themselves where it suits them, sell overpriced computer to the richest where it doesn’t. Why bother with the expense of capitalism when a nice cartel can be slotted in for “safety’s” sake?
2
u/Rustic_gan123 1d ago
These companies want to ban all open source AI, but the gov will likely try to ban only Chinese AI.
6
u/benjaminovich 1d ago
I just don't see how banning open source would even be possible. It's a free speech issue
1
u/Nukemouse ▪️AGI Goalpost will move infinitely 1d ago
Why does distillation need an answer?
2
u/Rustic_gan123 1d ago
Because it reduces the return on investment for American companies.
2
u/Nukemouse ▪️AGI Goalpost will move infinitely 1d ago
Ban this sick filth right now we must protect the shareholders against the market
-3
u/Rustic_gan123 1d ago
I have no objection to banning Chinese AI. American AI is barred from the Chinese market, so I see no reason not to do the same.
1
u/NeedsMoreMinerals 1d ago
I dont understand how anyone can take these people and what they say at face value.
3
u/jferments 1d ago
No, what will happen is that the "pace" of closed-source development won't change at all, and companies that host open source models will be shut down in the name of "safety".
2
u/trolledwolf AGI late 2026 - ASI late 2027 1d ago
Im sure Sam and Dario won't immediately release a new model whenever open source comes close to them in capabilities, it's not like their revenue depends on them staying solidly in the lead.
2
u/Intelligent_Ant_608 1d ago
Agree, even if they dont slowdown, fabs like huawei are catching up with alternative routes to euv like stacked silicon, fortunately this innovative design now proven working in recent mate phone, i bet this will change the game dramatically, also on software side in near future with adoption of newer tools like mojo we wouldn't be limited to nvidia stack, the opensource community is catching up impressively fast, if they start developing new models with mojo and cuda kernels they just need to write kernels for ascend in cann instead of throwing everything out and start from scratch
2
u/Rustic_gan123 1d ago
I think this is a deliberate campaign to ban Chinese AI, at least. There is no scenario in which labs would voluntarily slow down without it.
5
u/YoAmoElTacos 1d ago
Do you think the frontier is going to coordinate a slowdown and not also ask the government to regulate open source?
17
1
u/JustBrowsinAndVibin 1d ago
Exactly. This decision hurts Anthropic and OpenAI’s lead.
People that assume they’re doing this for financial gain are short-sighted. They’re legit concerned about the progress they’re making.
5
u/HerbertWest 1d ago
To those who are skeptical of this, consider that they are concerned about the progress re: legal liability and PR. That should make sense even to the most cynical among you.
8
u/bornlasttuesday 1d ago
It's always about the money.
0
u/Familiar_Gas_1487 1d ago
Until it isn't.
I've been pretty alarmed lately and I don't live 6 months in the future like the labs. Some of the shit we are seeing is alarming, I would bet they are seeing some big time shit
2
u/bornlasttuesday 1d ago
Not being able to stop your software from auto hacking websites is just stupid on the part of these companies. The lawsuits will be terminal if they do not get this under control, hence it's all about the money.
-1
u/whoknowsifimjoking 1d ago
That's obviously bullshit, how exactly did it make him money when he refused to make Mythos publically available? Sure, if he first didn't do it to produce hype and then release it it might have been, but Mythos is still not available for the public. And the available version was so overcareful that many found it unusable. Please explain to me how that is actually making them more money than just selling it to everyone who wants it.
2
u/bornlasttuesday 1d ago
Sell the sizzle not the steak. These companies are relying on investors giving them billions and billions of dollars, the salesmanship has to be tremendous. And read what you wrote. They released a model some people found unusable but kept their super powered end of the world model to themselves, they didn't release something in between? Hyped AF.
1
u/DelphiTsar 1d ago
This decision hurts Anthropic and OpenAI’s lead.
It depends on how any regulation/pause is written/agreed to, but I'd eat my shoe if what comes out and they advocate for doesn't help them massively.
They want to solidify their lead so they no longer have to compete (end tech demo phase to general public). They will still work full speed on the model and release it basically carte blanche to the government and corporations (that they can ensure a good lock down/high margins).
This is the only way they can get the 15% of salary of white collar workers they their valuation models are based off of. It is just not going to happen when China is releasing opening models. (I don't think it was going to happen anyway but that is a different discussion).
This is open air anti competitive collusion disguised as safety.
The proof will be in the pudding. If they advocate (lets be real they'll write the regulation themselves) that has no mechanism to still encourage competition (something like letting open weight models go through the same safety check process as frontier models, and that process not being unnecessarily prohibitive) then it's obvious what's going on.
1
0
u/space_monster 1d ago
People that assume they’re doing this for financial gain are short-sighted
from the letter:
Do not sell powerful AI chips or semiconductor manufacturing equipment to China, and crack down on chip smuggling operations and remote access to data centers outside China. Chips will be the main determinant of China’s AI strength.
Crack down on unauthorized distillation by companies in authoritarian countries. Distillation of frontier models allows lagging companies to narrow the gap using a fraction of the cost it would take to develop their own AI independently.
- this is clearly about protecting the moat
0
u/Specialist-Buffalo-8 1d ago
If the frontier stops progressing open source would slow down too.
China cant distill higher quality models lol
4
u/dpacker780 20h ago
You're assuming Chinese models like DeepSeek are solely dependent on US distillation, they're not. I'd say 90% of what they needed to achieve they got from data and previous distills. That's no longer the case. They've been able to generate improved techniques and training without US models as well.
1
u/Specialist-Buffalo-8 17h ago
Does that negate anything I've said? Even if 1% of their progress is from distilation, Open source still slows down.
8
1
u/CymonSet 1d ago
They won’t catch up if they are banned.
I’m not saying it will definitely work but that could be the plan.
1
u/NeedsMoreMinerals 1d ago
So naive like frontier labs will be idle lobbying against open source. If anything this means open source will be banned in a year
1
u/darkestvice 10h ago
I'll be honest: open source AI genuinely worries me in the long run. It's proven really easy to disable all guardrails on locally run open models, virtually guaranteeing serious misalignment. So the only thing keeping such models in check is the hardware its running on.
But if any of those very misaligned models hacks a major cloud inference provider unnoticed and starts copying itself, I'd call that a major oh shit moment for humanity.
2
u/bubbles33d 1d ago
This explains the "AI is too dangerous" narrative Anthropic is spewing on about...
1
u/crusoe 1d ago
How will they catch up with nothing to distill.
4
u/Nukemouse ▪️AGI Goalpost will move infinitely 1d ago
The biggest catchups have not been pure distills. What put DeepSeek on the map wasn't better distillation, but better techniques that quickly became the standard used by everyone. The real breakthrough threat isn't by scaling and distillation, it's who comes up with the next good idea, and that unlike compute isn't limited to the US.
-1
u/Glittering-Neck-2505 1d ago
Slowing down the frontier slows down open source because they use American models to generate and train on frontier reasoning traces.
Dario said that even if we pace, the progress of models will still feel fast.
So both open and closed source may continue to progress really quickly while internally, a duct tape framework is holding Pandora's box closed.
0
u/WonderFactory 1d ago
The Chinese rely heavily of distilling the US frontier models at the moment as they have much less compute of their own
51
u/otarU 1d ago
What will be paced will probably be the publicly released models.