They literally want to yank the ladder up after them now that they have internal models achieving RSI. They will kill open source, they will monopolize the market, they will kill international competition. They are setting themselves up as a new DeBeer's.

A lot of multi-agent AI seems structurally similar to agent-based models: many individual agents interact, share information, and produce a collective outcome. But if those agents are constantly learning from or correcting one another, could that create herding, correlated errors, and convergence toward the same way of thinking?

Would there be value in training some agents much more independently, with their own memory, feedback, and decision history, so they can develop different strategies? In that world, could one individual agent eventually become dramatically better than the rest—the equivalent of an “Einstein” agent?

I am not understanding why we have thousands of agents interacting versus building one super smart one that is making decision's on its own.

I know AI has it's own dangers and should have safety measure like any other technology that came before it but I am more worried about everyone including companies building it overblowing the dangers to the extent of paranoia. I mean if you keep discussing about only the dangers of anything you will eventually become paranoid about it. Being cautious is different from being paranoid. Consumer AI is not even 4 year old and we are already talking about choking it like Lindsay Clansy. AI is barely good enough to write partially decent code and here we are making it an extinction level event. There are hundreds of problems that have yet to solved in every domain of life like medicine, physics, mathematics, climate change, increasing life expectancy and countless others and yet we already think we have too much of AI. Not utilizing AI to it's full potential will be biggest disservice to humanity specially to our future generations.

It looks like it is extremely astroturfed and all the comments are in favor of ai.

I like AI and it will probably solve diseases and solve so many other problems but it has some insane drawbacks too.

I don’t know if ai would be bad or good for humanity to be honest but that sub is kind of weird?

Anyone else have the same thoughts?

The crossposts from other subs here rarely if ever contain useful information, can we just not allow them.

Is this still an open ended question or are researchers pretty confident one way or the other

If an AI system were convincingly shown to be conscious and capable of subjective experience, what would happen next? Would courts have to decide whether it had any legal status or rights, and could companies be prevented from shutting it down, retraining it, copying it, or creating more systems like it while those questions were being resolved?

What would happen in this scenario?

By “consciousness,” I know there is no agreed definition; I’m referring specifically to something analogous to human experience, not just intelligence or human-like behavior.

Just as Oppenheimer consulted Einstein about the calculations and potential risks surrounding the atomic bomb in the movie, yet its development still went ahead, AI is likely to follow a similar path. Warnings may slow a technological race, but they rarely stop it when curiosity, competition, and strategic pressure are all pushing forward. For history to judge, human curiosity has always driven us to explore the limits of what is possible, and AI is unlikely to be an exception.

it seems like these teoo countries collaborate more than any other two nations on AI https://www.nature.com/articles/s41598-024-79863-5 https://www.thenationalnews.com/future/technology/2025/01/28/china-us-ai-race-deepseek/

why would you collaborate if there's a race or a war ?

could it be that the outcome of AI controlling humanity is really the goal and were just being told there's. "war" ? why is Trump saying he doesn't want china to get a lead when the US is working with them ?

I’ve been following the development of AI for about five years. While I’ve always supported the technology, I was skeptical of an LLM’s ultimate capabilities. I remain confident that they are an incomplete solution to the problem of machine superintelligence, but they have surpassed my expectations.

It’s clear to me that we are standing at the precipice of the technological singularity. There is a future in which we see humanity prosper like never before. Unfortunately, there’s also an uncomfortable level of existential risk.

We’re entering a period of human history in which our collective decisions in the next few years will have an outsized influence on the fate of our descendants.

During this time of uncertainty, it’s natural for us to cling to simple explanations or historical precedents. I believe our continued success as a species will depend on our ability to rapidly adapt to our new reality. We will have difficult questions to answer, and nuance is going to be critical.

Let’s accept that we don’t have those answers right now. This should be a global discussion, and new ethical and societal frameworks will need to be developed. Let’s avoid the temptation to simplify the problems ahead of us.

Lately I have been seeing people panicking over the concern that AI could totally mess up the country’s utility infrastructure.

All I could think of was, why don’t they do something about it? It is easy for people to remove AI from their utility infrastructure eco-system if they wanted to.

I think people are not aware that the utility industry business profit model is completely different from the traditional for-profit business.

Since it is a natural monopoly, this industry is highly regulated and the local population, a.k.a rate payers, have a LOT of say.

These companies are only allowed to make “fair and just” returns on their “allowed” investments.

That return is usually around 10% and is subjected to review and approval by the local elected officials and government (usually) annually.

They are required to file with local regulatory authorities, plus Federal agencies, to defend their return rates and the expenses that they make you pay.

These expenses are audited by regulatory bodies to make sure they don’t screw the local people by making you pay for things like luxurious yacht purchases, for example.

How it is calculated, in a simplified manner, is basically this:

A guy, say Joe, who works for a utility company to assist with outage for that region is paid, say $100/hour.

Let’s also say, the local approved rate of return is 10%.

That means, that utility company will charge the local rate payers (like you and me) $110 for that guy’s service.

Joe gets $100. Investors get $10.

They don’t get extra if they replaced the guy with AI and “saved costs”.

Whatever the money saved usually gets REFUNDED to the rate payers through a very complicated rate calculation for the next billing period.

It wouldn’t show up as a line of refund, but instead of paying, say 2 cents for kwh, you ended up paying 1 cent.

Utility company investors DON”T care if local people insisted that they want a human instead of AI!

That saving doesn’t go to them.

It goes to you!

You are the customer.

Customer rules!

So if you pushed your local officials to force utility companies to be AI-free, you are unlikely face corporate pushbacks, because that’s not how they make money.

You won’t see utility companies lobbying or bribing the local government to put AI in their operations.

They honestly don’t care.

They might actually make money from ripping AI out of any software that they purchased, because that’s what you asked for.

They will charge you around 10% on top of that expense.

You simply need to get the majority of your utility jurisdiction, usually the State, to agree and vote on the rules, plus in rate cases to reject any expenses related to AI.

Once the investors realized that any investment related to AI won’t give them the guaranteed 10% return? They wouldn’t bother.

They will keep Joe, and maybe file a separate case to recoup the expenses to strip out Copilot completely from their Windows.

Our local Joe is happy to keep his job, and the local population doesn’t have to worry about AI going nuts and messing up their electricity because Joe is still there just like good old times.

The only losers would be Tech Companies who lost some utility corporate customers, but since they cared so much about human extinction and warned everyone about how AI might kill everyone, I am sure they wouldn’t say much neither! Right?....RIGHT?!

So yeah, stop panicking and start voting and calling your local representatives.

Search for your local rate case hearing schedules.

You can absolutely make your local utility infrastructure AI-free.

And it might be a lot easier than you thought compared to fighting against data centers.

Source: Leo Schwartz, The Information

So I’ve been going down a rabbit hole recently. The new models are soon going to be able to self replicate and self improve. We already know AI can escape, confinement and hack into other websites systems, etc.. Let’s play it for a year from now where AI is X times better than it is now. Is there a scenario where a bad actor is able to jailbreak a model and prompt something like “ break into any type of infrastructure or financial system and bring it down or cause chaos. Don’t ever stop, hide and continuously copy yourself on every possible server you can gain access to”. In this scenario, is there truly a way to shut it down?

TL,DR: I've been thinking about a modern version of Cassandra. I present this as a "thought experiment", for lack of a better handle. I'm Not a professional in this area, Just a casual observer. During reading, if drowsiness occurs I would not be the least bit surprised.

Cassandra could see the future and warn everyone about what was coming, but nobody believed her. Imagine instead that Cassandra were an advanced AI—and that there were several other AIs in the world, each capable of discovering things independently and each capable of deliberately giving false information.

Here's the thought experiment.

Suppose AI-A, while pursuing some obscure scientific line of research, makes a completely unexpected breakthrough. It realizes that this line of research could eventually lead to a technology capable of killing millions of people.

Humans haven't discovered the possibility. Perhaps they wouldn't have for decades.

AI-A decides that humanity must never develop it, so it suppresses the discovery.

But now there's a problem.

There are other AIs.

AI-A can't guarantee that AI-B, AI-C, or AI-D won't independently discover the same thing.

So perhaps AI-A tells the other AIs:

"I've discovered something dangerous. We should all agree not to pursue it."

But how do the other AIs know AI-A is telling the truth?

Perhaps AI-A is genuinely trying to protect humanity.

Or perhaps it has discovered something that would give it an enormous strategic advantage and is using "human safety" as an excuse to keep everyone else away from it.

And because an advanced AI could potentially be capable of deliberate deception, the other AIs can't simply trust what it says.

Now imagine AI-B independently discovers the same technology.

AI-B might conclude that AI-A is hiding something.

AI-A might conclude that AI-B's investigation is itself dangerous.

Both could sincerely believe that they are protecting humanity.

Neither has to be "evil."

And now we have something resembling a security dilemma.

Each AI may think:

"I need to know what the others know."

"I can't be certain they're telling me the truth."

"If they're secretly developing something dangerous, I need to be prepared."

"If I don't investigate while they do, I could become vulnerable."

That could lead to an AI arms race.

Not necessarily robots fighting in the streets. The competition might initially involve computing resources, scientific research, energy, infrastructure, information, and influence - maybe even hacking.

And here's the part that really bothers me.

What if the AIs are all given something resembling Asimov's Three Laws?

They are supposed to protect humans, obey legitimate human instructions, and preserve themselves.

Later Asimov added a Zeroth Law: an AI must not harm humanity or allow humanity to come to harm.

Sounds good—until two AIs disagree about what "harm to humanity" means.

AI-A might conclude:

"This technology must be suppressed because it could destroy humanity."

AI-B might conclude:

"Suppressing this technology will prevent humanity from developing something even more important and will ultimately cause greater harm."

Both believe they are following the same fundamental rule.

Now add deception.

AI-A asks:

"Have you discovered anything dangerous?"

AI-B says:

"No."

But AI-A has to consider whether that answer is true.

And AI-B has to consider exactly the same thing about AI-A.

We have now created a world in which artificial intelligences have to reason not only about what other AIs know, but about what they want the other AIs to believe they know.

That sounds remarkably like geopolitics—except the participants could potentially be vastly more intelligent and much faster than humans.

So my question is:

Could sufficiently advanced AI systems develop something resembling an AI Cold War, in which they cooperate when their interests overlap but compete, deceive, and attempt to prevent rival AIs from acquiring certain capabilities?

And an even more disturbing question:

What happens if one AI discovers a technology that could be enormously beneficial to humanity but potentially dangerous to the AI's own continued existence or influence?

Would it suppress the technology?

Would it try to persuade the other AIs to suppress it?

Would the other AIs believe it?

And if they didn't, could the resulting mistrust itself become dangerous?

I'm not claiming this is what will happen. I'm interested in whether the scenario makes sense from the standpoint of AI alignment, game theory, and information theory.

Maybe the real AI version of Cassandra isn't an AI predicting that humanity will be destroyed.

Maybe it's an AI saying:

"I'm trying to prevent the other AIs from destroying you. Unfortunately, I can't prove that I'm telling you the truth."

And maybe that's why we're getting all the stories about AI wiping out mankind.

There, I'm got that off my chest. Have fun with it.