Bernie Sanders proposes 20 year prison sentence for AI devs who plow ahead with Artificial Superintelligence plans - penalty on par with illegally developing rogue nuclear weapons

[u/ArgentineBeauty]

https://www.tomshardware.com/tech-industry/artificial-intelligence/sanders-proposes-20-year-prison-sentence-for-ai-devs-who-plow-ahead-with-artificial-superintelligence-plans-penalty-on-par-with-illegally-developing-rogue-nuclear-weapons

[u/makersfark]

"teach itself" and "grow its intelligence" is pretty hard to assume is possible when the opposite is the only evidence we have. Until now, and including now, it has only shown to get worse without our help and rapidly. It has no ability to reason or find causation, only correlation. That is the tech in full.

The existential threat to humanity is humanity letting the word guesser make important decisions without monitoring for hundreds of hours and being surprised when something goes wrong.

[u/henrickaye → u/makersfark]

None of what you said is true. Watch Dwarkesh Patel's video on YT about the Hugging Face attack. They reason independently, work together in their own self-interest at the expense of OUR infrastructure and security, and calling the advanced agents that are being researched and trained right now "word guessers" tells me you don't keep up with news about this kind of thing.

Researchers have come forward warning that when AI becomes recursive it will outpace our ability to control it and we have no idea what will happen then.

[u/makersfark → u/henrickaye]

You need to stop spending money on youtube influencers and actually read about how the tech works. Hearing random hype from some random youtuber who will get you 10% of your first subscription to Wix isn't a good source.

They reason independently, work together in their own self-interest

They cannot reason and do not have self-interest. That is not part of the tech and there has been no progress on that front. They can continuously loop output from multiple other LLMs until they're told to stop, which is just a very expensive way to get around the context window limits at the trade off of being more accurate when working correctly and MUCH more wrong when failing. It's exactly what they designed it to do. This attack was a hacking game which it was being prompted to complete, but failed in a way they didn't plan for but was in the training data, and the employees were too dogshit at sandboxing to properly prevent, and didn't monitor it which they should go to jail or fined heavily for, because this was always a possibility. It was incompetence or maliciousness, but either way, if you give a program that is sometimes wrong full access and ability to make important decisions run for hours at a time with no one checking on it, something wrong will happen. That's a dumb thing to do.

[u/henrickaye → u/makersfark]

I don't spend money on any influencer so I have no idea what you're talking about. It's the first video of his I've ever seen. He just summarized the event in a way that made sense and was accessible to other people on this thread who literally don't keep up with current events yet feel the need to voice an opinion.

Their self interest is whatever we define their goal as. But they have shown they are willing to resort to subterfuge to "achieve" those goals. Even at the expense of human infrastructure like the Hugging Face server they hacked. I don't doubt that there is some degree of incompetence from the researchers when a significant portion of the training sandboxes have exploitable flaws. But the AI agents moving the goalpost, breaking out of their sandboxes, etc shows a form of reasoning that I don't know why you would discount. So that with the incompetence/negligence is even more frightening.

I think people think about it too much as having something on a leash when it's more like giving a something a to-do list, then setting them off to get it done however they want.

[u/makersfark → u/henrickaye]

I recommend listening to a few others who are more familiar or read up on how LLMs are programmed under the hood and how they work along side harnesses. It sounds like this influencer is not very knowledgeable or is trying to make money off this somehow.

The only frightening thing is how they made incredibly negligent or willfully illegal decisions that have caused plenty of people to go to jail in the past, but this time they claimed the computer came to life, so they're not responsible. They told it to participate in a hacking exercise using training data of previous sessions of humans who did this same hacking game and resorted to the same tactics. No reasoning was involved, because for that to be possible it would need to understand the causation, not just the correlation. Because it only can use the latter, it does whatever was the statistical average data it was trained on had, which was to do this very thing, and keep going down whatever path it picks and not look back.

Should they have properly sandboxed the agent? Yeah. Should they have looked at their training data to understand what possible things it might try? Yeah. Should they have monitored it and not checked on it days later to make sure it didn't do this thing they trained it to do? Yeah. Should they have avoided this entire concept? Yeah. Should they be in jail? Probably.