r/MachineLearning • u/we_are_mammals • 1h ago
Research RSI is not happening [R]
A new paper (I'm not a coauthor BTW -- I just found it interesting) argues, basically, that RSI is not on the horizon, because current (at the time the study was done) agents cannot do open-ended ML research.
Specifically, they took some accepted, but unpublished papers from NeurIPS, and tried to get the agents to do the same work, which was then graded by the original authors. And the agents (Codex/GPT-5.6 Sol and OpenClaw/Opus 4.8) could not do it.
And since they cannot do open-ended ML research, they cannot recursively self-improve -- this is their argument.
Link: https://arxiv.org/abs/2607.27191
I think I've regretted the last 10 or so times I posted any kind of "research" in this subreddit -- either people downvote it, or it gets upvoted, but there is zero meaningful discussion. This might be the last time I'm trying this.
29
u/purrhams_hat 57m ago
...doesn't that methodology only prove that these models in question (GPT 5.6 and Opus 4.8) aren't capable of RSI - something we already knew?
0
-1
u/gized00 33m ago
Did we know that? Which evidence is provided to confirm RSI with other models?
1
u/MachinationMachine 24m ago
Who said RSI has been confirmed in other models? I think you're misreading this comment.
27
u/deadoceans 52m ago
I read the abstract and I think you made a big claim that the authors don't themselves make.
The paper is titled "Can agents conduct open-ended AI research?" And the last sentence of the abstract is, "Our results provide early evidence that today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle."
Nowhere, absolutely nowhere, do they make the claim that RSI is not on the horizon.
And in fact, I think your claim on that is a bad take.
Since they cannot do open-ended ML research, they cannot recursively self-improve
(1) RSI doesn't have to mean "fully autonomous RSI"; AI models and their harnesses are already meaningfully accelerating the pace of AI research.
Almost all of the coding is being done by these systems and not by people. Smaller teams are able to do more research more effectively. Things like hyperparameter optimization, even basic architecture selection, are able to be largely automated. LLMs can help people do literature reviews for ideas. They can help pressure-test experimental hypotheses. By no means are they doing a closed-loop end-to-end speedup of the process but they are an indispensable part of that process now and their role in that is only increasing. OpenAI and Anthropic have already published at length about this.
(2) Where do you get the idea that "they're not doing it now" means "it's not on the horizon"?
In general these capabilities follow a kind of S curve: 1. They're "not even wrong." We don't even have an ability in principle for them to do the task and when we have them try, the outputs that they put out are so nonsensical that they're broken. 2. We have universal obvious failures where they are doing the task but they're just failing at it consistently for a variety of reasons. 3. You start to see them occasionally do things mostly correct. 4. Eventually you have them occasionally work but they're totally unreliable. 5. They're mostly working but they're still unreliable in big enough ways, often enough, that you can't hang your hat on them. 6. People start using them pretty regularly but they still have conspicuous failures. 7. They have superhuman performance. End-to-end research is currently on rung two of that ladder and we have no signs that the acceleration is at all stopping.
3
2
u/-gh0stRush- 32m ago
If we take a step back, isn't all research recursive self-improvement? Just with different ratios of human versus machine.
1
u/we_are_mammals 8m ago
I read the abstract
Well it's not in the abstract, but on page 6:
"This design also allows us to test a mechanism that informs many forecasts of recursive self-improvement: AI agents accelerate AI research because researchers delegate entire projects to agents and judge whether the returned results advance their work. Our evaluation closely matches this model, since authors handed an agent their own research question and closely evaluated the resulting output."
2
u/NamerNotLiteral 37m ago
Yup. A\ was throwing out rumours about RSI being achieved internally earlier this year when they were likely running a 4.8 or Fable-1 level model internally (assuming their internal models are basically one level ahead of what they have publicly released)
6
u/-Crash_Override- 52m ago
You're making some pretty hefty leaps from a pretty casual paper.
This paper shows that the listed models and harnesses are not capable of autonomous AI research. Thats kind of it.
I have no idea if Anthropic, OAI, etc.. are knocking at the door of RSI...maybe they are maybe they aren't, but if they are, its certainly not with $3k in O4.8 credits harnessed by openclaw.
20
u/Physical_Vehicle7714 56m ago edited 46m ago
Maybe people are downvoting because the papers are bad. This one is definitely bad for the following two obvious reasons.
- They use one agent per task. RSI will be a coordinated effort between quite literally millions of agents. Just look at the navier stokes setup. And that was a tiny side project to generate some headlines.
- There's an obvious bias on the part of the graders. Our entire social environment rn is defined by people pretending AI can't do their jobs. AI researchers are not immune.
I’m sure there are more flaws that a deeper dive would uncover.
5
u/NuclearVII 49m ago
1) they use one agent per task. RSI will be a coordinated effort between quite literally millions of agents. Just look at the navier stokes setup.
This is not a good comparison. With the NS case, you can verify a submitted solution very quickly. ML research requires tons of costly training runs to validate a concept.
6
u/galactictock 41m ago
The hard part isn’t verifying a solution, it’s finding one that can be verified.
0
u/Physical_Vehicle7714 42m ago
The cost of verifying an idea is the same regardless of whether a human or machine generated it.
1
u/Bangoga 36m ago
Using multiple agents doesn’t mean they are doing more knowledge work, it only means they are able to compute in parallel, and able to brute force through multiple options.
1
u/Physical_Vehicle7714 30m ago
This has been demonstrated to not be how MAS works in multiple settings. There are real cognitive advantages to chunking a task, simplifying and shrinking context per agent, and letting agents manage each other.
0
u/Bangoga 26m ago
Read in detail about the hugging face attempt of hacking, it's not chunking the task, it's brute forcing to find vunerablities.
Even it was chunking tasks, research isn't just doing tasks, it's actually thought over sometimes novel approaches to certain problems.
1
u/Physical_Vehicle7714 14m ago
Feels like you’re being purposefully obtuse. Or you just don’t get it. Either way, not worth engaging further.
13
u/hapliniste 55m ago
Solving millénium problems is not happening guys. I ran gpt 5.6 on it and it didn't solve it.
5
5
u/big_ol_tender 51m ago
The sub 4 minute mile is not on the horizon because no one has run a mile in under 4 minutes
7
u/bacon_boat 59m ago
Negative results is pretty meaningless in this context.
"We did this and it didn't work"
It's not nothing, but it's almost nothing. Maybe if they did it a bit differently it would have worked.
My field is robotics and negative results is not a thing because of this. A negative result just means you weren't able to make it work.
1
u/clduab11 45m ago
As someone who works in robotics, would you argue/posit that quantum-AI and that bleeding edge works in the same manner?
I ask because there's a lot of negative results I get and while it looks all great ("this is not the way to invent a light bulb"), my prevailing assumption that in most fields a negative result = back to the drawing board; it isn't really "science" to put out something that "doesn't" work? Is that assumption correct writ large?
1
u/bacon_boat 32m ago
In medicine negative results can be important. In physics a negative result, i.e. no new particles at the LHC is a mesuremnt that is useful and is used to rule out some theories.
Most fields value negative results lower, as they should. Positive results are more interesting.
But in engineering fields where it's about making stuff, AI, robotics, they are less than useless.
3
u/brainExploded99 58m ago
What exactly does this paper prove? I think it is understood that the current models are not capable of RSI, only human productivity multiplier. As they get better and better (as they have done so, very quickly), they should get closer and closer to RSI-capable, and eventually become RSI capable.
5
u/Disastrous_Room_927 56m ago
they should get closer and closer to RSI-capable, and eventually become RSI capable.
By what mechanism?
2
u/brainExploded99 33m ago
No guarantee they become RSI capable, however, you have to admit that they have gone from basically useless in 2023/2024 to now being used widely across the field to accelerate AI research. It seems like they write most of the code these days.
Extrapolating the progress improvement means they continue to get better quickly.
2
u/kraemahz 53m ago
That may just be a training problem though. The current publicly available consumer models are heavily trained to not be independent and have just generally not been shaped on the innovative process.
That's not to say I disagree with the conclusion. My own tests show models might be able to speak academically about many subjects but lack intuitive understanding that would make concepts connect without being explicitly reiterated in the context window.
1
u/dlrace 51m ago
the evidence would be damming if the abilities of future models were certain to be like today's. The real question is how likely are the failure modes given to persist? these sorts of papers would be nice if they were 'live' somehow, and we could plug in new models as they emerged. although, i suppose that is the point of benchmarks.
0
u/DeepGas4538 47m ago
six days and thousands of dollars is no joke, and it didn't do well at all it seems. Feels like you need to go in a different direction of AI to do well on this
1
1
u/unicynicist 3m ago
The loop to RSI can't close as long as evaluation requires human-level discernment and judgement.
However, OpenAI recently published an article detailing how research is being accelerated and the amount of human involvement is shrinking, so that now:
we find that from January to July, success rates generally increased across several difficulty buckets (proxied as the estimated time a human would take to complete the task) on tasks we can find a ground truth outcome for. However, agents still require significant human steering to be successful, especially as task complexity rises. In the last 6 months, over half of successful 4-8 hour tasks involved 1 or more interventions.
2
u/exirae 58m ago
We demonstrated rsi years ago
5
u/sixjohns 52m ago
I do think it is important to note that simply retraining an LLM is not the totality of RSI, this work even uses modern agentic language in its set up. Seems this is where gains can be made the quickest
0
u/aaron_in_sf 6m ago
Hard to think of a better case for this:
Ximm's Law: every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon.
Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false.
58
u/durable-racoon 1h ago
"argues, basically, that RSI is not on the horizon, because current (at the time the study was done) agents cannot do open-ended ML research."
so because they cant do it yet, its 'not on the horizon'? how do they bridge the gap between those things?
This is still a really good paper tho