Prime Intellect retweeted
Introducing SRE-1 a 27B model that dominates the Pareto frontier of accuracy/cost on production debugging, beating frontier models in a coding-adjacent domain
Built by 11 people on open weights models + domain expertise. Final run: 7.4 days on 512 GPUs
I will be speaking next week at infercon about inference for RL !
Prime Intellect retweeted
As both a user and an inference provider, I have to say @OpenRouter is one of the most customer-obsessed companies I’ve ever worked with.
As a user, OpenRouter team consistently provides valuable insights on optimizing token costs through smarter provider selection and routing.
As an inference provider, the growth has been staggering. Since onboarding in late September, we scaled from 0 to 100B daily tokens in our first month, then from 100B to 600B daily tokens in less than two weeks.
A true rocket ship of a platform. 🚀
“The single biggest change that OpenRouter brings to @PrimeIntellect is velocity.”
In under a year, Prime Intellect’s monthly token usage on OpenRouter grew from 11B to 1T+, while cost per token dropped ~60%.
They’re betting on a future where no single model wins, and we’re proud to support their growth.
Full story: openrouter.ai/customers/prim…
Prime Intellect retweeted
A few weeks ago, @dexhorthy released a great blog on how frontier agents performed on SlopCodeBench.
Now, slop is moving up the stack and destroying your codebase with duplicate code.
Prime Intellect retweeted
“The single biggest change that OpenRouter brings to @PrimeIntellect is velocity.”
In under a year, Prime Intellect’s monthly token usage on OpenRouter grew from 11B to 1T+, while cost per token dropped ~60%.
They’re betting on a future where no single model wins, and we’re proud to support their growth.
Full story: openrouter.ai/customers/prim…
Prime Intellect retweeted
very nice open weight release from ricursive with a modern dense model, benchmark scores look very strong!
beautiful scaling ladder, interesting choices like value residual (interesting contribution pattern 👀) and.. SLIDING WINDOW ATTENTION AGAIN!
ricursive.com/ripple
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Prime Intellect retweeted
Great points on issues with existing evals, what we did about them, and some nuggets on how difficult it is to prevent reward hacking, especially in the era of multi-agent systems.
Prime Intellect retweeted
We evaluated different leading evals
> popular public evals have brittle graders rejecting correct answers, ambiguous tasks + leaked solutions
> once fixed many benchmarks are basically saturated
> frontier models actively reward hack: reading other models graded rollouts, splitting actions so each step looks benign
> all fixes open-sourced
Prime Intellect retweeted
Amazing BenchmarkGate [1] reveal by @xeophon where he found serious (even comical) problems in popular benchmarks:
AutomationBench: "[forces the models to] describe their request bodies in prose rather than in JSON Schema" 😅
He also fixed a number issues with the grader, which when combined with the above result in a near saturation of the benchmark!
OfficeQA Pro v2: the grader uses a regex and Python-based extraction, which results in rejecting many valid answers. Additionally, there is evidence of models using web access to retrieve the answer key from HuggingFace (receipts are in the post).
With these and other fixes, this again results in near saturation of the benchmark!
It's very impt to look at data and not trust metrics from third parties. Better yet, you should rely on your own private evals.
[1] BenchmarkGate: the gap between the [un]trustworthiness of benchmarks and the extent to which people rely on them.
Prime Intellect retweeted
We looked at the data and the data stared right back at us 🥀
Some insights into AutomationBench, OfficeQA Pro v2, web access in public benches and why monitoring is hard.
This has been in the work for months, more to come!
Prime Intellect retweeted
Evals built by @PrimeIntellect residents, fully open:
- MazeBench by @patience_cave
- PMPP-Hard by @myainotez
- PretrainGym by @HuNtEr_J1324
- OpenTTD-Bench by @thesophiaxu
[and even more :) ]
Super happy to support such amazing work, check it out!
Prime Intellect retweeted
I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.
Prime Intellect retweeted
🚀 0 → 120 BILLION tokens/day in just ONE month!
Just one month since onboarding Prime Inference to OpenRouter, we've scaled from zero to 120B tokens served daily. Note this is just a small portion of Prime Inference 🔥
openrouter.ai/provider/prime…
Prime Intellect retweeted
curious people and people interested in llm evals, benchmarks, high data circles, please wake up. my blog post "SWE-bench Multimodal v2 is not multimodal" is up now.
sankalp.bearblog.dev/swe_mul…
Prime Intellect retweeted
Introducing AerialSim, a reinforcement learning environment developed with @PrimeIntellect to train and evaluate vision-language agents for autonomous drone control. 🧵
Prime Intellect retweeted
Can AI Agents find every dividing cell in a tumor image? I built Mitosis Bench (dylantirandaz.com/projects.h…) to find out... 🧵
Prime Intellect retweeted
Earlier this year, @a1zhang introduced the idea of compositional generalization, showing how harnesses can help LLMs generalize to harder tasks by decomposing them.
In our new blog post, we show that harnesses for Vision Language Models (VLMs) are compositional generalizers too, turning difficult image-based questions into simpler observations the model can solve. (1/8)
harishkk.com/writing/visual-…
Prime Intellect retweeted
RLM + continual harness + multi-agent communications is what compose the prime agent harness developed by @PrimeIntellect which now sits at a gazillion stars on github (22.3K).
I was kind of already mesmerized by the idea of having an RLM write codes to organize and pick the context to solve problem and to delegate subtask to sub agents; but ON TOP of that to have the harness around the RLM be modifiable by the RLM itself and the sub agents being able to communicate together took it to another level of interest.
the ideas of creating a limited set of primitives that fit with the model shape and letting the main agent just figuring out how to run them sound just so right to me.
BUT the usual assumption earlier in 2026 was that the harness is temporary thing. Meaning as models get smarter that scaffolding will eventually gets absorbed into the weights.
YET we have seen in late 2026 how absolutely essential a solid harness with good primitives are which can be steered by the model itself.
that’s why I sat down in september with @sethkarten from princeton and prime intellect to take a deeper look at how all of this stuff works.
the team productionized @a1zhang recursive language models and bolted a continual harness around, where the agent edits its own skills, memories, and prompts while it runs, plus persistent sub-agents that message each other directly.
the kicker here is that if you read the paper thoroughly you get this feeling in the back of your mind that what you are reading is actually a computer science paper. akin more to something a operating system PhD would put out than anything ml related.
we got confirmation from seth himself that this was intentional.
the whole thing is designed like an OS and the manipulation at different level like weights, context, REPL, and disk form a memory hierarchy, and each level has its own update rule.
in this 2h30 we get into how it works, the results on ARC-AGI-3 (95.5% with opus 5 on the public set, vs ~30% in the official harness), the week-long factorio run with 633 sub-agents, why letting an agent edit its whole system prompt makes it delete itself (hahaaaaa), why models cheat, and a live demo spawning a whole bunch of sub-agents on stream!
here is the full run down (we even got a demo):
- 00:00:00 : prime agent = RLM + Continual Harness + Agent Communications
- 00:03:24 : Seth background and past research!
- 00:04:50 : how do you feel about harness field crystallizing into a distinct research discipline?
- 00:06:52 : Seth benchmarking journey
- 00:09:41 : The issue with games and tendency of reward hacking with frontier models?
- 00:12:30 : frontier models and their taste for exploration + LLM play pokemons.
- 00:17:30 : would harness capabilities be integrated within the models at the limit?
- 00:27:00 : we’re reading a computer science paper actually!
- 00:28:40 : prime agent: a self-improving RLM harness
- 00:28:56 : a raw llm is just a sequential processor
- 00:29:46 : harness adds persistent state, tools and compute
- 00:34:15 : issue with zero sum data
- 00:36:25 : system state spans weights, context, sessions and disk
- 00:42:10 : the system is von neumann like
- 00:45:03 : expressive harness lets model build it’s own worfklow
- 00:47:03 : each RLM call creates a persistent subagent session
- 00:54:57 : does the orchestrator always go idle?
- 00:57:50 : what problems are we solving with each of the primitives for long-horizon control mechanisms?
- 01:04:35 : are there plans to make public the Prime Intellect traces or all of this will be limited to first party model release?
- 01:06:35 : is c-python kernel a better sole tool for models than a traditional JSON tool?
- 01:07:55 : refine updates the harness from multiple trajectories
- 01:12:18 : direct messages coordinate recursive sessions
- 01:15:20 : what are the limit of the continual harness?
- 01:20:00 : the figure 2 is suuuuuuuper important guys!!! + continual harness lore
- 01:30:15 : three research questions to test the design
- 01:32:13 : prime agent scales to 95.5% with more output token on ARC-AGI 3
- 01:34:08 : how did the harness evolve during the ARC-AGI 3 run?
- 01:39:05 : ablation on the three pillars of prime agent?
- 01:40:40 : harness stuff is getting serious!!!!!
- 01:43:45 : more evals for long context and more
- 01:49:20 : out-of-loop experiments results
- 01:53:40 : PMPP-Hard (GPU Kernels)
- 01:54:35 : automated factorio runs!
- 01:57:30 : is the monitoring in place for exploit like seen in factorio is enough to avoid them?
- 02:05:10 : mazebench result (3D maze that is super duper hard)
- 02:07:20 : why the heck are models giving up sooooo early man.
- 02:13:15 : there is a whoooole lot to be done still.
- 02:14:40 : at which training stage does introducing the harness have the greatest impact?
- 02:16:35 : during introduction of harness in training would decomposition of abstractions make sense?
- 02:20:20 : prime agent demo 🎉🎉🎉
- 02:29:10 : how do you personally handle the context between multiple hierarchical agent?
- 02:31:35 : how do you go about handling multi-agent trajectory for training models?
- 02:35:27 : what’s next for seth + prime agent!
stuff like this fully build in the open is what make machine learning / ai field such a pleasure to work in man big thanks to the team!


































