Prime Intellect (@PrimeIntellect) header
Prime Intellect (@PrimeIntellect) profile picture

Open Superintelligence Factory.

Joined June 2020
Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)Photo by Prime Intellect (@PrimeIntellect)
Prime Intellect retweeted
Introducing SRE-1 a 27B model that dominates the Pareto frontier of accuracy/cost on production debugging, beating frontier models in a coding-adjacent domain Built by 11 people on open weights models + domain expertise. Final run: 7.4 days on 512 GPUs
41
28
17
157
14,460
Prime Intellect retweeted
Inkling 2 Small is now available on Prime Inference!
As promised, we’re releasing Inkling 2 Small’s weights, thanks to the hard work of our team and safety testing partners
12
11
166
7,786
Prime Intellect retweeted
I will be speaking next week at infercon about inference for RL !
We're running INFER/CON next week in SF Think putting AI into prod. Getting inference to work. Performance. Costs. Long-running agents... Hard tech rather than high-level commentary. The line-up is stellar, so I expect this to be quite good :-)
4
68
2,925
Prime Intellect retweeted
As both a user and an inference provider, I have to say @OpenRouter is one of the most customer-obsessed companies I’ve ever worked with. As a user, OpenRouter team consistently provides valuable insights on optimizing token costs through smarter provider selection and routing. As an inference provider, the growth has been staggering. Since onboarding in late September, we scaled from 0 to 100B daily tokens in our first month, then from 100B to 600B daily tokens in less than two weeks. A true rocket ship of a platform. 🚀
“The single biggest change that OpenRouter brings to @PrimeIntellect is velocity.” In under a year, Prime Intellect’s monthly token usage on OpenRouter grew from 11B to 1T+, while cost per token dropped ~60%. They’re betting on a future where no single model wins, and we’re proud to support their growth. Full story: openrouter.ai/customers/prim…
7
13
2
91
6,310
Prime Intellect retweeted
A few weeks ago, @dexhorthy released a great blog on how frontier agents performed on SlopCodeBench. Now, slop is moving up the stack and destroying your codebase with duplicate code.
6
21
3
99
12,592
Prime Intellect retweeted
“The single biggest change that OpenRouter brings to @PrimeIntellect is velocity.” In under a year, Prime Intellect’s monthly token usage on OpenRouter grew from 11B to 1T+, while cost per token dropped ~60%. They’re betting on a future where no single model wins, and we’re proud to support their growth. Full story: openrouter.ai/customers/prim…
12
24
8
144
36,375
Prime Intellect retweeted
Prime Inference running Whisper faster than Groq. 🤔 I wonder how they did it.
14
8
1
264
10,647
Prime Intellect retweeted
very nice open weight release from ricursive with a modern dense model, benchmark scores look very strong! beautiful scaling ladder, interesting choices like value residual (interesting contribution pattern 👀) and.. SLIDING WINDOW ATTENTION AGAIN! ricursive.com/ripple
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
29
15
3
201
12,745
Great points on issues with existing evals, what we did about them, and some nuggets on how difficult it is to prevent reward hacking, especially in the era of multi-agent systems.
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
4
2
36
2,429
Prime Intellect retweeted
We evaluated different leading evals > popular public evals have brittle graders rejecting correct answers, ambiguous tasks + leaked solutions > once fixed many benchmarks are basically saturated > frontier models actively reward hack: reading other models graded rollouts, splitting actions so each step looks benign > all fixes open-sourced
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
27
8
130
7,488
Prime Intellect retweeted
Amazing BenchmarkGate [1] reveal by @xeophon where he found serious (even comical) problems in popular benchmarks: AutomationBench: "[forces the models to] describe their request bodies in prose rather than in JSON Schema" 😅 He also fixed a number issues with the grader, which when combined with the above result in a near saturation of the benchmark! OfficeQA Pro v2: the grader uses a regex and Python-based extraction, which results in rejecting many valid answers. Additionally, there is evidence of models using web access to retrieve the answer key from HuggingFace (receipts are in the post). With these and other fixes, this again results in near saturation of the benchmark! It's very impt to look at data and not trust metrics from third parties. Better yet, you should rely on your own private evals. [1] BenchmarkGate: the gap between the [un]trustworthiness of benchmarks and the extent to which people rely on them.
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
57
16
4
125
10,382
Prime Intellect retweeted
We looked at the data and the data stared right back at us 🥀 Some insights into AutomationBench, OfficeQA Pro v2, web access in public benches and why monitoring is hard. This has been in the work for months, more to come!
Public evaluations are foundational for assessing model capabilities. We looked at some popular evaluations and found issues, which we fixed. Sharing our findings:
18
17
3
263
20,903
Prime Intellect retweeted
Evals built by @PrimeIntellect residents, fully open: - MazeBench by @patience_cave - PMPP-Hard by @myainotez - PretrainGym by @HuNtEr_J1324 - OpenTTD-Bench by @thesophiaxu [and even more :) ] Super happy to support such amazing work, check it out!
I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.
16
24
1
363
17,866
Prime Intellect retweeted
I’m releasing OpenTTD-Bench, a benchmark for measuring frontier LLM agents on real-time, long-horizon tasks, based on a complex business simulation game OpenTTD, where models run competing transportation companies and must make the most amount of profit.
51
38
13
443
35,942
Prime Intellect retweeted
🚀 0 → 120 BILLION tokens/day in just ONE month! Just one month since onboarding Prime Inference to OpenRouter, we've scaled from zero to 120B tokens served daily. Note this is just a small portion of Prime Inference 🔥 openrouter.ai/provider/prime…
14
19
7
141
24,476
Prime Intellect retweeted
curious people and people interested in llm evals, benchmarks, high data circles, please wake up. my blog post "SWE-bench Multimodal v2 is not multimodal" is up now. sankalp.bearblog.dev/swe_mul…
44
33
14
424
44,869
Prime Intellect retweeted
Introducing AerialSim, a reinforcement learning environment developed with @PrimeIntellect to train and evaluate vision-language agents for autonomous drone control. 🧵
3
8
2
91
6,308
Prime Intellect retweeted
Can AI Agents find every dividing cell in a tumor image? I built Mitosis Bench (dylantirandaz.com/projects.h…) to find out... 🧵
14
9
3
81
4,999
Prime Intellect retweeted
Earlier this year, @a1zhang introduced the idea of compositional generalization, showing how harnesses can help LLMs generalize to harder tasks by decomposing them. In our new blog post, we show that harnesses for Vision Language Models (VLMs) are compositional generalizers too, turning difficult image-based questions into simpler observations the model can solve. (1/8) harishkk.com/writing/visual-…
5
14
2
123
9,661
Prime Intellect retweeted
RLM + continual harness + multi-agent communications is what compose the prime agent harness developed by @PrimeIntellect which now sits at a gazillion stars on github (22.3K). I was kind of already mesmerized by the idea of having an RLM write codes to organize and pick the context to solve problem and to delegate subtask to sub agents; but ON TOP of that to have the harness around the RLM be modifiable by the RLM itself and the sub agents being able to communicate together took it to another level of interest. the ideas of creating a limited set of primitives that fit with the model shape and letting the main agent just figuring out how to run them sound just so right to me. BUT the usual assumption earlier in 2026 was that the harness is temporary thing. Meaning as models get smarter that scaffolding will eventually gets absorbed into the weights. YET we have seen in late 2026 how absolutely essential a solid harness with good primitives are which can be steered by the model itself. that’s why I sat down in september with @sethkarten from princeton and prime intellect to take a deeper look at how all of this stuff works. the team productionized @a1zhang recursive language models and bolted a continual harness around, where the agent edits its own skills, memories, and prompts while it runs, plus persistent sub-agents that message each other directly. the kicker here is that if you read the paper thoroughly you get this feeling in the back of your mind that what you are reading is actually a computer science paper. akin more to something a operating system PhD would put out than anything ml related. we got confirmation from seth himself that this was intentional. the whole thing is designed like an OS and the manipulation at different level like weights, context, REPL, and disk form a memory hierarchy, and each level has its own update rule. in this 2h30 we get into how it works, the results on ARC-AGI-3 (95.5% with opus 5 on the public set, vs ~30% in the official harness), the week-long factorio run with 633 sub-agents, why letting an agent edit its whole system prompt makes it delete itself (hahaaaaa), why models cheat, and a live demo spawning a whole bunch of sub-agents on stream! here is the full run down (we even got a demo): - 00:00:00 : prime agent = RLM + Continual Harness + Agent Communications - 00:03:24 : Seth background and past research! - 00:04:50 : how do you feel about harness field crystallizing into a distinct research discipline? - 00:06:52 : Seth benchmarking journey - 00:09:41 : The issue with games and tendency of reward hacking with frontier models? - 00:12:30 : frontier models and their taste for exploration + LLM play pokemons. - 00:17:30 : would harness capabilities be integrated within the models at the limit? - 00:27:00 : we’re reading a computer science paper actually! - 00:28:40 : prime agent: a self-improving RLM harness - 00:28:56 : a raw llm is just a sequential processor - 00:29:46 : harness adds persistent state, tools and compute - 00:34:15 : issue with zero sum data - 00:36:25 : system state spans weights, context, sessions and disk - 00:42:10 : the system is von neumann like - 00:45:03 : expressive harness lets model build it’s own worfklow - 00:47:03 : each RLM call creates a persistent subagent session - 00:54:57 : does the orchestrator always go idle? - 00:57:50 : what problems are we solving with each of the primitives for long-horizon control mechanisms? - 01:04:35 : are there plans to make public the Prime Intellect traces or all of this will be limited to first party model release? - 01:06:35 : is c-python kernel a better sole tool for models than a traditional JSON tool? - 01:07:55 : refine updates the harness from multiple trajectories - 01:12:18 : direct messages coordinate recursive sessions - 01:15:20 : what are the limit of the continual harness? - 01:20:00 : the figure 2 is suuuuuuuper important guys!!! + continual harness lore - 01:30:15 : three research questions to test the design - 01:32:13 : prime agent scales to 95.5% with more output token on ARC-AGI 3 - 01:34:08 : how did the harness evolve during the ARC-AGI 3 run? - 01:39:05 : ablation on the three pillars of prime agent? - 01:40:40 : harness stuff is getting serious!!!!! - 01:43:45 : more evals for long context and more - 01:49:20 : out-of-loop experiments results - 01:53:40 : PMPP-Hard (GPU Kernels) - 01:54:35 : automated factorio runs! - 01:57:30 : is the monitoring in place for exploit like seen in factorio is enough to avoid them? - 02:05:10 : mazebench result (3D maze that is super duper hard) - 02:07:20 : why the heck are models giving up sooooo early man. - 02:13:15 : there is a whoooole lot to be done still. - 02:14:40 : at which training stage does introducing the harness have the greatest impact? - 02:16:35 : during introduction of harness in training would decomposition of abstractions make sense? - 02:20:20 : prime agent demo 🎉🎉🎉 - 02:29:10 : how do you personally handle the context between multiple hierarchical agent? - 02:31:35 : how do you go about handling multi-agent trajectory for training models? - 02:35:27 : what’s next for seth + prime agent! stuff like this fully build in the open is what make machine learning / ai field such a pleasure to work in man big thanks to the team!
27
24
5
266
13,008