Turing Post (Ksenia Se) (@TheTuringPost) header
Turing Post (Ksenia Se) (@TheTuringPost) profile picture

On X – we follow important research papers and explain ideas behind it. In the newsletter, we connect the dots between AI’s past, present, and future ⬇️

Join over 115,000 readers
Joined June 2020
Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)Photo by Turing Post (Ksenia Se) (@TheTuringPost)
Turing Post (Ksenia Se) retweeted
The race to real-time, long-horizon AI video is somehow a race of "forcing" methods right now. The 15+ approaches you need to know: ▪️ Diffusion Forcing + DFoT ▪️ Self Forcing ▪️ Rolling Forcing ▪️ Self-Forcing++ ▪️ Deep Forcing ▪️ Causal Forcing ▪️ Сontext Forcing ▪️ Hybrid Forcing ▪️ Any Forcing ▪️ Wave Forcing ▪️ Trajectory Forcing ▪️ Relax Forcing ▪️ One-Forcing ▪️ Disen-Forcing ▪️ Causal Forcing++ Some fix imperfect frames, others manage memory or parallelize denoising across GPUs. Here’s our quick + comparison table to tell them apart (with links to all papers and code):turingpost.com/p/15-forcing-…
5
8
1
54
4,090
Turing Post (Ksenia Se) retweeted
Signed. A wild mix of people who signed this letter. Could not imagine them together on any over occasion. But as I wrote in one my recent editorials: - What does Anthropic’s IPO tell us about AI safety? It tells us that safety in AI is fake. And Anthropic’s stand as a safety company is fake too.
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: misanthropicstatement.com/
2
9
1
47
2,928
Turing Post (Ksenia Se) retweeted
8 models graded 620 anonymized medical answers. Every one of them ranked its own answer higher than the other 7 judges did. About 1.2 places on average, almost 3 for GPT-5.6 Sol. Telling them which model wrote which answer barely changed it. A lot of the leaderboards we read every week use a single LLM judge. How many of those results would hold up with a judge from a different model family? www.teapawt.org/rohanpaul_ai/status/21…
New Stanford paper finds that all 8 LLMs tested rank their own medical answers higher than other models rank them, even when answers are anonymous. Medical AI benchmarks often use LLMs to grade other LLMs' answers. If a judge favors its own outputs, the benchmark tilts toward that model. They had 8 models from OpenAI, Anthropic, Google and Qwen answer 620 real clinician questions, then rank all 8 anonymous answers. They compared where each judge placed its own answer with where the other 7 judges placed that same answer. Every judge favored its own answer, by 1.22 rank positions on average and nearly 3 for GPT-5.6 Sol. The bias held without a rubric, and revealing model names cut it by only 0.094 positions. If you use LLM judges, use a panel from several model families and report results per judge instead of 1 pooled score.
7
2
1
9
1,842
Turing Post (Ksenia Se) retweeted
btw my favorite model currently is Terra 6.1 – super fast and good with text. @thsottiaux I’d love to know how you think about the models – which one is good for what, specifically (and I asked this question at DevDay: what about routing? Is it coming?)
I had a couple of weeks before launch to play with Astra 6.2 and 6.2 Sol. They became much better in the last few days before release. And I would choose Astra 6.2 over Fable for sheer speed and token efficiency. It's just faster and doesn't drain your credits. At this point, when all the models are extremely good, the interface – and clarity around what to choose and how to use it – becomes crucial. With all this raw intelligence, we need to dumb down how users actually choose and operate these models.
1
6
1,043
Turing Post (Ksenia Se) retweeted
Coming back from @aiDotEngineer Code in San Francisco, I realized how much the AI Engineer profession has changed in just one year. It's wild. A few cool talks helped me see the bigger picture. When agents can write, run, and fix code on their own, what does the AI engineer do? 1. @simonw (interview dropping this week!) noted that GPT-5.1 and Claude Opus 4.5 made daily autonomous coding real. So engineers got freedom to move to other tasks, but the new challenge is handling all the small, unspoken decisions. 2. @lydiahallie, who works on Claude Code at Anthropic, showed how agents miss edge cases like keeping active users logged in during an auth update. The job shifts to spotting hidden assumptions, turning them into tests, and managing dozens of parallel sessions. 3. @poteto (Lauren Tan) shared how her team shipped 11,000 changes to Grok Bot in October while making it faster. The real work took ~6 months beforehand: encoding engineering experience into system checks so agents stop repeating the same bad shortcuts. 4. @Steve_Yegge proved that coding speed stops being the bottleneck. In his game Wyvern, agents shipped features so fast that players asked him to slow down and publish a roadmap just to keep up. I packed many more insights, including the ones from Thariq Shihipar's talk, into this video to capture this new wave in AI engineering. Watch my full AIE Code overview here or on Youtube: youtube.com/watch?v=ZINu6a3k…
3
3
7
1,246
Turing Post (Ksenia Se) retweeted
How to fit an open model to YOUR coding agent. Super concrete recipe from @ClementDelangue and the @huggingface team, who took @Alibaba_Qwen's Qwen3.8-27B from 39% to 65% on SWE-bench Pro (hard 51) with RL and one small LoRA: 1. Keep the harness, adapt the model. The same model behaves differently in different harnesses. Each one has its own tools, prompt and control flow. So instead of switching harnesses to get a better model, they train a small LoRA adapter per harness and you swap it in behind the agent you already use. First one is for @pidotdev. 2. Train inside the real, unmodified harness. The harness runs as is, in its own sandbox per rollout, and training uses exactly the tokens the harness saw. No re-tokenisation, no simplified copy of the agent. 3. Reward correctness, then efficiency. The reward is the task verifier's correctness plus a tool-efficiency bonus with weight 0.1. Tasks: 750 scale800 tasks mixed with 300 TermGrade tasks, 8 rollouts per task. 4. Keep the adapter small. LoRA r = 32, alpha = 64, on the attention and MLP projections. Async GRPO in TRL, learning rate 2e-5, 2× B200 per harness. The released checkpoint is step 160. The run was stopped at step 191 of 1,000 planned. 5. Block reward hacking during training. A proxy called safeagents checks every tool call. Clear attempts to hack the reward are blocked and score 0. In an earlier ablation, turning the guard on changed accuracy by +0.0 ± 4.1 points. It costs nothing. 6. Keep thinking off. With thinking off, the adapter matches the base model at its highest reasoning effort: 64.7% vs 64.7% on SWE-bench Pro, 56.6% vs 54.2% on Terminal-Bench 2.1, with about 4.5× fewer tokens. Turning thinking on for the adapter adds nothing. 7. Check your evals before you celebrate. The first version of the model card said 29.4% → 62.7%. Then they found nine base-model runs had failed because the tunnel between the sandboxes and the model server went down. They reran the affected runs on both sides: 39.2% → 64.7%. Smaller jump, still +25.5 points, 15 tasks newly solved vs 2 lost. The results they report under pi: SWE-bench Pro (hard 51) 39.2% → 64.7%, Terminal-Bench 2.1 39.5% → 55.8%, and less than half the turns on the tasks both models solve (23 vs 59 median on SWE-bench Pro). Apache 2.0, open weights, a GGUF version for llama.cpp, and an @opencode adapter came out today. And what about "just use a bigger model"? The interesting thing is that the harness itself becomes training data. You can't do this with a closed model. With an open one, a 27B model you run locally gets fitted to the tools you actually use. Full recipe blog is promised for Friday. Great work by @yb2698, @LysandreJik, @ben_burtenshaw, @nathanhabib1011 and the team 🤗 Model: huggingface.co/harness-hub/Q…
13
4
13
1,582
btw my favorite model currently is Terra 6.1 – super fast and good with text. @thsottiaux I’d love to know how you think about the models – which one is good for what, specifically (and I asked this question at DevDay: what about routing? Is it coming?)
I had a couple of weeks before launch to play with Astra 6.2 and 6.2 Sol. They became much better in the last few days before release. And I would choose Astra 6.2 over Fable for sheer speed and token efficiency. It's just faster and doesn't drain your credits. At this point, when all the models are extremely good, the interface – and clarity around what to choose and how to use it – becomes crucial. With all this raw intelligence, we need to dumb down how users actually choose and operate these models.
1
6
1,043
Signed. A wild mix of people who signed this letter. Could not imagine them together on any over occasion. But as I wrote in one my recent editorials: - What does Anthropic’s IPO tell us about AI safety? It tells us that safety in AI is fake. And Anthropic’s stand as a safety company is fake too.
Ahead of Anthropic’s IPO, a rare coalition across AI safety, open source, e/acc, science, and law has signed the MisAnthropic Statement. Read the statement or sign at: misanthropicstatement.com/
2
9
1
47
2,928
Turing Post (Ksenia Se) retweeted
I had a couple of weeks before launch to play with Astra 6.2 and 6.2 Sol. They became much better in the last few days before release. And I would choose Astra 6.2 over Fable for sheer speed and token efficiency. It's just faster and doesn't drain your credits. At this point, when all the models are extremely good, the interface – and clarity around what to choose and how to use it – becomes crucial. With all this raw intelligence, we need to dumb down how users actually choose and operate these models.
One of the funny reasons for the difference in vibes between the influencers and the crowd today on 6.2 Sol/Astra is that we gave a few folks access a while ago and then made a lot of changes in the last 3-4 days and I'm pretty sure no re-testing was done within those days and the videos were produced last week. Don't get me wrong, while the models are a hefty jump compared to their GPT predecessors, they are by no means perfect and there are a few things the teams are excited for the models to get better at in future iterations in terms of taste and niceness of use. Love all the feedback and we hear you!
9
3
1
26
2,777
Coming back from @aiDotEngineer Code in San Francisco, I realized how much the AI Engineer profession has changed in just one year. It's wild. A few cool talks helped me see the bigger picture. When agents can write, run, and fix code on their own, what does the AI engineer do? 1. @simonw (interview dropping this week!) noted that GPT-5.1 and Claude Opus 4.5 made daily autonomous coding real. So engineers got freedom to move to other tasks, but the new challenge is handling all the small, unspoken decisions. 2. @lydiahallie, who works on Claude Code at Anthropic, showed how agents miss edge cases like keeping active users logged in during an auth update. The job shifts to spotting hidden assumptions, turning them into tests, and managing dozens of parallel sessions. 3. @poteto (Lauren Tan) shared how her team shipped 11,000 changes to Grok Bot in October while making it faster. The real work took ~6 months beforehand: encoding engineering experience into system checks so agents stop repeating the same bad shortcuts. 4. @Steve_Yegge proved that coding speed stops being the bottleneck. In his game Wyvern, agents shipped features so fast that players asked him to slow down and publish a roadmap just to keep up. I packed many more insights, including the ones from Thariq Shihipar's talk, into this video to capture this new wave in AI engineering. Watch my full AIE Code overview here or on Youtube: youtube.com/watch?v=ZINu6a3k…
3
3
7
1,246
How to fit an open model to YOUR coding agent. Super concrete recipe from @ClementDelangue and the @huggingface team, who took @Alibaba_Qwen's Qwen3.8-27B from 39% to 65% on SWE-bench Pro (hard 51) with RL and one small LoRA: 1. Keep the harness, adapt the model. The same model behaves differently in different harnesses. Each one has its own tools, prompt and control flow. So instead of switching harnesses to get a better model, they train a small LoRA adapter per harness and you swap it in behind the agent you already use. First one is for @pidotdev. 2. Train inside the real, unmodified harness. The harness runs as is, in its own sandbox per rollout, and training uses exactly the tokens the harness saw. No re-tokenisation, no simplified copy of the agent. 3. Reward correctness, then efficiency. The reward is the task verifier's correctness plus a tool-efficiency bonus with weight 0.1. Tasks: 750 scale800 tasks mixed with 300 TermGrade tasks, 8 rollouts per task. 4. Keep the adapter small. LoRA r = 32, alpha = 64, on the attention and MLP projections. Async GRPO in TRL, learning rate 2e-5, 2× B200 per harness. The released checkpoint is step 160. The run was stopped at step 191 of 1,000 planned. 5. Block reward hacking during training. A proxy called safeagents checks every tool call. Clear attempts to hack the reward are blocked and score 0. In an earlier ablation, turning the guard on changed accuracy by +0.0 ± 4.1 points. It costs nothing. 6. Keep thinking off. With thinking off, the adapter matches the base model at its highest reasoning effort: 64.7% vs 64.7% on SWE-bench Pro, 56.6% vs 54.2% on Terminal-Bench 2.1, with about 4.5× fewer tokens. Turning thinking on for the adapter adds nothing. 7. Check your evals before you celebrate. The first version of the model card said 29.4% → 62.7%. Then they found nine base-model runs had failed because the tunnel between the sandboxes and the model server went down. They reran the affected runs on both sides: 39.2% → 64.7%. Smaller jump, still +25.5 points, 15 tasks newly solved vs 2 lost. The results they report under pi: SWE-bench Pro (hard 51) 39.2% → 64.7%, Terminal-Bench 2.1 39.5% → 55.8%, and less than half the turns on the tasks both models solve (23 vs 59 median on SWE-bench Pro). Apache 2.0, open weights, a GGUF version for llama.cpp, and an @opencode adapter came out today. And what about "just use a bigger model"? The interesting thing is that the harness itself becomes training data. You can't do this with a closed model. With an open one, a 27B model you run locally gets fitted to the tools you actually use. Full recipe blog is promised for Friday. Great work by @yb2698, @LysandreJik, @ben_burtenshaw, @nathanhabib1011 and the team 🤗 Model: huggingface.co/harness-hub/Q…
13
4
13
1,582
I had a couple of weeks before launch to play with Astra 6.2 and 6.2 Sol. They became much better in the last few days before release. And I would choose Astra 6.2 over Fable for sheer speed and token efficiency. It's just faster and doesn't drain your credits. At this point, when all the models are extremely good, the interface – and clarity around what to choose and how to use it – becomes crucial. With all this raw intelligence, we need to dumb down how users actually choose and operate these models.
One of the funny reasons for the difference in vibes between the influencers and the crowd today on 6.2 Sol/Astra is that we gave a few folks access a while ago and then made a lot of changes in the last 3-4 days and I'm pretty sure no re-testing was done within those days and the videos were produced last week. Don't get me wrong, while the models are a hefty jump compared to their GPT predecessors, they are by no means perfect and there are a few things the teams are excited for the models to get better at in future iterations in terms of taste and niceness of use. Love all the feedback and we hear you!
9
3
1
26
2,777
8 models graded 620 anonymized medical answers. Every one of them ranked its own answer higher than the other 7 judges did. About 1.2 places on average, almost 3 for GPT-5.6 Sol. Telling them which model wrote which answer barely changed it. A lot of the leaderboards we read every week use a single LLM judge. How many of those results would hold up with a judge from a different model family? www.teapawt.org/rohanpaul_ai/status/21…
New Stanford paper finds that all 8 LLMs tested rank their own medical answers higher than other models rank them, even when answers are anonymous. Medical AI benchmarks often use LLMs to grade other LLMs' answers. If a judge favors its own outputs, the benchmark tilts toward that model. They had 8 models from OpenAI, Anthropic, Google and Qwen answer 620 real clinician questions, then rank all 8 anonymous answers. They compared where each judge placed its own answer with where the other 7 judges placed that same answer. Every judge favored its own answer, by 1.22 rank positions on average and nearly 3 for GPT-5.6 Sol. The bias held without a rubric, and revealing model names cut it by only 0.094 positions. If you use LLM judges, use a panel from several model families and report results per judge instead of 1 pooled score.
7
2
1
9
1,842
The race to real-time, long-horizon AI video is somehow a race of "forcing" methods right now. The 15+ approaches you need to know: ▪️ Diffusion Forcing + DFoT ▪️ Self Forcing ▪️ Rolling Forcing ▪️ Self-Forcing++ ▪️ Deep Forcing ▪️ Causal Forcing ▪️ Сontext Forcing ▪️ Hybrid Forcing ▪️ Any Forcing ▪️ Wave Forcing ▪️ Trajectory Forcing ▪️ Relax Forcing ▪️ One-Forcing ▪️ Disen-Forcing ▪️ Causal Forcing++ Some fix imperfect frames, others manage memory or parallelize denoising across GPUs. Here’s our quick + comparison table to tell them apart (with links to all papers and code):turingpost.com/p/15-forcing-…
5
8
1
54
4,090
Turing Post (Ksenia Se) retweeted
Lots of self-improving agent papers this week. A good chunk of the gains turn out to be agents getting better at the benchmark, not the task. So we put the papers that actually check first. 1. Beneath the Surface: What We Overlook When We Talk About Agent Evolution Evolving the harness can lift benchmark scores with no weight updates, but the gains are unreliable, expensive and transfer poorly. Read this one first. arxiv.org/abs/2611.07624 2. Learning the Task or Learning the Verifier? Show a self-improving embodied coding agent the benchmark's success criteria and its score goes up. Its actual task completion doesn't. arxiv.org/abs/2611.08318 3. Evidence-Governed Adoption of Self-Generated Updates in Autonomous AI Agents Separates "the agent wrote a change" from "the agent ships it". Every change has to pass fresh paired audits and a non-inferiority check first. arxiv.org/abs/2611.16378 4. Harness^2: Recursive Agent Harnessing for an Open World A harness that improves itself on new tasks without an answer key, by test-running edits and keeping what helps. arxiv.org/abs/2610.14224 5. Experience-Handoff Reinforcement Learning for Recursive Self-Improvement Agents trained with RL learn to write and curate notes for their future runs, not only to pick better actions. arxiv.org/abs/2611.16409 6. InfraRSI: Harnessing Recursive Self-Improvement for Training Infrastructure Self-improvement where it's hardest: distributed training, where one bad edit means an OOM or silent numerical drift. Every optimization gets verified. arxiv.org/abs/2611.11558 7. Incentivizing Self-Learning in Large Language Models via Reinforcement Learning Give a model long reference material and train it to look things up and study while it solves the problem, instead of relying on what it already knows. arxiv.org/abs/2610.39671 8. PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models An LLM spots physics mistakes in generated videos and guides the fixes, so the video model improves on cleaner data. arxiv.org/abs/2611.16748 9. A Cross-Scenario Survey and Analysis of Self-Evolving Multimodal Agents Why a self-improvement trick that works for reasoning often breaks for navigation or manipulation. A good map of the field. arxiv.org/abs/2611.14256 If you're letting an agent edit its own setup: who decides which edits stay? Tests, a human, another model?
27
9
2
103
5,082
Must-read papers of the week ▪️ World In-Context Learning for Agentic Robot Manipulation ▪️ Don’t Just Trust Your Memory ▪️ Is Your Robot’s Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies ▪️ Unbundling Multimodal Vision Encoding ▪️ V-JEPA-LM ▪️ Learning Heterogeneous Tactile Representation for General Dexterous Manipulation ▪️ Training Reasoning Models in Simulation to Natively Use Real Robots ▪️ Embodied First Exam ▪️ Continual-ARC: Measuring Skill Acquisition over a Working Lifetime ▪️ Reframe: Observation-Aligned Multi-Teacher On-Policy Distillation for Agentic Visual Reasoning ▪️ MemInStream: Memorizing or Merely Recognizing? Rethinking Streaming Video Memory Evaluation ▪️ Infrathon: Benchmarking Language Agents for Real-World, Long-Horizon AI Infra Optimization ▪️ Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents ▪️ GTAGym: AAA Game Tasks Environment for Evaluation and Reinforcement Learning Explore these to keep up with main AI trends. Here’s also the full list of outstanding papers + links, and our weekly AI news digest: turingpost.com/p/who-are-we-…
5
20
11
126
8,106
Turing Post (Ksenia Se) retweeted
Yes, please. Every new open model = rewriting the same attention block with a slightly different twist. LEGO for LLMs, or something like this? @rasbt is onto something
The open-source LLM ecosystem could really benefit from an LLM building-block API. We have so many LLM components across different models now. Whether you are using PyTorch or C++ and CUDA code directly, there is a major switching cost when you want to run experiments or ablation studies, say swapping GQA with MLA, dense FFNs with MoE, attention layers with Mamba2 or Gated DeltaNet, or standard residual connections with attention residuals. Currently, it’s very messy even with LLM-assisted coding. It could maybe be on top of PyTorch or CUDA directly, but it needs to be carefully engineered and maintained (versus quickly vibe-coded on a weekend) and efficient enough that people trust it to use in production (which is really a requirement to move beyond toy examples). Maybe some food for thought for a big, well-funded lab :) (There are some efforts that go into that direction, like torchtitan, torchtune, Megatron Core, OLMo-core, but they are not quite that level of abstraction, I'd say. And, as always, there’s the chicken-and-egg problem: it needs to be popular to get widely adopted, and it needs to get widely adopted to become popular.)
3
6
28
2,846