Shivay Lamba retweeted
Many people have asked what @inferact is building and what it means for @vllm_project.
This is the answer: Inferact will build an autonomous inference system around vLLM, one that improves the engine and improves itself.
Shivay Lamba retweeted
Introducing Gemini 4 Neon! Pushing the frontier across domains and punching above its model class in a meaningful way. To me, Neon feels like the right balance of intelligence, speed, and usability. Available now across @GoogleAIStudio, the API, and more!
Shivay Lamba retweeted
2,000+ people. Two days. The first SGLang Summit.
Together, we announced SGLang v1.0, released Miles v0.2, and launched the SGLang Consortium with nine founding members: @nvidia, @AMD, @intel, @Google, @Meta, @thinkymachines, @Cloudflare, @modal, and @radixark.
SGLang is now used by 332 organizations. At Meta, it powers production inference for products serving billions of users. Teams from Modal, Cloudflare, LinkedIn, and others shared their production lessons.
Across 90+ technical talks and six hands-on workshops, the community explored what comes next for inference, reinforcement learning, and AI agents.
To everyone building, deploying, and contributing: thank you for making this possible.
Explore the announcements, engineering lessons, and moments from SGLang Summit 2026 in the link below.
Shivay Lamba retweeted
Crazy...this is Kubernetes moment for AI Agents.
Run millions of agents on Kubernetes. Idle agents go to sleep and wake up in under 500ms. 10x more agents on the same hardware.
100% Opensource.
github.com/agent-substrate/s…
Shivay Lamba retweeted
Introducing Wally Code: UNLIMITED coding at ₹1,000 a month for students. Same model, about 12x cheaper than @SarvamAI's API and 21x cheaper than @deepseek_ai's own API, for an agent running all month.
Sarvam got picked to build India's own model and launched a per-token API.
This is what India deserves, built with no grants and no subsidies.
Sarvam got picked by the government in April 2025 to build India's own foundation model. They got a ₹98.68 crore GPU subsidy for 4,096 H100s and raised $234M at a $1.5B valuation in June.
Now they've opened Token Factory to everyone. 8 models on it. 1 is theirs. 6 are Chinese open weights (DeepSeek, Kimi, GLM) and 1 is Google's Gemma.
So India's sovereign AI lab is mostly serving Chinese models from Indian GPUs. That's a hosting business, and plenty of companies already run one.
I wanted Sarvam to be the lab that ships frontier research out of India. After that much capital and subsidized compute, this is not it.
Shivay Lamba retweeted
What an enriching @KubeCon_ NA 2026!
From the talks: @kubeflow graduation panel @mesheryio ContribFest to speaking at ArgoCon and BackstageCon on AI-powered self-healing, observability, and agentic AI security, to project booths
A packed week in SLC. Here's what I was up to 👇
Shivay Lamba retweeted
Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me.
I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA.
I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation.
Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO.
Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours.
Jensen
GPT-6.2 Astra—a frontier model built for your hardest projects. It sets new highs on evals across professional work and scientific research at less than a third of the cost per task of GPT‑6 Astra.
You’ll see the difference in things like slideshows, websites, and 3D designs.
We’re expanding free access to our frontier models for up to 200,000 scientists worldwide, with training and support.
Applications open Nov. 23. By default, we don’t use data shared through the program to train our models.
openai.com/index/expanding-c…
Today, we release d1.2 and d1.2-ultra, the state-of-the-art omnimodal decision models. d1.2 and d1.2-ultra both can process text, images, and audio with new output capabilities.
> #1 on the @ArtificialAnlys Decision Index (75)
> Read a value from the input (text, number or date)
> Find every passage of a kind in a text (spans)
> Locate things in a picture (box for one, boxes for every instance)
> Estimate a number (number with a range)
> Pick every option that applies (choice)
1/ 🧵
Shivay Lamba retweeted
The big theme at @KubeCon_ this week was inference.
And "Inference" is such a loaded term, so let's think about each piece and where it matters for production teams.
1. GPUs and memory: Making sure you have enough, but at the same time, how it's allocated and where/how it can be shared. Dynamic Resource Allocation (DRA) was talked about a lot here. It handles the GPU slicing, and KEDA/Karpenter handle the scaling.
2. vLLM to serve LLMs and help with KV caching via PagedAttention. The goal is to cache memory efficiently, which should reduce memory waste and enhance throughput.
3. Inference-aware Gateways: AI Gateways with the Gateway API Inference extension capabilities, like @agentgateway, for routing to LLMs.
4. Distributed inference/scheduling: llm-d, which sits between vLLM for cache-aware request scheduling.
5. Isolation: Agents running on this infrastructure need a way to be truly sandboxed, with controls around ingress and egress. The sandbox layer needs to come from a lower-level engine that runs efficiently and helps implement not only isolation, but scale. @kagent_dev + Agent Substrate.
6. Where it all runs: @kubernetesio
This six-layer stack is inference infrastructure and agent infrastructure to serve LLMs and Agents efficiently.
#agenticai #ai #kubernetes
Shivay Lamba retweeted
📢 Meet Qwen4-Max, your new AI teammate for work, creativity, and discovery!
Qwen4-Max is built on a new MoE architecture with native vision and a 1M-token context window.
Highlights:
- Outperforms Qwen3.8-Max with up to ~50% fewer tokens, while average cost per task drops over 60% in our internal evaluation.
- Advances real-world productivity through greater proficiency with professional tools, stronger long-horizon execution, and higher-quality deliverables.
- Handles real tasks, including 3D modeling, game development, manufacturing, finance, law, healthcare, and frontier scientific discovery.
- $1.5/M tokens for input, $4.5/M tokens for output, and $0.075/M tokens for implicit caching — all lower than Qwen3.8-Max, with implicit caching down by 70%.
The API is live on QwenCloud. You can also try Qwen4-Max on Qwen Studio, QwenWork and Qoder.
Start building today! 🚀
- Blog: qwen.ai/blog?id=qwen4-max
- API: qwencloud.com/models/qwen4-m…
- Token Plan: qwencloud.com/pricing/token-…
More to come! The open weights of smaller Qwen4 models will be released. Stay tuned. 👀
Shivay Lamba retweeted
264 people spent Day 0 of KubeCon NA building with us at the Cloud Native Hackathon in Salt Lake City, and it was easily the most engaging Day 0 event of the week.
The room had CTOs, directors, platform and AI engineers, FDEs and a lot of first-time builders, all hacking side by side.
Huge thanks to @dash0hq, @SonarSource, @RadarK8s and @elastic search for backing the builders, to Kelsey Hightower and all 12 judges for spending the day with the teams, and to everyone who showed up and shipped something.
Here’s the recap, and the next one is already in the works.
Shivay Lamba retweeted
#kubecon #cloudnativecon job board is bigger than last year!
Shivay Lamba retweeted
Fortune is reporting that OpenAI is so convinced that LLMs will become brains for robots that it has disbanded its robotics team.
Shivay Lamba retweeted
Agent Substrate 101 🐙
Agent Substrate's site says 500+ resumes per second.
On one 24-vCPU node I measured about 37. Then I spent two days finding out why.
I wrote a free manual on Agent Substrate v1.0.0, the Kubernetes runtime for stateful agent actors. Every number in it comes from a run I recorded, and the 24 recordings ship with the manual.
Some of what we(my agent and I) found:
> An idle actor is never suspended. We left one alone for 900 s and it stayed RUNNING. You build the idle policy yourself.
> Resume cost follows the memory that holds data. 0.22 s at 0 MiB, 2.16 s at 1,000 MiB.
> 300 resumes at once all worked, 600 of 600, with a 7.3 s median. The rate stayed near 37 per second. More API server or router replicas did not raise it. About 100 gVisor restores run in parallel, and each slows from 0.27 s to 2.7 s.
> Four API server replicas used up the 100 connections of the bundled Postgres. Resumes failed and actors crashed.
> kubectl drain crashes a running actor. A suspended one moves to the other node by itself. Suspend first, then drain.
> With the bucket down, a sync suspend crashed a healthy actor and lost its last calls. The default suspend lost nothing.
> A resume that takes more than 10 s returns 504 first. The resume keeps going, and the next request gets 200 with the state kept. Retry, don't revert.
> An actor without a SIGTERM handler crashed at every upgrade roll, 4 of 4. With a handler it survived every one.
> kagent turns have no bound by default. One ran 10 minutes after the client gave up. Set maxTokens or timeout on the ModelConfig.
75 figures, about 75,000 words, free:
aiengineeringfromscratch.com…




































