Inference serving is increasingly open source. But inference optimization still happens inside each provider, and the optimized deployment remains tied to its proprietary stack. This has to change.
Introducing Presets: an open-source toolkit for agent-based inference optimization and a portable format for the result.
Deploying optimized Kimi K3 to any cloud, Kubernetes cluster, or bare-metal fleet, whether on NVIDIA, AMD, or other silicon, should be as simple as deploying a Docker image.
dstack.ai/blog/presets/
Getting training and inference to run well on a GPU cluster is mostly an orchestration problem.
dstack is an open-source control plane for that: training, inference optimization with presets, and production-grade deployment, in the cloud or data center.
As an example, presets more than doubled GLM-5.3 throughput on @CrusoeAI NVIDIA GB200 NVL72 over the SGLang cookbook recipe: dstack.ai/blog/glm53-gb200-n…
Read @CrusoeAI's post walking through it step by step, from setting up the rack to deploying the presets: crusoe.ai/resources/blog/ser…
23
How fast can you self-host GLM-5.3 coding agents on NVIDIA GB200 NVL72?
With dstack presets, we tuned @lmsysorg's SGLang on @CrusoeAI: as fast as the fastest OpenRouter endpoints, ~3x lower p50 TTFT, and cheaper than buying the same tokens.
dstack.ai/blog/glm53-gb200-n…
GLM-5.3 coding agents on an @NVIDIA GB200 NVL72 fleet on Crusoe Cloud: a 0.29 s median first token vs 0.99 s for the median of 47 OpenRouter endpoints, and 137 vs 81 tok/s.
A @dstack_ai agent designed the layout. KV cache picked the winner: crusoe.ai/resources/blog/ser…
How fast can you self-host GLM-5.3 coding agents on NVIDIA GB200 NVL72?
With dstack presets, we tuned @lmsysorg's SGLang on @CrusoeAI: as fast as the fastest OpenRouter endpoints, ~3x lower p50 TTFT, and cheaper than buying the same tokens.
dstack.ai/blog/glm53-gb200-n…
For the step-by-step NVIDIA GB200 NVL72 setup on @CrusoeAI, from creating the VMs and enabling IMEX to registering them as a dstack fleet, see their post:
crusoe.ai/resources/blog/ser…
135
Last week, we released dstack 0.22.8!
The Kubernetes backend now supports NVIDIA MIG, letting you run training and inference on GPU slices instead of allocating entire GPUs.
Plus cross-project fleet exports, improvements to SSH fleets, Nebius, services, and more.
github.com/dstackai/dstack/r…
dstack 0.22.7 is out!
Orchestrating training or inference on Kubernetes? We keep making our Kubernetes integration more reliable and easier to configure.
This update adds automatic cleanup of orphaned resources, plus improved proxy jump reliability and configuration.
github.com/dstackai/dstack/r…
dstack 0.22.7 is out!
Orchestrating training or inference on Kubernetes? We keep making our Kubernetes integration more reliable and easier to configure.
This update adds automatic cleanup of orphaned resources, plus improved proxy jump reliability and configuration.
github.com/dstackai/dstack/r…
dstack 0.22.6 is out!
Long-awaited automatic detection and cleanup of orphaned instances to avoid unnecessary costs. Supported on AWS, Azure, GCP, Nebius, and RunPod, with more GPU clouds coming.
Plus improvements to Offers and Events in the UI.
github.com/dstackai/dstack/r…
AMD ❤️ dstack
An open-source, ready-to-deploy stack for running AI token factories on @AMD GPUs.
At Advancing AI 2026, @dstackai demonstrated distributed training and prefill/decode-disaggregated inference of a Qwen model on AMD Instinct MI300X GPUs.
Its open-source control plane works across GPU clouds, Kubernetes, and bare-metal clusters, letting people and agents work with the same infrastructure for development, training, and inference. More recently, dstack introduced a preview of an agent-based toolkit for optimizing inference for specific models, including Qwen3.8-27B.
Watch the dstack demo 🔽
dstack retweeted
Biggest learning at PyTorchCon this week is that engineers are so hungry for alternatives to Slurm and Kubernetes. As soon as you mention it, you get their attention.
Super excited about the conference! Also great to see at least 5 GPU hardware vendors here. It’s the first time I’ve seen so many at such a small event. Exciting times for inference ahead!
dstack retweeted
In San José tonight? Come hang out! We’re throwing a @PyTorch Conference afterparty with @CrusoeAI and the @lmsysorg SGLang
luma.com/6j69xyal
dstack retweeted
Looking forward to @PyTorch Conf this week. Hanging out at the @dstackai booth, and Tue evening at our meetup with @CrusoeAI and @lmsysorg SGLang: luma.com/6j69xyal
In San Jose and SF all week, happy to grab coffee.
dstack 0.22.4 is out!
This release greatly improves server stability and performance, with significantly faster offer lookups and better responsiveness under heavy workloads.
Also included: UI improvements to Launch, Offers, and project quotas in dstack Sky.
github.com/dstackai/dstack/r…
How many coding agents can one 4xH200 node serve with Qwen3.8-Flash-Next?
With dstack presets, we tuned @lmsysorg's SGLang on Claude Code sessions: 37% lower TTFT, 74% lower p99 ITL vs the cookbook recipe.
Read about the benchmark and how preset optimization works: dstack.ai/blog/qwen38-flash-…
Huge thanks to @jarvislabsai for partnering!





