RouterPlus (YC F26) retweeted
been putting together a slack group of engineers and folks handling inference at their companies. calling it The Inference Club.
getting the first few folks together. if this is your world, would love to have you.
comment or dm to get in
RouterPlus (YC F26) retweeted
AI teams overpay for inference twice: once for models bigger than they need, and again for idle capacity they don’t use. @useRouterPlus is the extended inference team for fast-growing AI companies. We test models and providers on your traffic and get providers to compete for your work.
Make your inference better. Go to routerplus.com
RouterPlus (YC F26) retweeted
call for startups : corgi for ai inference.
anyone know of a fast way to underwrite insurance for inference contracts ? we are seeing many mid six to seven figure contracts where we need this desperately.
ideally it should be from someone who truly understants the inference markets, variance in providers and can run their own evals if need be. also better if the cost of insurance can be baked in the token price itself somehow
RouterPlus (YC F26) retweeted
if you wanna make the most of the proliferation of inference providers and get the best price possible - reach out to us. today we have a network of 40+ that we work with
we have been able to get our customers something even as extreme as 70% off of public listing price with BETTER SLAs than they had before
routerplus.com/capacity/#inf…
There is an absolute barrage of inference providers popping up that get to $1M/mo extremely quickly.
This is the rough economics of inference providers:
Even if you rent on-demand B300s at $8/GPU-hr, many open source models (GLM 5.3, Kimi K3, DeepSeek 4.1 Flash / Pro) can get you >$10-20/GPU-hr at reasonable throughputs (100tok/s). DeepInfer from DeepSeek directly shows they can get $38/GPU-hr on off-peak hours. Every chip / model has a different throughput / interactivity curve, but there is a lot of arbitrage between even high rental spot prices and token prices. New chips (Vera Rubins) also change this meaningfully. Your ability to be making gross margin relies on your utilization and how good your stack is.
Gateways like OpenRouter, Vercel and more help solve that problem by aggregating demand to you. May inference providers discount their tokens to get longer term commits too. I've even heard stories of selling back Chinese model tokens to Chinese labs!
One thorn in the economics of these players is that many Chinese open source model labs are now asking for rev share if you're at a certain scale, espp for pre-release access to their models for day 1 support. That further pushes the threshold to be in the money.
The barrier to entry seems low and falling. There are even stories of smaller ones who can "vibe code" a way to stand up OSS models on a cluster they rent. And as a result, we're seeing 10s of such companies pop up almost every week and immediately skyrocketing revenue at low-ish gross margin and raising rounds on the back of that.
Unsurprisingly, because the actual service provided seems commodity, this eventually drives up compute prices and drives down token prices. In theory, it would seem like neoclouds who have the lowest cost of compute and themselves have started inference businesses would win the race to the bottom (Crusoe Coreweave Nebius and GMI all have inference businesses), with larger players (Modal Fireworks Baseten) being second because they are well capitalized to make bigger compute commits cheaply and have demand (high utilization), but the reality is the pie is growing so quickly that even upstarts are absolutely crushing it and growing aggressively (inference dot net, DeepInfra, Wafer, Relace, Morph).
Strategies vary across the board: neoclouds want to serve inference, inference players want to be neoclouds and do higher margin reserved instances / post training, and some want to go heterogeneous hardware or creatively flexible compute constructs.
The risks are: do open source models continue to be competitive? Will their creators rent seek more? Will their shape change compute needs? How do different chips change economics? Who gets compute in a supply crunch?
Below is a aggressive sketch of how gross margin scales with utilization for various models / chip sets.
For now, there are many inference providers and they’re all ripping!
RouterPlus (YC F26) retweeted
Krea 3 by @krea_ai is live on @useRouterPlus
great to have partnered with the team for day 0 support
would love to see examples of what you folks do with it on RouterPlus !! What models would you like us to see support next ?
Krea 3, the new image editing model from @krea_ai, is now on @useRouterPlus .
High-quality edits in under 6 seconds. Reimagine the scene while keeping your subject consistent, or blend multiple reference images with a simple text prompt.
RouterPlus (YC F26) retweeted
CDMs are the future! Was great working with @scaledown_team on this release. Scaledown's purpose built CDMs - classify, extract and summarize are now available on @useRouterPlus
You already gave the LLM the answer. So why is it writing it out from scratch?
Introducing a new AI model architecture: Constrained Decoding Models (CDMs)
LLMs were built to write. Most production workloads just need an answer: fast, consistent, and affordable.
CDMs are purpose-built for 1 task each. They read your input once and skip the token-by-token generative loop that makes LLMs slow and expensive.
Not just a decision model, but a family of 3 CDMs live today:
• Classify: define your labels and get back a calibrated probability + reason for each. Used for routing, tagging, and intent detection.
• Extract: describe the entities in natural language and get back well-formed JSON with confidence scores + citations. Used for web scraping, document processing, and PII redaction.
• Summarize: pass in long text, images, or PDFs and get back a consistent summary under 1,000 tokens. Used for context compaction, metadata generation, and transcript digests.
Built for production:
• 1M tokens of multimodal (text + image) input, 31x larger than Jev
• 77.2% on DecisionBench vs Jev's 72%
• 200x faster, 467x cheaper than frontier LLM APIs
• $0.05/M input tokens. Output is free (yes, even for summarize)
Live on your favorite platforms today:
@OpenRouter @vercel_dev @LiteLLM @LangChain @TrustedRouter @itsArthurAI @useRouterPlus
Get 50M input tokens for free (for the first 50 hours) at link below 👀
RouterPlus (YC F26) retweeted
Self hosting has inference providers alternative now. So unless you have reasons to own hardware and self host or you are post training, everyone doesnt need to download anymore. You can access these services like subscription. Routers make switching, discovery and experimentation far more convenient than you downloading and trying.
Routers are also catalog now, an active one and not a passive listing. People now go to marketplaces like @openrouter, @vercel , @useRouterPlus for their discovery vs huggingface
h/t to @ParikhRajesh07
We've been monitoring a decline in HuggingFace downloads across the top open LLMs we track for interconnects. It looks like there was a slight change in counting of downloads, as we saw a ~30% download rate drop for many top models, but this is continuing over time. What else do people think it may be?
RouterPlus (YC F26) retweeted
Nimble is available on @useRouterPlus!
Launching Decision models on @useRouterPlus. You can now use various system 1 decision models such as Jev, @perplexity_ai Decider, @bespokelabsai Nimble, Mercury Decide, Clef, Kev 4B and Decider 2B with 3 models free !!
app.routerplus.com/playgroun…
RouterPlus (YC F26) retweeted
btw at @useRouterPlus we have been using numerous techniques to ensure that inference providers are ACTUALLY delivering what they claim - this involves running relatively expensive live evals , checking quality and also measuring symmetric KL divergence to monitor deviations from actual expected behaviour
since a lot of inference contracts go through us we can actively monitor for such discrepencies and hold providers accountable. its only done now for certain larger contracts - but we expect to roll it out to the wider platform soon
Yesterday I posted about inference price war pushing margin down. But I since learned inference scammers are a thing with shady price manipulations:
As inference providers,
- You can lower your input token price 100x cheaper but up your output price 5x. You rank higher in openrouter in price but tots cost is much higher.
- You can quantize the model to serve cheaply but users are buying a shittier model.
- You can retain and sell user data elsewhere to make more money.
- You can secretly swap to DeepSeek v4 flash but say it’s GLM 5.3 10% of the time.
Therefore it’s hard to know
1. You get the model you ask for
2. You pay for the tokens you actually burn across cache input, input, output and
3. Your data is safe.
I’m sure openrouter is working on it AND baseten, fireworks, together and other enterprise brands will continue to thrive even with higher pricing because they sell peace of mind.
RouterPlus (YC F26) retweeted
Buying inference today means picking between two bad options.
Commit for a year and pay for capacity you don't use. Or go serverless and get rate limited the moment you grow.
But usage arrives in bursts: a weekend eval, a two-week backfill, US business hours.
And not all tokens are equal. You might need BF16, not quietly quantized FP8. Served in the US. A latency target. Zero retention. Fast tokens for your product, cheap batched tokens for the rest.
The capacity exists. It's just scattered across big clouds and long-tail providers you've never heard of.
So we built a marketplace.
Introducing the @useRouterPlus managed inference marketplace.
Post the inference you need, when you need them. Commitment you are comfortable with. Providers bid. You pick the one that meets your spec.
we're already on track for 1 trillion tokens a month.
routerplus.com/capacity
RouterPlus (YC F26) retweeted
we are starting to host small gatherings around AI and inference with the first of the series tonight at 6:30pm at Hogpatch ( pizzas and snacks included ! ). Great chance to hang out with folks thinking and building in the space. luma link below
ps : we will be rebranding to @userouterplus, would love to show you what we’re upto as well!
RouterPlus (YC F26) retweeted
loved the GTM brainstorming with the guys. @XLN1999 and @alokbishoyi97 are the team I'd bet my life savings on. @EVO__HQ is gonna be the biggest force in managing inference loads, will come back to this post after the demo day, and again by '28 when they hit a billion in ARR (pardon for my lack of vision if you guys get there faster)
Kill the space, my Gs.
small get-together by friends and well wishers at @evo__hq before @XLN1999 and I set off to SF for YC fall batch
RouterPlus (YC F26) retweeted
inference engg chart porn inside @EVO__HQ dashboards
cc @XLN1999
RouterPlus (YC F26) retweeted
Replying to @useRouterPlus
@EVO__HQ got into YC!!
we are eager to talk with
- engineers who are hungry to become the best in the world at inference, posttraining and distributed systems.
- ops folks who have worked with datacentres and inference providers
RouterPlus (YC F26) retweeted
deployed and served custom models for a big @evo__hq customer over the past week
insane and rapid learnings
understood their traffic patterns, setup evals - did lot of ablation runs on multiple models
eventually we weren't getting enough savings, so we posttrained a model - first SFT and then other RL policies
then came inference optimization setup end to end , including custom spec dec models as per their data distribution, tuning vLLM configs, quantization etc
then more work around compute allocation , inference capacity planning / warming up as per traffic profiles etc
all orchestrated by our in house autoresearch / AI engineer
all this to ensure our customers get best bang for buck
we are now rapidly onboarding inference providers who can reliably service our growing demand
and if you are someone whose agentic/AI workload spends are more than $20k a month, then do reach out. We would love to figure what we could do for you!
RouterPlus (YC F26) retweeted
preach ! been telling ya’ll
also the most important part is the last line. “the challenge is customizing them to an org/team/individuals requires data”
this is exactly what @evo__hq solves for btw
RouterPlus (YC F26) retweeted
“There is no best model for every task. Routing, harnesses, evals, and the right mix of open and proprietary models can radically change the economics.”
exactly the thesis of us at @evo__hq btw :)
At Databricks, AI coding token spend is growing exponentially.
1. The “efficiency frontier” matters. On Databricks Coding Bench, GLM 5.2, Opus 4.8, and GPT 5.6-Sol sit on that frontier: the best quality per dollar.
2. We see cost regressions when comparing Opus 5.0 to 4.8. Newer model doesn’t always mean more efficient.
3. Hard budgets are the wrong primitive. Your biggest AI spenders may also be your most AI-leveraged engineers.
4. There is no best model for every task. Routing, harnesses, evals, and the right mix of open and proprietary models can radically change the economics.
We’re investing heavily in all 4, both for ourselves and for our customers.
RouterPlus (YC F26) retweeted
Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads.
It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload.
That can mean more than choosing just a single model or routing through a classifier. @EVO__HQ can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints.
As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated.
Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads.
EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.
RouterPlus (YC F26) retweeted
always great to hear folks making good use of @evo__hq's autoresearch engine
T - 48 hours untill we launch the next iteration that the team has been working on, putting to use all the learnings from our users
RouterPlus (YC F26) retweeted
If you want early access to not just yet another router™, but a system that automatically sets up evals, optimizes your workloads, and continuously finds the best model x provider for YOUR USECASE , then my DMs are open.
We already have a few teams in the beta, and they’ve reduced inference costs by 30–60% with no quality regressions.
@evo__hq




























