MLflow (@MLflow) header
MLflow (@MLflow) profile picture

The open source developer platform to build AI applications and models with confidence.

San Francisco, CA
Joined August 2018
Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)Photo by MLflow (@MLflow)
Hey all! Thanks for joining us. Where are you tuning in from? 🌐
1
64
Most teams hand-tune agent prompts until it breaks at scale. This cookbook shows a measured loop: domain judges, expert alignment, and GEPA optimization on MLflow traces. What's inside: • Tool-calling LangGraph agent you can adapt • Domain judges with pass/fail rubrics • Expert alignment via mlflow.log_feedback() + MemAlign • GEPA optimization: 40% → 85% pass rate in the example Run it once, compare baseline vs optimized in the UI. 👉 Check out the cookbook: mlflow.org/cookbook/agent-al…
3
22
1,459
Reminder: Find out what's new in MLflow LIVE this Wednesday 📅 Join us November 18 at 4:00 PM PT to explore the latest in AI observability, AIOps, AI Governance, and developer experience. We'll cover MLflow 3.18 features including Jev, Issue Detection, and the new Eval UX, followed by a live Q&A session. Register 👉 luma.com/mlflow-t8h9
1
4
417
Losing track of which prompt or model worked best? In this MLflow A–Z episode, MLflow Ambassador @shrinathsuresh covers E for experiment tracking. If you're changing models, prompts, and configurations, MLflow helps you see which version worked best by logging prompts, parameters, evaluation scores, latency, and token usage so every run is comparable. 🔗 Experiment tracking docs: mlflow.org/docs/latest/ml/tr… 🔗 Watch the full MLflow A–Z series: youtube.com/playlist?list=PL…
1
5
495
MLflow 3.18.0 is now available! 🎉 🔍 Automatic Issue Detection finds recurring agent failures across your traces, from wrong answers to unusual spending. 📚 The experimental Agent Skill Registry shares reusable skills with coding agents like Claude Code, Codex, and Cursor. 📊 Evaluation comparisons now pin a baseline and surface regressions. ✅ Experimental online evaluation runs your Python scorers on new traces on a self-hosted server. Full release: mlflow.org/releases/3.18.0/
4
1
6
577
Five coding agents usually means five API keys and one invoice nobody can explain. @2twitme shows how to govern that with MLflow AI Gateway. Agents point at one gateway, and it runs guardrails, enforces a budget, and traces every call. 👉 Learn more: mlflow.org/blog/govern-codin… #MLflow #GenAI #LLMOps #OpenSource
1
1
5
644
MLflow Ambassador @shrinathsuresh covers D for Dataset Tracking in this MLflow A-Z clip. Log training and evaluation datasets, version them, and associate those versions with your experiments. Skipping that is a huge mistake for GenAI, because the models are probabilistic. 🎥 youtube.com/watch?v=BoTbLWWV…
1
1
469
MLflow AI Gateway has budget policies. You set a daily, weekly, or monthly spend limit. 👇 When it hits, you can send a webhook to Slack, PagerDuty, or any HTTP alerting and keep serving traffic, or block requests until the window resets. The limit applies to every app that routes through the gateway. Get started: pip install 'mlflow[genai]' 🔗 mlflow.org/blog/gateway-budg…
6
504
This week at ODSC AI West, Debu Sinha (@databricks) is running a workshop on Evaluating LLM Applications with MLflow. Teams already run evals. The workshop is the part after: did the model actually get better, which samples regressed, and is the delta statistically significant or just noise. You build that pipeline in open source MLflow, from mlflow.genai.evaluate() through run comparison and a CI gate that fails the build when a regression is real. 🗓️ Wed, Oct 28 🕓 11:00 AM–12:00 PM PT 📍 Hyatt Regency SFO 🔗 odsc.ai/speakers-portfolio/e…
1
1
2
504
This week at ODSC AI West, Samraj Moorjani (@databricks) is giving a 30-min demo on Hidden Gems in Agent Traces. Traces show what the agent did. The talk is about what you do next: find repeating failure modes, auto-detect them across a large volume of traces, and check the fix with evals you can rerun. All in open source MLflow. 🗓️ Tues, Oct 27 🕓 11:05–11:35 AM PT 📍 Hyatt Regency SFO 🔗 Schedule: odsc.ai/west/expo/
4
3
2
5
618
Teams keep adding agent harnesses. Claude Code, Codex, Cursor, and the rest each keep their own conventions, so observing a run across them is messy. @2twitme walks through that harness layer, then how @omnigent_ai puts coding agents and custom agents behind one common API. From there, @opentelemetry traces land in MLflow so you can see how the agent actually behaved. Watch the full walkthrough on how to trace and debug AI coding agents with Omnigent and MLflow. 🎥 youtube.com/watch?v=vvJTyd-e…
10
3
13
828
Part 3 of the Self-Improvement & MLflow series by MLflow Ambassador Jonathan Bown walks through automating the verification layer of a self-improving AI workflow with MLflow and CI/CD. A coding agent reads traces at scale and can propose changes to itself, its skills, tools, memory, or the verifier. CI then checks each change before it is approved automatically or by a person. Read Part 3 👇 opendatascience.com/self-imp…
1
2
5
534
Scattered API keys. Different SDKs. No clear view of usage or cost. MLflow AI Gateway provides one @OpenAI-compatible endpoint, encrypted provider keys, and traces for requests, latency, tokens, and cost. How it works 👇 mlflow.org/blog/mlflow-ai-ga… #MLflow #LLMOps
6
425
What happens when every team has its own agents in production? You need more than LLMs. You need a control plane. For the third video in the MLflow A–Z series, MLflow Ambassador @shrinathsuresh covers C for Control Plane. He explains how MLflow AI Gateway brings access control, budgets, guardrails, traffic routing, and usage tracking into one place. 🔗 Learn more: mlflow.org/blog/gateway-guar… #MLflow #GenAI #LLMOps
2
1
2
463
The Claude SDK makes it fast to build an agent. With native @opentelemetry export and MLflow, it is just as fast to know whether the agent works. 🔹 Trace every model call by pointing a standard OTel exporter at MLflow 🔹 See each run as one trace, with model calls and tool calls under one agent span 🔹 Find expensive runs from token usage on the trace 🔹 Score every run with an LLM judge that inspects the trace 🔹 Compare system prompts with the same test prompts and judge 🔗 mlflow.org/blog/claude-sdk-o… #MLflow #Claude #OpenTelemetry
1
1
7
538
Part 2 of MLflow Ambassador Jonathan Bown’s series on self-improving AI workflows with MLflow is live. He walks a support agent through the loop: traces, feedback, expectations, and judges in; a versioned prompt and an evaluation dataset out; then a production alias move so the agent updates without code changes. @mlflow.test turns that dataset into a quality gate. When the manual pass stops scaling, a coding agent with MLflow skills proposes the themes, scorers, and tests. A human still accepts the changes. CI can run them as MLflow Evaluation runs. 🔗 opendatascience.com/self-imp… #MLflow #GenAI #ODSC
1
1
12
683
Claude Platform now supports native @opentelemetry tracing across its agent ecosystem. Bring traces from the Claude family into one backend using OTLP, the OpenTelemetry Protocol: 🔹 Messages API and Claude SDKs: model requests and tool-runner activity 🔹 Claude Code and Agent SDK: agent steps, tool calls, and subagent activity 🔹 Managed Agents: agent turns, model runs, and platform-run tools We published a guide covers setup OpenTelemetry export across the Claude family, including Claude Platform UI and SDK configuration. 👉 Setup guide: mlflow.org/blog/claude-otel-… #MLflow #opensource #otel
3
2
10
608