Measure end-to-end response time vs inference time
On the model metrics tab, you can now use the dropdown menu to toggle between two different views for model inference time: End-to-end response time includes time for cold starts, queuing, and...
Changelog | Page 6
See our latest feature releases, product improvements and bug fixes
Oct 9, 2023
Sep 29, 2023
Track both active and starting up replicas
The replica count chart on the model metrics page is now broken out into “active” and “starting up” replicas. An active replica has loaded the model for inference and is actively responding to...
Sep 25, 2023
Wake scaled to zero models
ML models deployed on Baseten can automatically scale to zero when not in use so that you’re not paying for unnecessary idle GPU time. When a scaled to zero model is invoked, it spins up on a new...
Sep 7, 2023
Get 500 response code on model invocation error
Models deployed on Baseten using Truss 0.7.1 or later can now send the 500 response code when there is an error during model invocation. This change only affects newly deployed models. Any exception...
Sep 7, 2023
Manage API keys with new dashboard
We’ve updated your API key management panel with four key changes: Dropped randomly generated key names (e.g. legit-artichoke ) Instead, the first 8 characters of the key are displayed to make it...
Aug 24, 2023
Download the Truss of any model in your Baseten workspace
Every model deployed on Baseten uses Truss , our open-source framework for packaging models with their dependencies, hardware requirements, and more. You can now securely download the Truss of any...
Aug 22, 2023
Identifiable icon for inactive models
A small but mighty change: a 🚫 icon now clearly marks inactive models in the model dropdown and model version sidebar. The green dot for active models and moon icon for scaled-to-zero models remain...
Aug 14, 2023
New model metrics views
Get cleaner, more accurate insights into your model’s performance and load with the refreshed model metrics charts in each model’s overview tab.. Monitor requests per minute and both mean and peak...
Aug 7, 2023
Stream model output
Baseten now supports streaming model output. Instead of having to wait for the entire output to be generated, you can immediately start returning results to users with a sub-one-second...
Jul 22, 2023
Stop a model deployment
If you deploy a model by accident or want to shut down a failed deployment quickly, you can now stop any model deployment before it finishes on the model page. Stopping a model deployment puts that...
