Flexible instance types per model deployment
Model deployments now support changing instance types, enabling you to experiment with different hardware configurations and use specific hardware for staging, development, and production environments.
We also added more environment details to the UI, highlighting autoscaling settings, promotion settings, and instance types. You can find a list of all supported instance types in our docs here.
Changing instance types for new deployments
Updating the instance type for a deployment creates a new deployment with the specified instance type. When you run truss push from the CLI on an existing model, it will respect any changes made to the resources field in your config.yaml.
Changing instance types on promotion
When promoting a deployment to an environment, you now have the option to keep its instance type or use the instance type of the environment you're promoting to.
The instance type of the environment will be used by default. You can opt not to use the environment instance type via the promotion dialog in the Baseten UI, or in the latest version of the Truss CLI using the --no-preserve-env-instance-type flag (for example, truss push --environment production --no-preserve-env-instance-type).
