Baseten @ re:Invent
Join us at AWS re:Invent and experience Baseten’s AI inference platform, offering industry-leading performance, security, and scale for organizations building AI products. With features like on-chip model optimization, autoscaling, and a developer-friendly interface, Baseten enables rapid, secure, and cost-effective deployment, ensuring you can maintain competitive user experiences while adhering to strict compliance requirements.
To learn more:
Visit us at booth 1522 and grab your "Artificially Intelligent” merch
Schedule a demo and see how we can speed up your AI models at scale
Grab Coffee with one of our Executives and learn more about what Baseten can do for you
Have a need for speed? Register for our exotic car driving experience on Tuesday Dec 3
Join our lightning Talk with Phil Howes Dec 4 @ 4:30PM Session AIM 248-S
We look forward to seeing you in December!
Host
Baseten
New observability features: activity logging, LLM metrics, and metrics dashboard customization
We added three new observability features for improved monitoring and debugging: an activity log, LLM metrics, and customizable metrics dashboards.
How we built production-ready speculative decoding with TensorRT-LLM
Our TensorRT-LLM Engine Builder now supports speculative decoding, which can improve LLM inference speeds.
A quick introduction to speculative decoding
Speculative decoding improves LLM inference latency by using a smaller model to generate draft tokens that the larger target model can accept during inference.
Introducing our Speculative Decoding Engine Builder integration for ultra-low-latency LLM inference
Our new Speculative Decoding integration can cut latency in half for production LLM workloads.
Generally Available: The fastest, most accurate and cost-efficient Whisper transcription
At Baseten, we've built the most performant (1000x real-time factor), accurate, and cost-efficient speech-to-text pipeline for production AI audio transcription
Introducing Custom Servers: Deploy production-ready model servers from Docker images
Deploy production-ready model servers on Baseten directly from any Docker image using just a YAML file.
Create custom environments for deployments on Baseten
Test and deploy ML models reliably with production-ready custom environments, persistent endpoints, and seamless CI/CD.
Introducing canary deployments on Baseten
Our canary deployments feature lets you roll out new model deployments with minimal risk to your end-user experience.
