Whisper & Whisky Tasting
Join us for a whisky tasting for AI experts in San Francisco!
There will be:
Networking: Connect with fellow experts who are passionate about Gen AI.
A headshot photo booth: Join us upstairs to take your headshot photo.
Drinks & good food: Indulge in a spread of food while you sip Japanese Whisky.
Host
Baseten
NYC Tech Breakfast Club: ML Engineering Leaders
Join us for the NYC Tech Breakfast Club for ML Engineers
Driving model performance optimization: 2024 highlights
Baseten's model performance team works to optimize customer models for latency, throughput, quality, cost, features, and developer efficiency.
New observability features: activity logging, LLM metrics, and metrics dashboard customization
We added three new observability features for improved monitoring and debugging: an activity log, LLM metrics, and customizable metrics dashboards.
How we built production-ready speculative decoding with TensorRT-LLM
Our TensorRT-LLM Engine Builder now supports speculative decoding, which can improve LLM inference speeds.
A quick introduction to speculative decoding
Speculative decoding improves LLM inference latency by using a smaller model to generate draft tokens that the larger target model can accept during inference.
