NYC Tech Breakfast Club: ML Engineering Leaders
We're back! To kick off 2025 Tech Breakfast Club is teaming up with Baseten to host a Tech Breakfast Club for CTOs and machine learning leaders in New York.
If you're an AI leader living in New York, come join for a hearty breakfast and good conversation.
Host
Baseten
NYC Tech Breakfast Club: ML Engineering Leaders
Join us for the NYC Tech Breakfast Club for ML Engineers
Driving model performance optimization: 2024 highlights
Baseten's model performance team works to optimize customer models for latency, throughput, quality, cost, features, and developer efficiency.
New observability features: activity logging, LLM metrics, and metrics dashboard customization
We added three new observability features for improved monitoring and debugging: an activity log, LLM metrics, and customizable metrics dashboards.
How we built production-ready speculative decoding with TensorRT-LLM
Our TensorRT-LLM Engine Builder now supports speculative decoding, which can improve LLM inference speeds.
A quick introduction to speculative decoding
Speculative decoding improves LLM inference latency by using a smaller model to generate draft tokens that the larger target model can accept during inference.
