How to build function calling and JSON mode for LLMs
We introduced built-in functionality for function calling and structured output for LLM deployments with our TensorRT-LLM Engine Builder. In this webinar, we’ll dive into how we built it!
What you'll learn:
Understanding structured output and function calling: Learn how these features ensure schema-compliant model outputs and enable LLMs to select and execute specific tools.
Building JSON mode and tool use: Dive into the implementation details, including defining schemas and tools, building a state machine, and using logit biasing to force valid output.
Hands-on demo: See these features in action with live demos and a comparison with alternative methods.
Watch now on-demand!
Host
Speakers
Announcing Baseten’s $75M Series C
Baseten raised a $75M Series C to power mission-critical AI inference for leading AI companies.
How multi-node inference works for massive LLMs like DeepSeek-R1
Running DeepSeek-R1 on H100 GPUs requires multi-node inference to connect the 16 H100s needed to hold the model weights.
Testing Llama 3.3 70B inference performance on NVIDIA GH200 in Lambda Cloud
The NVIDIA GH200 Superchip combines an NVIDIA Hopper GPU with an ARM CPU via high-bandwidth interconnect
Baseten Chains is now GA for production compound AI systems
Baseten Chains delivers ultra-low-latency compound AI at scale, with custom hardware per model and simplified model orchestration.
How to deploy low-latency compound AI systems at scale with Baseten Chains
Learn how to deploy ultra-low-latency compound AI with seamless model orchestration, custom autoscaling, and optimized hardware.
San Jose Sharks Vs Carolina Hurricanes @ GTC
Join us in a private suite for the Hurricanes vs Sharks game during NVIDIA GTC.
NYC Tech Breakfast Club: ML Engineering Leaders
Tech Breakfast Club is teaming up with Baseten to host a Tech Breakfast Club for CTOs and machine learning leaders in New York.
Private, secure DeepSeek-R1 in production in US & EU data centers
Dedicated deployments of DeepSeek-R1 and DeepSeek-V3 offer private, secure, high-performance inference that's cheaper than OpenAI
