"Inference Engineering" is now available. Get your copy here
Blog

Inference engineering

AllResearchModel performanceAI engineeringInfrastructureNewsCommunityAI modelsProductFoundations
Model performance

How to train custom EAGLE-3 heads for speculative decoding

Model Performance Team
eagle 3
Model performance

Sub-3 millisecond named entity recognition (NER) inference

Michael Feil
1 other
Sub-3 millisecond named entity recognition (NER) inference
Model performance

I spent 31 hours on the math behind TurboQuant so you don't have to

Ali Taha
Turboquant
Model performance

How we built the fastest GLM 5 API

Tri Dao
2 others
GLM-5
Model performance

Four Bits: 4-Bit Quantization for Inference Optimization in FLUX.2 [dev]

Ali Taha
four bits
Model performance

How we built the fastest Kimi K2.5 on Artificial Analysis

Tri Dao
3 others
kimi k2.5
Model performance

How to run LLM performance benchmarks (and why you should)

Alex Ker
1 other
LLM performance metrics
Model performance

Open-Sourcing Baseten’s Suffix Automaton MTP Accelerator

Mahmoud Hassan
1 other
Open-sourcing Baseten’s Suffix Automaton MTP Accelerator
Model performance

Wan 2.2 video generation in less than 60 seconds

Faraz Shahsavan
3 others
Wan 2.2 video generation in less than 60 seconds
12345
Model performance | Blog