"Inference Engineering" is now available. Get your copy here
Blog

Inference engineering

AllResearchModel performanceAI engineeringInfrastructureNewsCommunityAI modelsProductFoundations
Research

Dense, on-policy, or both?

Max Kirkby
1 other
17
Research

Repeated KV cache for long-running agents

Charles O'Neill
1
Position

The Michael Scott Paper Company of AI

Charles O'Neill
2
Research

Distillation without the dark

Max Kirkby
2 others
3
Position

Continual learning and the post monolith AI era

Charles O'Neill
1 other
4
Research

BYO SWE-grep: automatically train blazing fast search sub-agents on your knowledge base (Pt. 1)

Charles O'Neill
1 other
5
Research

Lumina: building self-improving evaluation through customer-in-the-loop refinement

Charles O'Neill
5 others
5
Research

Introducing RadixMLP: Intra-batch deduplication for causal transformers

Michael Feil
3
Research

Upweight the strategy, not the tokens: faster training with explicit reasoning through RGT (Rationale-Guided Training)

Charles O'Neill
2 others
3
Research

Attention-based attribution: what your model is actually looking at

Charles O'Neill
4 others
4
Research

Training loss predicts evaluation performance, even for non-verifiable tasks

Charles O'Neill
1 other
5
Research

Robust, sample efficient SFT with prompt mutations

Harry Partridge
1 other
5
Research

Iterative SFT (iSFT): dense reward learning

Charles O'Neill
4 others
5
Research

Write small, learn forever: rank-1 LoRA for continual learning

Charles O'Neill
3 others
5
Research

Practical LoRA Research

Max Kirkby
1 other
7
Research

Do transformers notice their own mistakes? Finding a linear hallucination detector inside LLMs

Mudith Jayasekara
4 others
4
Research

Resurrecting the salmon: seeing clearer inside LLMs with domain-specific SAEs

Charles O'Neill
2 others
2
Research

Why mechanistic interpretability needs a paradigm inversion

Charles O'Neill
2 others
5
Research

Towards infinite context windows: neural KV cache compaction

Charles O'Neill
2 others
2