Bland AI breaks latency barriers with record-setting speed using Baseten

Company overview

Based in San Francisco, Bland AI is transforming how enterprises handle phone calls using AI. Their platform analyzes post-call data and facilitates real-time conversations powered by AI models. Large enterprises rely on Bland AI to deliver seamless, human-like phone interactions that are scalable, reliable, and incredibly fast.

Challenges

Reducing latency for seamless conversations

From the start, Bland AI knew that delivering a voice AI solution that people genuinely loved would depend on latency performance.

Isaiah Granet, Co-Founder and CEO of Bland AI, and his team were determined to solve the issue of response delays, which were initially as high as three seconds. These delays make interactions feel clunky and unsatisfying, turning away potential enterprise clients who expected seamless, real-time conversations.

Latency was the make-or-break factor for us. We knew that unless we could get AI responses under 400 milliseconds, the experience wouldn’t be good enough for enterprises.

Scaling infrastructure to meet rapid customer growth

As Bland AI quickly gained traction, they faced the challenge of scaling their infrastructure to meet surging demand. With usage growing by 50x in just five months, Bland AI needed a platform that could scale with them while maintaining performance and reliability.

The team knew they couldn’t afford downtime or technical issues, especially as enterprises relied on their platform for mission-critical operations.

We experienced incredible initial growth and needed to ensure that our infrastructure could scale without compromising quality. We were always focused on speed, and that’s what led us to working with Baseten. We knew we needed a crazy response time to put us on the map.

Solution

Achieving record-breaking speed with Baseten

Bland AI partnered with Baseten to overcome its latency challenges and deliver an industry-defining solution. By January 2024, Bland AI had launched Bland Turbo, a breakthrough product that achieved an end-to-end AI call latency of under 400 milliseconds. This didn’t just meet the demands of enterprises—it shattered industry benchmarks. 

When we hit sub-400 milliseconds with Baseten, people didn’t believe it was real. We broke the record by such a wide margin that it seemed impossible to some.

This performance allowed Bland AI to provide real-time conversations that felt natural and fluid, giving them a clear competitive edge in the market.

Scaling seamlessly with traffic-based autoscaling

As Bland AI scaled rapidly, Baseten’s traffic-based autoscaling enabled the platform to handle the increased usage without interruptions. Over the next five months, Bland AI saw 50x growth in usage, and Baseten’s infrastructure scaled seamlessly to support this surge. 

Baseten allowed us to grow without worrying about managing infrastructure. Their autoscaling handled our growth seamlessly, and we could focus on delivering a great experience for our users.

This ensured that Bland AI could meet the needs of its enterprise customers while maintaining the high quality of service they demanded.

Results

Sub-400 millisecond end-to-end response time

With Baseten, Bland AI achieved a sub-400 millisecond end-to-end response time for their voice AI product, breaking industry records and setting a new standard for AI-driven conversations.

Baseten helped us cross the line into an entirely new realm of AI performance. Under 400 milliseconds wasn’t just fast—it was groundbreaking.

This achievement positioned Bland AI as a leader in the voice AI market, and their product became known for its unprecedented speed.

Scaled to 50x growth in usage over five months

Bland AI’s rapid growth required a flexible and scalable infrastructure. Baseten’s autoscaling enabled Bland to handle a 50x growth in usage without any technical breakdowns.

If we had tried to scale this ourselves, we would have been overwhelmed. Baseten handled the growth for us.

This scalability ensured Bland could continue to deliver reliable, real-time voice AI as their customer base expanded.

Reliable uptime during peak demand

With Baseten, Bland AI maintained flawless reliability, even during peak demand. Granet noted that their system had never experienced an outage while using Baseten’s platform, even with high concurrency.

We’ve never had an outage with Baseten. The platform has been rock solid, giving us the confidence to scale quickly without worrying about downtime.

This reliability has been critical as Bland positioned itself as the go-to solution for large enterprises that depend on uninterrupted service for mission-critical operations.