Ultravox
State of the art multi-modal LLM developed by Fixie AI
Deploy Ultravox behind an API endpoint in seconds.
Deploy modelExample usage
You can stream responses from this LLM using OpenAI chat completions.
Input
1from openai import OpenAI
2
3model_id = ""
4baseten_api_key = os.environ["BASETEN_API_KEY"]
5
6client = OpenAI(
7 api_key=baseten_api_key,
8 base_url=f"https://bridge.baseten.co/{model_id}/direct/v1"
9)
10
11response = client.chat.completions.create(
12 model="fixie-ai/ultravox-v0.2",
13 messages=[
14 {"role": "system", "content": "You are a helpful assistant."},
15 {"role": "user", "content": "Tell me about the Ultravox model."}
16 ],
17 stream=True
18)
19
20for chunk in response:
21 print(chunk.choices[0].delta)JSON output
1[
2 "The",
3 "ultravox",
4 "model",
5 "is",
6 "..."
7]If you prefer to get the full output without streaming, that is also supported.
Input
1import requests
2
3model_id = ""
4baseten_api_key = os.environ["BASETEN_API_KEY"]
5
6resp = requests.post(
7 f"https://model-{model_id}.api.baseten.co/production/predict",
8 headers={"Authorization": f"Api-Key {baseten_api_key}"},
9 json={"messages":
10 [{
11 "role": "user",
12 "content": [
13 {"type": "text", "text": "Summarize the following: <|audio|>"},
14 {"type": "image_url", "image_url": {"url": f"data:audio/wav;base64,{base64_wav}"}}
15 ]
16 }],
17 "stream": False
18 },
19)
20
21resp = resp.json()
22print(resp['choices'][0]['message']['content'])JSON output
1{
2 "id": "cmpl-9b123ee253bc476082bbadef90fd336e",
3 "object": "chat.completion",
4 "created": 1720733794,
5 "model": "fixie-ai/ultravox-v0.2",
6 "choices": [
7 {
8 "index": 0,
9 "message": {
10 "role": "assistant",
11 "content": "To go to go build a computer the type of computers and solve problems that normal computers can't and to this day we're focused on that and if you look at all the the problems that that and the markets that we opened up as a result, it's you know things like Ah computational drug design um Ah weather simulation, materials design . These are all things of where we are."
12 },
13 "logprobs": "None",
14 "finish_reason": "stop",
15 "stop_reason": "None"
16 }
17 ],
18 "usage": {
19 "prompt_tokens": 161,
20 "total_tokens": 239,
21 "completion_tokens": 78
22 }
23}
