> ## Documentation Index
> Fetch the complete documentation index at: https://docs.withhopper.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Audio understanding

`audio-perception-1` takes audio and an instruction through `POST /v1/chat/completions` and returns text or JSON.

## Request

* `model` (string, required) — `audio-perception-1`.
* `messages` (array, required) — `system`, `user` and `assistant` messages. User content is a string or an array of `text` and `input_audio` parts.
* `input_audio` — `{"data": "<base64>", "format": "wav" | "mp3"}`. Up to 25 MB per part and 10 minutes per request.
* `response_format` (object, optional) — `{"type": "json_schema", "json_schema": {...}}` or `{"type": "json_object"}`. Output is validated against the schema.
* `temperature` (number, optional, default 0), `top_p` (number, optional), `seed` (integer, optional).
* `max_tokens` or `max_completion_tokens` (integer, optional, default 1024, max 4096).

```python Python theme={null}
import base64
import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.withhopper.com/v1",
    api_key=os.environ["HOPPER_API_KEY"],
)

audio = base64.b64encode(open("call.wav", "rb").read()).decode()

response = client.chat.completions.create(
    model="audio-perception-1",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Classify the primary expressed emotion in this clip."},
                {"type": "input_audio", "input_audio": {"data": audio, "format": "wav"}},
            ],
        }
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "emotion",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "primary_emotion": {"enum": ["happy", "neutral", "sad"]},
                    "confidence": {"type": "number"},
                    "evidence": {"type": "string"},
                },
                "required": ["primary_emotion", "confidence", "evidence"],
            },
        },
    },
)
print(json.loads(response.choices[0].message.content))
```

```typescript TypeScript theme={null}
import { readFile } from "node:fs/promises";
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.withhopper.com/v1",
  apiKey: process.env.HOPPER_API_KEY,
});

const audio = (await readFile("call.wav")).toString("base64");

const response = await client.chat.completions.create({
  model: "audio-perception-1",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "Did the customer get their task done? Reply as JSON with key done (boolean)." },
        { type: "input_audio", input_audio: { data: audio, format: "wav" } },
      ],
    },
  ],
  response_format: { type: "json_object" },
});
console.log(JSON.parse(response.choices[0].message.content!));
```

```bash cURL theme={null}
curl https://api.withhopper.com/v1/chat/completions \
  -H "Authorization: Bearer $HOPPER_API_KEY" \
  -H "Content-Type: application/json" \
  -d @- <<EOF
{
  "model": "audio-perception-1",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "How would a human rate this call, 1 to 10? Reply with the number."},
      {"type": "input_audio", "input_audio": {"data": "$(base64 < call.wav | tr -d '\n')", "format": "wav"}}
    ]
  }]
}
EOF
```

## Response

```json theme={null}
{
  "id": "chatcmpl-2fe2f2982f56499182ae8a86d5909797",
  "object": "chat.completion",
  "model": "audio-perception-1",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "{\"primary_emotion\": \"sad\", \"confidence\": 0.85, \"evidence\": \"Subdued, hesitant voice while disputing an unexpected charge.\"}"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 725,
    "completion_tokens": 49,
    "total_tokens": 774,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "audio_seconds": 30
  }
}
```

Audio counts as input tokens, about 20 per second.

## Limits

* 8 requests per second. Over that: `429 rate_limit_exceeded` with `Retry-After`.
* An output that misses `response_format` after a retry: `422 response_format_mismatch`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.