--- title: "Real-time API Reference" section: "Live translation" order: 1 sidebarLabel: "API Reference" --- Pinch real-time translation is a single WebSocket connection. You stream audio in and receive transcripts and translated speech back on the same socket. No SDK — works from any language that can open a WebSocket. A session runs in one of three modes, chosen when you create it: | Mode | Send | Receive | Parameters | | --- | --- | --- | --- | | **Speech to speech** (default) | audio | transcripts + translated speech | — | | **Speech to text** | audio | transcripts | `audioOutputEnabled=false` | | **Text to speech** | text | speech | `mode=tts` — see [Text to speech](#text-to-speech-mode) | **Base URL:** `https://ws.startpinch.com` --- ## Flow 1. `GET /v1/session` — authenticate, get a short-lived `ws_url`. 2. Open a WebSocket to `ws_url`. 3. Wait for `{"type":"ready"}`. 4. Send audio as binary frames. 5. Receive transcripts (JSON) and, in speech-to-speech mode, translated speech (binary) on the same socket. 6. Close the socket when done. --- ## 1. Create a session ``` GET /v1/session ``` Returns a pre-signed WebSocket URL valid for 60 s. The WebSocket itself carries no further auth. ### Request headers | Header | Value | Required | | --- | --- | --- | | `Authorization` | `Bearer ` | Yes | ### Query parameters All query parameters become the session's metadata. | Param | Type | Default | Description | | --- | --- | --- | --- | | `sourceLanguage` | string | `en` | Language of the incoming audio, ISO-639-1 (`en`, `de`, `zh`). Regional tags are accepted (`en-US`); only the primary subtag is used. `auto` detects it per utterance on `relay-2`. See [Detecting the source language](#detecting-the-source-language). | | `languageHints` | string | — | Comma-separated ISO-639-1 codes the speech is expected in, e.g. `en,de,fr`. Read only with `sourceLanguage=auto`. | | `targetLanguage` | string | `es` | Language to translate into. | | `audioOutputEnabled` | bool | `true` | `true` = translated speech + transcripts, `false` = transcripts only. | | `voiceId` | string | — | Numeric voice ID from [`GET /v1/voices`](#voices), or `auto` to give each speaker their own matched voice. See [Voices](#voices). | | `voiceType` | string | `male` | `male` or `female`. Picks a default voice when `voiceId` is not set. | | `modelName` | string | `relay-1` | Recognition model: `relay-1` or `relay-2`. See [modelName](#modelname). | | `finalizeMode` | string | `stable` | `stable` = multiple finals per segment as spans lock in; `sentence` = one final per sentence. See [Finalize mode](#finalize-mode). | | `twoWay` | bool | `false` | Listen for both `sourceLanguage` and `targetLanguage`, detect which one each utterance is in, and translate into the other. See [Two-way sessions](#two-way-sessions). | | `context` | string | — | Free-text description of the session, up to 2 000 characters. Biases recognition and translation toward the names and jargon it mentions. See [Session context](#session-context). | | `sourceLanguageLabel` | string | — | Overrides how the source language is described to the translator. | | `targetLanguageLabel` | string | — | Overrides how the target language is described to the translator. Use it for register or persona, e.g. `professional English`, `casual Spanish`. | | `mode` | string | `s2tt` | `s2tt` (speech in) or `tts` (text in). See [Text to speech](#text-to-speech-mode). | Invalid combinations are rejected at the start of the WebSocket session with an [`error`](#error) frame, not at `GET /v1/session`. ### Try it
GET /v1/session
### Response (200) ```json { "session_id": "ss_01JF...", "ws_url": "wss://ws.startpinch.com/session?token=...", "expires_in": 60, "model_version_id": "a1b2c3d4e5f6" } ``` Open `ws_url` within `expires_in` seconds. `model_version_id` identifies the exact server build serving this session. Log it alongside any user-reported issue so support can map the report back to the build that produced it. ### Error responses | Code | Meaning | | ---- | ------- | | 401 | Missing or invalid API key. | | 402 | Insufficient balance. Add funds in the [Developer Portal](https://portal.startpinch.com/dashboard/developers). | | 403 | API key not entitled to real-time translation. | | 503 | Capacity exhausted — retry after a short backoff. | --- ## 2. Open the WebSocket Open a WebSocket to `ws_url`. No additional headers — the URL is pre-signed. An expired or invalid `ws_url` fails the WebSocket handshake with HTTP `401`; create a new session. The first frame from the server is: ```json {"type": "ready", "session_id": "ss_01JF...", "worker_id": "wkr_..."} ``` Do not send audio before `ready`. After `ready`, the socket is fully bidirectional. If the session parameters cannot be served — an unknown `modelName`, a voice ID that does not exist, a `targetLanguage` without voice output while `audioOutputEnabled=true` — the server sends an [`error`](#error) frame immediately after `ready` and closes. --- ## 3. Audio in — client → server Binary frames of raw PCM audio: - **Format:** PCM16 little-endian, mono - **Sample rate:** 16 kHz - **Frame size:** 10–40 ms recommended (160–640 samples = 320–1280 bytes) No container, no headers — only the sample bytes. Smaller frames feel more interactive; larger frames add latency but reduce overhead. ### Push-to-talk When the user releases the mic, send `{"type":"finalize"}`. The server flushes the current segment, emits `is_final` transcripts (and translated speech, in speech-to-speech mode), and keeps the session open for the next press. See [`finalize`](#finalize). Sending ~300 ms of silence (PCM16 zeros) after release also works, with more tail latency. --- ## 4. Audio out — server → client When `audioOutputEnabled=true`, binary frames of translated speech: - **Format:** float32 little-endian, mono - **Sample rate:** 24 kHz - **Frame size:** ~80 ms (1920 samples) Play frames as they arrive. Each frame is a standalone chunk. In speech-to-text mode (`audioOutputEnabled=false`) no binary frames are sent. --- ## 5. Events — server → client JSON text frames. Dispatch on `type`. | `type` | When | | --- | --- | | [`ready`](#2-open-the-websocket) | Once, first frame after connecting. | | [`original_transcript`](#original_transcript) | Recognised source-language speech, interim and final. | | [`translated_transcript`](#translated_transcript) | Translation of the current segment, interim and final. | | [`metadata_ack`](#metadata_ack) | Reply to [`update_metadata`](#update_metadata). | | [`error`](#error) | The session could not be started, or a message could not be handled. | | [`speech_end`](#text-to-speech-mode) | Text-to-speech mode only: one utterance's audio is complete. | | [`speak_ack`](#text-to-speech-mode) | Text-to-speech mode only: a `speak` with a `correlation_id` was accepted. | Transcript events share these fields: | Field | Description | | --- | --- | | `correlation_id` | Groups an original transcript with its translation. A new segment gets a new ID. | | `is_final` | `false` = may still be revised; `true` = stable. See [Interim vs final](#interim-vs-final). | | `source_language`, `target_language` | Session languages. | | `detected_language` | Language of the current utterance. Equal to `source_language` when one is set; with [`sourceLanguage=auto`](#detecting-the-source-language) it is what the model heard, and in [two-way sessions](#two-way-sessions) whichever of the pair is being spoken. `null` on the first interim frames of an `auto` utterance, before detection lands. | | `timestamp` | Unix seconds (float). | And, on `relay-2`, which separates speakers: | Field | Description | | --- | --- | | `speaker` | Opaque label for who was talking (`"1"`, `"2"`, …). The same label means the same person for the life of the session. Not a name; do not display it raw. | | `start_ms`, `end_ms` | Where this utterance sits in the session's audio, milliseconds from the first sample you sent. | | `voiceId` | Under [`voiceId=auto`](#voiceidauto--one-voice-per-speaker): the voice this speaker's translation is spoken in, once matched. | These fields are **omitted** when they do not apply, never sent as `null`. On `relay-1` there is no `speaker` at all; treat a missing `speaker` as "unknown", not as "one speaker". ### `original_transcript` Recognised source-language speech. `text` is the canonical field. ```json { "type": "original_transcript", "text": "Hello, how are you?", "is_final": true, "correlation_id": "seg_a8b2c3d4e5f6", "source_language": "en", "target_language": "es", "detected_language": "en", "timestamp": 1770933604.048, "speaker": "1", "start_ms": 1240, "end_ms": 2810, "voiceId": "16" } ``` ### `translated_transcript` Translation of the current segment. `text` is the translated text (same as `translated_text`); `original_text` is the source-language text that produced it. ```json { "type": "translated_transcript", "text": "Hola, ¿cómo estás?", "translated_text": "Hola, ¿cómo estás?", "original_text": "Hello, how are you?", "is_final": true, "correlation_id": "seg_a8b2c3d4e5f6", "source_language": "en", "target_language": "es", "detected_language": "en", "timestamp": 1770933604.945, "speaker": "1", "start_ms": 1240, "end_ms": 2810, "voiceId": "16" } ``` ### Interim vs final - `is_final: false` — partial, may be revised as more audio arrives. Use for live-feedback UI. - `is_final: true` — stable; never revised. Cadence depends on [`finalizeMode`](#finalize-mode). ### `metadata_ack` Sent in reply to [`update_metadata`](#update_metadata). `applied` names the keys that took effect. ```json {"type": "metadata_ack", "applied": {"context": true}} ``` ### `error` ```json {"type": "error", "error": "voiceId is not a valid Pinch voice. Call GET /v1/voices for the list of available voice IDs.", "timestamp": 1770933600.0} ``` An `error` sent right after `ready` means the session parameters were rejected; the server closes the socket afterwards. An `error` later in the session refers to the message that caused it, and the socket stays open. --- ## 6. Messages — client → server JSON text frames. | `type` | Modes | Purpose | | --- | --- | --- | | [`finalize`](#finalize) | speech in | Commit the current segment now. | | [`update_metadata`](#update_metadata) | all | Change session parameters mid-stream. | | [`speak`](#text-to-speech-mode) | text to speech | Synthesise a line of text. | Unknown message types are ignored. ### `finalize` ```json {"type": "finalize"} ``` Commits whatever has been spoken so far without closing the session: 1. The server waits ~300 ms for audio frames still in flight to land — your client needs no tail buffering or trailing silence. 2. The current segment is finalised: you receive `original_transcript` and `translated_transcript` with `is_final: true`, and translated speech when `audioOutputEnabled=true`. 3. The session stays open. The next audio frame starts a fresh segment with a new `correlation_id`. ### `update_metadata` ```json {"type": "update_metadata", "metadata": {"context": "Acme Corp Q3 all-hands. CEO Priya Ramanathan."}} ``` The server replies with [`metadata_ack`](#metadata_ack). What is applied live depends on the mode: | Mode | Applied live | | --- | --- | | Speech in (`s2tt`) | `context` | | Text to speech (`tts`) | `voiceId`, `targetLanguage` | Other keys — `sourceLanguage`, `targetLanguage`, `voiceId`, `modelName`, `twoWay`, … in a speech session — cannot change on a running session. To change them, send `finalize`, close the socket, and create a new session. --- ## 7. Close Close the WebSocket with code `1000` when done. Server close codes: | Code | Meaning | | ---- | ------- | | 1000 | Normal closure, including after a session-start `error`. | | 1001 | Server shutting down — create a new session. | | 4503 | No capacity on the server that accepted the socket — create a new session. | | 4500 | Internal error. | --- ## modelName Selects the model that recognises speech. Both translate the same way; they differ in what the recognition step can tell you. | `modelName` | Separates speakers | `start_ms` / `end_ms` | `voiceId=auto` | `sourceLanguage=auto` | | --- | --- | --- | --- | --- | | `relay-1` | no | no | no | no | | `relay-2` | **yes** | yes | yes | yes | Pick `relay-2` when you need to know **who** said something — one transcript per person in a meeting, per-speaker captions, or a distinct output voice per participant — or when you do not know in advance **which language** will be spoken. It adds [`speaker`, `start_ms` and `end_ms`](#5-events--server--client) to transcript events; `relay-1` omits them. Everything above recognition — finalisation, [`finalizeMode`](#finalize-mode), [`context`](#session-context), an explicit `voiceId` — behaves the same on both. An unrecognised `modelName` is rejected rather than defaulted, so a typo surfaces as an `error` instead of as unexpected behaviour and cost. --- ## Voices The voice library has 70 voices, each with a numeric ID, a name and a gender. List it — no auth needed: ```bash curl https://ws.startpinch.com/v1/voices curl https://ws.startpinch.com/v1/voices?gender=female ``` ```json { "voices": [ { "id": 1, "name": "Petra", "gender": "female" }, { "id": 2, "name": "Ulysses", "gender": "male" } ] } ``` Pass an `id` as the `voiceId` session parameter. **Every voice speaks every voice-output language**, so a voice pinned to a speaker keeps the same identity when `targetLanguage` changes. If `voiceId` is not set, `voiceType` (`male`, default, or `female`) selects a default voice for that gender. A `voiceId` that does not exist fails the session with an [`error`](#error). It is not ignored and not swapped for a default. ### `voiceId=auto` — one voice per speaker `voiceId=auto` gives each speaker the library voice that sounds most like them, instead of one voice for the whole session. It requires a model that separates speakers: `modelName=relay-2`. With `relay-1` the session is rejected with an `error`. - A speaker's first lines are already spoken in a matched voice — the first decision is made after about half a second of their speech, usually settling gender and little more. The match is revisited as more of the speaker is heard and locked after about 4 s. - Until a speaker's first decision lands, their lines use the session default (`voiceType`) and carry no `voiceId`. - Matching never delays transcripts or audio. A decision that lands late applies from the speaker's next line. - Two speakers can be given the same voice. Speaker separation occasionally splits one person into two labels; sharing a voice is the right outcome then. Transcript events carry the matched voice as [`voiceId`](#5-events--server--client), a string in the same ID space as `GET /v1/voices`, so you can show it or pin it explicitly in a later session. A few voices with a strong regional accent are never chosen by `auto`; they remain valid as an explicit `voiceId`. --- ## Finalize mode Controls when `is_final: true` is emitted. Set with the `finalizeMode` query parameter. - **`stable`** (default) — multiple finals per segment, each a prefix-extension of the previous, emitted as soon as a span is locked in. Lowest time to first stable text. - **`sentence`** — one final per sentence boundary. Best for transcript logging or downstream processing. `twoWay=true` always uses `sentence`. Partials (`is_final: false`) stream in both modes. --- ## Detecting the source language `sourceLanguage=auto` leaves the input language open. Each utterance is recognised in whatever language is spoken and translated from there; `detected_language` on its events carries that language. Requires `modelName=relay-2`; with `relay-1` the session is rejected with an `error`. ``` GET /v1/session?sourceLanguage=auto&languageHints=en,de,fr&targetLanguage=es&modelName=relay-2 ``` - `languageHints` narrows what the model listens for. Pass the languages you expect, comma-separated; the more you leave out, the more the first words of each utterance decide. Without it, any supported language. - The first interim frames of an utterance can arrive with `detected_language: null`. Translation waits for the detection; the original transcript does not. - An utterance already in `targetLanguage` is passed through: `translated_transcript` carries the original text and no audio is produced for it. - `source_language` on events echoes `auto`. - `auto` and `twoWay` cannot be combined. Use `twoWay` when the session is a conversation between two known languages and both sides need translating; use `auto` when one side speaks and you do not know what. --- ## Two-way sessions `twoWay=true` turns `sourceLanguage` and `targetLanguage` into a pair. The session listens for both, detects which one each utterance is in, and translates into the other. `detected_language` on each event tells you which side was spoken. Two-way sessions always use `finalizeMode=sentence`. With `audioOutputEnabled=true`, both languages must have voice output — see [Supported languages](/docs/supported-languages). --- ## Session context Pass a free-text description of the session as `context` to bias recognition and translation toward the names, products and jargon you mention. ```json { "context": "Acme Corp Q3 all-hands. CEO Priya Ramanathan, CFO Marco Bianchi. Topics: ARR, churn, EMEA expansion." } ``` Helpful for: - **Proper nouns** the model would not otherwise know — people, products, companies, places. - **Domain-specific jargon** that is easily misheard. Any language. Up to 2 000 characters. Send an empty string to clear. Context can be changed mid-session with [`update_metadata`](#update_metadata). --- ## Text to speech mode A session created with `mode=tts` skips recognition and translation. You send text; the server synthesises it and returns speech on the same socket, in the same [audio format](#4-audio-out--server--client) as speech-to-speech. Use it when you already have the text and want to choose the voice per line — an interpreter that assigns one voice per participant, for instance. Create the session with `mode=tts` plus the starting voice and language: ``` GET /v1/session?mode=tts&targetLanguage=es&voiceId=16 ``` `audioOutputEnabled=false` is rejected in this mode. After the usual `ready`, the server sends a second one for the mode: ```json {"type": "ready", "mode": "tts"} ``` Then send one `speak` message per utterance: ```json {"type": "speak", "text": "Hola, ¿cómo estás?", "voiceId": "16", "language": "es", "correlation_id": "line-42"} ``` | Field | Required | Description | | --- | --- | --- | | `text` | yes | Text to synthesise. | | `voiceId` | no | Voice for this and following lines. | | `language` | no | Language of the text, for this and following lines. | | `correlation_id` | no | Echoed back in `speak_ack`. | `voiceId` and `language` are sticky: omit them and the session keeps what it last used. Both can also be changed with [`update_metadata`](#update_metadata). For each utterance the server sends: 1. `{"type": "speak_ack", "correlation_id": "line-42"}` — only if you set a `correlation_id`. 2. Binary audio frames. 3. `{"type": "speech_end"}` once the utterance's audio is complete, so you can tell a finished line from one still arriving. Binary frames sent to a `tts` session are answered with an `error`. --- ## Languages - **Input** (`sourceLanguage`): see [Supported languages](/docs/supported-languages), or `auto` on `relay-2` — see [Detecting the source language](#detecting-the-source-language). - **Voice output** (`targetLanguage` with `audioOutputEnabled=true`): the languages marked with voice output on that page. Requesting one without voice output returns an `error`; pass `audioOutputEnabled=false` for transcripts only. - **Text output** (`audioOutputEnabled=false`): every supported language. --- ## Minimal client (Python) ```python import asyncio, json, httpx, sounddevice as sd, websockets API_KEY = "pk_..." BASE = "https://ws.startpinch.com" async def main(): async with httpx.AsyncClient() as http: r = await http.get( f"{BASE}/v1/session", params={"sourceLanguage": "en", "targetLanguage": "es", "modelName": "relay-2", "voiceId": "auto"}, headers={"Authorization": f"Bearer {API_KEY}"}, ) r.raise_for_status() ws_url = r.json()["ws_url"] async with websockets.connect(ws_url) as ws: ready = json.loads(await ws.recv()) assert ready["type"] == "ready" async def mic_to_ws(): with sd.RawInputStream(samplerate=16000, channels=1, dtype="int16", blocksize=320) as s: while True: buf, _ = s.read(320) await ws.send(bytes(buf)) async def ws_to_out(): out = sd.RawOutputStream(samplerate=24000, channels=1, dtype="float32") out.start() async for frame in ws: if isinstance(frame, (bytes, bytearray)): out.write(frame) else: evt = json.loads(frame) if evt.get("type") == "translated_transcript" and evt["is_final"]: print(evt.get("speaker"), evt["text"]) elif evt.get("type") == "error": raise RuntimeError(evt["error"]) await asyncio.gather(mic_to_ws(), ws_to_out()) asyncio.run(main()) ``` --- ## Minimal client (TypeScript / Node) ```ts import WebSocket from "ws"; const BASE = "https://ws.startpinch.com"; const key = process.env.PINCH_API_KEY!; const params = new URLSearchParams({ sourceLanguage: "en", targetLanguage: "es", modelName: "relay-2", voiceId: "auto", }); const r = await fetch(`${BASE}/v1/session?${params}`, { headers: { Authorization: `Bearer ${key}` }, }); const { ws_url } = await r.json(); const ws = new WebSocket(ws_url); ws.on("message", (data, isBinary) => { if (isBinary) { // float32 LE @ 24 kHz — pipe to audio output } else { const evt = JSON.parse(data.toString()); if (evt.type === "translated_transcript" && evt.is_final) console.log(evt.speaker, evt.text); if (evt.type === "error") console.error(evt.error); } }); // After {type:"ready"}, send PCM16LE @ 16 kHz: ws.send(int16Buffer, { binary: true }); ```