Services

STT, LLM and TTS providers are all processors behind small interfaces. Swapping one is a one-line change, because nothing in the pipeline depends on which provider you picked.

The pattern

Every provider is a Config struct plus a constructor:

stt := deepgram.NewSTT(deepgram.Config{APIKey: key})
llm := anthropic.NewLLM(anthropic.Config{APIKey: key, Model: "claude-sonnet-5"})
tts := cartesia.NewTTS(cartesia.Config{APIKey: key, VoiceID: id})

Library packages read no environment variables and take no functional options. Configs are validated with go-playground/validator tags, so a bad config fails at construction rather than mid-call. Reading env vars, flags or config files is your app’s job. See examples/.

Most fields have sensible defaults: an empty Model picks the provider’s recommended one, and SampleRate: 0 inherits the transport’s rate.

The config type name varies by provider. A provider that offers one service names it plainly Config; one that offers several qualifies the extras. So it is deepgram.Config for STT but deepgram.TTSConfig for TTS, cartesia.Config for TTS but cartesia.STTConfig for STT, and chat.STTConfig / chat.LLMConfig / chat.TTSConfig for all three. Check the package, or let the compiler tell you.

Providers

Pick any per category.

Each lives in provider/<name>, or provider/<vendor>/<name> when a vendor offers several (provider/azure/speech, provider/openai/chat, provider/aws/bedrock). There are more in the tree than listed above; browse provider/ for the current set. Per-provider runnable examples are in examples/voice/ .

Coverage is uneven: the providers used by the examples get the most exercise, and some of the others are thinly tested. Bug reports naming a specific provider are especially useful.

The interfaces

You rarely implement these, but knowing their shape explains what a provider can and cannot do.

STT

Two flavors. Streaming is what you want for conversation:

type Connector interface {
    Connect(ctx context.Context, sampleRate int) (Stream, error)
}

type Stream interface {
    Send(audio []byte) error
    Recv() ([]Result, error)
    Close() error
}

stt.StreamService wraps a Connector into a processor: audio in, interim and final TranscriptionFrames out.

Segment STT transcribes a finished buffer instead, for providers with no streaming API:

type Transcriber interface {
    Transcribe(ctx context.Context, audio []byte, sampleRate int) (string, error)
}

Segment STT emits no interim transcriptions. It still works with WithTurnTaking(), which gates on a finalized transcript, but start strategies that key on partial transcripts never fire, so turn starts fall back to VAD alone. chat.NewSTT is segment-based; deepgram.NewSTT streams.

LLM

type Generator interface {
    Generate(ctx context.Context, convo *frames.LLMContext, emit Emit) error
}

Stream deltas to emit until done or ctx is canceled. Cancellation is an interruption: honor it, or barge-in stalls for up to three seconds.

Tool-capable providers implement ToolGenerator as well:

type ToolGenerator interface {
    GenerateWithTools(ctx context.Context, convo *frames.LLMContext, sink Sink) error
}

llm.Base runs the tool loop automatically when the context carries tools and the generator supports them.

TTS

type Synthesizer interface {
    SampleRate() int
    Synthesize(ctx context.Context, text string, emit func(pcm []byte) error) error
}

Providers that return word timings also implement WordTimestamps, which is what lets TTSTextFrames align to the audio actually being spoken, and therefore what lets an interrupted response be recorded truncated rather than whole.

Sentences in the language spoken

A TTS service is given the model’s output a sentence at a time, and where a sentence ends depends on the language: “bzw.” ends nothing in German, and an abbreviation list is what tells “Dr. Smith” from the end of a sentence. The service finds its boundaries in the language its settings name, English when they name none, with rules for some forty languages; a TTSUpdateSettingsFrame changing the language applies from the next text, and keeps what is already buffered. An aggregator built by hand takes its language too, as text.NewSimpleAggregator(frames.TextTypeSentence, "de").

Pronunciations

A voice guesses a name or a term it does not know from its spelling. PronunciationTransformIPA builds a text transform from a word-to-IPA mapping, and each provider writes the IPA in its own markup: Cartesia as inline phoneme blocks, ElevenLabs as SSML <phoneme> tags, Inworld as IPA between slashes and Deepgram Aura-2 as an inline pronunciation object. One mapping therefore works with any of them. A word the provider cannot use is reported once, when the transform is built, and spoken as written.

pronounce := t.PronunciationTransformIPA(map[string]string{"Metformin": "mɛtˈfɔɹmɪn"})
t.SetTextTransformers(stripMarkdown, pronounce) // last, so nothing rewrites its markup

Register it last, so no later transform rewrites the markup it inserts. The ElevenLabs tags are read only by eleven_flash_v2 and eleven_turbo_v2, and on the WebSocket service only with EnableSSMLParsing on; with any other setup the transform is skipped with a warning and the words are spoken as written. Deepgram Flux has no pronunciation markup.

Tool calling

Register a handler by name, then advertise the tool on the context:

llm.RegisterFunction("get_order_status", func(ctx context.Context, p llm.FunctionCallParams) error {
    var in struct{ OrderID string `json:"order_id"` }
    if err := json.Unmarshal(p.Arguments, &in); err != nil {
        return err
    }
    status, err := lookup(ctx, in.OrderID)
    if err != nil {
        return err
    }
    return p.Result(ctx, status, nil)
})

convo.SetTools([]frames.Tool{{
    Name:        "get_order_status",
    Description: "Look up the status of an order by its ID.",
    Parameters:  json.RawMessage(`{
        "type": "object",
        "properties": {"order_id": {"type": "string"}},
        "required": ["order_id"]
    }`),
}})

Parameters is a raw JSON-Schema object. A handler reports what it produced through p.Result rather than returning it, because a call can have more than one thing to say. A returned error is reported as a non-fatal pipeline error and puts nothing in the tool’s mouth, so a failure the model should see belongs in the result instead.

A handler that blocks must honor ctx, the same interruption rule as Generate: the call’s context is canceled when the user barges in.

The call and the message answering it are written together the moment the call starts, so the conversation is valid at every instant, and the result replaces that placeholder in place. See LLM context .

A tool can carry its own handler instead, which saves keeping two lists in step:

convo.SetTools([]frames.Tool{{
    Name:        "get_order_status",
    Description: "Look up the status of an order by its ID.",
    Parameters:  schema,
    Handler:     lookupOrder,
}})

The handler is registered when the toolset is advertised and dropped when it stops being advertised, so what the model can call and what answers are the same set. A handler registered by hand always wins and is never dropped.

Tuning how calls run

Register the empty name for a catch-all that takes any call no named handler claims. UnregisterFunction withdraws a handler, HasFunction asks whether a call would be claimed, and OnFunctionCallsStarted / OnFunctionCallsCanceled report which calls a response started and which an interruption took away.

Speech-to-speech

A single model replaces the STT → LLM → TTS trio:

s2s := realtime.New(realtime.Config{APIKey: key})

pipeline.New(t.Input(), s2s, t.Output())

The three implementations are provider/openai/realtime, provider/google/live and provider/aws/novasonic.

Each takes its session’s configuration in its Config. For OpenAI Realtime that is SessionProperties: the voice, the transcription of the user’s speech, turn detection (server or semantic VAD, or turned off so the pipeline’s own turn detection drives the session), noise reduction and reasoning. An LLMUpdateSettingsFrame carrying a realtime.Settings delta changes them while the pipeline runs.

Lower latency and better prosody, at the cost of the per-stage control you get from three separate services. You cannot inspect the transcript before the model answers, or swap just the voice.

A frontend with a backend

llmwithbackend.New pairs a fast conversational LLM, the frontend, with a slower, stronger one that has the tools, the backend. It is alpha: its API is likely to change between releases. Put it where the LLM goes:

convo := frames.NewLLMContext("You fix failing tests in the repository.")
convo.SetTools([]frames.Tool{readFile, runTests})
backend, err := llmworker.NewBackend(llmworker.BackendConfig{LLM: anthropicLLM, Context: convo})
llm, err := llmwithbackend.New(llmwithbackend.Config{Frontend: openaiLLM, Backend: backend})

pipeline.New(t.Input(), stt, agg.User(), llm, tts, t.Output(), agg.Assistant())

The frontend gets one built-in tool, delegate, which hands the request to the backend and returns at once, so the frontend acknowledges while the backend works. Everything the backend writes comes back as a message in the frontend’s conversation marked Backend:. Its model begins a message with >> when the user should hear it, and the frontend then runs on it; everything else it writes is kept as notes on the work. The user can keep talking meanwhile: a second request joins the work in progress, and a correction changes or stops it.

The frontend’s system instruction says who the assistant is, the backend’s says what work it does, and each gets a pairing instruction appended that explains how the two work together. llmwithbackend.ConnectorConfig changes the wording, how requests are composed, and whether each exchange is traced to the client as an RTVI server message. The frontend has to carry built-in tools: the OpenAI chat completions, Anthropic and Gemini services do.

A text frontend spends a whole inference deciding to call delegate before the backend can start. llmwithbackend.PredictiveDelegation can save that time: a fast yes/no classifier, such as classifier/jev, races the frontend on each user turn, and a “yes” that comes before the frontend’s first output cancels its inference and delegates the turn at once. The cost is money, not time: the classifier runs on every turn, and an early delegation abandons a frontend request already under way.

judge, err := jev.New(jev.Config{APIKey: jevKey})
pd, err := llmwithbackend.NewPredictiveDelegation(llmwithbackend.PredictiveDelegationConfig{Classifier: judge})
llm, err := llmwithbackend.New(llmwithbackend.Config{
    Frontend: openaiLLM, Backend: backend, PredictiveDelegation: pd,
})

It needs a text frontend and a request strategy whose delegate call takes no arguments, such as the default transcript strategy.

Switching at runtime

pipeline.NewServiceSwitcher routes to one of several services, changed mid-call by pushing a ManuallySwitchServiceFrame:

sw, err := pipeline.NewServiceSwitcher(
    []processor.Processor{fastLLM, smartLLM},
    pipeline.NewManualStrategy, // or pipeline.NewFailoverStrategy
)
...
task.QueueFrame(frames.NewManuallySwitchServiceFrame(smartLLM))

The second argument is the strategy that decides when the active service changes. NewManualStrategy changes it only when asked. NewFailoverStrategy additionally moves to the next service when the active one reports a non-fatal error, which is the cheap way to survive a provider outage. Implement pipeline.SwitcherStrategy for anything else, and pass its constructor: the switcher builds the strategy over the services it manages, so the two cannot drift apart.

Every service is started and kept warm, but only the active one receives data and only the active one is heard from: the metadata a service broadcasts about itself reaches the rest of the pipeline only while it is in use, and a switch asks the new service to describe itself again.

A switch request naming a service this switcher does not manage travels on, so several switchers can sit in one pipeline and each pick up its own.

Useful for escalating to a stronger model when a conversation gets hard, without rebuilding the pipeline.

See Writing a service to add a provider.