Services

STT, LLM and TTS providers are all processors behind small interfaces. Swapping one is a one-line change, because nothing in the pipeline depends on which provider you picked.

The pattern

Every provider is a Config struct plus a constructor:

stt := deepgram.NewSTT(deepgram.Config{APIKey: key})
llm := anthropic.NewLLM(anthropic.Config{APIKey: key, Model: "claude-sonnet-5"})
tts := cartesia.NewTTS(cartesia.Config{APIKey: key, VoiceID: id})

Library packages read no environment variables and take no functional options. Configs are validated with go-playground/validator tags, so a bad config fails at construction rather than mid-call. Reading env vars, flags or config files is your app’s job. See examples/.

Most fields have sensible defaults: an empty Model picks the provider’s recommended one, and SampleRate: 0 inherits the transport’s rate.

The config type name varies by provider. A provider that offers one service names it plainly Config; one that offers several qualifies the extras. So it is deepgram.Config for STT but deepgram.TTSConfig for TTS, cartesia.Config for TTS but cartesia.STTConfig for STT, and chat.STTConfig / chat.LLMConfig / chat.TTSConfig for all three. Check the package, or let the compiler tell you.

Providers

Pick any per category.

Each lives in provider/<name>, or provider/<vendor>/<name> when a vendor offers several (provider/azure/speech, provider/openai/chat, provider/aws/bedrock). There are more in the tree than listed above; browse provider/ for the current set. Per-provider runnable examples are in examples/voice/ .

Coverage is uneven: the providers used by the examples get the most exercise, and some of the others are thinly tested. Bug reports naming a specific provider are especially useful.

The interfaces

You rarely implement these, but knowing their shape explains what a provider can and cannot do.

STT

Two flavors. Streaming is what you want for conversation:

type Connector interface {
    Connect(ctx context.Context, sampleRate int) (Stream, error)
}

type Stream interface {
    Send(audio []byte) error
    Recv() ([]Result, error)
    Close() error
}

stt.StreamService wraps a Connector into a processor: audio in, interim and final TranscriptionFrames out.

Segment STT transcribes a finished buffer instead, for providers with no streaming API:

type Transcriber interface {
    Transcribe(ctx context.Context, audio []byte, sampleRate int) (string, error)
}

Segment STT emits no interim transcriptions. It still works with WithTurnTaking(), which gates on a finalized transcript, but start strategies that key on partial transcripts never fire, so turn starts fall back to VAD alone. chat.NewSTT is segment-based; deepgram.NewSTT streams.

LLM

type Generator interface {
    Generate(ctx context.Context, convo *frames.LLMContext, emit Emit) error
}

Stream deltas to emit until done or ctx is canceled. Cancellation is an interruption: honor it, or barge-in stalls for up to three seconds.

Tool-capable providers implement ToolGenerator as well:

type ToolGenerator interface {
    GenerateWithTools(ctx context.Context, convo *frames.LLMContext, sink Sink) error
}

llm.Base runs the tool loop automatically when the context carries tools and the generator supports them.

TTS

type Synthesizer interface {
    SampleRate() int
    Synthesize(ctx context.Context, text string, emit func(pcm []byte) error) error
}

Providers that return word timings also implement WordTimestamps, which is what lets TTSTextFrames align to the audio actually being spoken, and therefore what lets an interrupted response be recorded truncated rather than whole.

Tool calling

Register a handler by name, then advertise the tool on the context:

llm.RegisterFunction("get_order_status", func(ctx context.Context, p llm.FunctionCallParams) error {
    var in struct{ OrderID string `json:"order_id"` }
    if err := json.Unmarshal(p.Arguments, &in); err != nil {
        return err
    }
    status, err := lookup(ctx, in.OrderID)
    if err != nil {
        return err
    }
    return p.Result(ctx, status, nil)
})

convo.SetTools([]frames.Tool{{
    Name:        "get_order_status",
    Description: "Look up the status of an order by its ID.",
    Parameters:  json.RawMessage(`{
        "type": "object",
        "properties": {"order_id": {"type": "string"}},
        "required": ["order_id"]
    }`),
}})

Parameters is a raw JSON-Schema object. A handler reports what it produced through p.Result rather than returning it, because a call can have more than one thing to say. A returned error is reported as a non-fatal pipeline error and puts nothing in the tool’s mouth, so a failure the model should see belongs in the result instead.

A handler that blocks must honor ctx, the same interruption rule as Generate: the call’s context is canceled when the user barges in.

The call and the message answering it are written together the moment the call starts, so the conversation is valid at every instant, and the result replaces that placeholder in place. See LLM context .

A tool can carry its own handler instead, which saves keeping two lists in step:

convo.SetTools([]frames.Tool{{
    Name:        "get_order_status",
    Description: "Look up the status of an order by its ID.",
    Parameters:  schema,
    Handler:     lookupOrder,
}})

The handler is registered when the toolset is advertised and dropped when it stops being advertised, so what the model can call and what answers are the same set. A handler registered by hand always wins and is never dropped.

Tuning how calls run

Register the empty name for a catch-all that takes any call no named handler claims. UnregisterFunction withdraws a handler, HasFunction asks whether a call would be claimed, and OnFunctionCallsStarted / OnFunctionCallsCanceled report which calls a response started and which an interruption took away.

Speech-to-speech

A single model replaces the STT → LLM → TTS trio:

s2s := realtime.New(realtime.Config{APIKey: key})

pipeline.New(t.Input(), s2s, t.Output())

The three implementations are provider/openai/realtime, provider/google/live and provider/aws/novasonic.

Lower latency and better prosody, at the cost of the per-stage control you get from three separate services. You cannot inspect the transcript before the model answers, or swap just the voice.

Switching at runtime

pipeline.NewServiceSwitcher routes to one of several services, changed mid-call by pushing a ManuallySwitchServiceFrame:

sw, err := pipeline.NewServiceSwitcher(
    []processor.Processor{fastLLM, smartLLM},
    pipeline.NewManualStrategy, // or pipeline.NewFailoverStrategy
)
...
task.QueueFrame(frames.NewManuallySwitchServiceFrame(smartLLM))

The second argument is the strategy that decides when the active service changes. NewManualStrategy changes it only when asked. NewFailoverStrategy additionally moves to the next service when the active one reports a non-fatal error, which is the cheap way to survive a provider outage. Implement pipeline.SwitcherStrategy for anything else, and pass its constructor: the switcher builds the strategy over the services it manages, so the two cannot drift apart.

Every service is started and kept warm, but only the active one receives data and only the active one is heard from: the metadata a service broadcasts about itself reaches the rest of the pipeline only while it is in use, and a switch asks the new service to describe itself again.

A switch request naming a service this switcher does not manage travels on, so several switchers can sit in one pipeline and each pick up its own.

Useful for escalating to a stronger model when a conversation gets hard, without rebuilding the pipeline.

See Writing a service to add a provider.