Skip to main content

​
Overview

The OpenAI Realtime WebRTC transport implementation enables real-time audio communication directly with the OpenAI Realtime API using WebRTC voice-to-voice service. It handles media device management, audio/video streams, and state management for the connection.

​
Installation

Add the OpenAI transport package to your project: 
.package(url: "https://github.com/pipecat-ai/pipecat-client-ios-openai-realtime.git", from: "0.0.1"),

// Add to your target dependencies
.target(name: "YourApp", dependencies: [
    .product(name: "PipecatClientIOSOpenAIRealtimeWebrtc", package: "pipecat-client-ios-openai-realtime")
],

​
Usage

Create a client: 
let rtviClientOptions = RTVIClientOptions.init(
    enableMic: currentSettings.enableMic,
    enableCam: false,
    params: .init(config: [
        .init(
            service: "llm",
            options: [
                .init(name: "api_key", value: .string(openaiAPIKey)),
                .init(name: "initial_messages", value: .array([
                    .object([
                        "role": .string("user"), // "user" | "system"
                        "content": .string("Start by introducing yourself.")
                    ])
                ])),
                .init(name: "session_config", value: .object([
                    "instructions": .string("You are Chatbot, a friendly and helpful assistant who provides useful information, including weather updates."),
                    "voice": .string("echo"),
                    "input_audio_noise_reduction": .object([
                        "type": .string("near_field")
                    ]),
                    "turn_detection": .object([
                        "type": .string("semantic_vad")
                    ])
                ])),
            ]
        )
    ])
)
self.rtviClientIOS = RTVIClient.init(
    transport: OpenAIRealtimeTransport.init(options: rtviClientOptions),
    options: rtviClientOptions
)
try await rtviClientIOS.start()
Currently, invalid session configurations will result in the OpenAI connection being failed.

​
API Reference

Demo

Simple Chatbot OpenAI Demo

Source

iOS OpenAI Realtime WebRTC

Pipecat iOS Client Reference

Complete API documentation for the OpenAI transport implementation