Your First Conversation

FirstConversation is the canonical introductory sample for STConversationRunner. It is intentionally small: its job is to expose the SpeechTiles conversation lifecycle without hiding it behind application architecture.

What you will build

The sample asks for the user's name, uses that application-supplied value in a later prompt, asks a yes/no question, branches declaratively, and finishes at CF_END.

CF_START
    |
node.configure
    |
node.askName
    |
node.askContinue
   /        \
 yes        no
  |          |
node.        node.
explain      goodbyeNo
  |
node.goodbyeYes
   \        /
      CF_END

Along the way you will see the boundary between declarative Conversation behavior and application logic.

The project

The canonical sample contains six files:

The application imports the framework as:

import STConversation

Both JSON files must be application-target resources.

1. Prepare the application

In Xcode, open the application target’s Info settings and supply NSMicrophoneUsageDescription (Privacy - Microphone Usage Description) and NSSpeechRecognitionUsageDescription (Privacy - Speech Recognition Usage Description). For example: “Microphone access lets you speak to the conversation” and “Speech recognition converts spoken responses to text.” On sandboxed macOS targets, enable Signing & Capabilities → App Sandbox → Hardware → Audio Input. Usage descriptions explain permission requests; they do not grant access by themselves.

The sample requires microphone and speech-recognition privacy usage descriptions.

For a macOS target using App Sandbox, the canonical sample also requires Audio Input and Outgoing Connections (Client). Depending on signing configuration, Audio Input may also be required under Hardened Runtime resource access.

The sample prepares SpeechTiles when the view appears rather than waiting for the user to press Run. This keeps permission acquisition and configuration separate from conversation execution.

.task {
    controller.prepare()
}

Run remains disabled until preparation reaches the Ready state.

2. Create and initialize STConversationRunner

ConversationController owns one runner:

private let conversation = STConversationRunner()

Preparation begins with logging, initialization, callback registration, and audio-environment configuration:

conversation.setLogLevel(.debug)

guard conversation.initialize() == .success else {
    // Handle initialization failure.
    return
}

guard conversation.setConversationCallback(self) == .success else {
    // Handle callback-registration failure.
    return
}

let result = conversation.configureAudioEnvironment(
    mode: .voiceChat,
    options: [.allowBluetoothHFP, .allowBluetoothA2DP]
)

initialize() establishes the Conversation runtime and required media infrastructure. Callback registration and audio-environment configuration require an initialized runner.

Reference: initialize(), setConversationCallback(), and configureAudioEnvironment().

3. Resolve permissions before enabling Run

The sample requests microphone permission first and speech-recognition permission second.

A request can complete immediately or return .permissionPending. Pending does not mean denied. The sample waits for conversationReceivedPermissionResult(permission:status:) before continuing preparation.

This distinction matters because permission acquisition is asynchronous on systems where the user must respond to an OS prompt.

Reference: requestMicrophonePermission(), requestSpeechRecognitionPermission(), and conversationReceivedPermissionResult.

4. Load the two configuration resources

Check the result of each synchronous load before enabling Run. For example:

guard conversation.loadAudioPromptModules(jsonData: prompts) == .success else {
    // Report the prompt-module loading failure.
    return
}
guard conversation.loadConversation(jsonData: flow) == .success else {
    // Report the conversation configuration loading failure.
    return
}

This sample passes bundled JSON data directly, so it does not need setResourceLocation for these two calls. File-based loading can instead use setLanguageFolder and the applicable URL overloads. The default language folder is en-US; for file-based resources the language-specific directory is searched before the parent directory.

After permissions are available, the sample reads the bundled JSON files and loads them into the runner:

conversation.loadAudioPromptModules(jsonData: prompts)
conversation.loadConversation(jsonData: flow)

Prompt Modules are loaded first, followed by the Conversation. Both loading operations require the Conversation runtime to be idle.

This illustrates the documentation boundary used throughout SpeechTiles:

• the SDK Reference explains how and when Swift loads configuration;
• the Configuration Reference explains the JSON structure, defaults, precedence, and runtime behavior.

Reference: loadAudioPromptModules (JSON), loadConversation (JSON), configuration/Prompt-Modules/prompt-module.yaml, and the Conversation Configuration Reference.

5. Start at CF_START and configure the Conversation

FirstConversation.json begins with CF_START, which routes to node.configure.

The configure module establishes the sample's resource language, Apple speech-recognition settings, speaker output, conversational audio processing, and an initial CollectedName runtime field:

{
  "id": "node.configure",
  "type": "configure",
  "configureParams": {
    "resourceLanguage": "en-US",
    "speechRecognitionEngine": "apple",
    "speechRecognitionLocale": "en-US",
    "audioOutput": "speaker",
    "audioProcessingMode": "conversational",
    "runtimeFields": [
      {
        "name": "CollectedName",
        "value": ""
      }
    ]
  },
  "goTo": {
    "default": "node.askName"
  }
}

The runtime field is deliberately initialized in configuration and later replaced by application code after name recognition.

Reference: configuration/Conversation/configure-params.yaml.

6. Ask for a name with Interactive Capture

node.askName is an interactiveCapture module. It combines an initial dialog prompt, retry prompts, speech recognition, VAD/recording behavior, and retry-limit transitions.

The initial prompt references pm.askName:

{
  "type": "initial",
  "playbackMode": "sequence",
  "recognizeWhilePlaying": true,
  "promptID": "pm.askName"
}

The corresponding Prompt Module is intentionally simple:

{
  "id": "pm.askName",
  "playbackMode": "single",
  "text": "Hello! What is your name?"
}

This demonstrates an important separation: the Conversation controls when and why the prompt is used, while the Prompt Module owns the reusable media/text content.

Reference: configuration/Conversation/interactive-capture.yaml, configuration/Conversation/dialog-prompt.yaml, and configuration/Prompt-Modules/prompt-module.yaml.

7. Receive recognition in the application

The callback must update the runtime field before returning when the next conversation module immediately consumes it. The following illustrates the relevant operation (using the session’s existing runner and its application-defined name parser):

// Inside conversationReceivedSpeechRecognition(moduleID:data:):
guard moduleID == "node.askName",
      let hypothesis = data.recognitionData?.hypothesis,
      !hypothesis.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty else { return }
// Parse/validate the name using application logic.
let name = hypothesis.trimmingCharacters(in: .whitespacesAndNewlines)
_ = conversation.setRuntimeField("CollectedName", value: name)
// Publish any SwiftUI changes separately, without delaying this update.

This simplified snippet demonstrates callback timing, not production-grade name parsing. A runtime field named CollectedName is referenced from configuration as $[CollectedName].

The sample separates SpeechTiles' synchronous callback path from SwiftUI state. ConversationController remains @MainActor; ConversationSession and its STConversationCallback conformance are explicitly nonisolated.

When node.askName produces a final hypothesis, the session parses the name and updates CollectedName synchronously before conversationReceivedSpeechRecognition returns. UI publication is separately delivered to MainActor.

This removes any dependency on the timing gap between the name-capture module and the next module. The name parser remains intentionally small teaching code rather than production name-understanding logic.

8. Interpolate application data into a Prompt Module

The next prompt uses the runtime field directly:

{
  "id": "pm.askContinue",
  "playbackMode": "single",
  "text": "Nice to meet you, $[CollectedName]. Would you like to hear what SpeechTiles can do?"
}

At runtime, $[CollectedName] is resolved from Conversation state. This is the bridge from application/domain data back into declarative conversation content.

Reference: configuration/Prompt-Modules/prompt-module.yaml.

9. Branch declaratively on yes or no

node.askContinue uses userInputTransitions:

"userInputTransitions": [
  { "comparator": "starts", "input": "yes",  "goTo": "node.explainSpeechTiles" },
  { "comparator": "starts", "input": "yeah", "goTo": "node.explainSpeechTiles" },
  { "comparator": "starts", "input": "sure", "goTo": "node.explainSpeechTiles" },
  { "comparator": "starts", "input": "no",   "goTo": "node.goodbyeNo" }
]

No application code is required for this branch. SpeechTiles has enough declarative information to make the decision itself.

The module also sets:

"unmatchedInputIsUnrecognized": true

If recognition successfully returns speech such as "what are you talking about?", but that speech does not match an allowed yes/no transition, SpeechTiles treats it as unrecognized conversational input and follows the existing retry behavior.

That is different from recognition failure, and different again from application-level semantic validation.

Reference: configuration/Conversation/user-input-transition.yaml and configuration/Conversation/interactive-capture.yaml.

10. Allow barge-in

recognizeWhilePlaying and the recording/playback options enable the requested interaction pattern, but reliable barge-in also depends on the audio environment and the device’s supported voice-processing and echo-control behavior.

Both Interactive Capture modules recognize while their dialog prompts are playing. Their recording configuration includes:

"voiceActivityDetection": {
  "enabled": true,
  "minimumSpeechDurationMs": 200
},
"recordingPlayback": {
  "recordWhilePlaying": true,
  "stopPlaybackWhenSpeechDetected": true
}

The canonical sample keeps the 200 ms minimum speech duration because it was selected for responsive prompt interruption in this sample. Other recording/VAD properties are explicitly configured where the sample needs them.

Reference: configuration/Conversation/audio-recording.yaml, configuration/Conversation/recording-playback-params.yaml, and the corresponding STMediaRunner recording reference.

11. Run and stop

Distinguish three events: .success from runConversation() accepts the request, reaching CF_END completes the configured conversation path, and stopConversation() requests an application-initiated stop. Update the UI from the appropriate lifecycle callbacks rather than assuming the run has finished when its initial request succeeds.

Once preparation is complete, starting execution is intentionally small:

let result = conversation.runConversation()

A .success result means the run request was accepted; Conversation execution continues through modules, callbacks, recognition, and media events.

The Stop button calls stopConversation(). When CF_END finishes, the sample returns its UI state to Ready so the Conversation can be run again.

Reference: runConversation() and stopConversation().

12. Understand the responsibility boundary

FirstConversation demonstrates one of the central SpeechTiles design boundaries:

SpeechTiles handles deterministic conversational mechanics when it has enough information. Application code remains responsible for domain semantics when it does not.

13. Test the complete flow

Validation checklist: launch and reach Ready; accept microphone and speech-recognition permissions; run the name prompt and confirm $[CollectedName] substitution; exercise both yes and no branches; try unmatched speech and confirm retry; interrupt a prompt with speech to test barge-in; stop during execution; and run again after completion. Also verify that permission denial and malformed or missing configuration files produce understandable failures without enabling Run.

1. Launch the application and wait for Ready.
2. Press Run.
3. Answer the name prompt with "My name is Antoine" or another name.
4. Verify that the UI displays the recognized name and that the next prompt uses it.
5. At the yes/no prompt, say "what are you talking about?" and verify that SpeechTiles asks for yes or no again.
6. Say "yes" and verify that the explanation and goodbye path reach CF_END.
7. Run again and say "no" to verify the negative branch.
8. Begin speaking while a prompt is still playing and verify responsive barge-in.

What this sample intentionally does not do

FirstConversation is a teaching sample, not the showcase application. It intentionally avoids AI/NLU, sophisticated name parsing, business rules, camera interaction, backend submission, and complex interruption handling.

Once this sample is understood, move into the full Conversation and Prompt Module Configuration References and the STConversationRunner API Reference. Richer samples can then demonstrate application semantics and more advanced orchestration without burdening the first-run experience.