Overview

# Audio & Speech Settings

Last Updated: 2026-10-09 | For: Teachers & Content Creators | Level: Intermediate to Advanced

Overview

The tables below summarize the JSON fields that control speech recognition (SR) and text-to-speech (TTS) behavior in TDF files. API keys, language, voice, and learner-audio defaults live in tutor.setspec; units can override whether SR is enabled and which TTS mode is used; card-delivery timing lives in deliverySettings.

On mobile, the same controls stack vertically:

Dashboard audio indicators

The lesson's headphones and microphone indicators reflect the effective audio settings. An available API key alone does not mean the feature is enabled. The indicator's tooltip distinguishes an enabled feature, a missing key, a feature disabled for the lesson, and a feature turned off in the learner's settings. Check both the lesson configuration and the learner's Audio Settings when diagnosing an inactive indicator.

Speech Recognition (Microphone Input)

Text-to-Speech & Audio Prompts

The Test Voice control uses the free documentation samples for the ten offered en-US-Standard-A through en-US-Standard-J voices at https://docs.cloud.google.com/text-to-speech/docs/audio/<voice>.wav. It does not call paid synthesis or require a provider key. Starting another preview stops the previous one; leaving Audio Settings stops playback. Playback failures appear in the existing inline TTS status area in the interface language. The October 9 staging2 acceptance record includes Chrome sample playback and inline failure display for candidate a3155584. That result applies to the representative repair batch; it does not establish every browser or production deployment. See Security Audit Operations for the evidence boundary.

Unit Overrides

Use top-level unit fields to vary audio behavior within one lesson:

{
  "unitname": "Spoken practice",
  "audioInputEnabled": "true",
  "audioPromptMode": "all",
  "learningsession": { "clusterlist": "0-9" }
}

Unit values override the matching lesson values. Personal microphone and spoken-audio choices remain gates: a unit cannot force a learner to use a disabled audio feature. Language, credentials, voices, rates, volume, grammar behavior, and sensitivity continue to come from lesson or profile settings.

Delivery Timing & Feedback

Language ownership

Interface language, authored lesson language, recognized-response language, and spoken-output language are separate settings. Platform prompts use the interface locale's voice mapping. Authored speech uses textToSpeechLanguage or an explicit voice locale; missing required language information produces an error. An SR-enabled lesson must declare speechRecognitionLanguage. Changing the interface language does not translate a lesson or authorize microphone use. Authors should test the actual language and voice configuration for their lesson.

Timing behavior in the current source contract:

• Trial timers begin when the card content starts fading in. Fade-in and fade-out are part of the configured duration.
• Pre-recorded prompt audio and timeuntilaudio are included in the configured response window. Avoid very short drill values when prompt media must play before a learner can reasonably answer.
• Generated TTS can push an expired timer forward only until active playback finishes; it does not restart the timer from the beginning.
• Text-entry activity and speech-recognition activity reset the response timeout.

Older deployed builds can start the response countdown after prerecorded question audio finishes. Their exported reveal-to-timeout latency therefore includes audio plus the configured response window: approximately four seconds of audio followed by drill = 10000 can yield a 14-15-second timeout without any input. This differs from the current contract above. Use the collection build to select the interpretation; see Timer Behavior Depends on the Collection Build.

Exported Time generally marks first input on a card attempt; Problem Start Time marks reveal. Start and end latencies are durations from reveal, and equal start/end latency on a no-input timeout results from substituting the timeout for missing first input. See Interpreting Trial Timing for the complete field definitions and formulas.

API Key Usage

How API Keys Work

See Glossary for canonical definitions of embedded API keys, profile flow, and experiment logins.

Experiment Logins (Embedded Keys)

In experiment logins (and other routes that set useEmbeddedAPIKeys to true), the lesson's embedded keys (speechAPIKey, textToSpeechAPIKey) are sent directly to Google for both SR and TTS.

Normal Profile Flow

In the normal profile flow (useEmbeddedAPIKeys remains false):

• Text-to-Speech still uses the embedded textToSpeechAPIKey when present; if the call fails the browser's speechSynthesis API handles speech output.
• Speech Recognition first tries the learner's saved key (Meteor.user().speechAPIKey). If none is stored, prompt the learner to enter one or switch to a flow that enables useEmbeddedAPIKeys; otherwise SR requests only reach Google automatically during experiment sessions.

Configuration Notes

Important Reminders

• Field names are case-sensitive. Many legacy TDF values are serialized as strings, but current delivery display settings also accept booleans/numbers where the schema defines them.
• When no textToSpeechAPIKey is present, MoFaCTS falls back to the browser's built-in speechSynthesis for any enabled prompts.
• Learners can override prompt modes (silent, question, feedback, all) and mic use during a session; the values above only establish the starting configuration.
• Runtime behavior depends on valid media references; unresolved audio/media references can fail delivery instead of silently skipping.
• Speech matching preserves numeric distinctions. Close-response matching checks the numbers in the recognized response against the target before accepting a phonetic match. The current number-word normalization handles common English forms; do not assume equivalent number-word handling in every language.

Example Configuration

{
  "setspec": {
    "name": "ExampleLesson",
    "lessonname": "Example Lesson with Audio",
    "audioInputEnabled": "true",
    "audioInputSensitivity": "20",
    "speechAPIKey": "YOUR_GOOGLE_SPEECH_API_KEY",
    "speechIgnoreOutOfGrammarResponses": "true",
    "speechOutOfGrammarFeedback": "Please say one of the possible answers",
    "textToSpeechAPIKey": "YOUR_GOOGLE_TTS_API_KEY",
    "audioPromptMode": "question",
    "audioPromptVoice": "en-US-Standard-B",
    "audioPromptQuestionSpeakingRate": "1.2",
    "audioPromptFeedbackSpeakingRate": "1.0"
  }
}

Common Use Cases

Enable Speech Recognition Only

{
  "setspec": {
    "speechAPIKey": "YOUR_API_KEY",
    "audioInputEnabled": "true",
    "audioInputSensitivity": "20"
  }
}

Enable Text-to-Speech Only

{
  "setspec": {
    "textToSpeechAPIKey": "YOUR_API_KEY",
    "audioPromptMode": "question",
    "audioPromptQuestionSpeakingRate": "1.0"
  }
}

Enable Both SR and TTS

{
  "setspec": {
    "speechAPIKey": "YOUR_SPEECH_API_KEY",
    "audioInputEnabled": "true",
    "textToSpeechAPIKey": "YOUR_TTS_API_KEY",
    "audioPromptMode": "all"
  }
}

Strict Answer Matching for Speech

{
  "setspec": {
    "speechAPIKey": "YOUR_API_KEY",
    "audioInputEnabled": "true",
    "speechIgnoreOutOfGrammarResponses": "true",
    "speechOutOfGrammarFeedback": "That answer is not in the expected set. Please try again."
  },
  "unit": [
    {
      "deliverySettings": {
        "autostopTranscriptionAttemptLimit": 3
      }
    }
  ]
}

Troubleshooting

Use the canonical troubleshooting page: Troubleshooting.

Quick checks:

1. Confirm lesson-level audio settings are present and valid.
2. Confirm referenced media resolves in the assigned package scope.
3. Confirm browser permissions and HTTPS requirements are met.

See Also

• Content Creation Reference Tables - Audio/speech field definitions
• Student Overview - Student speech recognition setup
• Glossary - API key flow definitions
• Troubleshooting - SR/TTS troubleshooting