ElevenLabs & AI Segments

​

What you can do

• Define one or more segments with start and end time (in seconds).
• Assign an ElevenLabs voice per segment using a Voice ID from your ElevenLabs account.
• Write a prompt template that can include placeholders filled from data you pass on the page.
• Control voice volume and optionally duck (lower) the main video audio while the AI voice plays.
• Preload audio shortly before each segment so playback feels instant when the viewer reaches that part of the video.

​

Prerequisites

• TrackPlay player v2 / Customize V2 for the video you want to enhance.
• An ElevenLabs account and an API key (API keys in ElevenLabs).
• Optional: Voice IDs from ElevenLabs for each voice you want to use (Voices or your cloned voices).

​

Step 1: Connect ElevenLabs

​

From Integrations (recommended)

1. Open your TrackPlay dashboard and go to Integrations.
2. Click Add integration (or find ElevenLabs on the grid).
3. Click Connect on ElevenLabs, then Configure.
4. Paste your ElevenLabs API key and click Save.

​

From Workspace settings

1. Go to Settings → ElevenLabs (or your workspace’s ElevenLabs page).
2. Paste your API key and save.

​

Step 2: Configure AI Segments on a video

1. Open the video → Customize (V2).
2. Open the AI Segments tab.
3. Turn Enable AI Segments on.
4. Set Preload seconds (how many seconds before each segment’s start time the player begins requesting/generated audio). Default is often 30 seconds.
5. Click Add segment and configure:

6. Save / Deploy the video so the updated options are baked into the player build.

​

Personalized prompts with trackplay_data

Hey {name}, thanks for watching. Here is something for you.

window.trackplay_data = {
  name: 'Sam',
  email: 'sam@example.com',
  // any keys you want; empty values are ignored for substitution
};

• Use {key} where key matches a property on trackplay_data.
• If a key is missing, the placeholder may be left as-is in the text.

​

Viewer experience: sound and autoplay

• The player does not request or play AI segment audio while the main video is muted (typical muted autoplay state).
• After the viewer unmutes (or otherwise enables audible playback), the player can preload and play AI segments as configured.

​

How generation and delivery work (overview)

• When a viewer is in range to need a segment, the player calls TrackPlay’s events infrastructure (e.trackplay.io) with segment_id, current video_time, session/device IDs, and trackplay_data (dynamic fields for your prompt).
• The server loads voice ID and prompt template from your saved video settings (not from the browser). It checks that video_time is inside the segment’s preload window, then builds the spoken line by filling {placeholders} from trackplay_data. Same pattern as on the page.
• You cannot send arbitrary text or a different voice from the client; that prevents abuse (e.g. via curl).
• The service uses your workspace’s ElevenLabs API key, then caches audio by voice + resolved text so repeat views don’t re-bill ElevenLabs.
• Cached audio is served from TrackPlay’s CDN; viewers receive a URL to the MP3.

​

Analytics events

​

Troubleshooting

​

“No audio” or segments never play

• Confirm ElevenLabs is connected and the integration status is ready after saving the key.
• Confirm AI Segments is enabled and the video was deployed after changes.
• Ensure the viewer has unmuted the video if you use muted autoplay.
• Check Voice ID and prompt in ElevenLabs (invalid voice or empty prompt can cause failures).

​

Audio late or missing on first watch

• Increase Preload seconds so generation can finish before the segment starts (especially for longer lines).
• First-ever generation for a given text+voice can take longer than cached playback.

​

Placeholders not replaced

• Verify window.trackplay_data is set and keys match {placeholder} names exactly.
• Keys with empty values may be skipped.

​

Testing on localhost

• Playback verification ties requests to analytics events in Elasticsearch. Right at the beginning of playback, events may still be buffered before they appear in search. TrackPlay allows generation for a short window after video_time 0 in that case, as long as workspace, video, session, and device IDs are valid and rate limits pass.
• If you self-host the events server, you can tune AI_SEGMENT_EARLY_PLAYBACK_MAX_VIDEO_TIME (seconds, default 60, clamped to a maximum of 180; set 0 to disable the bypass). The window is deliberately tight. A wider one turns the endpoint into free unauthenticated text-to-speech for anyone who finds it.

​

Related documentation

• Integration configuration: overview of all integrations.
• Dynamic options: runtime player options.
• Domain whitelist: allowed embed domains.
• Video events: player-side event hooks.

​

Support