| 1 | # Multiple Alternative Audio Tracks
|
| 2 | ## General
|
| 3 | m3u8 manifests with multiple audio streams will have those streams added to `video.js` in an `AudioTrackList`. The `AudioTrackList` can be accessed using `player.audioTracks()` or `tech.audioTracks()`.
|
| 4 |
|
| 5 | ## Mapping m3u8 metadata to AudioTracks
|
| 6 | The mapping between `AudioTrack` and the parsed m3u8 file is fairly straight forward. The table below shows the mapping
|
| 7 |
|
| 8 | | m3u8 | AudioTrack |
|
| 9 | |---------|------------|
|
| 10 | | label | label |
|
| 11 | | lang | language |
|
| 12 | | default | enabled |
|
| 13 | | ??? | kind |
|
| 14 | | ??? | id |
|
| 15 |
|
| 16 | As you can see m3u8's do not have a property for `AudioTrack.id`, which means that we let `video.js` randomly generate the id for `AudioTrack`s. This will have no real impact on any part of the system as we do not use the `id` anywhere.
|
| 17 |
|
| 18 | The other property that does not have a mapping in the m3u8 is `AudioTrack.kind`. It was decided that we would set the `kind` to `main` when `default` is set to `true` and in other cases we set it to `alternative` unless the track has `characteristics` which include `public.accessibility.describes-video`, in which case we set it to `main-desc` (note that this `kind` indicates that the track is a mix of the main track and description, so it can be played *instead* of the main track; a track with kind `description` *only* has the description, not the main track).
|
| 19 |
|
| 20 | Below is a basic example of a mapping
|
| 21 | m3u8 layout
|
| 22 | ``` JavaScript
|
| 23 | {
|
| 24 | 'media-group-1': [{
|
| 25 | 'audio-track-1': {
|
| 26 | default: true,
|
| 27 | lang: 'eng'
|
| 28 | },
|
| 29 | 'audio-track-2': {
|
| 30 | default: false,
|
| 31 | lang: 'fr'
|
| 32 | },
|
| 33 | 'audio-track-3': {
|
| 34 | default: false,
|
| 35 | lang: 'eng',
|
| 36 | characteristics: 'public.accessibility.describes-video'
|
| 37 | }
|
| 38 | }]
|
| 39 | }
|
| 40 | ```
|
| 41 |
|
| 42 | Corresponding AudioTrackList when media-group-1 is used (before any tracks have been changed)
|
| 43 | ``` JavaScript
|
| 44 | [{
|
| 45 | label: 'audio-tracks-1',
|
| 46 | enabled: true,
|
| 47 | language: 'eng',
|
| 48 | kind: 'main',
|
| 49 | id: 'random'
|
| 50 | }, {
|
| 51 | label: 'audio-tracks-2',
|
| 52 | enabled: false,
|
| 53 | language: 'fr',
|
| 54 | kind: 'alternative',
|
| 55 | id: 'random'
|
| 56 | }, {
|
| 57 | label: 'audio-tracks-3',
|
| 58 | enabled: false,
|
| 59 | language: 'eng',
|
| 60 | kind: 'main-desc',
|
| 61 | id: 'random'
|
| 62 | }]
|
| 63 | ```
|
| 64 |
|
| 65 | ## Startup (how tracks are added and used)
|
| 66 | > AudioTrack & AudioTrackList live in video.js
|
| 67 |
|
| 68 | 1. `HLS` creates a `MasterPlaylistController` and watches for the `loadedmetadata` event
|
| 69 | 1. `HLS` parses the m3u8 using the `MasterPlaylistController`
|
| 70 | 1. `MasterPlaylistController` creates a `PlaylistLoader` for the master m3u8
|
| 71 | 1. `MasterPlaylistController` creates `PlaylistLoader`s for every audio playlist
|
| 72 | 1. `MasterPlaylistController` creates a `SegmentLoader` for the main m3u8
|
| 73 | 1. `MasterPlaylistController` creates a `SegmentLoader` for a potential audio playlist
|
| 74 | 1. `HLS` sees the `loadedmetadata` and finds the currently selected MediaGroup and all the metadata
|
| 75 | 1. `HLS` removes all `AudioTrack`s from the `AudioTrackList`
|
| 76 | 1. `HLS` created `AudioTrack`s for the MediaGroup and adds them to the `AudioTrackList`
|
| 77 | 1. `HLS` calls `MasterPlaylistController`s `useAudio` with no arguments (causes it to use the currently enabled audio)
|
| 78 | 1. `MasterPlaylistController` turns off the current audio `PlaylistLoader` if it is on
|
| 79 | 1. `MasterPlaylistController` maps the `label` to the `PlaylistLoader` containing the audio
|
| 80 | 1. `MasterPlaylistController` turns on that `PlaylistLoader` and the Corresponding `SegmentLoader` (master or audio only)
|
| 81 | 1. `MediaSource`/`mux.js` determine how to mux
|
| 82 |
|
| 83 | ## How tracks are switched
|
| 84 | > AudioTrack & AudioTrackList live in video.js
|
| 85 |
|
| 86 | 1. `HLS` is setup to watch for the `changed` event on the `AudioTrackList`
|
| 87 | 1. User selects a new `AudioTrack` from a menu (where only one track can be enabled)
|
| 88 | 1. `AudioTrackList` enables the new `Audiotrack` and disables all others
|
| 89 | 1. `AudioTrackList` triggers a `changed` event
|
| 90 | 1. `HLS` sees the `changed` event and finds the newly enabled `AudioTrack`
|
| 91 | 1. `HLS` sends the `label` for the new `AudioTrack` to `MasterPlaylistController`s `useAudio` function
|
| 92 | 1. `MasterPlaylistController` turns off the current audio `PlaylistLoader` if it is on
|
| 93 | 1. `MasterPlaylistController` maps the `label` to the `PlaylistLoader` containing the audio
|
| 94 | 1. `MasterPlaylistController` maps the `label` to the `PlaylistLoader` containing the audio
|
| 95 | 1. `MasterPlaylistController` turns on that `PlaylistLoader` and the Corresponding `SegmentLoader` (master or audio only)
|
| 96 | 1. `MediaSource`/`mux.js` determine how to mux
|