Timed Tracks

Improved access to the content and the playback controls is only one half of the problem; your content still needs to be accessible to be useful. To this end, both the audio and video elements allow timed text tracks to be embedded using the HTML5 track element.

If you’re wondering what timed text tracks are, though, you’re probably more familiar with their practical names, like captions, subtitles, and descriptions. A timed track provides the instructions on how to synchronize text (or its rendering) with an audio or video resource: to overlay text as a video plays, to include synthesized voice descriptions, to provide signed descriptions, to allow navigation within the resource, etc.

As I touched on when talking about accessibility at the start of the guide, don’t underestimate the usefulness of subtitles and captions. They are not a niche accessibility need. There are many cases where a reader would prefer not to be bothered with the noise while reading, are reading in an environment where it would bother others to enable sound, or are unable to hear clearly or accurately what is going on because of background noise (e.g., on a subway, bus, or airplane). The irritation they will feel at having to return to the video later when they are in a more amenable environment pales next to someone who is not provided any access to that information.

It probably bears repeating at this point, too, that subtitles and captions are not the same thing, and both have important uses that necessitate their inclusion. Subtitles provide the dialogue being spoken, whether in the same language as in the video or translated, and there’s typically an assumption the reader is aware which person is speaking. Captions, however, are descriptive and provide ambient and other context useful for someone who can’t hear what else might be going on in the video in addition to the dialogue (which typically will shift location on the screen to reflect the person speaking).

A typical aside at this point would be to show a simple example of how to create one of these tracks using one of the many available technologies, but plenty of these kinds of examples abound on the Web. Understanding a bit of the technology is not a bad thing, but, similar to writing effective descriptions for images, the bigger issue is having the experience and knowledge about the target audience to create meaningful and useful captions and descriptions. These issues are outside the realm of EPUB 3, so the only advice I’ll give is if you don’t have the expertise, engage those who do. Transcription costs are probably much less than you’d expect, especially considering the small amounts of video and audio ebooks will likely include.

We’ll instead turn our attention to how these tracks can be attached to your audio or video content using the track element. The following example shows a subtitle and caption track being added to a video:

<video width="320" height="180" controls="controls">
    <source src="video/v001.webm" type='video/webm; codecs="vp8, vorbis"'/>
    <track
        kind="subtitles"
        src="video/captions/en/v001.vtt"
        srclang="en"
        label="English"/>
    <track
        kind="captions"
        src="video/captions/en/v001.cc.vtt"
        srclang="en"
        label="English"/>
</video>

The first three attributes on the track element provide information about the relation to the referenced video resource: the kind attribute indicates the nature of the timed track you’re attaching; the src attribute provides the location of the timed track in the EPUB container; and the srclang attribute indicates the language of that track.

The label attribute differs in that it provides the text to render when presenting the options the reader can select from. The value, as you might expect, is that you aren’t limited to a single version of any one type of track so long as each has a unique label. We could expand our previous example to include translated French subtitles as follows:

<video width="320" height="180" controls="controls">
    <source src="video/v001.webm" type='video/webm; codecs="vp8, vorbis"'/>
    <track
        kind="subtitles"
        src="video/captions/en/v001.vtt"
        srclang="en"
        label="English"/>
    <track
        kind="captions"
        src="video/captions/en/v001.cc.vtt"
        srclang="en"
        label="English"/>
    <track
        kind="subtitles"
        src="video/captions/fr/v001.vtt"
        srclang="fr"
        label="Fran&#xE7;ais"/>
</video>

I’ve intentionally only used the language name for the label here to highlight one of the prime deficiencies of the track element for accessibility purposes, however. Different disabilities have different needs, and how you caption a video for someone who is deaf is not necessarily how you might caption it for someone with cognitive disabilities, for example.

The weak semantics of the label attribute are unfortunately all that is available to convey the target audience. The HTML5 specification, for example, currently includes the following track for captions (fixed to be XHTML-compliant):

<track
    kind="captions"
    src="brave.en.hoh.vtt"
    srclang="en"
    label="English for the Hard of Hearing"/>

You can match the kind of track and language to a reader’s preferences, but you can’t make finer distinctions about who is the intended audience without reading the label. Machines not only haven’t mastered the art of reading, but native speakers find many ways to say the same thing, scuttling heuristic tests.

The result is that reading systems are going to be limited in terms of being able to automatically enable the appropriate captioning for any given user. In reality, getting one caption track would be a huge step forward compared to the Web, but it takes away a tool from those who do target these reader groups and introduces a frustration for the readers in that they have to turn on the proper captioning for each video.

I mentioned the difference between subtitles and captions at the outset, but the kind attribute can additionally take the following two values of note:

• descriptions — specifying this value indicates that the track contains a text description of the video. A descriptions track is designed to provide missing information to readers who can hear the audio but not see the video (which includes blind and low-vision readers, but also anyone for whom the video display is obscured or not available). The track is intended to be voiced by a text-to-speech engine.

• chapters — a chapters track includes navigational aid within the resource. If your audio or video is structured in a meaningful way (e.g., scenes), adding a chapters track will enable readers of all abilities to more easily navigate through it.

But now I’m going to trip you up a bit. The downside of the track element that I’ve been trying to hold off on is that it remains unsupported in browser cores as of writing (at least natively), which means EPUB readers also may not support tracks right away. There are some JavaScript libraries that claim to be able to provide support now (polyfills, as they’re colloquially called), but that assumes the reader has a JavaScript-enabled reading system.

Embedding the tracks directly in your video resources is, of course, another option if native support does not materialize right away.