But those concerns aside, that’s all there is to basic text and audio synchronization. So, as you can now see, no reading system witchcraft was required to synchronize the text document with its audio track! Instead, the audio playback is controlled by timestamps that precisely determine how an audio recording is mapped to the text structure. Whether synchronizing down to the word or moving through by paragraph, this process doesn’t change.

To synchronize the first three words “Call me Ishmael” in the first paragraph, for example, we simply repeat the process of matching element ids and audio offsets:

<par>
    <text src="chapter_001.xhtml#c01w00001"/>
    <audio
        src="audio/mobydick_001_002_melville.mp4"
        clipBegin="0:00:29.269"
        clipEnd="0:00:29.441"/>
</par>
<par>
    <text src="chapter_001.xhtml#c01w00002"/>
    <audio
        src="audio/mobydick_001_002_melville.mp4"
        clipBegin="0:00:29.441"
        clipEnd="0:00:29.640"/>
</par>
<par>
    <text src="chapter_001.xhtml#c01w00003"/>
    <audio
        src="audio/mobydick_001_002_melville.mp4"
        clipBegin="0:00:29.640"
        clipEnd="0:00:30.397"/>
</par>

You’ll notice each clipEnd matches the next element’s clipBegin here because we have a single continuous playback track. Finding each of these synchronization points manually is not so easy, though, as you might imagine.

Synchronizing to the sentence level, however, means only one synchronization point is required for all the words the sentence contains, thereby reducing the time and complexity of the process several magnitudes. The par is otherwise constructed exactly like the previous example:

<par>
    <text src="chapter_001.xhtml#c01s0002"/>
    <audio
        src="audio/mobydick_001_002_melville.mp4"
        clipBegin="0:00:30.397"
        clipEnd="0:00:44.783"/>
</par>

The process of creating overlays is only complicated by the time and text synchronizations involved, as is no doubt becoming clearer. Moving up another level, paragraph level synchronization reduces the process several more magnitudes as all the sentences can be skipped. Here’s the single entry we’d only have to make for the entire 28s second paragraph:

<par>
    <text src="chapter_001.xhtml#c01p0002"/>
    <audio
        src="audio/mobydick_001_002_melville.mp4"
        clipBegin="0:01:46.450"
        clipEnd="0:02:14.138"/>
</par>

The complexity isn’t only limited to the number of entries and finding the audio points, however, otherwise technology would easily overcome the problem. Narrating at a heading, paragraph, or even sentence level can be done relatively easily with trained narrators, as each of these structures provides a natural pause point for the person reading, a simplifier not provided when performing word-level synchronization.

A real-world recording scenario, for example, would typically involve the narrator loading their ebook and synchronizing the text in the recording application as they narrate to speed up this process immensely (e.g., using the forward arrow or spacebar each time they start a new paragraph to have the recording program automatically set the new synchronization point). Performing the synchronization at the natural pause points is not problematic in this scenario, as the person reading is briefly not focused on that task and/or the person assisting has enough of a break to cleanly resynchronize. Trying to narrate and synchronize at the word level, however, is a tricky process to perform effectively, as people naturally talk more fluidly than any process can keep up with, even if two people are involved.