Logo
  • TALK TO US
  • BLOGS
  • HIRE US
  • COMMUNITY
We build React web apps
Talk to us
Community initiatives
Views on technology
Video on demand with language switching, using HLS
Switch the language on the site as well as on the video being played, without reloading the video or the web page. The language selector would be at the top level of the page, and selecting a new language will switch the HLS audio stream as well as load the localised text for the language.

Wouldn’t it be great, if videos continued playing while we changed the language selection on the web page?

Further, if only the audio stream playing is downloaded, we can have localisation for multiple languages without worrying about download size/speed.

There are two basic ideas here:

  1. Single button for localisation of web page and video.
  2. Separation of the audio streams from the video. So only the audio of the language chosen is streamed from the backend to the user’s browser.

To do this, we need to have one centralised language button, which is integrated with the video player controls. It isn’t very difficult to achieve this. And secondly to separate the video and audio streams on the backend. Which is also easy.

Here’s demo video, demonstrating this.

Separate audio and video streams using HLS (Http live streaming)

HLS or Http live streaming is a protocol supported by all browsers and devices, which gives it an edge over Dash which isn’t supported by Apple devices (Sad but true) as of Sept, 2020.

The HLS protocol creates smaller segments of a media file which is incrementally streamed to the user. This is beneficial as the end user does not need to download the entire media file, which could be of a large size.

To achieve our objective of switching between audio streams, while downloading a single audio stream. There are three main steps to perform on the media files.

  1. Remove the original audio from the media file, creating a video only and audio only file.

This is needed so we could create a separate independent audio stream from the original audio. Having a video only file, helps us create a video only stream, so if the user selects an alternate language the original audio doesn’t interfere.

A single HLS segment could contain audio and video OR it could contain only video OR only audio. As we need to switch between multiple audio streams, we will have the video segments without any audio and audio streams for each language.

  1. Assuming we have multiple audio translations available as separate audio files. We would have to create HLS audio streams for each of them.
  2. The last step would be to create an HLS playlist, with the audio and video streams. This playlist would use the EXT-X-MEDIA tags for each of the audio streams. Something like below:
#EXT-X-MEDIA:TYPE=AUDIO,URI="eng.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="stream_4",DEFAULT=YES,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="hin.m3u8",GROUP-ID="default-audio-group",LANGUAGE="hi",NAME="stream_5",AUTOSELECT=YES,CHANNELS="2"

and the video stream would refer to the multiple audio streams, using the ‘GROUP-ID’, like below:

#EXT-X-STREAM-INF: AUDIO="default-audio-group"

with this kind of playlist, we have separated the video stream from the multiple audio streams.

Single button click for page and audio selection.

To be able to select the localisation language and the audio stream with a single button. We would need to integrate our media player to the localisation selector. Lets see, how this can be accompolished using React on the frontend.

For our example, we will use react-i18next for localisation of the page. For the player we will use hls.js.

Lets say, we have two languages, english and hindi on our web page. The selectors for the languages could be like below.

        <select onChange={handleChange}>
          <option value={0}> {t('language.en')} </option>
          <option value={1}> {t('language.hi')} </option>
        </select>

When the user selects the first option, he selects english (en). The second option would be hindi (hi). The {t('language.en')} is the syntax from react-i18next. It essentially will display localised text on the browser, depending on the selected language.

On selection of a language, we will trigger the handleChange function. This function will change the language on the web page and also would change the audio stream selected. The code for this function, could be as below:

  const handleChange = (e) => {
    if (e.target.value === '0') {
      i18n.changeLanguage('en');
    } else if (e.target.value === '1') {
      i18n.changeLanguage('hi');
    }
    hlsRef.current.audioTrack = e.target.value;
  };

The if-else check is to change the language on the web page using the react-i18next’s changeLanguage function. (This is part of the react-118next’s documentation). The hlsRef.current.audioTrack = e.target.value; is the assignment, which changes the audio stream to either english (0) or hindi (1). The 0 or 1 here maps to the order of the audio streams in our playlist.

If you are wondering, what is the hlsRef here, this comes from the hls.js library. Here is some sample react code, to explain that:

  let hlsRef = useRef();

  useEffect(() => {
    if (Hls.isSupported()) {
      let hls = new Hls();
      ...
      hls.on(Hls.Events.MEDIA_ATTACHED, function () {
        console.log('video and hls.js are now bound together !');
        hls.loadSource(videoSrc);
        hls.on(Hls.Events.MANIFEST_PARSED, function () {
          hlsRef.current = hls;
          hlsRef.current.media.controls = true;
        });
        ...
      });
    }
  }, []);

This was a high level description of how to achieve a great localisation experience for a web page with videos and needing to support multiple languages.

If you have any questions, you could contact me on twitter @jkntji

© 2020 Tech47