What is Ultra-low latency video streaming?

 When you watch the live broadcast on the Internet, it usually takes 30 seconds to view the scenes delivered by the streamer. That is, even if you send a chat message now to the streamer, you can't see the streamer's response unless 30 seconds have elapsed. Therefore, you should keep watching for 30 seconds after sending a chat message so that you can communicate with the streamer. The chat message is in text format so it's sent immediately, but the streamer responds with a video stream, so it takes 30 seconds to reach you. This is called the "Latency".

 Until now, the broadcasting was the one-way broadcast from the Content provider to the Viewer, so there was no major inconvenience to viewers and the 30-second latency was not a big deal. However, interactive broadcasting, which is an increasing trend these days, requires viewers to participate and intervene in broadcasting in near real-time, such as Games, Stocks, and etc. If the latency of 30 seconds as in existing broadcasting, people will suffer from discomfort and will not even use it. The service can't be maintained. 

Why does latency occur during streaming?

 Commonly, Live Stream works according to the flow. Please refer to the inserted figure.

1. Video source is the original video, such as your camera or game screen.
2. Encoder encodes the original video and transmits it to the Internet. Usually, the protocol used for Encoder Output is RTMP or MPEG-TS / UDP. 
3. Ingest section receives the video from the encoder.
4. Transcoding section converts the video quality by changing the codec or adjusting the bit rate and resolution.
5. Packaging section packages the video in a format the player can play. HLS or MPEG-Dash is most often used when playing media on the web or mobile without plug-ins, so it's usually packaged in TS format and generates TS chunks of N seconds.
6. Chunks that has been packaged in the packaging section are delivered to the Origin Server.
7. Go through the Edge server.
8. Play in the Player.

 Video has latency little by little through many sections, but the biggest cause is the length of the chunk created in the Packaging section. Because HLS traditionally has recommended three chunks of about 10 seconds, which is a 30-second latency. Therefore, there is latency as long as the chunk length.

 The next big reason is the buffering of devices such as Encoder, Ingest, Transcoding or Player. Buffering is required to solve frequent transmission interruptions due to network jitter, and latency occurs as much as the time of buffering. Therefore, buffering should be minimized in all sections to reduce latency.

What is Ultra-Low Latency?

 Chunk type streaming, such as HLS or MPEG-Dash, has a latency of 30 seconds or more, but reducing the chunk size while minimizing reliability can reduce latency to about 10 seconds. For this reason, major streaming service providers can support streaming with a latency of 6 to 10 seconds.

 Using RTMP or the Smooth Streaming protocol instead of traditional chunks can deliver video with a latency of about 2 or 3 seconds. It's usually called Low Latency. And delivering video with less than a second of latency is called Ultra-Low Latency, which can be seen as a result of a new category according to technological advancement.

What is needed for ultra-low latency?

 Improving the protocol between the Edge and Player sections is the most important to provide Ultra-low latency streaming service. Because if the chunk is excessively reduced to implement Ultra-Low Latency in HLS or MPEG-Dash, which is a traditional streaming protocol, the Player must frequently request a new chunk list to the Edge server, which degrades the quality of service very much. Of course, building up the performance of the device and reducing buffering in all sections in the above-mentioned "Why does latency occur during streaming?" should be basically preceded.

 Recently, the media industry regards WebRTC and CMAF with MPEG-Dash(hereinafter CMAF) as a new alternative. Because WebRTC and CMAF enable Ultra-low latency of less than 1 second, and they are protocols that can be played without plug-ins on most modern browsers.

• WebRTC is a free open source project that provides real-time communication from web browsers and mobile applications standardizing by W3C and IETF. It's a P2P(peer-to-peer) protocol created for direct communication between devices, but it can also be applied to streaming servers.
• CMAF(Common Media Application Format) is supporting by Google and Akamai, and Delivery uses MPEG-Dash, but it's a protocol that reconfigured the Chunk format for low latency. 

How can Ultra-low latency streaming be created?

 It's very difficult to implement protocols and various features directly for Ultra-low latency streaming service. Thus, AirenSoft has released OvenMediaEngine(OME), which is an ultra-low latency streaming server, as the Open source in order to contribute to the development of the media industry. OME supports RTMP input and WebRTC output for ultra-low latency, and traditional output is also possible such as HLS, MPEG-Dash, and RTMP for general streaming. And we are developing more various protocols than now such as CMAF, SRT to secure more convenient usability and is still actively developing to add various functions necessary for streaming service operation.

 OME is implemented as one system from the Ingest to Edge server section. Since the Live transcoder is included in the streaming server, it's a streaming server that anyone can use with simple setting without a separate system.

 As you can see in AirenSoft 's GitHub, there is OvenPlayer which has released like OME. OvenPlayer is HTML5 player and one of AirenSoft's Open Source Projects. It supports WebRTC, MPEG-Dash, HLS, RTMP, and you can simply embed the player into your web page only by inserting a few lines of source code.

---

 Also, this post has been moved from AirenSoft Blog. If you read this article and are interested in our technology, please visit our blog.