HTTP Live Streaming (HLS) is a technology to play streaming video via the HTTP protocol developed by Apple. An HLS video stream is encoded to H.264 and AAC and is played on any compatible device, browser or player.
Web Call Server converts to HLS video received from other supported sources of broadcasting such as web cameras and professional video capturing device, SIP calls and so on.
Supported platforms and browsers
- Video: H.264
- Audio: AAC
- The browser connects to the server via the Websocket protocol and sends the publish command.
- The browser captures the microphone and the camera and sends the WebRTC stream to the server.
- The second browser establishes a connection via HTTP.
- The second browser receives the HLS stream and plays this stream on the page.
Quick manual on testing
Publishing of a video stream on the server and playing it via HLS in a browser
1. For the test we use:
- WCS server;
- the Two Way Streaming web application to publish the stream
- the HLS Player Minimal web application to publish the stream
2. Open the Two Way Streaming web application. Click Connect, then Publish. Copy the identifier of the stream:
3. open the HLS Player Minimal web application. In the Stream field paste the identifier of the stream and click Play. The stream starts playing:
Below is the call flow when using the HLS Player Minimal example to play a stream via HLS
1. Querying the server and playing.
Configuring the HLS URL.
Starting the playback.
2. Receiving the HLS stream from the server
Streams which can be played as HLS
Every stream published to WCS server with certain name, can be played as HLS
Since build 5.2.771, stream URI can be set for RTSP
or for VOD live translation from file
In this case a stream will be captured by source URI, and after publishing to server the stream will be played as HLS. Note that URI should be encoded, all the characters except latin letters and digits must be replaced with character code.
When HLS subscriber connects to CDN Edge, if a stream with certain name or URI is already published to some Origin server, the stream from CDN will be played as HLS for this subscriber. If there is no stream with such name or URI in CDN, Edge will try to capture stream locally to play as HLS.
HLS playback authentication using REST hook
A client authentication for HLS playback can be setup if necessary. To do this, the following parameter should be set in flashphoner.properties file
At client side, the parameter must be added to HLS URL to pass to WCS server a token obtained for example from backend server. The name of the parameter is defined with the following setting
In this case, stream URL should be formed as follows
REST hook /playHLS must be implemented on backend server in defaultApp application. WCS server will send query to backend with token received from client
Backend server should return 200 OK if token is checked successfully, and 403 Forbidden if token is wrong. In its turn, client will receive either HLS stream or 401 Unauthorized.
defines token caching interval in seconds (10 seconds by default). /playHLS queries with certain token will not be sent to backend until the token is in cache i.e. either there is stream subscriber with this token or caching interval is not expired. If caching interval parameter is set to 0
/playHLS queries are sent to backend on every HTTP GET request from client.
HLS authentication setting can be changed without server restart. In this case
hls_auth_enabled affects existing subscribers and
hls_auth_token_cache affects new subscribers only.
Custom backend application usage for HLS playback authentication
Since build 5.2.1008 it is possible to set backend application key in HLS URL, for example
In this case REST hook /playHLS will be sent to backend application with defined key (customAppKey in the example above).
Unauthorized access to HLS segments prevention
In this case, randomly generated suffix is added to every segment file name, for example
So, brute force iteration over segments is escaped.
Note that suffix is not added to preloader segments.
Adding cross-domain access control HTTP headers for HLS playback
By default, the followin access control headers are added to 200 OK response to HTTP GET request:
If necessary, for example, if HLS content and HLS player page are in different domains, custom access control headers can be added using the following parameter in flashphoner.properties file:
In this case, the headers listed in the parameter will be added to 200 OK response:
Mask support in ACAO header
Sometimes, for example, when load balancer AWS LB is used, it is necessary to set request origin address including port in ACAO header sent in response to GET request, for example
However, origin address is not always known while configuring the server. Therefore, since build 5.2.755 mask support in ACAO header can be enabled
This feature is enabled by default. In this case, if '*' character is set in the following parameter
server returns ACAO header with full request origin address in response to GET request
This can be disabled if necessary
Using nginx as reverse proxy for HLS playback
In some cases nginx web server can be used as reverse proxy for HLS playback from WCS server. Usually, it may require if HTTP headers adding does not help to workaround cross domain request restrictions in some browsers.
For example, if browser requires HLS player page and HLS stream to be in the same domain your.domain and on the same port 443 (HTTPS), nginx should be set up as follows:
It also may be useful to cache HLS stream. In this case nginx should be additionally set up as follow:
http section of /etc/nginx.conf settings file proxy cache parameters are set
server section of site settings file caching of HLS segments is set, playlist should not be cached:
Returning of static HTML pages on HLS port
Another way to workaround cross domain requests restrictions in browser is to return a static content, player page for example, on the same port that returns HLS content. To enable this feature, the following parameter should be set in flashphoner.properties file
The player page should be in directory defined by the following parameter
In this case (by default) the path to the player page files is set relative to WCS installation directory. A full path may also be set, for example
If static content returning is enabled, browser will display the HLS player page by URL . If this feature is disabled, server will return 404 Not found error by such URL.
Preloader for HLS stream playback
When first HLS subscribes connects to a stream on WCS server, especilally to CDN stream, it may take a time to split a stream to HLS segmets and to form a playlist. As a result, Safari browser on iOS devices may not be able to play HLS stream on the first attempt. To successfully play HLS stream in this case, the HLS preloader was added since build 5.2.371. The default preloader looks like this:
Since build 5.2.408 preloaders are divided by stream aspect ratio: 16:9, 4:3, 2:1
Default preloader segments are placed to
/usr/local/FlashphonerWebCallserver/hls/.preloader folder when server is started
The minimal preloader segment duration is 2 seconds by default, and can be set in milliseconds with the following parameter
Disabling HLS preloader
HLS preloader can be disabled if necessary, this feature is available since build 5.2.396. To disable HLS preloader, the following parameter is used
Custom preloader configuration
To replace the default preloader to thecustom one, do the following:
1. Choose video clip (logo for example) in three aspect ratios: 16:9, 4:3, 2:1
2. Encode video to H264, add audio track to the clip, set GOP and remove B-frames using ffmpeg
3. Download and install HLS tools from Apple site
4. Prepare custom preloader HLS segments with desired duration, for example 2 seconds
This step should be repeated for all aspect ratios.
5. Make a folder for custom preloader
6. Unpack preloader segments from archive prepared on step 4
This step should also be repeated for all aspect ratios.
7. Set custom preloader folder and duration in server settings
HLS subscription management using REST API
REST query should be HTTP/HTTPS POST request as follows:
- test.flashphoner.com - WCS server address
- 8081 - WCS server standard REST / HTTP port
- 8444 - standard HTTPS port
- rest-api - mandatory URL part
- /hls/startup - REST method used
REST queries and response sates
200 - OK
404 - Stream not found
500 - Internal error
Start HLS agent for the stream
200 – OK
404 – Not found
Find all streams having HLS agents
200 – OK
404 – Not found
Stop or restart HLS agent for the stream
Stream published name
1. If HLS agent for the stream is started by REST query /hls/startup, and there are no active HLS subscribers, agent will stop after the following timeout in seconds
By default, the timeout is 5 minutes. Also it concerns HLS agents which are started automatically for streams published using the following parameter
2. If HLS agent for the stream is stopped by REST query /hls/terminate, and there are active HLS subscribers, this agent will be restarted. In this case, active HLS subscribers must reconnect to the stream.
HLS ABR support
Master playlist file name can be set using the following parameter
Browser should get master playlist by URL
- wcs_address - WCS server address
- streamName - stream name
- index.m3u8 - master playlist file name
When master playlist is requested for the stream, server checks if streams are published according to transcoding profiles listed in cdn_profiles.yml file, for example:
All the streams published by profiles on server, will be added to master playlist, for example:
Then browser switches between HLS streams listed in master playlist depending on channel bandwidth.
When browser requests master playlist for the certain stream, transcoded stream must already be published on server and must be cut to HLS segments
To provide HLS streams by profiles, the following should be done:
1. On a standalone server:
1.1. Periodically check if streams are transcoded to parameters set by profiles, and launch transcoding if necessary using REST API
1.2. Periodically launch HLS cut for the streams, for example
2. On an Edge server in CDN periodically request HLS streams by transcoding profiles, for example
Actual HLS ABR implementation in build 5.2.585 and newer
Since build 5.2.585 HLS ABR implementation is significally changed. As usual, HLS ABR can be used in CDN only, but Edge server captures all the transcoded streams for ABR manifest stream variants within the same mediasession to syhchronize the stream variants. This requires to configure Tanscoder nodes and Edge nodes simultaneously, and adds some limits. Let's explore it below.
Transcoder node settings
To synchronize all the variants of the same stream, encoding should be aligned on Transcoder node
Also, FPS filter should be enabled
All stream variants key frames (GOPs) should be synchronized. For example, we will send key frame every 2 seconds for 25 fps stream
HLS Edge node settings
HLS preloader and streams resizing should be disabled on HLS Edge node
The transcoding profiles should be set as follows in cdn_profiles.yml file
Note that audio parameters can be set for the first profile only because those parameters should be identical for all the profiles, and will be applyed according to the first profile.
Then HLS ABR should be enabled
Client should request ABR manifest giving a stream name with a special suffix
The suffix can be set with the following parameter
The playlist contains links to stream variants playlists, a client can switch between then
If stream name contains no suffix, HLS will be played without ABR support.
Transcoding to higher resolutions prevention
If stream transcoding to higher resolutions is disabled on HLS ABR Edge server
then stream variants conforming to higher resolution profiles will not be cut for the stream, and will not be vailable to a player.
1. HLS Edge can be used for HLS streaming only, client sessions of another kinds will not work.
2. Stream recording, snapshots, mixing, stream capturing from another server and other captured stream management functions will not work.
HLS segments storage
By default, HLS segments are written to server hard drive to the /usr/local/FlashphonerWebCallServer/hls folder. Starting from build 5.2.687, the folder for saving segments can be changed using the following parameter
(Preloader folder is configured separately with parameter hls_preloader_dir.)
The number of stored segments corresponds to the specified playlist size, 10 by default
The less segments number, the less playback latency. However subscribers with poor channel could request HLS segments which are already gone from playlist and disk if playlist is short. To fix it, since build 5.2.581 the certain number of segments can be stored on disk after they gone from playlist. This feature can be enabled by the following parameter
By default, 5 last segments will be stored
For example, if playlist contains 3 segments
3 current segments and 5 previous segments will be stored on disk
Under high load, for example on HLS streaming dedicated server, segments reading from hard drive to send them to subscribers can increase the latency. In this case, HLS segments storing in memory should be enabled
Segments will be read from memory to send them to subscribers, and will also be written to hard drive for debugging purposes. Note this will require more Java heap memory.
Debug logs for HLS session
For an error report, debug logging can be enabled for HLS sessions using CLI
Note that config flashphoner.properties will be overwritten after the command.
1. Non-recoverable freeze of HLS stream played in iOS Safari through a CDN
Symptoms: one minute after publishing start image stops, sound continues to play
b) enable transcoding on server using the following option in flashphoner.properties file
b) if transcoding is undesirable, set the following option in flashphoner.properties file
In this case, clicks are possible in audio, but video will not stop.
2. HLS segments writing stops when playing stream published in Firefox browser.
Symptoms: a few minutes after playback start HLS segments stop writing, in that case the stream directory in hls directory is not deleted, and messages in server log continue to appear
Publisher must publish stream again to recover.
Solution: use another browser to publish the stream which supposed to be played via HLS.
3. No video for the first subscriber when playing HLS in Safari on iOS 12.4
Symptoms: no video for the first subscriber when playing a stream as HLS in Safari on iOS 12.4, however video is played normally if there are other HLS-subscribers to the stream
Solution: set the minimum HLS playlist segments count not less than 2
4. No video for any subscriber when playing RTMP stream as HLS in Safari on iOS 12.4 if the following parameter is set to
Symptoms: no video for any subscriber when playing RTMP stream as HLS in Safari on iOS 12.4
Soluition: use mono sound when a file with stereo sound track is published, for example, set the followint command line options for ffmpeg
5. If stream transcoded by CDN is played as HLS, and if stream aspect ratio is changed during transcoding, HLS preloader is shown by original stream aspect ratio
Symptoms: when stream transcoded is requested by setting profile name, test-640x480p for example, 16x9 preloader is shown if original stream is 1280x720
Solution: turn on aspect ratio preserving on Transcoder CDN nodes
6. If the source stream contains B-fames, the picture can twitch in some players
Symptoms: a strong picture twitching while playing a stream via HLS, this may lokk like low FPS
Solution: update WCS to build 5.2.863, where the problem is solved