Since build 5.2.607 real-time stream mixer function is added. This function with MCU support is designed for real-time video confereincing. Unlike the previous implementation, a real-time mixer does not stop the output stream when some of incoming streams is late (usually due to packet loss or channel interference), and does not wait for bad quality stream to restore.
Configuration
Real-time mixer is enabled by default
mixer_realtime=true
Mixer automatic creation when publishing a stream named like user1#room1 is also enabled by default
mixer_auto_start=true
For conferencing, MCU support should be enabled
mixer_mcu_audio=true mixer_mcu_video=true
It is recommended to reduce mixer idle timeout
mixer_idle_timeout=10000
Stream name and speech indicator displaying in stream picture caption can be enabled if necessary
mixer_display_stream_name=true
The rest of mixer settings are aslo supported. Note that it is not recommended to enable custom losless video processor because rel-time mixer functions will not work in this case.
Real-time mixer tuning
Real-time mixer can be tuned using the following parameters
Parameter | Default value | Description |
---|---|---|
mixer_audio_silence_threshold | -50.00 | Incoming stream audio silence level in Db |
mixer_debug_mode | false | Adds some debug info to stream picture caption |
mixer_in_buffering_ms | 200 | Incoming stream video buffer in milliseconds |
mixer_incoming_time_rate_lower_threshold | 0.95 | Relative incoming stream time to mixer time rate lower threshold |
mixer_incoming_time_rate_upper_threshold | 1.05 | Relative incoming stream time to mixer time rate upper threshold |
mixer_video_stable_fps_threshold | 15 | Incoming stream FPS threshold, video buffer will be disabled for streams with low fps |
Testing
1. For test we use
- demo server demo.flashphoner.com;
- Chrome browser;
- MCU Client web application example for conferencing.
2. Open MCU Client web application page. Enter user name user1 and room name room1
2. Click Join. A stream from your web camera will be published and added to mixer, then mixer output stream will be displayed without your microphone audio
3. Open MCU Client application page in another brower tab/browser/PC. Enter user name user2 and room name room1
4. Click Join. A stream from your web camera will be published and added to mixer, then mixer output stream will be displayed with videos from both users but audio from user1 only
5. On user1 page two videos also are playing with user2 audio only
6. Click Leave to leave the room in both tabs/browsers
Call flow
Call flow for conferencing example based on real-time mixer with MCU function is described on MCU Client page.
Incoming streams tuning recommendations
When delay occurs in the incoming stream from one of participants, realtime mixer will freeze that stream. The following is recommended to minimize incoming stream delays:
1. For RTMP streams, adjust encoding parameter so that:
- client encoder perfomance to be enough to send picture frames in time
- stream resolution and bitrate to fit to publishers channel from client to server
2. For WebRTC streams, do not raise minimum video bitrate threshold higher than webrtc_cc_min_bitrate
server configuration parameter defines. By default, lower video bitrate threshold is set to 30 kbps
webrtc_cc_min_bitrate=30000
The publisher client browser will adopt the stream to channel quality drops. The lower bitrate the lower picture quality, but the participant stream will not be freezed in this case.