...

Mixer captions font size can be changed if necessary depending on mixer outpout output stream resolution:

for video streams, caption font size if 20pt by default

...

Since build 5.2.741 caption text colour and backgroung background colour can be changed with the following parameters

...

Code Block

theme	RDark

mixer_text_background_opacity=100

Background opacity shold should be set in percents: 0 for full transparency, 100 for full opacity (text background will be coloured to defined colour). The default value is 100 (fully opaque).

...

Since build 5.2.770 it is possible to set captions font using the following param,eterparameter

Code Block

theme	RDark

mixer_text_font=Serif

...

If the chosen font contains no characters which are in caption text, those charecters characters will not be displayed.

Caption text autoscaling

Since build 5.2.709 caption text can be autoscaled to fit the picture. The feature is enabled by default with thr the following parameter

Code Block

theme	RDark

mixer_text_autoscale=true

...

Since build 5.2.770 it is possible to disable room name displaying (which is set to sream stream name while automatically ading adding stream to mixer). This can be done with the following parameter

...

In this case, for stream named "user1#room1" only "user1" will be displayed (without room name and special character).

Speech indicator management

Speech indicator frame thikness can be set (6 pixels by default)

Code Block

theme	RDark

mixer_voice_activity_frame_thickness=6

The frame is displayed outside astream picture by default. However, if stream pictures are too close, foe example, if CenterNoPaddingGridLayout is used, the frame can affect neighbor pictures

Image Removed

In this case, inner speech frame displaying should be enabled

Code Block

theme	RDark

mixer_voice_activity_frame_position_inner=true

Image Removed

By default, if audio only track from some stream was added to mixer, this stream is shown in separate frame with speech indicator (see above). Sound only can be added to mixer without showing a separate participant (for example, to comment or to dub) with the following parameter if necessary

Code Block

theme	RDark

mixer_show_separate_audio_frame=false

Since build 5.2.741 speech indicator frame colour can be changed with the following parameter

Code Block

theme	RDark

mixer_voice_activity_colour=#FF0000

A colour can be set as hexadecimal value with # or 0x prefix, in #RRGGBB form. Using the settings above, speech indicator frame will be red.

Mixer background management

By default, mixer applies black background. Since build 5.2.645 it is possible to set PNG picture file which will be used as mixer background. For example, to change the background to blue, prepare blue.png picture and set the following parameter

Code Block

theme	RDark

mixer_video_background_filename=blue.png

Background picture file should be placed to /usr/local/FlashphonerWebCallServer/conf if full path is not set. The file can be placed to any folder on the server, in this case full path to the file should be set to the parameter

Code Block

theme	RDark

mixer_video_background_filename=/opt/media/blue.png

Background picture will be automatically scaled to mixer output stream resolution

Image Removed

Audio only streams picture aspect ratio management

By default, audio only streams picture aspect ratio is set according to mixer one (16:9)

Code Block

theme	RDark

mixer_audio_only_width=640
mixer_audio_only_height=360
mixer_video_width=1280
mixer_video_height=720

It can be changed together with mixer if necessary, for example to 4:3

Code Block

theme	RDark

mixer_audio_only_width=640
mixer_audio_only_height=480
mixer_video_width=1280
mixer_video_height=960

Note that audio only streams picture aspect ratio settings are applied only with WCS restart.

Real-time mixer tuning

Real-time mixer can be tuned using the following parameters

...

Parameter

...

Default value

...

Description

...

mixer_in_buffering_ms

...

200

...

Incoming stream video buffer in milliseconds

...

Testing

1. For test we use

demo server demo.flashphoner.com;
Chrome browser;
MCU Client web application example for conferencing.

2. Open MCU Client web application page. Enter user name user1 and room name room1

Image Removed

2. Click Join. A stream from your web camera will be published and added to mixer, then mixer output stream will be displayed without your microphone audio

Image Removed

3. Open MCU Client application page in another brower tab/browser/PC. Enter user name user2 and room name room1

Image Removed

4. Click Join. A stream from your web camera will be published and added to mixer, then mixer output stream will be displayed with videos from both users but audio from user1 only

Image Removed

5. On user1 page two videos also are playing with user2 audio only

Image Removed

6. Click Leave to leave the room in both tabs/browsers

Call flow

Call flow for conferencing example based on real-time mixer with MCU function is described on MCU Client page.

Incoming streams tuning recommendations

When delay occurs in the incoming stream from one of participants, realtime mixer will freeze that stream. The following is recommended to minimize incoming stream delays:

1. For RTMP streams, adjust encoding parameter so that:

client encoder perfomance to be enough to send picture frames in time
stream resolution and bitrate to fit to publishers channel from client to server

2. For WebRTC streams, do not raise minimum video bitrate threshold higher than webrtc_cc_min_bitrate server configuration parameter defines. By default, lower video bitrate threshold is set to 30 kbps

Code Block

theme	RDark

webrtc_cc_min_bitrate=30000

The publisher client browser will adopt the stream to channel quality drops. The lower bitrate the lower picture quality, but the participant stream will not be freezed in this case.

Adding one stream to two or more realtime mixers simultaneously

Since build 5.2.732 one stream can be added to two ore more realtime mixers simultaneously. Note that realtime mixer should be enabled

Code Block

theme	RDark

mixer_realtime=true

and custom losless videoprocessor should be disabled

Code Block

theme	RDark

mixer_lossless_video_processor_enabled=false

Known issues

1. Real-time mixer functions are disabled when custom losless videoprocessor is used, mixer incoming streams cannot be played

Symptoms: stream stops playing after adding to mixer

Solution: do not use custom losless videoprocessor with real-time mixer

Code Block

theme	RDark

mixer_lossless_video_processor_enabled=false

2. To display stream captions in mixer, it would be necessary to install fontconfig library

Symptoms: streams cannot be added to mixer with the following exception in server log

Code Block

theme	RDark

09:17:11,756 ERROR           MixerAgent - MIXER-AGENT-mixer://mixervmixdr52-9d46cd04-5867-4d74-a9d9-baf67f74e7d2 Mixer closed due to error
java.lang.InternalError: java.lang.reflect.InvocationTargetException
	at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:86)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:310)
	at java.desktop/sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)
	at java.desktop/sun.font.SunFontManager.getInstance(SunFontManager.java:247)
	at java.desktop/sun.font.FontDesignMetrics.getMetrics(FontDesignMetrics.java:265)
	at java.desktop/java.awt.Font.getStringBounds(Font.java:2606)
	at java.desktop/java.awt.Font.getStringBounds(Font.java:2516)
	at com.flashphoner.media.N.A.A.A(Unknown Source)
	at com.flashphoner.media.mixer.video.presentation.Canvas.computeTextScales(Unknown Source)
	at com.flashphoner.media.mixer.video.presentation.Canvas.writeNative(Unknown Source)
	at com.flashphoner.media.N.A.A(Unknown Source)
	at com.flashphoner.media.N.D.D(Unknown Source)
	at com.flashphoner.media.N.D.A(Unknown Source)
	at com.flashphoner.server.remote.B.B.Г™(Unknown Source)
	at com.flashphoner.server.remote.B.B.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
	at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:84)
	... 14 more
Caused by: java.lang.NullPointerException
	at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1262)
	at java.desktop/sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:225)
	at java.desktop/sun.awt.FontConfiguration.init(FontConfiguration.java:107)
	at java.desktop/sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:719)
	at java.desktop/sun.font.SunFontManager$2.run(SunFontManager.java:367)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:310)
	at java.desktop/sun.font.SunFontManager.<init>(SunFontManager.java:312)
	at java.desktop/sun.awt.FcFontManager.<init>(FcFontManager.java:35)
	at java.desktop/sun.awt.X11FontManager.<init>(X11FontManager.java:56)
	... 20 more

Solution: install fontconfig library

Code Block

language	bash
theme	RDark

sudo yum install -y fontconfig

3. A participant name can be drawn over speech indicator frame if there are many participants in mixer

Symptoms: for small participant pictures (in desktop layout, for example) participants name can be drawn over speech indicator frame

Solution: enable inner speech indicator frame position and decrease frame thickness

...

theme	RDark

...

Custom stream label displaying

Since build 5.2.816 it is possible to display custom stream label in mixer instead of stream and room name. In fact, almost any name can be assigned including national characters.

A label should be set in stream name while publishing, for example

Code Block

language	js
theme	RDark

session.createStream({
    streamName: "test1#m1?label=John Doe",
    display: localDisplay,
    ...
}).publish();

Image Added

In this case the label will be displayed in mixer, including audio only streams

Image Added

National characters are supported along with the following special characters

Code Block

theme	RDark

~!@#№$%^*()-_,.;:[]{}<>/|\

Character '+' will be replaced by whitespace, i.e. in the example above stream can be published with label

Code Block

theme	RDark

test1#m1?label=John+Doe

Setting or changing stream label using REST API

Since build 5.2.1635 it is possible to set a custom stream label while adding a stream to mixer with /mixer/add REST API query

Code Block

language	js
theme	RDark

{
 "uri": "mixer://mixer1",
 "remoteStreamName": "stream1",
 "streamLabel": "John Doe"
}

The label may be changed for the stream already added to the mixer by /mixer/set_stream_label query

Code Block

language	js
theme	RDark

{
 "uri":"mixer://m1",
 "remoteStreamName":"stream1",
 "streamLabel": "Mr. John Doe"
}

or

Code Block

language	js
theme	RDark

{
 "uri":"mixer://m1",
 "remoteMediaSessionId":"95bf2be8-f459-4f62-9a7f-c588f33e0ad3",
 "streamLabel": "Mr. John Doe"
}

The label may be cleared by passing an empty string as streamLabel

Code Block

language	js
theme	RDark

{
 "uri":"mixer://m1",
 "remoteMediaSessionId":"95bf2be8-f459-4f62-9a7f-c588f33e0ad3",
 "streamLabel": ""
}

In this case the stream name will be displayed as published.

Caption location

Obsoleted settings in builds from 5.2.844 to 5.2.1077

Warning
This setting is not supported since build 5.2.1079!

Since build 5.2.844, it is possible to change location of video participant's caption. By default, caption is located in left bottom corner of video.

It is possible to display caption above video using the following parameter

Code Block

theme	RDark

mixer_text_outside_frame=TOP

Image Added

or under video

Code Block

theme	RDark

mixer_text_outside_frame=BOTTOM

Image Added

The distance between video and its caption can be changed using the following parameter

Code Block

theme	RDark

mixer_text_outside_frame_padding=70

Captions placement above or below a picture affects picture placement even if captions diplaying is disabled.

Actual settings since build 5.2.1079

Since build 5.2.1079 the following parameter was added to set caption text alignment realtive to frame picture

Code Block

theme	RDark

mixer_text_align=BOTTOM_LEFT

Possible values

Value	Caption placement
TOP_LEFT	Top left corner
TOP_CENTER	Top center
TOP_RIGHT	Top rightcorner
CENTER	In a picture center
BOTTOM_LEFT	Bottom left corner (by default)
BOTTOM_CENTER	Bottom center
BOTTOM_RIGHT	Bottom right corner
EXTERNAL_TOP_CENTER	Centerd above a picture
EXTERNAL_BOTTOM_CENTER	Centered below a picture

By default, captions are placed in bottom left corner of a picture frame.

The values EXTERNAL_TOP_CENTER and EXTERNAL_BOTTOM_CENTER replace obsoleted mixer_text_outside_frame parameter. For example, the following value

Code Block

theme	RDark

mixer_text_align=EXTERNAL_BOTTOM_CENTER

will act like mixer_text_outside_frame=BOTTOM

Image Added

Speech indicator management

Thickness and position

Speech indicator frame thickness can be set (6 pixels by default)

Code Block

theme	RDark

mixer_voice_activity_frame_thickness=6

The frame is displayed outside astream picture by default. However, if stream pictures are too close, foe example, if CenterNoPaddingGridLayout is used, the frame can affect neighbor pictures

Image Added

In this case, inner speech frame displaying should be enabled

Code Block

theme	RDark

mixer_voice_activity_frame_position_inner=true

Image Added

Audio only streams displaying

By default, if audio only track from some stream was added to mixer, this stream is shown in separate frame with speech indicator (see above). Sound only can be added to mixer without showing a separate participant (for example, to comment or to dub) with the following parameter if necessary

Code Block

theme	RDark

mixer_show_separate_audio_frame=false

Before build 5.2.965, the stream for dubbing must contain audio and video, and must be added to mixer with the following /mixer/add REST query

Code Block

language	js
theme	RDark

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "stream_dub",
    "hasVideo":false,
    "hasAudio":true
}

Since build 5.2.965, audio only stream also will not be displayed if this parameter is used

Since build 5.2.1359, the parameter mixer_show_separate_audio_frame affects stream in mixer displaying when audio data are received before video. In this case, a frame with speech indicator will be displayed for such stream when default settings are used, then video will be displayed after first video data receiving. If the following parameter is set

Code Block

theme	RDark

mixer_show_separate_audio_frame=false

then speech indicator frame will not be displayed until video data arrive.

Avatar pictures displaying for audio only streams

Warning

In the builds 5.2.1710 - 5.2.1725 the feature is supported in Ubuntu 20.04 and other systems with glibc 2.31 and newer only

Since build 5.2.1727 the feature is supported in Centos 7.6 (glibc 2.17) and above

Since build 5.2.1710 it is possible to set an avatar picture in PNG (with transparency), JPG or BMP formats to an audio only stream in mixer. A picture may be set while adding a stream to mixer by REST API query /mixer/add :

Code Block

language	js
theme	RDark

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "user2",
    "hasVideo": false,
    "hasAudio": true,
    "avatar": "https://mystorage.com/storage/avatar.png"
}

or by REST API query /mixer/set_stream_avatar for the audio only stream which already has added to the mixer:

Code Block

language	js
theme	RDark

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "user2",
    "avatar": "https://mystorage.com/storage/avatar.png"
}

The following ways to set a picture are supported:

HTTP URL: https://mystorage.com/storage/avatar.png
file URI: file:///opt/avatar.png
local file: /opt/avatar.png

Audio only stream in mixer is displaying as follows

Image Added

If stream names displaying in mixer is enabled, audio only stream name will be displayed like video stream one.

Avatar picture may be removed by REST API query /mixer/remove_stream_avatar :

Code Block

language	js
theme	RDark

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "user2"
}

Since WCS build 5.2.1858 it is possible to set avatar pictures automatically by stream name, not using REST API. In this case avatar picture files should be placed to the folder

Code Block

theme	RDark

avatar_dir=/usr/local/FlashphonerWebCallServer/avatar

A file name should be equal to stream name. For example, the picture file stream1.png will be applied to the stream named stream1. If the stream was added to the mixer automatically, e.g. stream1#mixer1, the mixer name will not be used to get the picture, only the stream name before # character.

Warning
Avatar picture setting with REST API query `/mixer/set_stream_avatar` has a higher priority then automatic picture setting

Frame colour

Since build 5.2.741 speech indicator frame colour can be changed with the following parameter

Code Block

theme	RDark

mixer_voice_activity_colour=#FF0000

A colour can be set as hexadecimal value with # or 0x prefix, in #RRGGBB form. Using the settings above, speech indicator frame will be red.

Frame displaying when silence is in a stream

By default, speech indicator frame may blink at normal participant speech rate, which can be incomfortable for viewing. For this reason, since build 5.2.775 it is possible to set a time interval in which the frame will be displayed since silence is detected in the stream. The feature can be enabled using the following parameter

Code Block

theme	RDark

mixer_voice_activity_switch_delay=500

In this case, speech indicator frame will still be displayed in 500 milliseconds since silence is detected in the stream.

By default, this interval is set to 0, frame will be hidden without delay.

Mixer background management

By default, mixer applies black background. Since build 5.2.645 it is possible to set PNG picture file which will be used as mixer background. For example, to change the background to blue, prepare blue.png picture and set the following parameter

Code Block

theme	RDark

mixer_video_background_filename=blue.png

Background picture file should be placed to /usr/local/FlashphonerWebCallServer/conf if full path is not set. The file can be placed to any folder on the server, in this case full path to the file should be set to the parameter

Code Block

theme	RDark

mixer_video_background_filename=/opt/media/blue.png

Background picture will be automatically scaled to mixer output stream resolution

Image Added

Audio only streams picture aspect ratio management

By default, audio only streams picture aspect ratio is set according to mixer one (16:9)

Code Block

theme	RDark

mixer_audio_only_width=640
mixer_audio_only_height=360
mixer_video_width=1280
mixer_video_height=720

It can be changed together with mixer if necessary, for example to 4:3

Code Block

theme	RDark

mixer_audio_only_width=640
mixer_audio_only_height=480
mixer_video_width=1280
mixer_video_height=960

Note that audio only streams picture aspect ratio settings are applied only with WCS restart.

Multithreading support and hign load optimizations

Since build 5.2.793 multithreading support is added to optimize a mixer for a big participants amount. This feature can be enabled by the following parameters

Code Block

theme	RDark

mixer_type=MULTI_THREADED_NATIVE
mixer_mcu_multithreaded_mix=true

CPU threads count to mix audio amd video can be adjusted by the following parameters

Code Block

theme	RDark

mixer_audio_threads=10
mixer_video_threads=4

Usually, MCU mixer encodes one video stream and a many ausio streams: two per every participant plus one common audio track. Therefore, it is recommended to set more CPU threads for audio encoding than for video (as shown above). Additionally, if freezes occur in MCU mixer output stream, it is recommended to enable multithreaded delivery of mixing result to the main WCS engine, to send it to subscribers.

Code Block

theme	RDark

mixer_mcu_multithreaded_delivery=true

If participants count is lower than mixing threads count (for example, 3 participants), only one CPU thread will be used for mixing.

Real-time mixer tuning

Real-time mixer can be tuned using the following parameters

Parameter	Default value	Description
mixer_audio_silence_threshold	-50.00	Incoming stream audio silence level in Db
mixer_debug_mode	false	Adds some debug info to stream picture caption
mixer_in_buffering_ms	200	Incoming stream video buffer in milliseconds
mixer_incoming_time_rate_lower_threshold	0.95	Relative incoming stream time to mixer time rate lower threshold
mixer_incoming_time_rate_upper_threshold	1.05	Relative incoming stream time to mixer time rate upper threshold
mixer_video_stable_fps_threshold	15	Incoming stream FPS threshold, video buffer will be disabled for streams with low fps
audio_mixer_max_delay	300	A maximum time interval to wait for any audio data in the incoming stream, in milliseconds, when this time is expired, the stream will not be mixed until some media data receiving

Testing

1. For test we use

demo server demo.flashphoner.com;
Chrome browser;
MCU Client web application example for conferencing.

2. Open MCU Client web application page. Enter user name user1 and room name room1

Image Added

2. Click Join. A stream from your web camera will be published and added to mixer, then mixer output stream will be displayed without your microphone audio

Image Added

3. Open MCU Client application page in another brower tab/browser/PC. Enter user name user2 and room name room1

Image Added

4. Click Join. A stream from your web camera will be published and added to mixer, then mixer output stream will be displayed with videos from both users but audio from user1 only

Image Added

5. On user1 page two videos also are playing with user2 audio only

Image Added

6. Click Leave to leave the room in both tabs/browsers

Call flow

Call flow for conferencing example based on real-time mixer with MCU function is described on MCU Client page.

Incoming streams tuning recommendations

When delay occurs in the incoming stream from one of participants, realtime mixer will freeze that stream. The following is recommended to minimize incoming stream delays:

1. For RTMP streams, adjust encoding parameter so that:

client encoder perfomance to be enough to send picture frames in time
stream resolution and bitrate to fit to publishers channel from client to server

2. For WebRTC streams, do not raise minimum video bitrate threshold higher than webrtc_cc_min_bitrate server configuration parameter defines. By default, lower video bitrate threshold is set to 30 kbps

Code Block

theme	RDark

webrtc_cc_min_bitrate=30000

The publisher client browser will adopt the stream to channel quality drops. The lower bitrate the lower picture quality, but the participant stream will not be freezed in this case.

Adding one stream to two or more realtime mixers simultaneously

Since build 5.2.732 one stream can be added to two ore more realtime mixers simultaneously. Note that realtime mixer should be enabled

Code Block

theme	RDark

mixer_realtime=true

and custom losless videoprocessor should be disabled

Code Block

theme	RDark

mixer_lossless_video_processor_enabled=false

Known issues

1. Real-time mixer functions are disabled when custom losless videoprocessor is used, mixer incoming streams cannot be played

Symptoms: stream stops playing after adding to mixer

Solution: do not use custom losless videoprocessor with real-time mixer

Code Block

theme	RDark

mixer_lossless_video_processor_enabled=false

2. To display stream captions in mixer, it would be necessary to install fontconfig library

Symptoms: streams cannot be added to mixer with the following exception in server log

Code Block

theme	RDark

09:17:11,756 ERROR           MixerAgent - MIXER-AGENT-mixer://mixervmixdr52-9d46cd04-5867-4d74-a9d9-baf67f74e7d2 Mixer closed due to error
java.lang.InternalError: java.lang.reflect.InvocationTargetException
	at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:86)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:310)
	at java.desktop/sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)
	at java.desktop/sun.font.SunFontManager.getInstance(SunFontManager.java:247)
	at java.desktop/sun.font.FontDesignMetrics.getMetrics(FontDesignMetrics.java:265)
	at java.desktop/java.awt.Font.getStringBounds(Font.java:2606)
	at java.desktop/java.awt.Font.getStringBounds(Font.java:2516)
	at com.flashphoner.media.N.A.A.A(Unknown Source)
	at com.flashphoner.media.mixer.video.presentation.Canvas.computeTextScales(Unknown Source)
	at com.flashphoner.media.mixer.video.presentation.Canvas.writeNative(Unknown Source)
	at com.flashphoner.media.N.A.A(Unknown Source)
	at com.flashphoner.media.N.D.D(Unknown Source)
	at com.flashphoner.media.N.D.A(Unknown Source)
	at com.flashphoner.server.remote.B.B.Г™(Unknown Source)
	at com.flashphoner.server.remote.B.B.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
	at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:84)
	... 14 more
Caused by: java.lang.NullPointerException
	at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1262)
	at java.desktop/sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:225)
	at java.desktop/sun.awt.FontConfiguration.init(FontConfiguration.java:107)
	at java.desktop/sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:719)
	at java.desktop/sun.font.SunFontManager$2.run(SunFontManager.java:367)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:310)
	at java.desktop/sun.font.SunFontManager.<init>(SunFontManager.java:312)
	at java.desktop/sun.awt.FcFontManager.<init>(FcFontManager.java:35)
	at java.desktop/sun.awt.X11FontManager.<init>(X11FontManager.java:56)
	... 20 more

Solution: install fontconfig library

Code Block

language	bash
theme	RDark

sudo yum install -y fontconfig

3. A participant name can be drawn over speech indicator frame if there are many participants in mixer

Symptoms: for small participant pictures (in desktop layout, for example) participants name can be drawn over speech indicator frame

Solution: enable inner speech indicator frame position and decrease frame thickness

Code Block

theme	RDark

mixer_voice_activity_frame_position_inner=true
mixer_voice_activity_frame_thickness=2

4. B-frames should be excluded from mixer input stream, or mixer input buffer should be increased

Symptoms: when mixer input stream is encoded by Main profile with B-frames, the mixer output stream is plaing unsmoothly, there are short freezes with sound pauses

Solution:

a) encode stream without B-frames (preferrable)

b) increase mixer input buffer (may give an additional delay)

Code Block

theme	RDark

mixer_in_buffering_ms=600

5. 60 FPS stream publishing to mixer gives additional CPU and system memory load

Symptoms: CPU load increasing and system memory (not Java heap) consumption growing while publishing 60 FPS stream to a mixer

Solution: publish 30 FPS streams to mixer or use more powerful server

6. Captions may twitch for many participants in MCU mixer

Symptoms: for many (over 10) participants in MCU mixer captions may periodically twitch

Solution: apply the following settings

Code Block

theme	RDark

mixer_text_bulk_write=false
mixer_text_bulk_write_with_buffer=false

7. When a number of participants are speaking simultaneously, some participants are less audible then others. The problem is typical for mixing WebRTC streams, and may ocuur in any solutions using WebRTC and audio mixing, in Discord for example: more data are encoded to one stream, more audio samples may be dropped due to fixed samplerate.

Symptoms: when three or more participants are speaking simultaneously, one of them is less audible then two others

Solution: raise the audio publishing bitrate on client to send more data per one participant

Code Block

language	js
theme	RDark

constraints: {
  audio: {
     bitrate: 128000
  }
}

and raise Opus encoding bitrate on server

Code Block

theme	RDark

opus.encoder.bitrate=128000

8. An incoming stream will be never encoded in realtime mixer if media traffic is stopped in the stream

Symptoms: an incoming stream freezes in the mixer when media traffic is stopped in the stream, for example, an application window captured by screen sharing is minimized to task bar

Solution: decrease a minimal FPS threshold for mixer incoming streams

Code Block

theme	RDark

mixer_video_stable_fps_threshold=0

9. Mixer output stream bitrate may be unstable when incoming stream bitrate or fps drops

Symptoms: mixer output stream bitrate becomes unstable when incoming stream bitrate or fps drops

Solution: update WCS to build 5.2.1843 or newer and set the parameter

Code Block

theme	RDark

h264_encoder_filler_data_padding=true

Space shortcuts

Page tree

Versions Compared

Old Version 13

New Version 33

Key

Caption text autoscaling

Speech indicator management

Mixer background management

Audio only streams picture aspect ratio management

Real-time mixer tuning

Testing

Call flow

Incoming streams tuning recommendations

Adding one stream to two or more realtime mixers simultaneously

Known issues

Custom stream label displaying

Setting or changing stream label using REST API

Caption location

Obsoleted settings in builds from 5.2.844 to 5.2.1077

Actual settings since build 5.2.1079

Speech indicator management

Thickness and position

Audio only streams displaying

Avatar pictures displaying for audio only streams

Frame colour

Frame displaying when silence is in a stream

Mixer background management

Audio only streams picture aspect ratio management

Multithreading support and hign load optimizations

Real-time mixer tuning

Testing

Call flow

Incoming streams tuning recommendations

Adding one stream to two or more realtime mixers simultaneously

Known issues

Space shortcuts

Page tree

Page History

Versions Compared

Old Version 13

New Version 33

Key

Caption text autoscaling

Speech indicator management

Mixer background management

Audio only streams picture aspect ratio management

Real-time mixer tuning

Testing

Call flow

Incoming streams tuning recommendations

Adding one stream to two or more realtime mixers simultaneously

Known issues

Custom stream label displaying

Setting or changing stream label using REST API

Caption location

Obsoleted settings in builds from 5.2.844 to 5.2.1077

Actual settings since build 5.2.1079

Speech indicator management

Thickness and position

Audio only streams displaying

Avatar pictures displaying for audio only streams

Frame colour

Frame displaying when silence is in a stream

Mixer background management

Audio only streams picture aspect ratio management

Multithreading support and hign load optimizations

Real-time mixer tuning

Testing

Call flow

Incoming streams tuning recommendations

Adding one stream to two or more realtime mixers simultaneously

Known issues