Page History

...

test1#m1?label=John+Doe

Caption location

Obsoleted settings in builds from 5.2.844 to 5.2.1077

...

Setting or changing stream label using REST API

Since build 5.2.

...

Since build 5.2.844, it is possible to change location of video participant's caption. By default, caption is located in left bottom corner of video.

It is possible to display caption above video using the following parameter

mixer_text_outside_frame=TOP

Image Removed

or under video

mixer_text_outside_frame=BOTTOM

Image Removed

The distance between video and its caption can be changed using the following parameter

mixer_text_outside_frame_padding=70

Captions placement above or below a picture affects picture placement even if captions diplaying is disabled.

Actual settings since build 5.2.1079

Since build 5.2.1079 the following parameter was added to set caption text alignment realtive to frame picture1635 it is possible to set a custom stream label while adding a stream to mixer with /mixer/add REST API query

{
 "uri": "mixer://mixer1",
 "remoteStreamName": "stream1",
 "streamLabel": "John Doe"
}

The label may be changed for the stream already added to the mixer by /mixer/set_stream_label query

{
 "uri":"mixer://m1",
 "remoteStreamName":"stream1",
 "streamLabel": "Mr. John Doe"
}

or

{
 "uri":"mixer://m1",
 "remoteMediaSessionId":"95bf2be8-f459-4f62-9a7f-c588f33e0ad3",
 "streamLabel": "Mr. John Doe"
}

The label may be cleared by passing an empty string as streamLabel 

{
 "uri":"mixer://m1",
 "remoteMediaSessionId":"95bf2be8-f459-4f62-9a7f-c588f33e0ad3",
 "streamLabel": ""
}

In this case the stream name will be displayed as published.

Caption location

Obsoleted settings in builds from 5.2.844 to 5.2.1077

Since build 5.2.844, it is possible to change location of video participant's caption. By default, caption is located in left bottom corner of video.

It is possible to display caption above video using the following parameter

mixer_text_outside_frame=TOP

Image Added

or under video

mixer_text_outside_frame=BOTTOM

Image Added

The distance between video and its caption can be changed using the following parameter

mixer_text_outside_frame_padding=70

Captions placement above or below a picture affects picture placement even if captions diplaying is disabled.

Actual settings since build 5.2.1079

Since build 5.2.1079 the following parameter was added to set caption text alignment realtive to frame picture

mixer_text_align=BOTTOM_LEFT

Possible values

By default, captions are placed in bottom left corner of a picture frame.

The values EXTERNAL_TOP_CENTER and EXTERNAL_BOTTOM_CENTER replace obsoleted mixer_text_outside_frame parameter. For example, the following value

mixer_text_align=EXTERNAL_BOTTOM_CENTER

will act like mixer_text_outside_frame=BOTTOM

Image Added

Speech indicator management

Thickness and position

Speech indicator frame thickness can be set (6 pixels by default)

mixer_voice_activity_frame_thickness=6

The frame is displayed outside astream picture by default. However, if stream pictures are too close, foe example, if CenterNoPaddingGridLayout is used, the frame can affect neighbor pictures

Image Added

In this case, inner speech frame displaying should be enabled

mixer_voice_activity_frame_position_inner=true

Image Added

Audio only streams displaying

By default, if audio only track from some stream was added to mixer, this stream is shown in separate frame with speech indicator (see above). Sound only can be added to mixer without showing a separate participant (for example, to comment or to dub) with the following parameter if necessary

mixer_show_separate_audio_frame=false

Before build 5.2.965, the stream for dubbing must contain audio and video, and must be added to mixer with the following /mixer/add REST query

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "stream_dub",
    "hasVideo":false,
    "hasAudio":true
}

Since build 5.2.965, audio only stream also will not be displayed if this parameter is used

Since build 5.2.1359, the parameter mixer_show_separate_audio_frame affects stream in mixer displaying when audio data are received before video. In this case, a frame with speech indicator will be displayed for such stream when default settings are used, then video will be displayed after first video data receiving. If the following parameter is set

mixer_show_textseparate_audio_align=BOTTOM_LEFT

Possible values

...

By default, captions are placed in bottom left corner of a picture frame.

The values EXTERNAL_TOP_CENTER and EXTERNAL_BOTTOM_CENTER replace obsoleted mixer_text_outside_frame parameter. For example, the following value

mixer_text_align=EXTERNAL_BOTTOM_CENTER

will act like mixer_text_outside_frame=BOTTOM

Image Removed

Speech indicator management

Thickness and position

Speech indicator frame thickness can be set (6 pixels by default)

mixer_voice_activity_frame_thickness=6

The frame is displayed outside astream picture by default. However, if stream pictures are too close, foe example, if CenterNoPaddingGridLayout is used, the frame can affect neighbor pictures

Image Removed

In this case, inner speech frame displaying should be enabled

mixer_voice_activity_frame_position_inner=true

Image Removed

Audio only streams displaying

By default, if audio only track from some stream was added to mixer, this stream is shown in separate frame with speech indicator (see above). Sound only can be added to mixer without showing a separate participant (for example, to comment or to dub) with the following parameter if necessary

mixer_show_separate_audio_frame=false

Before build 5.2.965, the stream for dubbing must contain audio and video, and must be added to mixer with the following /mixer/add REST query

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "stream_dub",
    "hasVideo":false,
    "hasAudio":true
}

...

frame=false

then speech indicator frame will not be displayed until video data arrive.

Avatar pictures displaying for audio only streams

Since build 5.2.1710 it is possible to set an avatar picture in PNG (with transparency), JPG or BMP formats to an audio only stream in mixer. A picture may be set while adding a stream to mixer by REST API query /mixer/add :

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "user2",
    "hasVideo": false,
    "hasAudio": true,
    "avatar": "https://mystorage.com/storage/avatar.png"
}

or by REST API query /mixer/set_stream_avatar for the audio only stream which already has added to the mixer:

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "user2",
    "avatar": "https://mystorage.com/storage/avatar.png"
}

The following ways to set a picture are supported:

• HTTP URL: https://mystorage.com/storage/avatar.png 
• file URI: file:///opt/avatar.png 
• local file: /opt/avatar.png 

Audio only stream in mixer is displaying as follows

Image Added

If stream names displaying in mixer is enabled, audio only stream name will be displayed like video stream one.

Avatar picture may be removed by REST API query /mixer/remove_stream_avatar :

{
    "uri": "mixer://mixer1",
    "remoteStreamName": "user2"
}

Since WCS build 5.2.1858 it is possible to set avatar pictures automatically by stream name, not using REST API. In this case avatar picture files should be placed to the folder

avatar_dir=/usr/local/FlashphonerWebCallServer/avatar

A file name should be equal to stream name. For example, the picture file stream1.png will be applied to the stream named stream1. If the stream was added to the mixer automatically, e.g. stream1#mixer1, the mixer name will not be used to get the picture, only the stream name before # character.

Frame colour

Since build 5.2.741 speech indicator frame colour can be changed with the following parameter

...

09:17:11,756 ERROR           MixerAgent - MIXER-AGENT-mixer://mixervmixdr52-9d46cd04-5867-4d74-a9d9-baf67f74e7d2 Mixer closed due to error
java.lang.InternalError: java.lang.reflect.InvocationTargetException
	at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:86)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:310)
	at java.desktop/sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)
	at java.desktop/sun.font.SunFontManager.getInstance(SunFontManager.java:247)
	at java.desktop/sun.font.FontDesignMetrics.getMetrics(FontDesignMetrics.java:265)
	at java.desktop/java.awt.Font.getStringBounds(Font.java:2606)
	at java.desktop/java.awt.Font.getStringBounds(Font.java:2516)
	at com.flashphoner.media.N.A.A.A(Unknown Source)
	at com.flashphoner.media.mixer.video.presentation.Canvas.computeTextScales(Unknown Source)
	at com.flashphoner.media.mixer.video.presentation.Canvas.writeNative(Unknown Source)
	at com.flashphoner.media.N.A.A(Unknown Source)
	at com.flashphoner.media.N.D.D(Unknown Source)
	at com.flashphoner.media.N.D.A(Unknown Source)
	at com.flashphoner.server.remote.B.B.Г™(Unknown Source)
	at com.flashphoner.server.remote.B.B.run(Unknown Source)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
	at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:84)
	... 14 more
Caused by: java.lang.NullPointerException
	at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1262)
	at java.desktop/sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:225)
	at java.desktop/sun.awt.FontConfiguration.init(FontConfiguration.java:107)
	at java.desktop/sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:719)
	at java.desktop/sun.font.SunFontManager$2.run(SunFontManager.java:367)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:310)
	at java.desktop/sun.font.SunFontManager.<init>(SunFontManager.java:312)
	at java.desktop/sun.awt.FcFontManager.<init>(FcFontManager.java:35)
	at java.desktop/sun.awt.X11FontManager.<init>(X11FontManager.java:56)
	... 20 more

Solution: install fontconfig library

sudo yum install -y fontconfig

3. A participant name can be drawn over speech indicator frame if there are many participants in mixer

Symptoms: for small participant pictures (in desktop layout, for example) participants name can be drawn over speech indicator frame

Solution: enable inner speech indicator frame position and decrease frame thickness

mixer_voice_activity_frame_position_inner=true
mixer_voice_activity_frame_thickness=2

4. B-frames should be excluded from mixer input stream, or mixer input buffer should be increased

Symptoms: when mixer input stream is encoded by Main profile with B-frames, the mixer output stream is plaing unsmoothly, there are short freezes with sound pauses

Solution:

a) encode stream without B-frames (preferrable)

b) increase mixer input buffer (may give an additional delay)

mixer_in_buffering_ms=600

5. 60 FPS stream publishing to mixer gives additional CPU and system memory load

Symptoms: CPU load increasing and system memory (not Java heap) consumption growing while publishing 60 FPS stream to a mixer

Solution: publish 30 FPS streams to mixer or use more powerful server

6. Captions may twitch for many participants in MCU mixer

Symptoms: for many (over 10) participants in MCU mixer captions may periodically twitch

Solution: apply the following settings

mixer_text_bulk_write=false
mixer_text_bulk_write_with_buffer=false

7. When a number of participants are speaking simultaneously, some participants are less audible then others. The problem is typical for mixing WebRTC streams, and may ocuur in any solutions using WebRTC and audio mixing, in Discord for example: more data are encoded to one stream, more audio samples may be dropped due to fixed samplerate.

Symptoms: when three or more participants are speaking simultaneously, one of them is less audible then two others

Solution:  raise the audio publishing bitrate on client to send more data per one participant

constraints: {
  audio: {
     bitrate: 128000
  }
}

and raise Opus encoding bitrate on server

opus.encoder.bitrate=128000.awt.X11FontManager.<init>(X11FontManager.java:56)
	... 20 more

Solution: install fontconfig library

sudo yum install -y fontconfig

3. A participant name can be drawn over speech indicator frame if there are many participants in mixer

Symptoms: for small participant pictures (in desktop layout, for example) participants name can be drawn over speech indicator frame

Solution: enable inner speech indicator frame position and decrease frame thickness

mixer_voice_activity_frame_position_inner=true
mixer_voice_activity_frame_thickness=2

4. B-frames should be excluded from mixer input stream, or mixer input buffer should be increased

Symptoms: when mixer input stream is encoded by Main profile with B-frames, the mixer output stream is plaing unsmoothly, there are short freezes with sound pauses

Solution:

a) encode stream without B-frames (preferrable)

b) increase mixer input buffer (may give an additional delay)

mixer_in_buffering_ms=600

5. 60 FPS stream publishing to mixer gives additional CPU and system memory load

Symptoms: CPU load increasing and system memory (not Java heap) consumption growing while publishing 60 FPS stream to a mixer

Solution: publish 30 FPS streams to mixer or use more powerful server

6. Captions may twitch for many participants in MCU mixer

Symptoms: for many (over 10) participants in MCU mixer captions may periodically twitch

Solution: apply the following settings

mixer_text_bulk_write=false
mixer_text_bulk_write_with_buffer=false

7. When a number of participants are speaking simultaneously, some participants are less audible then others. The problem is typical for mixing WebRTC streams, and may ocuur in any solutions using WebRTC and audio mixing, in Discord for example: more data are encoded to one stream, more audio samples may be dropped due to fixed samplerate.

Symptoms: when three or more participants are speaking simultaneously, one of them is less audible then two others

Solution:  raise the audio publishing bitrate on client to send more data per one participant

constraints: {
  audio: {
     bitrate: 128000
  }
}

and raise Opus encoding bitrate on server

opus.encoder.bitrate=128000

8. An incoming stream will be never encoded in realtime mixer if media traffic is stopped in the stream

Symptoms: an incoming stream freezes in the mixer when media traffic is stopped in the stream, for example, an application window captured by screen sharing is minimized to task bar

Solution: decrease a minimal FPS threshold for mixer incoming streams

mixer_video_stable_fps_threshold=0

9. Mixer output stream bitrate may be unstable when incoming stream bitrate or fps drops

Symptoms: mixer output stream bitrate becomes unstable when incoming stream bitrate or fps drops

Solution: update WCS to build 5.2.1843 or newer and set the parameter

h264_encoder_filler_data_padding=true