...
- HTTP: http://streaming.flashphoner.com:90918081/rest-api/mixer/startup
- HTTPS: https://streaming.flashphoner.com:88888444/rest-api/mixer/startup
Here:
- streaming.flashphoner.com - is the address of the WCS server
- 9091 8081 - the standard REST / HTTP port of the WCS server
- 88888444 - the standard HTTPS port
- rest-api - the required prefix
- mixer/startup - the REST-method used
...
REST-method | Example of REST query | Example of response | Response statuses | Description | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
/mixer/startup |
| 200 - OK 409 - Conflict 500 - Internal error | Creates a mixer the provided stream is published for | ||||||||
/mixer/add |
| 200 - OK 404 - Mixer not found 404 - Stream not found 500 - Internal error | Add the RTMP stream to the mixer | ||||||||
/mixer/remove |
| 200 - OK 404 - Mixer not found 404 - Stream not found 500 - Internal error | Remove the RTMP stream from the mixer | ||||||||
/mixer/find_all |
| 200 - OK 404 - Not found 500 - Internal error | Find all mixers | ||||||||
/mixer/terminate |
| 200 - OK 404 - Not found 500 - Internal error | Terminate operation of the mixer | ||||||||
/stream/startRecording |
| 200 - OK 404 - Not found 500 - Internal error | Start recording of the stream in the given media session | ||||||||
/stream/stopRecording |
| 200 - OK 404 - Not found 500 - Internal error | Stop recording the stream in the given media session |
...
Parameter name | Description | Example |
---|---|---|
uri | Unique identifier of the mixer | mixer://mixer1 |
localStreamName | Name of the output stream of the mixer | stream3 |
hasVideo | Mix video (true) or audio only (false) | false |
remoteStreamName | Name of the stream added to the mixer | stream1 rtmp://rtmp.flashphoner.com:1935/live/rtmp_stream1 |
mediaSessionId | Media session identifier | ce92b134-2468-4460-8d06-1ea3c5aabace |
status | Stream status | PROCESSED_LOCAL |
...
Parameter | Default value | Description |
---|---|---|
mixer_video_desktop_layout_inline_padding | 10 | Distance (padding) between windows of video streams in the lower line (below the screen sharing window) |
mixer_video_desktop_layout_padding | 30 | Distance (padding) between the screen sharing window and the lower line (the rest streams) |
mixer_video_enabled | true | Enables (by default) or disables video mixing |
mixer_video_grid_layout_middle_padding | 10 | Distance between windows of video streams in one line (without screen sharing window) |
mixer_video_grid_layout_padding | 30 | Distance between lines of windows (without screen sharing window) |
mixer_video_height | 720 | The image height of the mixer output stream |
mixer_video_layout_desktop_key_word | desktop | Keyword for the screen sharing stream |
mixer_video_width | 1280 | The image width of the mixer output stream |
record_mixer_streams | false | Turns on or off (default) recording of all mixer output streams |
Automatic mixer creation configuration
Automatic creation of mixers for streams with the '#' symbol in their name requires the application that handles input streams to register the handler: 'com.flashphoner.server.client.handler.wcs4.FlashRoomRecordingStreamingHandler'. Registering the handler can be done using the command line interface. For instance, for the flashStreamingApp application used to publish incoming RTMP streams this can be done with the following command:
...
You can read more about managing applications using the command line of the WCS server here.
Audio and video mixing configuration
By default, both video and audio streams are mixed. If audio only mixing is necessary, it should be set on mixer creation
Code Block | ||||
---|---|---|---|---|
| ||||
{
"uri": "mixer://mixer1",
"localStreamName": "stream3",
"hasVideo": "false"
} |
To switch off video mixing for all streams, this parameter should be set in flashphoner.properties file
Code Block | ||||
---|---|---|---|---|
| ||||
mixer_video_enabled=false |
In this case video mixing can be switched on for certain mixer on its creation.
Mixer output stream bufferization
In some cases, mixer output stream bufferization is needed. This feature is enabled with the following parameter in flashphoner.properties file
Code Block | ||||
---|---|---|---|---|
| ||||
mixer_out_buffer_enabled=true |
The buffer size is defined in milliseconds with parameter
Code Block | ||||
---|---|---|---|---|
| ||||
mixer_out_buffer_start_size=400 |
In this case, the buffer size is 400 ms.
Stream data fetching from buffer and sending period is defined in milliseconds with parameter
Code Block | ||||
---|---|---|---|---|
| ||||
mixer_out_buffer_polling_time=20 |
In this case, the period is 20 ms.
Changing bitrate of mixer output stream
When OpenH264 codec is used for transcoding, it is possible to change bitrate of mixer output stream with the following parameter in flashphoner.properties file
Code Block | ||
---|---|---|
| ||
mixer_video_bitrate_kbps=2000 |
By default, mixer output stream bitrate is set to 2 Mbps. If a channel bandwith between server and viewer is not enough, bitrate can be reduced, for example
Code Block | ||
---|---|---|
| ||
encoder_priority=OPENH264
mixer_video_bitrate_kbps=1500 |
If picture quality with default bitrate is low, or distortion occurs, it is recommended to rise mixer outout stream bitrate to 3-5 Mbps
Code Block | ||
---|---|---|
| ||
encoder_priority=OPENH264
mixer_video_bitrate_kbps=5000 |
Mixer output stream sound management
By default, mixer output stream sound is encoded to Opus with sample rate 48 kHz. These settings may be changed using the parameters in flashphoner.properties file. For example, to use mixer output stream in SIP call the following value can be set:
Code Block | ||
---|---|---|
| ||
audio_mixer_output_codec=pcma
audio_mixer_output_sample_rate=8000 |
In this case, sound will be encoded to PCMA (alaw) with sample rate 8 kHz.
Using custom lossless videoprocessor for incoming streams handling
To handle mixer incoming streams, if additional bufferizing or audio and video tracks syncronizing is required for example, the custom lossless videoprocessor may be used. This feature is enabled with the following parameter in flashphoner.properties file
Code Block | ||
---|---|---|
| ||
mixer_lossless_video_processor_enabled=true |
The maximum size of mixer buffer in milliseconds is set with this parameter
Code Block | ||
---|---|---|
| ||
mixer_lossless_video_processor_max_mixer_buffer_size_ms=200 |
By default, maximum mixer buffer size is 200 ms. After filling this buffer, the custom lossless videoprocessor uses its own buffer and waits for mixer buffer freeing. The period of mixer buffer checking is set in milliseconds with this parameter
Code Block | ||
---|---|---|
| ||
mixer_lossless_video_processor_wait_time_ms=20 |
By default, the mixer buffer checking period is 20 ms.
Note that using the custom lossless videoprocessor may degrade realtime perfomance.
When custom lossless videoprocessor is used, it is necessary to stop mixer with REST query /mixer/terminate to free all consumed resources. Mixer can be stopped also by stopping all incoming streams, in this case mixer will stop when following timeout in milliseconds expires
Code Block | ||
---|---|---|
| ||
mixer_idle_timeout=60000 |
By default, mixer will stop after 60 seconds if there are no active incoming streams.
Mixer output stream layout management
By default, three mixer output stream layouts are implemented:
1. Grid layout
This layout can be enabled with the following parameter in flashphoner.properties file
Code Block | ||
---|---|---|
| ||
mixer_layout_class=com.flashphoner.media.mixer.video.presentation.GridLayout |
2. Zero padding grid layout
This layout can be enabled with the following parameter
Code Block | ||
---|---|---|
| ||
mixer_layout_class=com.flashphoner.media.mixer.video.presentation.CenterNoPaddingGridLayout |
and works for input streams of equal resolution with the same aspect ratio only
3. Desktop (screen sharing) layout
This layout is enabled if one of mixer input streams has a name defined in the following parameter
Code Block | ||
---|---|---|
| ||
mixer_video_layout_desktop_key_word=desktop |
By default, desktop
name is used for screen sharing stream.
Custom mixer layout implementation
For more fine tuning of mixer layout, custom Java class should be developed to implement IVideoMixerLayout
interface, for example
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
package com.flashphoner.mixerlayout;
import com.flashphoner.sdk.media.IVideoMixerLayout;
import com.flashphoner.sdk.media.YUVFrame;
import java.awt.*;
import java.util.ArrayList;
public class TestLayout implements IVideoMixerLayout {
private static final int PADDING = 5;
@Override
public Layout[] computeLayout(YUVFrame[] yuvFrames, String[] strings, int canvasWidth, int canvasHeight) {
ArrayList<IVideoMixerLayout.Layout> layout = new ArrayList<>();
for (int c = 0; c < yuvFrames.length; c++) {
Point prevPoint = new Point();
Dimension prevDimension = new Dimension(canvasWidth, canvasHeight);
if (layout.size() > 0) {
prevPoint.setLocation(layout.get(c-1).getPoint());
prevDimension.setSize(layout.get(c-1).getDimension());
}
Point currentPoint = new Point((int) (prevPoint.getX()+prevDimension.getWidth()+PADDING),
(int)(prevPoint.getY()+prevDimension.getHeight()));
layout.add(new IVideoMixerLayout.Layout(currentPoint, new Dimension(canvasWidth/yuvFrames.length,
canvasHeight/yuvFrames.length), yuvFrames[c]));
}
return layout.toArray(new IVideoMixerLayout.Layout[layout.size()]);
}
} |
Then the class should be complied into byte code. To do this, create folder tree according to TestLayout class package name
Code Block | ||||
---|---|---|---|---|
| ||||
mkdir -p com/flashphoner/mixerlayout |
and execute the command
Code Block | ||||
---|---|---|---|---|
| ||||
javac -cp /usr/local/FlashphonerWebCallServer/lib/tbs-flashphoner.jar ./com/flashphoner/mixerlayout/TestLayout.java |
Now, pack the code compiled to jar file
Code Block | ||||
---|---|---|---|---|
| ||||
jar -cf testlayout.jar ./com/flashphoner/mixerlayout/TestLayout.class |
and copy this file to WCS libraries folder
Code Block | ||||
---|---|---|---|---|
| ||||
cp testlayout.jar /usr/local/FlashphonerWebCallServer/lib |
To use custom mixer layout class, set it to the following parameter in flashphoner.properties file
Code Block | ||
---|---|---|
| ||
mixer_layout_class=com.flashphoner.mixerlayout.TestLayout |
and restart WCS.
With this custom layout, mixer output stream for three input streams will look like:
Quick manual on testing
1. For this test we use:
...
9. Sending the WebRTC audio stream to the client
Known issues
1) . A mixer is not created is the name of the mixer contains symbols restricted for URI.
Symptoms: a mixer with the name like test_mixer does not create.
Solution: do not use disallowed symbols in the name of a mixer or a stream, especially if automatic mixer creation option is enabled. For instance, the name
...
cannot be used.
If streams of chat rooms are mixed, room names also cannot use restricted symbols.
2. Mixer output stream will be empty if transcoding is enabled on server on demand only.
Symptoms: video streams mixer created successfully, but black screen is played in mixer output stream.
Solution: for stream mixer to work transcoding should be enabled on server with the following parameter in flashphoner.properties file
Code Block | ||||
---|---|---|---|---|
| ||||
streaming_video_decoder_fast_start=true |