Overview

WCS allows mixing streams of active broadcasts. The output stream of the mixer can be recorded, played or republished using any of technologies supported by WCS.
Mixing is controlled using settings and REST API.

Supported protocols of input streams

• WebRTC
• RTMP
• RTSP

Output stream control capabilities

The mixer allows custom placing of video streams in the output frame. The stream with a certain name (by default desktop) is seen as screensharing and hence is placed in the center of the frame:

Automatically create a mixer when publishing the stream

If the name of the published RTMP stream has the '#' symbol, the server treats everything after that symbols as the name of the mixer that will be created when the stream is published. For instance, for the user1#room1 stream, the room1 mixer is created, and the stream is added to this mixer then. The stream name can also include the screen sharing keyword, for example, user1#room1#desktop

Operation flowchart

1. The browser connects to the server via the Websocket protocol and sends the publish command.
2. The browser sends the WebRTC stream1 to the server.
3. Flash Player establishes a connection via RTMP and sends the publish command.
4. Flash Player sends the RTMP stream2 to the server.
5. The REST client creates a mixer with the output stream3 using the query: /mixer/startup
6. The REST client adds stream1 to the mixer
7. The REST client adds stream2 to the mixer
8. The second browser establishes a connection via Websocket and sends the play command.
9. The second browser receives the WebRTC audio stream stream3 and plays that stream on the page.

REST queries

A REST-query must be an HTTP/HTTPS POST query in the following form:

• HTTP: http://streaming.flashphoner.com:9091/rest-api/mixer/startup
• HTTPS: https://streaming.flashphoner.com:8888/rest-api/mixer/startup

Here:

• streaming.flashphoner.com - is the address of the WCS server
• 9091 - the standard REST / HTTP port of the WCS server
• 8888 - the standard HTTPS port
• rest-api - the required prefix
• mixer/startup - the REST-method used

REST-methods and response statuses

{
  "uri": "mixer://mixer1",
  "localStreamName": "stream3",
  "hasVideo": "false"
}

{
  "uri": "mixer://mixer1",
  "remoteStreamName": "stream1"
}

{
  "uri": "mixer://mixer1",
  "remoteStreamName": "stream1"
}

{
"localMediaSessionId": "ce92b134-2468-4460-8d06-1ea3c5aabace",
"remoteMediaSessionId": null,
"localStreamName": "mixer1",
"remoteStreamName": null,
"uri": "mixer://mixer1",
"status": "PROCESSED_LOCAL",
"mediaSessions": [
  "95bf2be8-f459-4f62-9a7f-c588f33e0ad3",
  "693781de-cada-4589-abe1-c3ee55c66901"
],
}

{
  "uri": "mixer://mixer1"
}

{
  "mediaSessionId": "23d07fa1-3c74-4d6f-a0de-9b4bf83ce665"
}

{
  "mediaSessionId": "23d07fa1-3c74-4d6f-a0de-9b4bf83ce665"
}

Parameters

Sending the REST query to the WCS server

To send the REST query to the WCS server use a REST-client.

Configuration

Mixing can be configured using the following parameters in the flashphoner.properties settings file

Automatic creation of mixers for streams with the '#' symbol in their name requires the application that handles input streams to register the handler: 'com.flashphoner.server.client.handler.wcs4.FlashRoomRecordingStreamingHandler'. Registering the handler can be done using the command line interface. For instance, for the flashStreamingApp application used to publish incoming RTMP streams this can be done with the following command:

update app -m com.flashphoner.server.client.handler.wcs4.FlashRoomRecordingStreamingHandler -c com.flashphoner.server.client.handler.wcs4.FlashStreamingCallbackHandler flashStreamingApp

You can read more about managing applications using the command line of the WCS server here.

By default, both video and audio streams are mixed. If audio only mixing is necessary, it should be set on mixer creation

{
 "uri": "mixer://mixer1",
 "localStreamName": "stream3",
 "hasVideo": "false"
}

To switch off video mixing for all streams, this parameter should be set in flashphoner.properties file

mixer_video_enabled=false

In this case video mixing can be switched on for certain mixer on its creation.

Quick manual on testing

1. For this test we use:

• the demo server at demo.flashphoner.com;
• the Chrome browser and the REST-client to send queries to the server;
• the Two Way Streaming web application to publish input streams of the mixer;
• the Player web application to play the output stream of the mixer.

2. Open the page of the Two Way Streaming application. Publish the stream named stream1:

3. In another tab open the page of the Two Way Streaming application. Publish the stream named desktop:

4. Open the REST client. Send the /mixer/startup query and specify the URI of the mixer mixer://mixer1 and the output stream name stream3 in its parameters:

5. Send the /mixer/add query and specify the URI of the mixer mixer://mixer1 and the input stream name stream1 in its parameters:

6. Open the Player web application, specify the name of the output stream of the mixer stream3 in the Stream field and click Start:

7. Send /mixer/add and specify the URI of the mixer mixer://mixer1 and the input stream name desktop in its parameters:

8. In the output stream of the mixer you should see the desktop stream that imitates screen sharing and the stream stream1:

Call flow

Below is the call flow when using the mixer.

1. Publishing of the WebRTC stream stream1

2. Sending the WebRTC stream to the server

3. Publishing the RTMP stream stream2

4. Sending the RTMP stream to the server

5. Sending the /mixer/startup query to create the mixer://stream3 mixer with the output stream3

http://demo.flashphoner.com:9091/rest-api/mixer/startup
{
 "uri": "mixer://stream3",
 "localStreamName": "stream3"
}

6. Sending the /mixer/add query to add stream1 to the mixer://stream3 mixer

http://demo.flashphoner.com:9091/rest-api/mixer/add
{
 "uri": "mixer://stream3",
 "localStreamName": "stream3"
 "remoteStreamName": "stream1"
}

7. Sending the /mixer/add query to add stream2 to the mixer://stream3 mixer

http://demo.flashphoner.com:9091/rest-api/mixer/add
{
 "uri": "mixer://stream3",
 "localStreamName": "stream3"
 "remoteStreamName": "stream2"
}

8. Playing the WebRTC stream stream3

9. Sending the WebRTC audio stream to the client

Known issues

1) A mixer is not created is the name of the mixer contains symbols restricted for URI.

Symptoms: a mixer with the name like test_mixer does not create.
Solution: do not use disallowed symbols in the name of a mixer or a stream, especially if automatic mixer creation option is enabled. For instance, the name

user_1#my_room

cannot be used.
If streams of chat rooms are mixed, room names also cannot use restricted symbols.