Overview
WCS allows mixing streams of active broadcasts. The output stream of the mixer can be recorded, played or republished using any of technologies supported by WCS.
Mixing is controlled using settings and REST API.
Supported protocols of input streams
- WebRTC
- RTMP
- RTSP
Output stream control capabilities
The mixer allows custom placing of video streams in the output frame. The stream with a certain name (by default desktop) is seen as screensharing and hence is placed in the center of the frame:
Automatically create a mixer when publishing the stream
If the name of the published RTMP stream has the '#' symbol, the server treats everything after that symbols as the name of the mixer that will be created when the stream is published. For instance, for the user1#room1 stream, the room1 mixer is created, and the stream is added to this mixer then. The stream name can also include the screen sharing keyword, for example, user1#room1#desktop
Operation flowchart
- The browser connects to the server via the Websocket protocol and sends the publish command.
- The browser sends the WebRTC stream1 to the server.
- Flash Player establishes a connection via RTMP and sends the publish command.
- Flash Player sends the RTMP stream2 to the server.
- The REST client creates a mixer with the output stream3 using the query: /mixer/startup
- The REST client adds stream1 to the mixer
- The REST client adds stream2 to the mixer
- The second browser establishes a connection via Websocket and sends the play command.
- The second browser receives the WebRTC audio stream stream3 and plays that stream on the page.
REST queries
A REST-query must be an HTTP/HTTPS POST query in the following form:
- HTTP: http://streaming.flashphoner.com:9091/rest-api/mixer/startup
- HTTPS: https://streaming.flashphoner.com:8888/rest-api/mixer/startup
Here:
- streaming.flashphoner.com - is the address of the WCS server
- 9091 - the standard REST / HTTP port of the WCS server
- 8888 - the standard HTTPS port
- rest-api - the required prefix
- mixer/startup - the REST-method used
REST-methods and response statuses
REST-method | Example of REST query | Example of response | Response statuses | Description |
---|---|---|---|---|
/mixer/startup | { "uri": "mixer://mixer1", "localStreamName": "stream3" } | 200 - OK 409 - Conflict 500 - Internal error | Creates a mixer the provided stream is published for | |
/mixer/add | { "uri": "mixer://mixer1", "remoteStreamName": "stream1" } | 200 - OK 404 - Mixer not found 404 - Stream not found 500 - Internal error | Add the RTMP stream to the mixer | |
/mixer/remove | { "uri": "mixer://mixer1", "remoteStreamName": "stream1" } | 200 - OK 404 - Mixer not found 404 - Stream not found 500 - Internal error | Remove the RTMP stream from the mixer | |
/mixer/find_all | { "localMediaSessionId": "ce92b134-2468-4460-8d06-1ea3c5aabace", "remoteMediaSessionId": null, "localStreamName": "mixer1", "remoteStreamName": null, "uri": "mixer://mixer1", "status": "PROCESSED_LOCAL", "mediaSessions": [ "95bf2be8-f459-4f62-9a7f-c588f33e0ad3", "693781de-cada-4589-abe1-c3ee55c66901" ], } | 200 - OK 404 - Not found 500 - Internal error | Find all mixers | |
/mixer/terminate | { "uri": "mixer://mixer1" } | 200 - OK 404 - Not found 500 - Internal error | Terminate operation of the mixer | |
/stream/startRecording | { "mediaSessionId": "23d07fa1-3c74-4d6f-a0de-9b4bf83ce665" } | 200 - OK 404 - Not found 500 - Internal error | Start recording of the stream in the given media session | |
/stream/stopRecording | { "mediaSessionId": "23d07fa1-3c74-4d6f-a0de-9b4bf83ce665" } | 200 - OK 404 - Not found 500 - Internal error | Stop recording the stream in the given media session |
Parameters
Parameter name | Description | Example |
---|---|---|
uri | Unique identifier of the mixer | mixer://mixer1 |
localStreamName | Name of the output stream of the mixer | stream3 |
remoteStreamName | Name of the stream added to the mixer | stream1 rtmp://rtmp.flashphoner.com:1935/live/rtmp_stream1 |
mediaSessionId | Media session identifier | ce92b134-2468-4460-8d06-1ea3c5aabace |
status | Stream status | PROCESSED_LOCAL |
Sending the REST query to the WCS server
To send the REST query to the WCS server use a REST-client.
Configuration
Mixing can be configured using the following parameters in the flashphoner.properties settings file
Parameter | Default value | Description |
---|---|---|
mixer_video_desktop_layout_inline_padding | 10 | Distance (padding) between windows of video streams in the lower line (below the screen sharing window) |
mixer_video_desktop_layout_padding | 30 | Distance (padding) between the screen sharing window and the lower line (the rest streams) |
mixer_video_enabled | true | Enables (by default) or disables video mixing |
mixer_video_grid_layout_middle_padding | 10 | Distance between windows of video streams in one line (without screen sharing window) |
mixer_video_grid_layout_padding | 30 | Distance between lines of windows (without screen sharing window) |
mixer_video_height | 720 | The image height of the mixer output stream |
mixer_video_layout_desktop_key_word | desktop | Keyword for the screen sharing stream |
mixer_video_width | 1280 | The image width of the mixer output stream |
record_mixer_streams | false | Turns on or off (default) recording of all mixer output streams |
Automatic creation of mixers for streams with the '#' symbol in their name requires the application that handles input streams to register the handler: 'com.flashphoner.server.client.handler.wcs4.FlashRoomRecordingStreamingHandler'. Registering the handler can be done using the command line interface. For instance, for the flashStreamingApp application used to publish incoming RTMP streams this can be done with the following command:
update app -m com.flashphoner.server.client.handler.wcs4.FlashRoomRecordingStreamingHandler -c com.flashphoner.server.client.handler.wcs4.FlashStreamingCallbackHandler flashStreamingApp
You can read more about managing applications using the command line of the WCS server here.
Quick manual on testing
1. For this test we use:
- the demo server at demo.flashphoner.com;
- the Chrome browser and the REST-client to send queries to the server;
- the Two Way Streaming web application to publish input streams of the mixer;
- the Player web application to play the output stream of the mixer.
2. Open the page of the Two Way Streaming application. Publish the stream named stream1:
3. In another tab open the page of the Two Way Streaming application. Publish the stream named desktop:
4. Open the REST client. Send the /mixer/startup query and specify the URI of the mixer mixer://mixer1 and the output stream name stream3 in its parameters:
5. Send the /mixer/add query and specify the URI of the mixer mixer://mixer1 and the input stream name stream1 in its parameters:
6. Open the Player web application, specify the name of the output stream of the mixer stream3 in the Stream field and click Start:
7. Send /mixer/add and specify the URI of the mixer mixer://mixer1 and the input stream name desktop in its parameters:
8. In the output stream of the mixer you should see the desktop stream that imitates screen sharing and the stream stream1:
Call flow
Below is the call flow when using the mixer.
1. Publishing of the WebRTC stream stream1
2. Sending the WebRTC stream to the server
3. Publishing the RTMP stream stream2
4. Sending the RTMP stream to the server
5. Sending the /mixer/startup query to create the mixer://stream3 mixer with the output stream3
http://demo.flashphoner.com:9091/rest-api/mixer/startup { "uri": "mixer://stream3", "localStreamName": "stream3" }
6. Sending the /mixer/add query to add stream1 to the mixer://stream3 mixer
http://demo.flashphoner.com:9091/rest-api/mixer/add { "uri": "mixer://stream3", "localStreamName": "stream3" "remoteStreamName": "stream1" }
7. Sending the /mixer/add query to add stream2 to the mixer://stream3 mixer
http://demo.flashphoner.com:9091/rest-api/mixer/add { "uri": "mixer://stream3", "localStreamName": "stream3" "remoteStreamName": "stream2" }
8. Playing the WebRTC stream stream3
9. Sending the WebRTC audio stream to the client
Known issues
1) A mixer is not created is the name of the mixer contains symbols restricted for URI.
Symptoms: a mixer with the name like test_mixer does not create.
Solution: do not use disallowed symbols in the name of a mixer or a stream, especially if automatic mixer creation option is enabled. For instance, the name
user_1#my_room
cannot be used.
If streams of chat rooms are mixed, room names also cannot use restricted symbols.