...
Since build 5.2.1012 it is possible to record multiple streams to one file. Then streams can be extracted from this file and mixed by a special tool. Multiple streams can be recorded only to MP4 container (H264 + AAC)or, since build 5.2.1440, to MKV container. This feature is intended, for example, to record a video conference. In this case, unlike to to MCU mixer, stream mixing works while recording file is post-processed, this allows to aquire a less of server resources during the conference itself.
Multiple stream recording is managed by REST API.
Codecs support
MP4 container:
- H264
- AAC
MKV container:
- H264
- VP8
- Opus
- AAC
- PCMA
- PCMU
- G722
Multirecorder REST API
REST query must be HTTP/HTTPS POST query like this:
...
- streaming.flashphoner.com is WCS server address
- 8081 is a standard WCS REST / HTTP port
- 8444 is a standard WCS REST / HTTPS port
- rest-api is mandatory prefix
- /multipleRecorder/startup is REST method
REST methods and responses
REST query | Example of REST query | Example of REST response | Response states | Description | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
/multipleRecorder/startup |
| 409 - Conflict 500 - Internal error | Launch multiple streams recorder | ||||||||
/multipleRecorder/add |
| 404 - Not found 409 - Conflict 500 - Internal error | Add the stream with mediaSessionId to the recorder | ||||||||
/multipleRecorder/find_all |
| 404 - Not found 500 - Internal error | Find all recorders | ||||||||
/multipleRecorder/remove |
| 404 - Not found 500 - Internal error | Remove stream with mediaSessionId from recorder | ||||||||
/multipleRecorder/terminate |
| 404 - Not found 500 - Internal error | Stop multiple streams recorder |
...
Parameter name | Description | Example |
---|---|---|
uri | Recorder URI | multi-recorder://test-record |
mediaSessionId | Stream mediasession Id | 866a9910-fbfe-11eb-aae4-6f99b0c80a3a |
filename | Recording file name | multi-recorder___test-record.mp4 |
Recording file name
Multiple streams recording file name is formed by template. In this case:
1. {streamName} parameter is set according to recorder URI, replacing all the characters not allowed to use in file system to underline.
Container configuration
The feature is available since build 5.2.1440. By default, multiple recordings are stored to MP4 container
Code Block | ||
---|---|---|
| ||
multi_recorder_type=MP4 |
MKV container may be used if necessary (for example, if VP8+Opus streams are published in conference)
Code Block | ||
---|---|---|
| ||
multi_recorder_type=MKV |
Recording file name
Multiple streams recording file name is formed by template. In this case:
1. {streamName} parameter is set according to recorder URI, replacing all the characters not allowed to use in file system to underline.
2. {startTime}, {endTime} parameters cannot be set because they depend on stream timestamps, and we have a multiple streams with a different timestamps simultaneously. So it is recommended to use {startTimeMillis}, {endTimeMillis} parameters to add servers clock timestamps to file name.
...
A picture sample from the mixed file
Warning |
---|
Only MP4 recordings can be mixed |
Getting tracks information from multiple recording file
...
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
[ { "durationInMS": "78978", "trackEdits": [ { "endInMs": "63050", "type": "pause", "startInMs": "0" }, { "endInMs": "78978", "type": "media", "startInMs": "63050" } ], "channels": "2", "trackType": "AUDIO", "trackId": "3", "timescale": "44100", "streamName": "test", "trackCodec": "mp4a", "sampleRate": "44100", "mediaSessionId": "fbbf5b50-20ee-11ec-bf06-ef6ec6048b2c" }, { "durationInMS": "39708", "trackEdits": [ { "endInMs": "23150", "type": "media", "startInMs": "0" } ], "channels": "2", "trackType": "AUDIO", "trackId": "1", "timescale": "44100", "streamName": "test", "trackCodec": "mp4a", "sampleRate": "44100", "mediaSessionId": "c7bc1460-20ee-11ec-bf06-ef6ec6048b2c" }, { "durationInMS": "39791", "trackEdits": [ { "endInMs": "23233", "type": "media", "startInMs": "0" } ], "trackType": "VIDEO", "trackId": "0", "width": "640", "timescale": "90000", "streamName": "test", "trackCodec": "avc1", "mediaSessionId": "c7bc1460-20ee-11ec-bf06-ef6ec6048b2c", "height": "360" }, { "durationInMS": "63050", "trackEdits": [ { "endInMs": "39791", "type": "pause", "startInMs": "0" }, { "endInMs": "50191", "type": "media", "startInMs": "39791" } ], "trackType": "VIDEO", "trackId": "2", "width": "640", "timescale": "90000", "streamName": "test", "trackCodec": "avc1", "mediaSessionId": "ed3ebda0-20ee-11ec-bf06-ef6ec6048b2c", "height": "360" } ] |
Multiple recording file handling script
When multiple stream recording is finished, a specila handling script is automatically launched as set by the following parameter
...
Pulling a separate streams from MKV container
Since build 5.2.1440 a separate streams can be pulled from MKV container using multiple stream mixing tool:
Code Block | ||||
---|---|---|---|---|
| ||||
on_multiple_record_hook_script=on_multiple_record_hook.sh |
...
./offline_mixer_tool.sh |
...
--pull-streams ../records/multi-recorder___test-record.mkv |
In this case, MKV files will be created per each stream in multirecording:
Code Block | ||||
---|---|---|---|---|
| ||||
LOGGER_ENABLED=true |
Multithreaded encoding while mixing multiple stream recordings
Since build 5.2.1089 multithreaded encoding can be enabled while mixing multiple stream recordings. To enable this feature, add the following parameter to /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"multithreading": true
} |
Multiple stream recordings are mixed a twice faster when multithreaded encoding is enabled comparing to singe-threaded one.
Multiple recording data callback
Since build 5.2.1123 it is possible to send PORT query to a certain URL after multiple recording and mixing is finished. This allows to notify backend about file name to which chat room recording is mixed.
Callback URL should be set in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"callbackUrl": "http://backend.url/multiRecorderCallback"
} |
...
multi-recorder___test-record_fbbf5b50-20ee-11ec-bf06-ef6ec6048b2c.mkv
multi-recorder___test-record_c7bc1460-20ee-11ec-bf06-ef6ec6048b2c.mkv
multi-recorder___test-record_ed3ebda0-20ee-11ec-bf06-ef6ec6048b2c.mkv |
If a stream was removed from multirecorder and when added again, or was added a much later then other streams, the gaps will be filled by default to equalize all the pulled streams. This can be disabled if necessary
Code Block | ||
---|---|---|
| ||
multi_recorder_mkv_fill_gaps=false |
Multiple recording file handling script
When multiple stream recording is finished, a specila handling script is automatically launched as set by the following parameter
Code Block | ||
---|---|---|
| ||
on_multiple_record_hook_script=on_multiple_record_hook.sh |
By default, script will launch offline_mixer_tool.sh, passing multiple streams recording file name to it.
Since build 5.2.1023, on_multiple_record_hook.sh script writes only file mixing result to log file /usr/local/FlashphonerWebCallServer/logs/multi-record.log, to decrease hard disk I/O load while offline mixing tool is working. A detailed logging mat=y be enabled if necessary by setting the following variable in the script
Code Block | ||||
---|---|---|---|---|
| ||||
LOGGER_ENABLED=true |
Multithreaded encoding while mixing multiple stream recordings
Since build 5.2.1089 multithreaded encoding can be enabled while mixing multiple stream recordings. To enable this feature, add the following parameter to /usr/local/FlashphonerWebCallServer/bin/on_multiple_record_hook.sh script when launching conf/offline_mixer_tool.sh. Therefore, if WCS build 5.2.1123 is installed over a previous build, or if custom on_multiple_record_hook.sh is used, the script should be modified as follows:.json file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"multithreading": true
} |
Multiple stream recordings are mixed a twice faster when multithreaded encoding is enabled comparing to singe-threaded one.
Multiple recording data callback
Since build 5.2.1123 it is possible to send PORT query to a certain URL after multiple recording and mixing is finished. This allows to notify backend about file name to which chat room recording is mixed.
Callback URL should be set in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# This script copies a recorded stream to client/records FILE_NAME=$1 CREATION_MODIFICATION_TIME=$2 DURATION_IN_MS=$3 RECORDER_URI=$4 WCS_HOME={ ..., "callbackUrl": "http://backend.url/multiRecorderCallback" } |
A recording data to send are passed via /usr/local/FlashphonerWebCallServer
...
/bin/on_multiple_record_hook.sh script when launching offline_mixer_tool.sh. Therefore, if WCS build 5.2.1123 is installed over a previous build, or if custom on_multiple_record_hook.sh is used, the script should be modified as follows:
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# This script copies a recorded stream to client/records
FILE_NAME=$1
CREATION_MODIFICATION_TIME=$2
DURATION_IN_MS=$3
RECORDER_URI=$4
WCS_HOME=/usr/local/FlashphonerWebCallServer
LOG_FILE=$WCS_HOME/logs/multi-record.log
MIXER_TOOL=$WCS_HOME/tools/offline_mixer_tool.sh
# Set LOGGER_ENABLED to true to enable mixing debug logging
LOGGER_ENABLED=false
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Start mixing multiple recording file $FILE_NAME" >> $LOG_FILE
if $LOGGER_ENABLED; then
bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI >> $LOG_FILE 2>&1
else
bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI > /dev/null 2>&1
fi
CODE=$?
if [ "$CODE" -ne "0" ]; then
if [ "$CODE" -eq "64" ]; then
echo "ERROR: File to mix not found" >> $LOG_FILE
elif [ "$CODE" -eq "65" ]; then
echo "ERROR: Offline mixer config not found" >> $LOG_FILE
else
echo "ERROR: Offline mixer tool error code: $CODE" >> $LOG_FILE
fi
exit $CODE
fi
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Multiple recording file $FILE_NAME is mixed successfully" >> $LOG_FILE
exit 0 |
...
for RTMP streams with appropriate encoder settings
12. VLC player skips gaps when playing a stream pulled from multirecording in MKV container
Symptoms: gaps in a pulled stream are skipped when playing it in VLC
Solution: use the following parameter
Code Block | ||
---|---|---|
| ||
multi_recorder_mkv_fill_gaps=true |
13. It is recommended to provide a regular keyframes sending when WebRTC stream is recording to singe MKV file and to MKV multirecorder simultaneously
Symptoms: VLC jumps to time of stream addition to multirecorder when playing a single stream MKV recording
Solution: provide periodic key frames sending for WebRTC streams with the following parameter
Code Block | ||
---|---|---|
| ||
periodic_fir_request=true |
14. G722 audio can be a shortly distorted when pulling a stream file from MKV multirecording
Symptoms: If the stream with G722 audio was removed and then added again to the multirecorder, a short audio distortions can be heard when the stream was added to multirecorder
Solution: use Opus audio codec
15. PCMA audio may finish earlier than video after stream removing from multirecorder when pulling a stream file from MKV multirecording
Symptoms: if the stream with PCMA audio was removed from multirecorder, audio finishes earlier than video when playing the pulled stream file
Solution: use Opus audio codec