Page History

...

Since build 5.2.1012 it is possible to record multiple streams to one file. Then streams can be extracted from this file and mixed by a special tool. Multiple streams can be recorded only to MP4 container (H264 + AAC)or, since build 5.2.1440, to MKV container. This feature is intended, for example, to record a video conference. In this case, unlike to to MCU mixer, stream mixing works while recording file is post-processed, this allows to aquire a less of server resources during the conference itself.

Multiple stream recording is managed by REST API.

Codecs support

MP4 container:

H264
AAC

MKV container:

H264
VP8
Opus
AAC
PCMA
PCMU
G722

Multirecorder REST API

REST query must be HTTP/HTTPS POST query like this:

...

streaming.flashphoner.com is WCS server address
8081 is a standard WCS REST / HTTP port
8444 is a standard WCS REST / HTTPS port
rest-api is mandatory prefix
/multipleRecorder/startup is REST method

REST methods and responses

REST query

Example of REST query

Example of REST response

Response states

Description

/multipleRecorder/startup

Code Block

language	js
theme	RDark

{
 "uri": "multi-recorder://test-record"
}

409 - Conflict

500 - Internal error

Launch multiple streams recorder

/multipleRecorder/add

Code Block

language	js
theme	RDark

{
 "uri": "multi-recorder://test-record",
 "mediaSessionId": "866a9910-fbfe-11eb-aae4-6f99b0c80a3a"
}

404 - Not found

409 - Conflict

500 - Internal error

Add the stream with mediaSessionId to the recorder

/multipleRecorder/find_all

Code Block

language	js
theme	RDark

[
  {
    "mediaSessionsId": [
      "866a9910-fbfe-11eb-aae4-6f99b0c80a3a",
      "9f1e2530-fbfe-11eb-9ec1-77172ac14d86",
      "a970d0a0-fbfe-11eb-8fcc-912807bab442"
    ],
    "uri": "multi-recorder://test-record",
    "fileName": "multi-recorder___test-record.mp4"
  }
]

404 - Not found

500 - Internal error

Find all recorders

/multipleRecorder/remove

Code Block

language	js
theme	RDark

{
 "uri": "multi-recorder://test-record",
 "mediaSessionId": "866a9910-fbfe-11eb-aae4-6f99b0c80a3a"
}

404 - Not found

500 - Internal error

Remove stream with mediaSessionId from recorder

/multipleRecorder/terminate

Code Block

language	js
theme	RDark

{
 "uri": "multi-recorder://test-record"
}

404 - Not found

500 - Internal error

Stop multiple streams recorder

...

Parameter name	Description	Example
uri	Recorder URI	multi-recorder://test-record
mediaSessionId	Stream mediasession Id	866a9910-fbfe-11eb-aae4-6f99b0c80a3a
filename	Recording file name	multi-recorder___test-record.mp4

Recording file name

Multiple streams recording file name is formed by template. In this case:

1. {streamName} parameter is set according to recorder URI, replacing all the characters not allowed to use in file system to underline.

...

Container configuration

The feature is available since build 5.2.1440. By default, multiple recordings are stored to MP4 container

Code Block

theme	RDark

multi_recorder_type=MP4

MKV container may be used if necessary (for example, if VP8+Opus streams are published in conference)

Code Block

theme	RDark

multi_recorder_type=MKV

Recording file name

Multiple streams recording file name is formed by template. In this case:

1. {streamName} parameter is set according to recorder URI, replacing all the characters not allowed to use in file system to underline.

2. {startTime}, {endTime} parameters cannot be set because they depend on stream timestamps, and we have a multiple streams with a different timestamps simultaneously. So it is recommended to use {startTimeMillis}, {endTimeMillis} parameters to add servers clock timestamps to file name.

...

A picture sample from the mixed file

...

Since build 5.2.1049 tracks information from 1481, a streams can be mixed both from MP4 and MKV containers. The mixed stream is always recording to MP4 container.

Getting tracks information from multiple recording file

Since build 5.2.1049 tracks information from multiple recording file may be extracted using multiple stream mixing tool. To do this, the tool should be launched as follows:

...

Code Block

language	js
theme	RDark
title	Re-added stream track information example example
collapse	true

[
  {
    "durationInMS": "78978",
    "trackEdits": [
      {
        "endInMs": "63050",
        "type": "pause",
        "startInMs": "0"
      },
      {
        "endInMs": "78978",
        "type": "media",
        "startInMs": "63050"
      }
    ],
    "channels": "2",
    "trackType": "AUDIO",
    "trackId": "3",
    "timescale": "44100",
    "streamName": "test",
    "trackCodec": "mp4a",
    "sampleRate": "44100",
    "mediaSessionId": "fbbf5b50-20ee-11ec-bf06-ef6ec6048b2c"
  },
  {
    "durationInMS": "39708",
    "trackEdits": [
      {
        "endInMs": "23150",
        "type": "media",
        "startInMs": "0"
      }
    ],
    "channels": "2",
    "trackType": "AUDIO",
    "trackId": "1",
    "timescale": "44100",
    "streamName": "test",
    "trackCodec": "mp4a",
    "sampleRate": "44100",
    "mediaSessionId": "c7bc1460-20ee-11ec-bf06-ef6ec6048b2c"
  },
  {
    "durationInMS": "39791",
    "trackEdits": [
      {
        "endInMs": "23233",
        "type": "media",
        "startInMs": "0"
      }
    ],
    "trackType": "VIDEO",
    "trackId": "0",
    "width": "640",
    "timescale": "90000",
    "streamName": "test",
    "trackCodec": "avc1",
    "mediaSessionId": "c7bc1460-20ee-11ec-bf06-ef6ec6048b2c",
    "height": "360"
  },
  {
    "durationInMS": "63050",
    "trackEdits": [
      {
        "endInMs": "39791",
        "type": "pause",
        "startInMs": "0"
      },
      {
        "endInMs": "50191",
        "type": "media",
        "startInMs": "39791"
      }
    ],
    "trackType": "VIDEO",
    "trackId": "2",
    "width": "640",
    "timescale": "90000",
    "streamName": "test",
    "trackCodec": "avc1",
    "mediaSessionId": "ed3ebda0-20ee-11ec-bf06-ef6ec6048b2c",
    "height": "360"
  }
]

Multiple recording file handling script

When multiple stream recording is finished, a specila handling script is automatically launched as set by the following parameter

Code Block

theme	RDark

on_multiple_record_hook_script=on_multiple_record_hook.sh

By default, script will launch offline_mixer_tool.sh, passing multiple streams recording file name to it.

...

Pulling a separate streams from MKV container

Since build 5.2.1440 a separate streams can be pulled from MKV container using multiple stream mixing tool:

Code Block

language	bash
theme	RDark

./offline_mixer_tool.sh --pull-streams ../records/multi-recorder___test-record.mkv

In this case, MKV files will be created per each stream in multirecording:

Code Block

language	bash
theme	RDark

LOGGER_ENABLED=true

Multithreaded encoding while mixing multiple stream recordings

Since build 5.2.1089 multithreaded encoding can be enabled while mixing multiple stream recordings. To enable this feature, add the following parameter to /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file

Code Block

language	js
theme	RDark

{
  ...,
  "multithreading": true
}

Multiple stream recordings are mixed a twice faster when multithreaded encoding is enabled comparing to singe-threaded one.

Multiple recording data callback

Since build 5.2.1123 it is possible to send PORT query to a certain URL after multiple recording and mixing is finished. This allows to notify backend about file name to which chat room recording is mixed.

...

multi-recorder___test-record_fbbf5b50-20ee-11ec-bf06-ef6ec6048b2c.mkv
multi-recorder___test-record_c7bc1460-20ee-11ec-bf06-ef6ec6048b2c.mkv
multi-recorder___test-record_ed3ebda0-20ee-11ec-bf06-ef6ec6048b2c.mkv

If a stream was removed from multirecorder and when added again, or was added a much later then other streams, the gaps will be filled by default to equalize all the pulled streams. This can be disabled if necessary

Code Block

theme	RDark

multi_recorder_mkv_fill_gaps=false

Multiple recording file handling script

When multiple stream recording is finished, a specila handling script is automatically launched as set by the following parameter

Code Block

theme	RDark

on_multiple_record_hook_script=on_multiple_record_hook.sh

By default, script will launch offline_mixer_tool.sh, passing multiple streams recording file name to it.

Since build 5.2.1023, on_multiple_record_hook.sh script writes only file mixing result to log file /usr/local/FlashphonerWebCallServer/conf/offline_mixer.logs/multi-record.log, to decrease hard disk I/O load while offline mixing tool is working. A detailed logging mat=y be enabled if necessary by setting the following variable in the script

Code Block

language	bash
theme	RDark

LOGGER_ENABLED=true

Multi-threaded encoding while mixing multiple stream recordings

Since build 5.2.1089 multi-threaded encoding can be enabled while mixing multiple stream recordings. To enable this feature, add the following parameter to /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file

Code Block

language	js
theme	RDark

{
  ...,
   "callbackUrlmultithreading": "http://backend.url/multiRecorderCallback"
}

A recording data to send are passed via /usr/local/FlashphonerWebCallServer/bin/on_multiple_record_hook.sh script when launching offline_mixer_tool.sh. Therefore, if WCS build 5.2.1123 is installed over a previous build, or if custom on_multiple_record_hook.sh is used, the script should be modified as follows:

Code Block

language	bash
theme	RDark
title	on_multiple_record_hook.sh
collapse	true

# This script copies a recorded stream to client/records
FILE_NAME=$1
CREATION_MODIFICATION_TIME=$2
DURATION_IN_MS=$3
RECORDER_URI=$4
WCS_HOME=/usr/local/FlashphonerWebCallServer
LOG_FILE=$WCS_HOME/logs/multi-record.log
MIXER_TOOL=$WCS_HOME/tools/offline_mixer_tool.sh
# Set LOGGER_ENABLED to true to enable mixing debug logging
LOGGER_ENABLED=false

echo "[$(date '+%Y-%m-%d %H:%M:%S')] Start mixing multiple recording file $FILE_NAME" >> $LOG_FILE

if $LOGGER_ENABLED; then
    bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI >> $LOG_FILE 2>&1
else
    bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI > /dev/null 2>&1
fi

CODE=$?
if [ "$CODE" -ne "0" ]; then
    if [ "$CODE" -eq "64" ]; then
        echo "ERROR: File to mix not found" >> $LOG_FILE
    elif [ "$CODE" -eq "65" ]; then
        echo "ERROR: Offline mixer config not found" >> $LOG_FILE
    else
	    echo "ERROR: Offline mixer tool error code: $CODE" >> $LOG_FILE
	fi
	exit $CODE
fi
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Multiple recording file $FILE_NAME is mixed successfully" >> $LOG_FILE
exit 0

POST query contains a following JSON data:

Code Block

language	js
theme	RDark

POST /multiRecorderCallback HTTP/1.1
Content-Type: application/json
Content-Length: 463
Host: localhost
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.3.5 (java 1.5)
Accept-Encoding: gzip,deflate

{
  "multiRecorderCreationModificationTime":3724973476,
  "multiRecorderDurationInMS":44061,
  "multiRecorderFilePath":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin.mp4",
  "recorderUri":"multi-recorder://room-bace1f",
  "mixerParams":
  [
    {
      "path":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin_mixed.mp4",
      "durationInMs":44000,
      "creationModificationTime":3724973524
    }
  ]
}

Multiple recording file parameters:

multiRecorderCreationModificationTime - multiple recording file creation date and time
multiRecorderDurationInMS - multiple recording file duration in milliseconds
multiRecorderFilePath - multiple recording file path
recorderUri - multiple recording identifier, contains room name if RoomApi is used

Mixed file parameters:

path - mixed file path
durationInMs - mixed file duration in milliseconds
creationModificationTime - mixed file creation date and time

Minimal available disk space checking for multiple recordings mixing

Since build 5.2.1317 minimal available disk space checking is enabled for multiple recordings mixing. The mixing will not start or will be stopped when disk space becomes less then defined threshold value. The threshold is set by the following parameter in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file

Code Block

language	js
theme	RDark

{
  ...,
  "minAvailableSpace": "1G"
}true
}

Multiple stream recordings are mixed a twice faster when multi-threaded encoding is enabled comparing to singe-threaded one.

Threads count to use for multi-threaded encoding

Since build 5.2.1523 threads count for multi-threaded encoding can be set up. By default, threads count is equal to a half of CPUs available. For example, on 12 vCPU server 6 threads will be used

Code Block

language	js
theme	RDark

{
  ...,
  "threadCount": 6
}

If mixing takes a long time, threads count may be increased, but it is not recommended to set it more than CPUs count, which can be detected by the following command

Code Block

language	bash
theme	RDark

lscpu | grep -E "^CPU\(s\)"

Stream name displaying in mixed multiple record

By default, every stream name is displayed in mixed multiple record. This may be disabled if necessary by the following setting in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file

Code Block

language	js
theme	RDark

{
  ...,
  "mixerDisplayStreamName": false
}

When RoomApi conference streams are recorded, stream name includes a room name and a participant stream id, for example room-1882a6-bob-037c. Since build 5.2.1642 room name may be excluded by the following settings

Code Block

language	js
theme	RDark

{
  ...,
  "mixerDisplayStreamName": true,
  "mixerTextDisplayRoom": false,
  "labelReplaceRegex": "\\w+-\\w+-([^\\-]+)-\\w+",
  "labelReplaceWith":""
}

Where:

labelReplaceRegex - a regular expression to find items to be replaced in a stream name
labelReplaceWith - string to replace items found, empty string will exclude the items

In this case, for the example above, only participant name bob will be displayed.

Characters decoding in stream name

Since build 5.2.1751 a characters encoded at client side by encodeURIComponent() may be decoded during mixing a multiple record

Code Block

language	js
theme	RDark

{
  ...,
  "mixerDecodeStreamName": true
}

In this case a decoded characters available in the font used will be displayed, or a similar characters.

Multiple recording data callback

Since build 5.2.1123 it is possible to send PORT query to a certain URL after multiple recording and mixing is finished. This allows to notify backend about file name to which chat room recording is mixed.

Callback URL should be set in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file

Code Block

language	js
theme	RDark

{
  ...,
  "callbackUrl": "http://backend.url/multiRecorderCallback"
}

A recording data to send are passed via /usr/local/FlashphonerWebCallServer/bin/on_multiple_record_hook.sh script when launching offline_mixer_tool.sh. Therefore, if WCS build 5.2.1123 is installed over a previous build, or if custom on_multiple_record_hook.sh is used, the script should be modified as follows:

Code Block

language	bash
theme	RDark
title	on_multiple_record_hook.sh
collapse	true

# This script copies a recorded stream to client/records
FILE_NAME=$1
CREATION_MODIFICATION_TIME=$2
DURATION_IN_MS=$3
RECORDER_URI=$4
WCS_HOME=/usr/local/FlashphonerWebCallServer
LOG_FILE=$WCS_HOME/logs/multi-record.log
MIXER_TOOL=$WCS_HOME/tools/offline_mixer_tool.sh
# Set LOGGER_ENABLED to true to enable mixing debug logging
LOGGER_ENABLED=false

echo "[$(date '+%Y-%m-%d %H:%M:%S')] Start mixing multiple recording file $FILE_NAME" >> $LOG_FILE

if $LOGGER_ENABLED; then
    bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI >> $LOG_FILE 2>&1
else
    bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI > /dev/null 2>&1
fi

CODE=$?
if [ "$CODE" -ne "0" ]; then
    if [ "$CODE" -eq "64" ]; then
        echo "ERROR: File to mix not found" >> $LOG_FILE
    elif [ "$CODE" -eq "65" ]; then
        echo "ERROR: Offline mixer config not found" >> $LOG_FILE
    else
	    echo "ERROR: Offline mixer tool error code: $CODE" >> $LOG_FILE
	fi
	exit $CODE
fi
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Multiple recording file $FILE_NAME is mixed successfully" >> $LOG_FILE
exit 0

POST query contains a following JSON data:

Code Block

language	js
theme	RDark

POST /multiRecorderCallback HTTP/1.1
Content-Type: application/json
Content-Length: 463
Host: localhost
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.3.5 (java 1.5)
Accept-Encoding: gzip,deflate

{
  "multiRecorderCreationModificationTime":3724973476,
  "multiRecorderDurationInMS":44061,
  "multiRecorderFilePath":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin.mp4",
  "recorderUri":"multi-recorder://room-bace1f",
  "mixerParams":
  [
    {
      "path":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin_mixed.mp4",
      "durationInMs":44000,
      "creationModificationTime":3724973524
    }
  ]
}

Multiple recording file parameters:

multiRecorderCreationModificationTime - multiple recording file creation date and time
multiRecorderDurationInMS - multiple recording file duration in milliseconds
multiRecorderFilePath - multiple recording file path
recorderUri - multiple recording identifier, contains room name if RoomApi is used

Mixed file parameters:

path - mixed file path
durationInMs - mixed file duration in milliseconds
creationModificationTime - mixed file creation date and time

Minimal available disk space checking for multiple recordings mixing

Since build 5.2.1317 minimal available disk space checking is enabled for multiple recordings mixing. The mixing will not start or will be stopped when disk space becomes less then defined threshold value. The threshold is set by the following parameter in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file

Code Block

language	js
theme	RDark

{
  ...,
  "minAvailableSpace": "1G"
}

By default, disk space threshold is set to 1 Gb (same as for single stream recordings). If the treshold is reached while mixing is already working, the mixing will be stopped with keeping data already written to destination file. The file can be correctly played after that.

...

Code Block

theme	RDark

v=0
o=- 1988962254 1988962254 IN IP4 0.0.0.0
c=IN IP4 0.0.0.0
t=0 0
a=sdplang:en
m=audio 0 RTP/AVP 97 8 0 102 103 104 105 106 107 108 109 110 105 106 107 108 109 110
a=rtpmap:97 SPEEX/16000
a=rtpmap:8 PCMA/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:102 mpeg4-generic/48000/1
a=rtpmap:103 mpeg4-generic/44100/1
a=rtpmap:104 mpeg4-generic/32000/1
a=rtpmap:105 mpeg4-generic/24000/1
a=rtpmap:106 mpeg4-generic/22050/1
a=rtpmap:107 mpeg4-generic/16000/1
a=rtpmap:108 mpeg4-generic/12000/1
a=rtpmap:97 SPEEX/16000109 mpeg4-generic/11025/1
a=rtpmap:8110 PCMAmpeg4-generic/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:102 mpeg4-generic/48000/1
a=rtpmap:103 mpeg4-generic/44100/1
a=rtpmap:104 mpeg4-generic/32000/1
a=rtpmap:105 mpeg4-generic/24000/1
a=rtpmap:106 mpeg4-generic/22050/1
a=rtpmap:107 mpeg4-generic/16000/1
a=rtpmap:108 mpeg4-generic/12000/1
a=rtpmap:109 mpeg4-generic/11025/1
a=rtpmap:110 mpeg4-generic/8000/1
a=sendonly

11. Stream recording will start only when at least one key frame is received

Symptoms: recording file is not created when starting recording (via REST API for example), there are no errors in server logs, the stream is already playing in browsers (until viewers reconnect)

Solution: provide periodic key frames sending for WebRTC streams with the following parameter

Code Block

theme	RDark

periodic_fir_request=true

for RTMP streams with appropriate encoder settings

...

/1
a=sendonly

11. Stream recording will start only when at least one key frame is received

Symptoms: recording file is not created when starting recording (via REST API for example), there are no errors in server logs, the stream is already playing in browsers (until viewers reconnect)

Solution: provide periodic key frames sending for WebRTC streams with the following parameter

Code Block

theme	RDark

periodic_fir_request=true

for RTMP streams with appropriate encoder settings

Image Added

12. VLC player skips gaps when playing a stream pulled from multirecording in MKV container

Symptoms: gaps in a pulled stream are skipped when playing it in VLC

Solution: use the following parameter

Code Block

theme	RDark

multi_recorder_mkv_fill_gaps=true

13. It is recommended to provide a regular keyframes sending when WebRTC stream is recording to singe MKV file and to MKV multirecorder simultaneously

Symptoms: VLC jumps to time of stream addition to multirecorder when playing a single stream MKV recording

Solution: provide periodic key frames sending for WebRTC streams with the following parameter

Code Block

theme	RDark

periodic_fir_request=true

14. G722 audio can be a shortly distorted when pulling a stream file from MKV multirecording

Symptoms: If the stream with G722 audio was removed and then added again to the multirecorder, a short audio distortions can be heard when the stream was added to multirecorder

Solution: use Opus audio codec

15. PCMA audio may finish earlier than video after stream removing from multirecorder when pulling a stream file from MKV multirecording

Symptoms: if the stream with PCMA audio was removed from multirecorder, audio finishes earlier than video when playing the pulled stream file

Solution: use Opus audio codec

Space shortcuts

Page tree

Versions Compared

Old Version 58

New Version Current

Key

Codecs support

Multirecorder REST API

REST methods and responses

Recording file name

Container configuration

Recording file name

Getting tracks information from multiple recording file

Multiple recording file handling script

Pulling a separate streams from MKV container

Multithreaded encoding while mixing multiple stream recordings

Multiple recording data callback

Multiple recording file handling script

Multi-threaded encoding while mixing multiple stream recordings

Minimal available disk space checking for multiple recordings mixing

Threads count to use for multi-threaded encoding

Stream name displaying in mixed multiple record

Characters decoding in stream name

Multiple recording data callback

Minimal available disk space checking for multiple recordings mixing