...
2. Playback can start slightly earlier if start time is set, depending on closest key frame in the recording file.
...
REST hook for stream recording
...
Since build build 5.2.1012 it 1416 it is possible to record multiple streams to one file. Then streams can be extracted from this file and mixed by a special tool. Multiple streams can be recorded only to MP4 container (H264 + AAC). This feature is intended, for example, to record a video conference. In this case, unlike to MCU mixer, stream mixing works while recording file is post-processed, this allows to aquire a less of server resources during the conference itself.
Multiple stream recording is managed by REST API.
REST query must be HTTP/HTTPS POST query like this:
- HTTP: http://streaming.flashphoner.com:8081/rest-api/multipleRecorder/startup
- HTTPS: https://streaming.flashphoner.com:8444/rest-api/multipleRecorder/startup
Where:
- streaming.flashphoner.com is WCS server address
- 8081 is a standard WCS REST / HTTP port
- 8444 is a standard WCS REST / HTTPS port
- rest-api is mandatory prefix
- /multipleRecorder/startup is REST method
REST methods and responses
...
REST query
...
Example of REST query
...
Example of REST response
...
Response states
...
Description
/multipleRecorder/startup
...
Code Block | ||||
---|---|---|---|---|
| ||||
{
"uri": "multi-recorder://test-record"
} |
409 - Conflict
500 - Internal error
...
Launch multiple streams recorder
...
Code Block | ||||
---|---|---|---|---|
| ||||
{
"uri": "multi-recorder://test-record",
"mediaSessionId": "866a9910-fbfe-11eb-aae4-6f99b0c80a3a"
} |
...
404 - Not found
409 - Conflict
500 - Internal error
...
language | js |
---|---|
theme | RDark |
...
receive an events about a certain stream recording starting or stopping. WCS sends to a backend server the REST hook /StreamEvent
Code Block | ||||
---|---|---|---|---|
| ||||
URL:http://localhost:8081/apps/EchoApp/StreamEvent
OBJECT:
{
"nodeId" : "d2hxbqNPE04vGeZ51NPhDuId6k3hUrBB@192.168.1.39",
"appKey" : "defaultApp",
"sessionId" : "/192.168.1.83:49977/192.168.1.39:8443-591009c4-e051-4722-b34d-71cf2ade3bed",
"mediaSessionId" : "15de2290-4089-11ed-88fe-d78a87cf3386",
"type" : "startedRecording",
"payload" : {
"fileName" : "stream-15de2290-4089-11ed-88fe-d78a87cf3386-8mv1of1o4fni58k0qdomu52kru.mp4"
}
} |
when stream recording is started and
Code Block | ||||
---|---|---|---|---|
| ||||
URL:http://localhost:8081/apps/EchoApp/StreamEvent OBJECT: { "nodeId" : "d2hxbqNPE04vGeZ51NPhDuId6k3hUrBB@192.168.1.39", "appKey" : "defaultApp", "sessionId" : "/192.168.1.83:49977/192.168.1.39:8443-591009c4-e051-4722-b34d-71cf2ade3bed", "mediaSessionId" : "15de2290-4089-11ed-88fe-d78a87cf3386", "type" : "stoppedRecording", "payload" : { "fileName" : " |
...
stream-15de2290-4089-11ed-88fe-d78a87cf3386-8mv1of1o4fni58k0qdomu52kru.mp4" } |
...
404 - Not found
500 - Internal error
...
} |
when stream recording is stopped.
The StreamEvent
method must be added to a backend application configuration when updating WCS from previous builds
...
Code Block | ||||
---|---|---|---|---|
| ||||
{
"uri": "multi-recorder://test-record"
} |
...
404 - Not found
500 - Internal error
...
Parametes
...
Parameter name
...
Description
...
Example
...
mediaSessionId
...
Stream mediasession Id
...
866a9910-fbfe-11eb-aae4-6f99b0c80a3a
...
Recording file name
Multiple streams recording file name is formed by template. In this case:
1. {streamName} parameter is set according to recorder URI, replacing all the characters not allowed to use in file system to underline.
2. {startTime}, {endTime} parameters cannot be set because they depend on stream timestamps, and we have a multiple streams with a different timestamps simultaneously. So it is recommended to use {startTimeMillis}, {endTimeMillis} parameters to add servers clock timestamps to file name.
For example, with the following template
Code Block | ||||
---|---|---|---|---|
| ||||
stream_record_policy_template={streamName}-{startTime}-{startTimeMillis}-{endTime}-{endTimeMillis} |
the file name for recorder with URI
Code Block | ||||
---|---|---|---|---|
| ||||
"uri": "multi-recorder://test-record" |
will look as follows:
Code Block | ||
---|---|---|
| ||
multi-recorder___test-record--1-1628821032180--1-1628821151750.mp4 |
Here {startTime}, {endTime} are replaced by -1.
Multiple streams recording folder
By default, multiple streams recording files are stored in WCS_HOME/records forlder. Since build 5.2.1088, multiple streams recording folder can be set using the following parameter
Code Block | ||
---|---|---|
| ||
multi_record_dir=/usr/local/FlashphonerWebCallServer/records |
This folder should be writalble, For example, if the folder is set to
Code Block | ||
---|---|---|
| ||
multi_record_dir=/opt/media |
the folder access privilegies sholud be set as
Code Block | ||||
---|---|---|---|---|
| ||||
sudo chmod o+w /opt/media |
Multiple streams mixing tool
Only one stream can be played by default from multiple streams recording file. The streams should be mixed to view them all. The OfflineMP4Mixer tool should be used to mix, launching as follows:
Code Block | ||||
---|---|---|---|---|
| ||||
cd /usr/local/FlashphonerWebCallServer/tools
./offline_mixer_tool.sh multi-recorder___test-record--1-1628821032180--1-1628821151750.mp4 |
...
Code Block | ||||
---|---|---|---|---|
|
...
{
"uri": "multi-recorder://test-record",
"mediaSessionId": "866a9910-fbfe-11eb-aae4-6f99b0c80a3a"
}
...
404 - Not found
500 - Internal error
...
add app-rest-method defaultApp StreamEvent
add app-rest-method MyAppKey StreamEvent |
Multiple stream recording to one file with subsequent mixing
Since build 5.2.1012 it is possible to record multiple streams to one file. Then streams can be extracted from this file and mixed by a special tool. Multiple streams can be recorded only to MP4 container or, since build 5.2.1440, to MKV container. This feature is intended, for example, to record a video conference. In this case, unlike to MCU mixer, stream mixing works while recording file is post-processed, this allows to aquire a less of server resources during the conference itself.
Multiple stream recording is managed by REST API.
Codecs support
MP4 container:
- H264
- AAC
MKV container:
- H264
- VP8
- Opus
- AAC
- PCMA
- PCMU
- G722
Multirecorder REST API
REST query must be HTTP/HTTPS POST query like this:
- HTTP: http://streaming.flashphoner.com:8081/rest-api/multipleRecorder/startup
- HTTPS: https://streaming.flashphoner.com:8444/rest-api/multipleRecorder/startup
Where:
- streaming.flashphoner.com is WCS server address
- 8081 is a standard WCS REST / HTTP port
- 8444 is a standard WCS REST / HTTPS port
- rest-api is mandatory prefix
- /multipleRecorder/startup is REST method
REST methods and responses
REST query | Example of REST query | Example of REST response | Response states | Description | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
/multipleRecorder/startup |
| 409 - Conflict 500 - Internal error | Launch multiple streams recorder | ||||||||
/multipleRecorder/add |
| 404 - Not found 409 - Conflict 500 - Internal error | Add the stream with mediaSessionId to the recorder | ||||||||
/multipleRecorder/find_all |
|
...
|
...
|
...
A mixed file is placed to the same folder as original one, with _mixed suffix addition, for example
Code Block | ||
---|---|---|
| ||
multi-recorder___test-record--1-1628821032180--1-1628821151750_mixed.mp4 |
A picture sample from the mixed file
Getting tracks information from multiple recording file
Since build 5.2.1049 tracks information from multiple recording file may be extracted using multiple stream mixing tool. To do this, the tool should be launched as follows:
Code Block | ||||
---|---|---|---|---|
| ||||
./offline_mixer_tool.sh --show-tracks-info ../records/multi-recorder___test-record.mp4 |
...
| 404 - Not found 500 - Internal error | Find all recorders | |||||||||
/multipleRecorder/remove |
| 404 - Not found 500 - Internal error | Remove stream with mediaSessionId from recorder | ||||||||
/multipleRecorder/terminate |
|
...
...
|
...
|
...
| 404 - Not found 500 - Internal error | Stop multiple streams recorder |
Parametes
Parameter name | Description | Example |
---|---|---|
uri | Recorder URI | multi-recorder://test-record |
mediaSessionId | Stream mediasession Id | 866a9910-fbfe-11eb-aae4-6f99b0c80a3a |
filename | Recording file name | multi-recorder___test-record.mp4 |
Container configuration
The feature is available since build 5.2.1440. By default, multiple recordings are stored to MP4 container
Code Block | ||
---|---|---|
| ||
multi_recorder_type=MP4 |
MKV container may be used if necessary (for example, if VP8+Opus streams are published in conference)
Code Block | ||
---|---|---|
| ||
multi_recorder_type=MKV |
Recording file name
Multiple streams recording file name is formed by template. In this case:
1. {streamName} parameter is set according to recorder URI, replacing all the characters not allowed to use in file system to underline.
2. {startTime}, {endTime} parameters cannot be set because they depend on stream timestamps, and we have a multiple streams with a different timestamps simultaneously. So it is recommended to use {startTimeMillis}, {endTimeMillis} parameters to add servers clock timestamps to file name.
For example, with the following template
Code Block | ||||
---|---|---|---|---|
| ||||
stream_record_policy_template={streamName}-{startTime}-{startTimeMillis}-{endTime}-{endTimeMillis} |
the file name for recorder with URI
Code Block | ||||
---|---|---|---|---|
| ||||
"uri": "multi-recorder://test-record" |
will look as follows:
Code Block | ||
---|---|---|
| ||
multi-recorder___test-record--1-1628821032180--1-1628821151750.mp4 |
Here {startTime}, {endTime} are replaced by -1.
Multiple streams recording folder
By default, multiple streams recording files are stored in WCS_HOME/records forlder. Since build 5.2.1088, multiple streams recording folder can be set using the following parameter
Code Block | ||
---|---|---|
| ||
multi_record_dir=/usr/local/FlashphonerWebCallServer/records |
This folder should be writalble, For example, if the folder is set to
Code Block | ||
---|---|---|
| ||
multi_record_dir=/opt/media |
the folder access privilegies sholud be set as
Code Block | ||||
---|---|---|---|---|
| ||||
sudo chmod o+w /opt/media |
Multiple streams mixing tool
Only one stream can be played by default from multiple streams recording file. The streams should be mixed to view them all. The OfflineMP4Mixer tool should be used to mix, launching as follows:
Code Block | ||||
---|---|---|---|---|
| ||||
cd /usr/local/FlashphonerWebCallServer/tools
./offline_mixer_tool.sh multi-recorder___test-record--1-1628821032180--1-1628821151750.mp4 |
Offline mixer settings should be defined in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file. By default, the following settings are used:
Code Block | ||||
---|---|---|---|---|
| ||||
{
"hasVideo": "true",
"hasAudio": "true",
"mixerDisplayStreamName": true
} |
A mixed file is placed to the same folder as original one, with _mixed suffix addition, for example
Code Block | ||
---|---|---|
| ||
multi-recorder___test-record--1-1628821032180--1-1628821151750_mixed.mp4 |
A picture sample from the mixed file
Since build 5.2.1481, a streams can be mixed both from MP4 and MKV containers. The mixed stream is always recording to MP4 container.
Getting tracks information from multiple recording file
Since build 5.2.1049 tracks information from multiple recording file may be extracted using multiple stream mixing tool. To do this, the tool should be launched as follows:
Code Block | ||||
---|---|---|---|---|
| ||||
./offline_mixer_tool.sh --show-tracks-info ../records/multi-recorder___test-record.mp4 |
In this case, the tool will print tracks information in JSON form. See two participants room recording information example below:
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
[ { "durationInMS": "37282", "trackEdits": [ { "endInMs": "10014", "type": "pause", "startInMs": "0" }, { "endInMs": "2153437282", "type": "pausemedia", "startInMs": "10014" }, { ], "endInMschannels": "392742", "typetrackType": "mediaAUDIO", "startInMstrackId": "215341", } ], "channels": "2", "trackType": "AUDIO", "trackId": "3", "timescale"timescale": "44100", "streamName": "room-09f012-user2user1-f746e6ff", "trackCodec": "mp4a", "sampleRate": "44100", "mediaSessionId": "f74633a1e6ff54e0-1c2a-11ec-bba590e8-af8cf43275a879a2a32f3d9d" }, { "durationInMS": "3930337336", "trackEdits": [ { "endInMs": "2143437336", "type": "pausemedia", "startInMs": "0" }, ], { "trackType": "VIDEO", "endInMstrackId": "393030", "typewidth": "media320", "startInMstimescale": "2143490000", } ]"streamName": "room-09f012-user1-e6ff", "trackTypetrackCodec": "VIDEOavc1", "trackIdmediaSessionId": "2e6ff54e0-1c2a-11ec-90e8-79a2a32f3d9d", "widthheight": "320240" }, { "timescaledurationInMS": "9000039274", "streamNametrackEdits": "room-09f012-user2-f746", [ { "trackCodecendInMs": "avc1100", "mediaSessionIdtype": "f74633a1-1c2a-11ec-bba5-af8cf43275a8pause", "heightstartInMs": "2400" } ] |
Where:
durationInMS - track duration in milliseconds
trackType - track type: AUDIO or VIDEO
- trackId - track Id
- streamName - stream name containing this track
- mediaSessionId - stream media session Id
- timescale - track samples per second quantity
- trackCodec - track codec
- width, height - video track picture size by first key frame
- channels - audio track channels count
- sampleRate - audio track sample rate (usually equal to timescale parameter)
- trackEdits - track timeline description
Track timeline is described as a set of segments according to MP4 'edit lists` atom content, with the following parameters:
- startInMs - segment start time in milliseconds relative to file beginning
- endInMs - segment end time in milliseconds relative to file beginning
- type - segment type: mediadata (media) or pause (pause)
Any single track may be extracted from multiple recording file using ffmpeg or other MP4 editing tool.
Note that if a stream with the same name was added to multiple recorder, then removed from recorder and then added again, this stream will be repersented in the file as different tracks with subsequent track Ids, for example:
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
[ { "durationInMS": "78978", "trackEdits": [, { "endInMs": "21534", "type": "pause", "startInMs": "100" }, { "endInMs": "39274", "type": "media", "startInMs": "21534" {} ], "endInMschannels": "630502", "typetrackType": "pauseAUDIO", "startInMstrackId": "03", }"timescale": "44100", { "streamName": "room-09f012-user2-f746", "endInMstrackCodec": "78978mp4a", "typesampleRate": "media44100", "startInMsmediaSessionId": "63050f74633a1-1c2a-11ec-bba5-af8cf43275a8" }, ],{ "channelsdurationInMS": "239303", "trackTypetrackEdits": "AUDIO",[ "trackId": "3", { "timescale": "44100", "streamNameendInMs": "test21434", "trackCodec": "mp4a", "sampleRatetype": "44100pause", "mediaSessionId": "fbbf5b50-20ee-11ec-bf06-ef6ec6048b2c" }, { "durationInMSstartInMs": "397080", "trackEdits": [ }, { "endInMs": "2315039303", "type": "media", "startInMs": "021434" } ], "channelstrackType": "2VIDEO", "trackTypetrackId": "AUDIO2", "trackIdwidth": "1320", "timescale": "4410090000", "streamName": "testroom-09f012-user2-f746", "trackCodec": "mp4aavc1", "sampleRatemediaSessionId": "44100", "mediaSessionId": "c7bc1460-20eef74633a1-1c2a-11ec-bf06bba5-ef6ec6048b2caf8cf43275a8" }, { "durationInMSheight": "39791240", "trackEdits": [ { "endInMs": "23233", "type": "media", "startInMs": "0" } ], "trackType": "VIDEO", "trackId": "0", "width": "640", "timescale": "90000} ] |
Where:
durationInMS - track duration in milliseconds
trackType - track type: AUDIO or VIDEO
- trackId - track Id
- streamName - stream name containing this track
- mediaSessionId - stream media session Id
- timescale - track samples per second quantity
- trackCodec - track codec
- width, height - video track picture size by first key frame
- channels - audio track channels count
- sampleRate - audio track sample rate (usually equal to timescale parameter)
- trackEdits - track timeline description
Track timeline is described as a set of segments according to MP4 'edit lists` atom content, with the following parameters:
- startInMs - segment start time in milliseconds relative to file beginning
- endInMs - segment end time in milliseconds relative to file beginning
- type - segment type: mediadata (media) or pause (pause)
Any single track may be extracted from multiple recording file using ffmpeg or other MP4 editing tool.
Note that if a stream with the same name was added to multiple recorder, then removed from recorder and then added again, this stream will be repersented in the file as different tracks with subsequent track Ids, for example:
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
[ { "durationInMS": "78978", "streamNametrackEdits": "test",[ "trackCodec": "avc1", { "mediaSessionIdendInMs": "c7bc1460-20ee-11ec-bf06-ef6ec6048b2c63050", "heighttype": "360pause", }, { "durationInMSstartInMs": "630500", "trackEdits": [ }, { "endInMs": "3979178978", "type": "pausemedia", "startInMs": "063050" }, { "endInMs": "50191", ], "typechannels": "media", "startInMs": "39791" } ]2", "trackType": "VIDEOAUDIO", "trackId": "23", "widthtimescale": "64044100", "timescalestreamName": "90000test", "streamNametrackCodec": "testmp4a", "trackCodecsampleRate": "avc144100", "mediaSessionId": "ed3ebda0fbbf5b50-20ee-11ec-bf06-ef6ec6048b2c" }, { "heightdurationInMS": "36039708", "trackEdits": [ { "endInMs": "23150", "type": "media", "startInMs": "0" } ] |
Multiple recording file handling script
When multiple stream recording is finished, a specila handling script is automatically launched as set by the following parameter
Code Block | ||
---|---|---|
| ||
on_multiple_record_hook_script=on_multiple_record_hook.sh |
By default, script will launch offline_mixer_tool.sh, passing multiple streams recording file name to it.
Since build 5.2.1023, on_multiple_record_hook.sh script writes only file mixing result to log file /usr/local/FlashphonerWebCallServer/logs/multi-record.log, to decrease hard disk I/O load while offline mixing tool is working. A detailed logging mat=y be enabled if necessary by setting the following variable in the script
Code Block | ||||
---|---|---|---|---|
| ||||
LOGGER_ENABLED=true |
Multithreaded encoding while mixing multiple stream recordings
Since build 5.2.1089 multithreaded encoding can be enabled while mixing multiple stream recordings. To enable this feature, add the following parameter to /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"multithreading": true
} |
Multiple stream recordings are mixed a twice faster when multithreaded encoding is enabled comparing to singe-threaded one.
Multiple recording data callback
Since build 5.2.1123 it is possible to send PORT query to a certain URL after multiple recording and mixing is finished. This allows to notify backend about file name to which chat room recording is mixed.
Callback URL should be set in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"callbackUrl": "http://backend.url/multiRecorderCallback"
} |
A recording data to send are passed via /usr/local/FlashphonerWebCallServer/bin/on_multiple_record_hook.sh script when launching offline_mixer_tool.sh. Therefore, if WCS build 5.2.1123 is installed over a previous build, or if custom on_multiple_record_hook.sh is used, the script should be modified as follows:
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# This script copies a recorded stream to client/records
FILE_NAME=$1
CREATION_MODIFICATION_TIME=$2
DURATION_IN_MS=$3
RECORDER_URI=$4
WCS_HOME=/usr/local/FlashphonerWebCallServer
LOG_FILE=$WCS_HOME/logs/multi-record.log
MIXER_TOOL=$WCS_HOME/tools/offline_mixer_tool.sh
# Set LOGGER_ENABLED to true to enable mixing debug logging
LOGGER_ENABLED=false
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Start mixing multiple recording file $FILE_NAME" >> $LOG_FILE
if $LOGGER_ENABLED; then
bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI >> $LOG_FILE 2>&1
else
bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI > /dev/null 2>&1
fi
CODE=$?
if [ "$CODE" -ne "0" ]; then
if [ "$CODE" -eq "64" ]; then
echo "ERROR: File to mix not found" >> $LOG_FILE
elif [ "$CODE" -eq "65" ]; then
echo "ERROR: Offline mixer config not found" >> $LOG_FILE
else
echo "ERROR: Offline mixer tool error code: $CODE" >> $LOG_FILE
fi
exit $CODE
fi
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Multiple recording file $FILE_NAME is mixed successfully" >> $LOG_FILE
exit 0 |
POST query contains a following JSON data:
Code Block | ||||
---|---|---|---|---|
| ||||
POST /multiRecorderCallback HTTP/1.1
Content-Type: application/json
Content-Length: 463
Host: localhost
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.3.5 (java 1.5)
Accept-Encoding: gzip,deflate
{
"multiRecorderCreationModificationTime":3724973476,
"multiRecorderDurationInMS":44061,
"multiRecorderFilePath":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin.mp4",
"recorderUri":"multi-recorder://room-bace1f",
"mixerParams":
[
{
"path":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin_mixed.mp4",
"durationInMs":44000,
"creationModificationTime":3724973524
}
]
} |
Multiple recording file parameters:
- multiRecorderCreationModificationTime - multiple recording file creation date and time
- multiRecorderDurationInMS - multiple recording file duration in milliseconds
- multiRecorderFilePath - multiple recording file path
- recorderUri - multiple recording identifier, contains room name if RoomApi is used
Mixed file parameters:
- path - mixed file path
- durationInMs - mixed file duration in milliseconds
- creationModificationTime - mixed file creation date and time
Known issues
1. Maximum length of file name in all actual Linux file systems is limited to 255 characters. When record file is created, its name will be trimmed to this limit including extension and part number if rotation is enabled.
2. When stream published in chat room is recorded, file rotation will be automatically disabled, otherwise record files will not be merged.
3. Creation time will be set to MP4 recording metadata only.
4. In Amazon WCS instance, record files hook script requires sudo to execute any file operation.
Symptoms: record hook script does not perform any operation on record files
Solution: in Amazon WCS instance use sudo to make any file operation or call external script from record hook script, for example
Code Block | ||||
---|---|---|---|---|
| ||||
sudo cp $SRC_FILE $DST_FILE |
5. CPU load may grow and simultaneous streams recording may delay to finish on low powered servers when two channels audio is recording
Symptoms: all the CPU cores are loaded to 100% while a number of streams are recording simultaneously, and the recordings are finished with a big delay when streams stop.
Solution: disable two channels audio recording
Code Block | ||
---|---|---|
| ||
record_audio_codec_channels=1 |
6. Stream is playing normally via WebRTC, but recording may be corrupted when publishing H264 stream from Android Firefox on some devices
Symptoms: recording file has a small size and cannot be played or picture seems corrupted when stream is published from Android Firefox
Solution:
a) use VP8 to publish stream from Android Firefox
b) use Chrome or other browser to publish stream from this device
7. Some Android devices may publish WebRTC H264 stream with High profile even if there is no such profile in SDP while establishing WebRTC connection
Symptoms: High profile is displayed in MP4 recording file data
Solution: if there are any problems playing stream recording files with High profile, re-encode those files using ffmpeg for example, running it after recording is finished by on_record_hook.sh script.
8. The first record after the server start may be corrupted if Java machine does not initialize all the necessary modules in time
Symptoms: a long freeze in the first recording file after server start
Solution:
a) update WCS to build 5.2.1105
b) if the build 5.2.1105 and newer is already used, make sure WebRTC stack modules pre-initilization on server startup is enabled
Code Block | ||
---|---|---|
| ||
webrtc_pre_init=true |
9. webm recording files cannot be played in iOS Safari
Symptoms: file download starts instead of playback when clicking webm recording file link
...
],
"channels": "2",
"trackType": "AUDIO",
"trackId": "1",
"timescale": "44100",
"streamName": "test",
"trackCodec": "mp4a",
"sampleRate": "44100",
"mediaSessionId": "c7bc1460-20ee-11ec-bf06-ef6ec6048b2c"
},
{
"durationInMS": "39791",
"trackEdits": [
{
"endInMs": "23233",
"type": "media",
"startInMs": "0"
}
],
"trackType": "VIDEO",
"trackId": "0",
"width": "640",
"timescale": "90000",
"streamName": "test",
"trackCodec": "avc1",
"mediaSessionId": "c7bc1460-20ee-11ec-bf06-ef6ec6048b2c",
"height": "360"
},
{
"durationInMS": "63050",
"trackEdits": [
{
"endInMs": "39791",
"type": "pause",
"startInMs": "0"
},
{
"endInMs": "50191",
"type": "media",
"startInMs": "39791"
}
],
"trackType": "VIDEO",
"trackId": "2",
"width": "640",
"timescale": "90000",
"streamName": "test",
"trackCodec": "avc1",
"mediaSessionId": "ed3ebda0-20ee-11ec-bf06-ef6ec6048b2c",
"height": "360"
}
] |
Pulling a separate streams from MKV container
Since build 5.2.1440 a separate streams can be pulled from MKV container using multiple stream mixing tool:
Code Block | ||||
---|---|---|---|---|
| ||||
./offline_mixer_tool.sh --pull-streams ../records/multi-recorder___test-record.mkv |
In this case, MKV files will be created per each stream in multirecording:
Code Block | ||||
---|---|---|---|---|
| ||||
multi-recorder___test-record_fbbf5b50-20ee-11ec-bf06-ef6ec6048b2c.mkv
multi-recorder___test-record_c7bc1460-20ee-11ec-bf06-ef6ec6048b2c.mkv
multi-recorder___test-record_ed3ebda0-20ee-11ec-bf06-ef6ec6048b2c.mkv |
If a stream was removed from multirecorder and when added again, or was added a much later then other streams, the gaps will be filled by default to equalize all the pulled streams. This can be disabled if necessary
Code Block | ||
---|---|---|
| ||
multi_recorder_mkv_fill_gaps=false |
Multiple recording file handling script
When multiple stream recording is finished, a specila handling script is automatically launched as set by the following parameter
Code Block | ||
---|---|---|
| ||
on_multiple_record_hook_script=on_multiple_record_hook.sh |
By default, script will launch offline_mixer_tool.sh, passing multiple streams recording file name to it.
Since build 5.2.1023, on_multiple_record_hook.sh script writes only file mixing result to log file /usr/local/FlashphonerWebCallServer/logs/multi-record.log, to decrease hard disk I/O load while offline mixing tool is working. A detailed logging mat=y be enabled if necessary by setting the following variable in the script
Code Block | ||||
---|---|---|---|---|
| ||||
LOGGER_ENABLED=true |
Multi-threaded encoding while mixing multiple stream recordings
Since build 5.2.1089 multi-threaded encoding can be enabled while mixing multiple stream recordings. To enable this feature, add the following parameter to /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"multithreading": true
} |
Multiple stream recordings are mixed a twice faster when multi-threaded encoding is enabled comparing to singe-threaded one.
Threads count to use for multi-threaded encoding
Since build 5.2.1523 threads count for multi-threaded encoding can be set up. By default, threads count is equal to a half of CPUs available. For example, on 12 vCPU server 6 threads will be used
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"threadCount": 6
} |
If mixing takes a long time, threads count may be increased, but it is not recommended to set it more than CPUs count, which can be detected by the following command
Code Block | ||||
---|---|---|---|---|
| ||||
lscpu | grep -E "^CPU\(s\)" |
Stream name displaying in mixed multiple record
By default, every stream name is displayed in mixed multiple record. This may be disabled if necessary by the following setting in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json
file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"mixerDisplayStreamName": false
} |
When RoomApi conference streams are recorded, stream name includes a room name and a participant stream id, for example room-1882a6-bob-037c
. Since build 5.2.1642 room name may be excluded by the following settings
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"mixerDisplayStreamName": true,
"mixerTextDisplayRoom": false,
"labelReplaceRegex": "\\w+-\\w+-([^\\-]+)-\\w+",
"labelReplaceWith":""
} |
Where:
- labelReplaceRegex - a regular expression to find items to be replaced in a stream name
- labelReplaceWith - string to replace items found, empty string will exclude the items
In this case, for the example above, only participant name bob
will be displayed.
Characters decoding in stream name
Since build 5.2.1751 a characters encoded at client side by encodeURIComponent() may be decoded during mixing a multiple record
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"mixerDecodeStreamName": true
} |
In this case a decoded characters available in the font used will be displayed, or a similar characters.
Multiple recording data callback
Since build 5.2.1123 it is possible to send PORT query to a certain URL after multiple recording and mixing is finished. This allows to notify backend about file name to which chat room recording is mixed.
Callback URL should be set in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"callbackUrl": "http://backend.url/multiRecorderCallback"
} |
A recording data to send are passed via /usr/local/FlashphonerWebCallServer/bin/on_multiple_record_hook.sh script when launching offline_mixer_tool.sh. Therefore, if WCS build 5.2.1123 is installed over a previous build, or if custom on_multiple_record_hook.sh is used, the script should be modified as follows:
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
# This script copies a recorded stream to client/records
FILE_NAME=$1
CREATION_MODIFICATION_TIME=$2
DURATION_IN_MS=$3
RECORDER_URI=$4
WCS_HOME=/usr/local/FlashphonerWebCallServer
LOG_FILE=$WCS_HOME/logs/multi-record.log
MIXER_TOOL=$WCS_HOME/tools/offline_mixer_tool.sh
# Set LOGGER_ENABLED to true to enable mixing debug logging
LOGGER_ENABLED=false
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Start mixing multiple recording file $FILE_NAME" >> $LOG_FILE
if $LOGGER_ENABLED; then
bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI >> $LOG_FILE 2>&1
else
bash $MIXER_TOOL $FILE_NAME $CREATION_MODIFICATION_TIME $DURATION_IN_MS $RECORDER_URI > /dev/null 2>&1
fi
CODE=$?
if [ "$CODE" -ne "0" ]; then
if [ "$CODE" -eq "64" ]; then
echo "ERROR: File to mix not found" >> $LOG_FILE
elif [ "$CODE" -eq "65" ]; then
echo "ERROR: Offline mixer config not found" >> $LOG_FILE
else
echo "ERROR: Offline mixer tool error code: $CODE" >> $LOG_FILE
fi
exit $CODE
fi
echo "[$(date '+%Y-%m-%d %H:%M:%S')] Multiple recording file $FILE_NAME is mixed successfully" >> $LOG_FILE
exit 0 |
POST query contains a following JSON data:
Code Block | ||||
---|---|---|---|---|
| ||||
POST /multiRecorderCallback HTTP/1.1
Content-Type: application/json
Content-Length: 463
Host: localhost
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.3.5 (java 1.5)
Accept-Encoding: gzip,deflate
{
"multiRecorderCreationModificationTime":3724973476,
"multiRecorderDurationInMS":44061,
"multiRecorderFilePath":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin.mp4",
"recorderUri":"multi-recorder://room-bace1f",
"mixerParams":
[
{
"path":"/usr/local/FlashphonerWebCallServer/multirecords/stream-32c7edd7-37bf-4bf2-a58d-955679c5287e-mockLogin_mixed.mp4",
"durationInMs":44000,
"creationModificationTime":3724973524
}
]
} |
Multiple recording file parameters:
- multiRecorderCreationModificationTime - multiple recording file creation date and time
- multiRecorderDurationInMS - multiple recording file duration in milliseconds
- multiRecorderFilePath - multiple recording file path
- recorderUri - multiple recording identifier, contains room name if RoomApi is used
Mixed file parameters:
- path - mixed file path
- durationInMs - mixed file duration in milliseconds
- creationModificationTime - mixed file creation date and time
Minimal available disk space checking for multiple recordings mixing
Since build 5.2.1317 minimal available disk space checking is enabled for multiple recordings mixing. The mixing will not start or will be stopped when disk space becomes less then defined threshold value. The threshold is set by the following parameter in /usr/local/FlashphonerWebCallServer/conf/offline_mixer.json
file
Code Block | ||||
---|---|---|---|---|
| ||||
{
...,
"minAvailableSpace": "1G"
} |
By default, disk space threshold is set to 1 Gb (same as for single stream recordings). If the treshold is reached while mixing is already working, the mixing will be stopped with keeping data already written to destination file. The file can be correctly played after that.
REST hook for multiple stream recording
Since build 5.2.1416 it is possible to receive an events about a certain stream is added or removed to/from multiple recorder. WCS sends to a backend server the REST hook /StreamEvent
Code Block | ||||
---|---|---|---|---|
| ||||
URL:http://localhost:8081/apps/EchoApp/StreamEvent
OBJECT:
{
"nodeId" : "d2hxbqNPE04vGeZ51NPhDuId6k3hUrBB@192.168.1.39",
"appKey" : "defaultApp",
"sessionId" : "/192.168.1.83:49977/192.168.1.39:8443-591009c4-e051-4722-b34d-71cf2ade3bed",
"mediaSessionId" : "15de2290-4089-11ed-88fe-d78a87cf3386",
"type" : "addedToMultiRecording",
"payload" : {
"fileName" : "stream-0389ff08-7e45-4f00-a579-9d253319cba4-mockLogin.mp4",
"uri" : "multi-recorder://test-record"
}
} |
when stream is added to a multiple recorder and
Code Block | ||||
---|---|---|---|---|
| ||||
URL:http://localhost:8081/apps/EchoApp/StreamEvent
OBJECT:
{
"nodeId" : "d2hxbqNPE04vGeZ51NPhDuId6k3hUrBB@192.168.1.39",
"appKey" : "defaultApp",
"sessionId" : "/192.168.1.83:49977/192.168.1.39:8443-591009c4-e051-4722-b34d-71cf2ade3bed",
"mediaSessionId" : "15de2290-4089-11ed-88fe-d78a87cf3386",
"type" : "removedFromMultiRecording",
"payload" : {
"fileName" : "stream-0389ff08-7e45-4f00-a579-9d253319cba4-mockLogin.mp4",
"uri" : "multi-recorder://test-record"
}
} |
when stream is removed from multiple recorder.
The StreamEvent
method must be added to a backend application configuration when updating WCS from previous builds
Code Block | ||||
---|---|---|---|---|
| ||||
add app-rest-method defaultApp StreamEvent
add app-rest-method MyAppKey StreamEvent |
Known issues
1. Maximum length of file name in all actual Linux file systems is limited to 255 characters. When record file is created, its name will be trimmed to this limit including extension and part number if rotation is enabled.
2. When stream published in chat room is recorded, file rotation will be automatically disabled, otherwise record files will not be merged.
3. Creation time will be set to MP4 recording metadata only.
4. In Amazon WCS instance, record files hook script requires sudo to execute any file operation.
Symptoms: record hook script does not perform any operation on record files
Solution: in Amazon WCS instance use sudo to make any file operation or call external script from record hook script, for example
Code Block | ||||
---|---|---|---|---|
| ||||
sudo cp $SRC_FILE $DST_FILE |
5. CPU load may grow and simultaneous streams recording may delay to finish on low powered servers when two channels audio is recording
Symptoms: all the CPU cores are loaded to 100% while a number of streams are recording simultaneously, and the recordings are finished with a big delay when streams stop.
Solution: disable two channels audio recording
Code Block | ||
---|---|---|
| ||
record_audio_codec_channels=1 |
6. Stream is playing normally via WebRTC, but recording may be corrupted when publishing H264 stream from Android Firefox on some devices
Symptoms: recording file has a small size and cannot be played or picture seems corrupted when stream is published from Android Firefox
Solution:
a) use VP8 to publish stream from Android Firefox
b) use Chrome or other browser to publish stream from this device
7. Some Android devices may publish WebRTC H264 stream with High profile even if there is no such profile in SDP while establishing WebRTC connection
Symptoms: High profile is displayed in MP4 recording file data
Solution: if there are any problems playing stream recording files with High profile, re-encode those files using ffmpeg for example, running it after recording is finished by on_record_hook.sh script.
8. The first record after the server start may be corrupted if Java machine does not initialize all the necessary modules in time
Symptoms: a long freeze in the first recording file after server start
Solution:
a) update WCS to build 5.2.1105
b) if the build 5.2.1105 and newer is already used, make sure WebRTC stack modules pre-initilization on server startup is enabled
Code Block | ||
---|---|---|
| ||
webrtc_pre_init=true |
9. webm recording files cannot be played in iOS Safari
Symptoms: file download starts instead of playback when clicking webm recording file link
Solution: download recording file to device and play it in local player application
10. Recording file is not created when publishing RTMP audio only stream with default settings
Symptoms: audio only RTMP stream published to WCS is not recording
Solution: use SDP settings file flash_handler_publish.sdp
without video track
Code Block | ||
---|---|---|
| ||
v=0
o=- 1988962254 1988962254 IN IP4 0.0.0.0
c=IN IP4 0.0.0.0
t=0 0
a=sdplang:en
m=audio 0 RTP/AVP 97 8 0 102 103 104 105 106 107 108 109 110
a=rtpmap:97 SPEEX/16000
a=rtpmap:8 PCMA/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:102 mpeg4-generic/48000/1
a=rtpmap:103 mpeg4-generic/44100/1
a=rtpmap:104 mpeg4-generic/32000/1
a=rtpmap:105 mpeg4-generic/24000/1
a=rtpmap:106 mpeg4-generic/22050/1
a=rtpmap:107 mpeg4-generic/16000/1
a=rtpmap:108 mpeg4-generic/12000/1
a=rtpmap:109 mpeg4-generic/11025/1
a=rtpmap:110 mpeg4-generic/8000/1
a=sendonly |
11. Stream recording will start only when at least one key frame is received
Symptoms: recording file is not created when starting recording (via REST API for example), there are no errors in server logs, the stream is already playing in browsers (until viewers reconnect)
Solution: provide periodic key frames sending for WebRTC streams with the following parameter
Code Block | ||
---|---|---|
| ||
periodic_fir_request=true |
for RTMP streams with appropriate encoder settings
12. VLC player skips gaps when playing a stream pulled from multirecording in MKV container
Symptoms: gaps in a pulled stream are skipped when playing it in VLC
Solution: use the following parameter
Code Block | ||
---|---|---|
| ||
multi_recorder_mkv_fill_gaps=true |
13. It is recommended to provide a regular keyframes sending when WebRTC stream is recording to singe MKV file and to MKV multirecorder simultaneously
Symptoms: VLC jumps to time of stream addition to multirecorder when playing a single stream MKV recording
Solution: provide periodic key frames sending for WebRTC streams with the following parameter
Code Block | ||
---|---|---|
| ||
periodic_fir_request=true |
14. G722 audio can be a shortly distorted when pulling a stream file from MKV multirecording
Symptoms: If the stream with G722 audio was removed and then added again to the multirecorder, a short audio distortions can be heard when the stream was added to multirecorder
Solution: use Opus audio codec
15. PCMA audio may finish earlier than video after stream removing from multirecorder when pulling a stream file from MKV multirecording
Symptoms: if the stream with PCMA audio was removed from multirecorder, audio finishes earlier than video when playing the pulled stream file
Solution: use Opus audio codec