...
Code Block | ||
---|---|---|
| ||
rtp_activity_video=false |
If Chrome browser sends empty video due to web camera conflict
Some Chrome versions does not return an error if web camera is busy, but publish a stream with empty video (black screen). In this case, stream publishing can be stopped by two ways: using JavaScript and HTML5 on client, or using server settings.
Stopping a stream with empty video on client side
Videotrack that Chrome browsers creates for busy web camera, stops after no more than one second publishing, then stream is send without a videotrack. In this case videotrack state (readyState
variable) changes to ended
, and corresponding onended
event is generated that can be catched by web application. To use this event:
1. Add to web application script the registartion function for onended event handler, in which stream pub;ishing is stopped with stream.stop()
...
language | js |
---|---|
theme | RDark |
...
RTP activity can be checked for publishing streams only, not for playing streams.
Disable tracks activity checking by stream name
Since build 5.2.1784 it is possible to disable video and audio tracks activity checking for the streams with names matching a regular expression
Code Block | ||
---|---|---|
| ||
rtp_activity_audio_exclude=stream1
rtp_activity_video_exclude=stream1 |
The feature may be useful for streams in which a media traffic can stop for a long time, for example, screen sharing streams from an application window
Code Block | ||
---|---|---|
| ||
rtp_activity_audio_exclude=.*-screen$
rtp_activity_video_exclude=.*-screen$ |
In this case tracks activity checking will not be applied to the tracks named like conference-123-user-456-screen
If Chrome browser sends empty video due to web camera conflict
Some Chrome versions does not return an error if web camera is busy, but publish a stream with empty video (black screen). In this case, stream publishing can be stopped by two ways: using JavaScript and HTML5 on client, or using server settings.
Stopping a stream with empty video on client side
Videotrack that Chrome browsers creates for busy web camera, stops after no more than one second publishing, then stream is send without a videotrack. In this case videotrack state (readyState
variable) changes to ended
, and corresponding onended
event is generated that can be catched by web application. To use this event:
1. Add to web application script the registartion function for onended event handler, in which stream pub;ishing is stopped with stream.stop()
Code Block | ||||
---|---|---|---|---|
| ||||
function addVideoTrackEndedListener(localVideo, stream) {
var videoTrack = extractVideoTrack(localVideo);
if (videoTrack && videoTrack.readyState == 'ended') {
console.error("Video source error. Disconnect...");
stream.stop();
} else if (videoTrack) {
videoTrack.onended = function (event) {
console.error("Video source error. Disconnect...");
stream.stop();
};
}
} |
2. Add function to remove event handler when stream is stopped
Code Block | ||||
---|---|---|---|---|
| ||||
function removeVideoTrackEndedListener(localVideo) {
var videoTrack = extractVideoTrack(localVideo);
if(videoTrack) {
videoTrack.onended = null;
}
} |
3. Add function to extract videotrack
Code Block | ||||
---|---|---|---|---|
| ||||
function extractVideoTrack(localVideo) {
return localVideo.firstChild.srcObject.getVideoTracks()[0];
} |
4. Register event handler when publishing a stream
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({ name: streamName, display: localVideo, ... }).on(STREAM_STATUS.PUBLISHING, function (stream) { var videoTrack = extractVideoTrackaddVideoTrackEndedListener(localVideo, stream); if (videoTrack && videoTrack.readyState == 'ended') { setStatus("#publishStatus", STREAM_STATUS.PUBLISHING); console.error("Video source error. Disconnect..." onPublishing(stream); stream.stop(); } else if (videoTrack ... }).publish(); |
5. Remove event handler when stopping a stream
Code Block | ||||
---|---|---|---|---|
| ||||
function onPublishing(stream) { $("#publishBtn").text("Stop").off('click').click(function () { videoTrack.onended = function (event) {$(this).prop('disabled', true); console.error("Video source error. Disconnect..."removeVideoTrackEndedListener(localVideo); stream.stop(); }}).prop('disabled', false); } } |
2. Add function to remove event handler when stream is stopped
Code Block | ||||
---|---|---|---|---|
| ||||
function removeVideoTrackEndedListener(localVideo) {
var videoTrack = extractVideoTrack(localVideo);
if(videoTrack) {
videoTrack.onended = null;
}
} |
...
$("#publishInfo").text("");
} |
Videotrack activity checking on server side
Videotrack activity checking for streams published on server is enabled with the following parameters in flashphoner.properties file
Code Block | ||
---|---|---|
| ||
rtp_activity_video=true |
In this case, if there is no video in stream, its publishing will be stopped after 60 seconds.
Video only stream publishing with constraints
In some cases, video only stream should be published while microphone is busy, for example video is published while voice phone call. To prevent browser access request to microphone, set the constraints for video on;ly publishing:
Code Block | ||||
---|---|---|---|---|
| ||||
function extractVideoTrack(localVideo) session.createStream({ return localVideo.firstChild.srcObject.getVideoTracks()[0]; } |
4. Register event handler when publishing a stream
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({name: streamName, namedisplay: streamNamelocalVideo, displayconstraints: {video: localVideo,true, audio: false} ... }).on(STREAM_STATUS.PUBLISHING, function (stream) publish(); |
Audio only stream publishing
In most cases, it is enough to set the constraints to publish audio only stream:
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({ addVideoTrackEndedListener(localVideo, stream); name: streamName, setStatus("#publishStatus", STREAM_STATUS.PUBLISHING);display: localVideo, onPublishing(stream);constraints: {video: false, audio: true} ... }).publish(); |
...
Audio only stream publishing in Safari browser
When audio only stream is published from iOS Safari browser with constraints, browser does not send audio packets. To workaround this, a stream should be published with video, then video should be muted:
Code Block | ||||
---|---|---|---|---|
| ||||
function onPublishing(stream) { $("#publishBtn").text("Stop").off('click').click(function () { session.createStream({ name: streamName, display: localVideo, constraints: $(this).prop('disabled', true); removeVideoTrackEndedListener(localVideo);{video: true, audio: true} ... }).on(STREAM_STATUS.PUBLISHING, function (stream) { stream.stopmuteVideo(); }).prop('disabled', false); ... $("#publishInfo"}).textpublish(""); } |
Videotrack activity checking on server side
Videotrack activity checking for streams published on server is enabled with the following parameters in flashphoner.properties file
Code Block | ||
---|---|---|
| ||
rtp_activity_video=true |
In this case, iOS Safari browser will send emply video packets (blank screen) and audio packets.
Disable resolution constraints normalization in Safari browser
By default, WebSDK normalizes stream publishing resolution constraints set in Safari browser. In this case, if there is no video in stream, its publishing will be stopped after 60 seconds.
Video only stream publishing with constraints
In some cases, video only stream should be published while microphone is busy, for example video is published while voice phone call. To prevent browser access request to microphone, set the constraints for video on;ly publishingwidth or height is not set, or equal to 0, then the picture resolution is forced to 320x240 or 640x480. Since WebSDK build 0.5.28.2753.109 (hash 149855cc050bf7512817104fd0104e9cce760ac4), it is possible to disable normalization and pass resolution constarints to the browser as is. for example:
Code Block | ||||
---|---|---|---|---|
| ||||
publishStream = session.createStream({ ... namedisableConstraintsNormalization: streamNametrue, constraints { video: { display width: {ideal: localVideo1024}, constraints height: {video: true, audio: false} ... }).publish(); |
Audio only stream publishing
In most cases, it is enough to set the constraints to publish audio only stream:
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({
name: streamName,
display: localVideo,
constraints: {video: false, audio: true}
...
}).publish(); |
Audio only stream publishing in Safari browser
...
ideal: 768}
},
audio: true
}
}).on(STREAM_STATUS.PUBLISHING, function (publishStream) {
...
});
publishStream.publish(); |
H264 encoding profiles exclusion
Since build 5.2.620 some H264 encoding profiles can be excluded from remote SDP which is sent by server to browser. The profiles to exckude should be listed in the following parameter
Code Block | ||
---|---|---|
| ||
webrtc_sdp_h264_exclude_profiles=4d,64 |
In this case, Main (4d) and High (64) profiles will be excluded, but Baseline (42) still remains:
Code Block | ||
---|---|---|
| ||
a=rtpmap:102 H264/90000
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:125 H264/90000
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:127 H264/90000
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:108 H264/90000
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f |
This setting may be useful if some browser encodes B-frames for example using high profiles while hardware acceleration is enabled.
By default, no profiles will be excluded from SDP if they are supported by browser
Code Block | ||
---|---|---|
| ||
webrtc_sdp_h264_exclude_profiles= |
Content type management while publishing from Chromium based browser
In some cases, most browsers based on Chromium 91 agressively assess a publishing channel quality and drop publishing resolution lower than set in constraints, even if channel is enough to publish 720p or 1080p stream. To workaround this behaviour, since WebSDK build 2.0.180 videoContentHint option was added:
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({ name: streamName, display: localVideo, constraints: {videocacheLocalResources: true, audio: true} receiveVideo: false, ... }).on(STREAM_STATUS.PUBLISHING, function (stream) { receiveAudio: false, stream.muteVideo();videoContentHint: "detail" ... }).publish(); |
In this case, iOS Safari browser will send emply video packets (blank screen) and audio packets.
Disable resolution constraints normalization in Safari browser
...
In WebSDK builds before 2.0.242 this option is set to detail
by default and forces browsers to keep the publishing resolution as set in constraints. However, browser can drop FPS in this case when publishing stream from som USB web cameras. If FPS should be kept mo matter to resolution, the option should be set to motion
Code Block | ||||
---|---|---|---|---|
| ||||
publishStream = session.createStream({ ... disableConstraintsNormalizationname: truestreamName, constraints { videodisplay: {localVideo, widthcacheLocalResources: {ideal: 1024}true, heightreceiveVideo: {ideal: 768} false, receiveAudio: }false, audio: true videoContentHint: } }).on(STREAM_STATUS.PUBLISHING, function (publishStream) { "motion" ... }); publishStream.publish(); |
...
Since build 5.2.620 some H264 encoding profiles can be excluded from remote SDP which is sent by server to browser. The profiles to exckude should be listed in the following parameter
Code Block | ||
---|---|---|
| ||
webrtc_sdp_h264_exclude_profiles=4d,64 |
In this case, Main (4d) and High (64) profiles will be excluded, but Baseline (42) still remains:
Code Block | ||
---|---|---|
| ||
a=rtpmap:102 H264/90000
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:125 H264/90000
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:127 H264/90000
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:108 H264/90000
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f |
This setting may be useful if some browser encodes B-frames for example using high profiles while hardware acceleration is enabled.
By default, no profiles will be excluded from SDP if they are supported by browser
Code Block | ||
---|---|---|
| ||
webrtc_sdp_h264_exclude_profiles= |
Content type management while publishing from Chromium based browser
In some cases, most browsers based on Chromium 91 agressively assess a publishing channel quality and drop publishing resolution lower than set in constraints, even if channel is enough to publish 720p or 1080p stream. To workaround this behaviour, since WebSDK build 2.0.180 videoContentHint option was added:WebSDK build 2.0.242, videoContentHint
is set to motion
by default. The detail
or text
values should be set only for screen sharing streaming in browser.
Since WebSDK build 2.0.204 videoContentHint
selection is available in Media Device example
FPS management in Firefox browser
By default, Firefox is publishing video with maximum FPS shown by web camera driver fo requested resolution. This value is 30 FPS for most of modern web cameras. Publishing FPS can be defined more strictly if necessary. To do this, disable constraints normalization:
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({
...
disableConstraintsNormalization: true,
constraints: {
video: {
width: 640,
height: 360,
frameRate: { max: 15 }
},
audio: true
}
}).on(STREAM_STATUS.PUBLISHING, function (publishStream) {
...
}).publish(); |
Note that Foirefox can exclude the camera from the list while requesting camera and microphone access if camera driver does not provide a required combination of resolution and FPS. Also, Firefox can change a publishing resolution if there is only one resolution with required FPS in camera driver response.
Stereo audio publishing in browser
Audio bitrate should be more than 60000 bps to publish stereo in Opus codec from browser. This can be done by setting Opus codec parameters on client side
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({ name: streamName, display: remoteVideo, displayconstraints: localVideo,{ cacheLocalResourcesaudio: true,{ receiveVideo bitrate: false,64000 receiveAudio: false}, videoContentHint: "detail"... } ... }).publish(); |
By default, this option is set to detail
and forces browsers to keep the publishing resolution as set in constraints. However, browser can drop FPS in this case when publishing stream from som USB web cameras. If FPS should be kept mo matter to resolution, the option should be set to motion
or on server side
Code Block | ||
---|---|---|
| ||
opus_formats = maxaveragebitrate=64000;stereo=1;sprop-stereo=1; |
In this case, Firefox browser publishes stereo audio without additional setup.
Stereo audio publishing in Chrome based browsers
A certain client setup is required to publish stereo audio from Chrome. Thre is two ways to set this up depending on client implementation
Using Web SDK
If Web SDK is used in project, it is necessary to set the fiollowing constraint option:
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({ name: streamName, display: localVideo, display constraints: localVideo,{ cacheLocalResourcesaudio: true,{ receiveVideo stereo: false,true receiveAudio: false}, videoContentHint: "motion"... } ... }).publish(); |
Since WebSDK build 2.0.204 videoContentHint
selection is available in Media Device example
FPS management in Firefox browser
...
Using Websocket API
If only Websocket API is used in project, it is necessary to disable echo cancellation
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({ var constraints = { audio: { echoCancellation: false, googEchoCancellation: false }, ... disableConstraintsNormalization: true,}; constraints: {... navigator.getUserMedia(constraints, video:function (stream) { ... width: 640, height: 360, frameRate: { max: 15 } }}, reject); |
If echo cancellation is enabled, Chrome will publish mono audio even if stereo is set in Opus codec options.
How to bypass an encrypted UDP traffic blocking
Sometimes an encrypted UDP mediatraffic may be blocked by ISP. In this case, WebRTC stream publishing over UDP will fail with Failed by RTP activity
error. To bypass this, it is recommended to use TCP transport at client side
Code Block | ||||
---|---|---|---|---|
| ||||
session.createStream({ name: streamName, audiodisplay: truelocalVideo, } }).on(STREAM_STATUS.PUBLISHING, function (publishStream) {transport: "TCP" ... }).publish(); |
Note that Foirefox can exclude the camera from the list while requesting camera and microphone access if camera driver does not provide a required combination of resolution and FPS. Also, Firefox can change a publishing resolution if there is only one resolution with required FPS in camera driver responseAnother option is to use external or internal TURN server or publish a stream via RTMP or RTSP.
Known issues
1. If the web app is inside an iframe element, publishing of the video stream may fail.
...
17. iOS Safari 15.1 requires from another side to enable image orientation extension when publishing H264 stream
Symptoms: webpage crashe crashes in iOS Safari 15.1 when stream publishing is started (Webkit bugs https://bugs.webkit.org/show_bug.cgi?id=232381 and https://bugs.webkit.org/show_bug.cgi?id=231505)
...