Managing camera and microphone¶

WCS allows configuring the camera and the microphone from a browser. Let's see how this can be done and what parameters you can adjust when an audio and video stream is captured. We use the Media Devices web application as an example:

media_device_manager.html

manager.js

Microphone settings¶

1. Selecting the microphone from the list¶

code:

Flashphoner.getMediaDevices(null, true, MEDIA_DEVICE_KIND.INPUT).then(function (list) {
    list.audio.forEach(function (device) {
        ...
    });
    ...
}).catch(function (error) {
    $("#notifyFlash").text("Failed to get media devices");
});

2. Microphone switching while stream is publishing¶

code:

$("#switchMicBtn").click(function (){
    stream.switchMic().then(function(id) {
        $('#audioInput option:selected').prop('selected', false);
        $("#audioInput option[value='"+ id +"']").prop('selected', true);
    }).catch(function(e) {
        console.log("Error " + e);
    });
}).prop('disabled', !($('#sendAudio').is(':checked')));

3. Adjusting microphone gain¶

Attention

The feature works in Chromium-based browsers only

code:

$("#micGainControl").slider({
    range: "min",
    min: 0,
    max: 100,
    value: currentGainValue,
    step: 10,
    animate: true,
    slide: function (event, ui) {
        currentGainValue = ui.value;
        if(previewStream) {
            publishStream.setMicrophoneGain(currentGainValue);
        }
    }
});

4. Enabling error correction¶

Attention

This works for the Opus codec only

code:

if (constraints.audio) {
    constraints.audio = {
        deviceId: $('#audioInput').val()
    };
    if ($("#fec").is(':checked'))
        constraints.audio.fec = $("#fec").is(':checked');
    ...
}

5. Setting stereo/mono mode.¶

code:

if (constraints.audio) {
    constraints.audio = {
        deviceId: $('#audioInput').val()
    };
    ...
    if ($("#sendStereoAudio").is(':checked'))
        constraints.audio.stereo = $("#sendStereoAudio").is(':checked');
    ...
}

6. Setting audio bitrate¶

Attention

The value must be set in bits per second, for example 64000 for 64 kbps

code:

if (constraints.audio) {
    constraints.audio = {
        deviceId: $('#audioInput').val()
    };
    ...
    if (parseInt($('#sendAudioBitrate').val()) > 0)
        constraints.audio.bitrate = parseInt($('#sendAudioBitrate').val());
}

7. Turning off the microphone (mute).¶

code:

if ($("#muteAudioToggle").is(":checked")) {
    muteAudio();
}

Camera settings¶

1. Camera selection¶

code:

Flashphoner.getMediaDevices(null, true, MEDIA_DEVICE_KIND.INPUT).then(function (list) {
    ...
    list.video.forEach(function (device) {
        ...
    });
}).catch(function (error) {
    $("#notifyFlash").text("Failed to get media devices");
});

If audio devices access should not be requested while choosing a camera, getMediaDevices() function should be called with explicit constraints setting

Flashphoner.getMediaDevices(null, true, MEDIA_DEVICE_KIND.INPUT, {video: true, audio: false}).then(function (list) {
    ...
    list.video.forEach(function (device) {
        ...
    });
}).catch(function (error) {
    $("#notifyFlash").text("Failed to get media devices");
});

2. Switching cameras while stream is publishing¶

code:

$("#switchBtn").text("Switch").off('click').click(function () {
    stream.switchCam().then(function(id) {
        $('#videoInput option:selected').prop('selected', false);
        $("#videoInput option[value='"+ id +"']").prop('selected', true);
    }).catch(function(e) {
        console.log("Error " + e);
    });
});

Switching of the camera can be done "on the fly" during stream broadcasting. Here is how switching works:

On PC cameras switch in the order they are defined in the device manager of the operating system.
On Android, if Chrome is used, the default is the front camera. If Firefox is used, the default is the rear camera.
On iOS in the Safari browser, by default the front camera is selected, but in the drop-down the rear camera is the first.

3. Setting the video resolution¶

code:

constraints.video = {
    deviceId: $('#videoInput').val(),
    width: parseInt($('#sendWidth').val()),
    height: parseInt($('#sendHeight').val())
};
if (Browser.isSafariWebRTC() && Browser.isiOS() && Flashphoner.getMediaProviders()[0] === "WebRTC") {
    constraints.video.deviceId = {exact: $('#videoInput').val()};
}

4. Setting FPS¶

code:

if (constraints.video) {
    ...
    if (parseInt($('#fps').val()) > 0)
         constraints.video.frameRate = parseInt($('#fps').val());
}

5.Setting video bitrate¶

Attention

The bitrate values must be set in kbps

code:

if (constraints.video) {
    ...
    if (parseInt($('#sendVideoMinBitrate').val()) > 0)
        constraints.video.minBitrate = parseInt($('#sendVideoMinBitrate').val());
    if (parseInt($('#sendVideoMaxBitrate').val()) > 0)
        constraints.video.maxBitrate = parseInt($('#sendVideoMaxBitrate').val());
    ...
}

6. Setting CPU Overuse Detection¶

code:

if (!$("#cpuOveruseDetection").is(':checked')) {
    mediaConnectionConstraints = {
        "mandatory": {
            googCpuOveruseDetection: false
        }
    }
}

7. Turning off the camera (mute)¶

code:

if ($("#muteVideoToggle").is(":checked")) {
    muteVideo();
}

Testing camera and microhpone capturing locally¶

Local camera and microphone test is intended to check capturing in browser without publishing stream to server.

code:

function startTest() {
    Flashphoner.getMediaAccess(getConstraints(), localVideo).then(function (disp) {
        $("#testBtn").text("Release").off('click').click(function () {
            $(this).prop('disabled', true);
            stopTest();
        }).prop('disabled', false);

        window.AudioContext = window.AudioContext || window.webkitAudioContext;
        if (Flashphoner.getMediaProviders()[0] == "WebRTC" && window.AudioContext) {
            for (i = 0; i < localVideo.children.length; i++) {
                if (localVideo.children[i] && localVideo.children[i].id.indexOf("-LOCAL_CACHED_VIDEO") != -1) {
                    var stream = localVideo.children[i].srcObject;
                    audioContextForTest = new AudioContext();
                    var microphone = audioContextForTest.createMediaStreamSource(stream);
                    var javascriptNode = audioContextForTest.createScriptProcessor(1024, 1, 1);
                    microphone.connect(javascriptNode);
                    javascriptNode.connect(audioContextForTest.destination);
                    javascriptNode.onaudioprocess = function (event) {
                        var inpt_L = event.inputBuffer.getChannelData(0);
                        var sum_L = 0.0;
                        for (var i = 0; i < inpt_L.length; ++i) {
                            sum_L += inpt_L[i] * inpt_L[i];
                        }
                        $("#micLevel").text(Math.floor(Math.sqrt(sum_L / inpt_L.length) * 100));
                    }
                }
            }
        } else if (Flashphoner.getMediaProviders()[0] == "Flash") {
            micLevelInterval = setInterval(function () {
                $("#micLevel").text(disp.children[0].getMicrophoneLevel());
            }, 500);
        }
        testStarted = true;
    }).catch(function (error) {
        $("#testBtn").prop('disabled', false);
        testStarted = false;
    });
}

SDP parameters replacing¶

When publishing stream, an SDP parameters may be replaced if needed. In SDP replace field string template is set for search for the parameter to replace, and in 'with' field new parameter value is set.

To replace SDP parameters, a callback function is used that should be set on stream creation in sdpHook option of createStream() method:

stream creation code

publishStream = session.createStream({
    name: streamName,
    display: localVideo,
    cacheLocalResources: true,
    constraints: constraints,
    mediaConnectionConstraints: mediaConnectionConstraints,
    sdpHook: rewriteSdp,
    ...
})

rewriteSdp function code

function rewriteSdp(sdp) {
    var sdpStringFind = $("#sdpStringFind").val().replace('\\r\\n','\r\n');
    var sdpStringReplace = $("#sdpStringReplace").val().replace('\\r\\n','\r\n');
    if (sdpStringFind != 0 && sdpStringReplace != 0) {
        var newSDP = sdp.sdpString.toString();
        newSDP = newSDP.replace(new RegExp(sdpStringFind,"g"), sdpStringReplace);
        return newSDP;
    }
    return sdp.sdpString;
}

Rising up the bitrate of video stream published in Chrome browser¶

SDP parameters replacement allows to rise video streeam published bitrate. To do this, SDP parameter a=ftmp must be replaced by this template when publishing H264 stream:

a=fmtp:(.*) (.*)

to

a=fmtp:$1 $2;x-google-min-bitrate=2500

where 2500 is the bitrate in kilobits per second.

Similarly, video bitrate on start can be set (x-google-start-bitrate attribute) and maximum bitrate can be limited (x-google-max-bitrate attribute). Note that if minimum bitrate only is set, then resulting bitrate cannot be above 2500 kbps, probably maximum bitrate value is fixed on this level by default in Chrome browser. When higher bitrate values are required, for example, to publish high resolution streams, both minimum and maximum values must be explicitly set:

a=fmtp:$1 $2;x-google-max-bitrate=7000;x-google-min-bitrate=3000

In this case browser will try to keep bitrate in limits from 3000 to 7000 kbps when publishing a stream.

When publishing VP8 stream, SDP parameter a=rtpmap must be replaced by this template

a=rtpmap:(.*) VP8/90000\r\n

to

a=rtpmap:$1 VP8/90000\r\na=fmtp:$1 x-google-min-bitrate=3000;x-google-max-bitrate=7000\r\n

This feature is available in Chrome browser only.

Bandwidth management¶

SDP parameters replacement allows to set bandwidth for stream publishhed. To do this, SDP parameter c=IN must be replaced by this template when publishing stream

c=IN (.*)\r\n

to

c=IN $1\r\nb=AS:10000\r\n

Setting up codecs¶

When publishing the stream, some codecs which should not be used to publish the given stream may be stripped from WebRTC SDP, for example:

publishStream = session.createStream({
    ...
    stripCodecs: "h264,H264,flv,mpv"]
}).on(STREAM_STATUS.PUBLISHING, function (publishStream) {
    ...
});
publishStream.publish();

It may be useful when you need to find some workaround for bugs of a browser or if it conflicts with the given codec. For example, if H264 does not work in a browser, you can turn it off and switch to VP8 when publishing WebRTC stream.

Sound device selection¶

Sound output device can be selected (and switched "on the fly") while stream is playing in Chromium based browsers.

code:

Flashphoner.getMediaDevices(null, true, MEDIA_DEVICE_KIND.OUTPUT).then(function (list) {
    list.audio.forEach(function (device) {
        ...
    });
}).catch(function (error) {
    $('#audioOutputForm').remove();
});

Note that Firefox and Safari browsers always return empty output devices list, therefore sound device selection is not available for these browsers

WebRTC statistics displaying¶

A client application can get WebRTC statistics according to the standard while publishing or playing stream. The statistics can be displayed in browser, for example:

Statistics displaying while stream is publishing
Stream.getStats()
```
                            
```

Managing camera and microphone¶

Microphone settings¶

1. Selecting the microphone from the list¶

2. Microphone switching while stream is publishing¶

3. Adjusting microphone gain¶

4. Enabling error correction¶

5. Setting stereo/mono mode.¶

6. Setting audio bitrate¶

7. Turning off the microphone (mute).¶

Camera settings¶

1. Camera selection¶

2. Switching cameras while stream is publishing¶

3. Setting the video resolution¶

4. Setting FPS¶

5.Setting video bitrate¶

6. Setting CPU Overuse Detection¶

7. Turning off the camera (mute)¶

Testing camera and microhpone capturing locally¶

SDP parameters replacing¶

Rising up the bitrate of video stream published in Chrome browser¶

Bandwidth management¶

Setting up codecs¶

Sound device selection¶

WebRTC statistics displaying¶

Picture parameters management¶

Picture resolution management¶

Frame rate management¶

Seamless switching between web camera stream and screen share stream during publication¶

Known limits¶

Known issues¶

1. Microphone swithing does not work in Safari browser¶

2. iOS Safari freezes on playback when publisher changes a camera¶