WebRTC stream picture rotation¶

Overview¶

When WebRTC stream is published from mobile device, a device orientation can be changed. In this case, Android devices rotate picture itself, and iOS devices send picture orientation info in a stream

Orientation info	Rotation angle, degrees
0	0
1	90
2	180
3	270

A picture is rotated clockwise.

Publishing WebRTC stream with orientation info¶

To publish a WebRTC stream with orientation info, cvoExtension parameter should be passed to createStream() function

session.createStream({
    name: streamName,
    ...
    cvoExtension: true
}).publish();

Playing stream with orientation info as RTMP¶

Player side support is required to play a WebRTC stream with orientation info as RTMP. Thus, the following code should be added to Flash Streaming example:

public function onMetaData(info:Object):void {
    Logger.info("Video orientation " + info.orientation);
    revertVideoFarEndPos();
    if (info.orientation == 0) {
        videoFarEnd.rotation = 0;
    } else if (info.orientation == 1) {
        videoFarEnd.rotation = 90;
        videoFarEnd.x = videoFarEnd.x + videoFarEnd.width;
    } else if (info.orientation == 2) {
        videoFarEnd.rotation = 180;
        videoFarEnd.x = videoFarEnd.x + videoFarEnd.width;
        videoFarEnd.y = videoFarEnd.y + videoFarEnd.height;
    } else if (info.orientation == 3) {
        videoFarEnd.rotation = 270;
        videoFarEnd.y = videoFarEnd.y + videoFarEnd.height;
    }
    videoFarEndOrientation = info.orientation;
}

public function revertVideoFarEndPos():void {
    if (videoFarEndOrientation == 1) {
        videoFarEnd.x = videoFarEnd.x - videoFarEnd.width;
    } else if (videoFarEndOrientation == 2) {
        videoFarEnd.x = videoFarEnd.x - videoFarEnd.width;
        videoFarEnd.y = videoFarEnd.y - videoFarEnd.height;
    } else if (videoFarEndOrientation == 3) {
        videoFarEnd.y = videoFarEnd.y - videoFarEnd.height;
    }
}

If transcoding is used, for example if VP8+opus stream is published, it is necessary to set the following parameter in flashphoner.properties file for player to receive stream transcoded orientation info

video_filter_enable_rotate=true

If CDN is used, this parameter should be set on all CDN servers supposed to play RTMP stream from.

Mobile devices publishing issues¶

If mobile device screen rotation is enabled, a picture aspect ratio will change while device is turning from portrait to landscape and vice versa.

If mobile device screen rotation is disabled (for example, Lock Portait Orientation switch is set), a picture aspect ratio will remains the same`as stream was published, but picture will turn according to device current orientation.

Known issues¶

iOS Safari does not send picture orientation info when device is rotated to 180 degrees (upside down)
iOS 12 Safari stops sending video packets when device is rotated to 90 degrees right (from portrait to landscape)
A picture aspect ratio will change while device is turning from portrait to landscape and vice versa while publishing from Android device regardless of cvoExtension value