...

 profile3:
  audio:
    codec : opus
    rate : 48000
  video:
    height : 180

Transcoder node tuning to work under high load

The server CPU load increases while a large number of streams are transcoded. If the server CPU does not manage to process all the streams frames, frame encoding queues take a much of server RAM (not JVM heap!). As a result, memory leak may occur and, therefore, server can stop working depending on memory allocation libraries used by server operating system.

To prevent this, the following tune settings are recommended to be used together:

• suspend incoming streams decoding when encoding queues are full
• use jemalloc memory allocation library

Incoming streams decoding suspending

Incoming streams decoding suspending allows to smooth (but not fully prevent) peak server loads, if this is set for transcoder group. The feature is enabled with the following parameter

streaming_video_decoder_wait_for_distributors=true

Encoding queue size to suspend streams decoding is set in frames with the following parameter

streaming_video_decoder_wait_for_distributors_max_queue_size=3

Timeout to wait for queue freeing is set in milliseconds

streaming_video_decoder_wait_for_distributors_timeout=33

Using jemalloc memory allocation library

By default, glibc memory allocation library is used in most cases. To prevent memory leaks under high load it is recommended to set Transcoder nodes to use jemalloc library as follows (Centos 7 for example):

1. Install build prerequisites

yum -y install autoconf libtool pkg-config g++ make cmake bzip

2. Download library source code

wget https://github.com/jemalloc/jemalloc/releases/download/5.2.1/jemalloc-5.2.1.tar.bz2

3. Unpack the archive

tar -xvjf jemalloc-5.2.1.tar.bz2

4. Build the library

cd jemalloc-5.2.1
./configure && make && make install

5. Comment the following line in /usr/local/FlashphonerWebCallServer/bin/setenv.sh file

export MALLOC_ARENA_MAX=4

and add the following line

export LD_PRELOAD=/usr/local/lib/libjemalloc.so

6. Switch decoder to OpenH264 in flashphoner.properties file

decoder_priority=OPENH264,FF

7. Restart WCS.

Choosing a route for stream playback

...