...

-Xlog:gc*:/usr/local/FlashphonerWebCallServer/logs/gc-core-:time

5. Tune the wcsAdd the following setting to wcs-core.properties properties (for example, allocating 24G under memory heap):

# ZGC
-XX:+UnlockExperimentalVMOptions -XX:+UseZGC -Xms24g -Xmx24g

6. If hugepages is planning to use, add the following settings to wcs-core.properties:

• in JDK 12 or 14

-XX:+UseLargePages -XX:ZPath=/hugepages

• in JDK 15 and newer

 -XX:+UseLargePages -XX:ZPathAllocateHeapAt=/hugepages

6. Сonfigure ZGC Then configure hugepages according to the recommendations (the number of memory pages (2048K each) with a margin to the memory for heap (1,125 * 24G * 1024 / 2M)) and add the required parameters in the server startup (Centos example):

sudo mkdir /hugepages
sudo echo "echo 13824 >/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages" >>/etc/rc.local
sudo echo "mount -t hugetlbfs -o uid=0,mode=0757 nodev /hugepages" >>/etc/rc.local
sudo chmod +x /etc/rc.d/rc.local
sudo systemctl enable rc-local.service
sudo systemctl restart rc-local.service
sudo chmod o+w /hugepages

7.  After restarting the WCS, the gc-core.log log files show the periodic operation of the garbage collector. To understand the working model of Z Garbage Collector, you can see this presentation.

Physical memory allocation tuning on system level

When server is under a high load, there can be not enough a physical memory map areas which are avalable to a proccess in system by default. This can lead to JVM crash due to lack of native memory. In this case, crash log contains the following comment:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
...

To prevent such crashes, increase memory map areas count available to a procces with the following system parameter

sysctl -w vm.max_map_count=131072

and restart WCS.

Known issues

1. CPU load average is higher when ZGC is used, especially in JDK 15

Symptoms: CPU load average measured at system level (using htop for example) grows after update from JDK 12 or 14 to 15 if ZGC is used

Solution: use ZGC in JDK 12 or 14 only fro high loaded servers if GC pauses minimizing is required