Performance of RHEL for Databases on Microsoft Azure Cloud

At Red Hat, we strive to ensure that Red Hat Enterprise Linux (RHEL) performs its best regardless of where you choose to deploy it. In this post, we're going to discuss how to improve the performance of various benchmarks and database workloads in Microsoft Azure. In this blog, we will compare the performance database workloads on RHEL 8 and RHEL 7 running in Azure. We will also share some simple tunings for RHEL images being run in Azure to improve performance. 

CPU Performance Improvements

With RHEL 8.0 comes many upgrades and improvements, including a new kernel and compiler suite. These upgrades have shown performance improvements for many types of applications. While some applications are more sensitive to compiler improvements than others, the STREAM Copy Performance benchmark provides an example of an almost 15% improvement:

Network Performance Improvements

With the newer kernel, RHEL 8.0 introduces a number of networking improvements compared to RHEL 7.6. When combined with the Azure Accelerated Networking offering, the performance improvements are significant in both delivered bandwidth and packet rate. The performance gains differ based on the packet size. 

The 1st chart shows results at 16KByte packets. 16 KByte packets reach saturation of the 40 Gb NIC at 8/64 instance of uperf at 1500 MTU. The bottom chart is for tiny 64 Bytes and the metric is in transactions/sec. Thus since the RHEL 8 kernel is more efficient at network packet processing, we measured higher network throughput at 8 and 64 instances at 64 bytes.

Large packet bandwidth shows a significant improvement, with RHEL 8.0 able to fully utilize a network link with greater efficiency:

Small packet transaction rate also shows great improvement with an up to 50% increase:

As a note, we measure transaction rate with small packets to minimize the time spent in moving bits over the wire rather than the time spent processing the transaction. Larger packets give us a better view of what we can send over the wire and hence why we measure bandwidth in that scenario.

The Impact on Database Performance

The low level improvements we’ve just reviewed are important to note as they can and will be used by databases and other applications. In this section, we have evaluated the performance differences between RHEL 8 and RHEL 7 running database workloads in Azure. 

• We observed up to 133% gains in RHEL 8 compared to RHEL 7 for SCSI storage due to the introduction of the SCSI multi-queue storage elevator.

• In case of NVMe storage, multi-queue is available for both RHEL 7 and RHEL 8. RHEL 8 performed 10% better, due to filesystem and locking improvements in the RHEL 8 kernel.

The results below use the open source HammerDB benchmarking and load testing software driving Microsoft SQL Server. The following table shows the details for our setup:

MariaDB SCSi (Intel Broadwell) results with 8-SSDs showing mulit-q now enabled in RHEL8 VMs. On a different tier in Azure, results of MariaBD NvME (AMD) with 4-NvME shows a smaller performance gain due to filesystem improvement since multiq is enabled for NvME for both RHEL 7.6 and RHEL 8.0.

Other Performance Tunings 

For this exercise, we took a 4 virtual CPU 16G ram instance and looked at AVC Cache Thresholds and Dirty Ratios. 

We also investigated how much of an impact the selected VM size has on performance.

AVC Cache

By default, the SELinux AVC Cache Threshold (/sys/fs/selinux/avc/cache_threshold) is set to 512. This sets the number of cache entries that SELinux can have to a maximum of 512.

• In the case of MariaDB on RHEL 7.6, when we adjusted this value to 8,192, we saw an almost 10% performance increase

• When running the same test across 5 different VMs with  MSSQL-Server (using the HammerDB Benchmark ), the performance difference between 512 and 8192 for this setting was minimal.

• Openshift Container Platform has been running with a value of 8192 and has found a solid performance improvement for their workloads.

For database workloads on cloud platforms, we recommend updating the value of /sys/fs/selinux/avc/cache_tuning to 8192. We would also recommend upgrading to a higher tier of storage before switching to a larger VM size as the larger VMs are generally underutilized with the lower tier storage.

Conclusion

Red Hat Enterprise Linux 8.0 (RHEL 8) is a release that brings forth optimizations in the kernel. The RHEL 8 kernel reduces system-call related overhead which improves I/O performance for both the filesystem and network subsystems. 

The SCSI multi-queue I/O elevator enables Microsoft SQLserver OLTP workloads to improve by a factor of two, when compared to the single-queue I/O elevator shipped in RHEL 7. Changes to XFS filesystem journaling has enabled improvements of up to 10% in RHEL 8 as well. 

The networking stack for RHEL 8 has measured performance gains between 15-20%, primarily  due to including nftables (instead of iptables). RHEL 8 also shows better performance with small and medium-sized packets when compared to RHEL 7. 

Red Hat continues to work on optimizing the performance of Red Hat Enterprise Linux in order to deliver the best performance for your workloads!