피드 구독

This article was originally published on the Red Hat Customer Portal. The information may no longer be current.

A kernel bug announced on oss-security list claims to create a situation in which memory corruption can panic the system, by causing an integer used in determining the size of TCP send and receive buffers to be a negative value. Red Hat engineering sometimes backports security fixes and features from the current kernel, diverging the Red Hat Enterprise Linux kernel from upstream and causing some security issues to no longer apply. This blog post shows how to use live kernel debugging to determine if a system is at risk by this integer overflow flaw.

This walkthrough assumes that the reader has a Red Hat Enterprise Linux 7 guest system and basic knowledge of C programming.

Setting up the guest target to debug

The guest to be the target of the debugging session is a libvirt (or KVM/QEMU) style Virtual Machine. The guest virtual serial port should be mapped to the TCP port (TCP/1234) for use by GDB (the GNU Debugger).

Modifying the guest domain file

The virsh-edit command is intended to be a safe method of manipulating the raw XML that describes the guest, which is what we need to do in this circumstance. We need to configure the guest via the domain configuration file as there is no tickbox to enable what we need in virt-manager.

The first change is to set the XML namespace for QEMU, which sounds more complex than it is.

# virsh-edit your-virt-host-name

Find the domain directive and add the option xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'.

<domain type='kvm'
        xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' >

Add a new qemu:commandline tag inside domain which will allow us to pass a parameter to QEMU for this guest when starting.

<domain type='kvm'
       xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0' >
     <qemu:commandline>
          <qemu:arg value='-s'/>
     </qemu:commandline>

Save the file and exit the editor. Some versions of libvirt may complain that the XML has invalid attributes; ignore this and save the file anyway. The libvirtd daemon does not need to be restarted. The guest will need to be destroyed and restarted if it is already running.

The -s parameter is an abbreviation of -gdb tcp::1234. If you have many guests needing debugging on different ports, or already have a service running on port 1234 on the host, you can set the port in the domain XML file as shown below:

  <qemu:commandline>
        <qemu:arg value='-gdb'/>
        <qemu:arg value='tcp::1235'/>
  </qemu:commandline>

If it is working, the QEMU process on the host will be listening on the port specified as shown below:

[root@target]# netstat -tapn | grep 1234
tcp        0      0 0.0.0.0:1234            0.0.0.0:*               LISTEN      11950/qemu-system-x
Change /etc/default/grub on the guest

The guest kernel will need to be booted with new parameters to enable KGDB debugging facilities. Add the values kgdboc=ttyS0,115200. In the system shown here, a serial console is also running on ttyS0 with no adverse effects.

Use the helpful grubby utility to apply these changes across all kernels.

# grubby --update-kernel=ALL --args="console=ttyS0,115200 kgdboc=ttyS0,115200"
Downloading debuginfo packages

The Red Hat Enterprise Linux kernel packages do not include debug symbols, as the symbols are stripped from binary files at build time. GDB needs those debug symbols to assist programmers when debugging. For more information on debuginfo see this segment of the Red Hat Enteprise Linux 7 developer guide.

RPM packages containing the name 'debuginfo' contain files with symbols. These packages can be downloaded from Red Hat using yum or up2date.

To download these packages on the guest:

# debuginfo-install --downloadonly kernel-3.10.0-327.el7

This should download two files in the current directory on the host for later extraction and use by GDB.

Copy these files from the guest to the host for the host GDB to use them. I choose ~/kernel-debug/ as a sane location for these files. Create the directory if it doesn't already exist.

# mkdir -p ~/kernel-debug
# scp yourlogin@guest:kernel*.rpm ~/kernel-debug/

The final step on the guest is to reboot the target. At this point the system should reboot with no change in behavior.

Preparing the host to debug

The system which runs the debugger doesn't need to be the host that contains the guest. The debugger system must be capable of making a connection to the guest running on the specified port (1234). In this example these commands will be run on the host which contains the virtual machine.

Installing GDB

Install GDB on the host using a package manager.

# sudo yum -y install gdb
Extracting files to be used from RPMs

When Red Hat builds the kernel it strips debugging symbols from the RPMs. This creates smaller downloads and uses less memory when running. The stripped packages are the well-known RPM packages named like kernel-3.10.0-327.el7.x86_64.rpm. The non-stripped debug information is stored in debuginfo rpms, like the ones downloaded earlier in this document by using debuginfo-install. They must match the exact kernel version and architecture being debugged on the guest to be of any use.

The target does not need to match the host system architecture or release version. The example below can extract files from RPMs on any system.

# cd ~/kernel-debug
# rpm2cpio kernel-debuginfo-3.10.0-327.el7.x86_64.rpm | cpio -idmv
# rpm2cpio kernel-debuginfo-common-3.10.0-327.el7.x86_64.rpm | cpio -idmv

This extracts the files within the packages into the current working directory as they would be on the intended file system. No scripts or commands within the RPMs are ran. These files are not installed and the system package management tools will not manage them. This allows them to be used on other architectures, releases or distributions.

  • The unstripped kernel is the vmlinux file in ~/kernel-debug/usr/lib/debug/lib/modules/3.10.0-510.el7.x86_64/vmlinux
  • The kernel source is in the directory ~/kernel-debug/usr/src/debug/kernel-3.10.0-327.el7/linux-3.10.0-327.el7.x86_64/
Connecting to the target system from the remote system

Start GDB with the text user interface, passing as a parameter the path to the unstripped kernel binary (vmlinux) running on the target system.

# gdb -tui ~/kernel-debug/var/lib/kernel-3.10.0-327.el7/boot/vmlinux

<gdb prelude shows here>

GDB must be told where to find the target system. Type the following into the GDB session:

set architecture i386:x86-64:intel
target remote localhost:1234
dir ~/kernel-debug/usr/src/debug/kernel-3.10.0-327.el7/linux-3.10.0-327.el7.x86_64/

Commands entered at the (gdb) prompt can be saved in ~/.gdbinit to reduce repetitive entry.

At this point, if all goes well, the system should be connected to the remote GDB session.

The story so far...

Congratulations, you've made it this far. If you've been following along you should have setup a GDB session to a system running in libvirt and be able to recreate and begin investigation into flaws.

Join the dark side as next time when we validate an integer promotion comparison flaw.


저자 소개

UI_Icon-Red_Hat-Close-A-Black-RGB

채널별 검색

automation icon

오토메이션

기술, 팀, 인프라를 위한 IT 자동화 최신 동향

AI icon

인공지능

고객이 어디서나 AI 워크로드를 실행할 수 있도록 지원하는 플랫폼 업데이트

open hybrid cloud icon

오픈 하이브리드 클라우드

하이브리드 클라우드로 더욱 유연한 미래를 구축하는 방법을 알아보세요

security icon

보안

환경과 기술 전반에 걸쳐 리스크를 감소하는 방법에 대한 최신 정보

edge icon

엣지 컴퓨팅

엣지에서의 운영을 단순화하는 플랫폼 업데이트

Infrastructure icon

인프라

세계적으로 인정받은 기업용 Linux 플랫폼에 대한 최신 정보

application development icon

애플리케이션

복잡한 애플리케이션에 대한 솔루션 더 보기

Original series icon

오리지널 쇼

엔터프라이즈 기술 분야의 제작자와 리더가 전하는 흥미로운 스토리