PXE booting with UEFI hardware continued

This article is part two of two in a short series. In case you missed it, be sure to read the first article, How to set up PXE boot for UEFI hardware.

In the first article, you created a functioning PXE server, a DHCP server configured for delivering IP addresses to PXE booted systems, and a TFTP server to deliver a bootable system and ISO images. This article continues the task of finishing your environment and includes some troubleshooting tips for when things go wrong.

HTTP setup

An HTTP server is needed to provide the new server with the Kickstart file as well as the OS to be installed. It is possible to provide these files over HTTPS, but for most new servers, for simplicity, HTTP is used.

Here are the steps to set up the HTTP server:

1- Install the HTTP server:

# yum install -y httpd

2- Mount the Red Hat Virtualization Host ISO image (if it is not mounted):

# mount -t iso9660 /path/iso-file.iso /mnt -o loop,ro

3- Copy the Squash image and ISO files to the target locations:

# mkdir /var/www/html/rhv4.3/{ISO,image}

# cp -a /mnt /var/www/html/rhv4.3/ISO

# cp /mnt/Packages/redhat-virtualization-host-image-update* /tmp

# rpm2cpio redhat-virtualization-host-image-update* | cpio -idmv

# cp /tmp/usr/share/redhat-virtualization-host-version_number_version.squashfs.img /var/www/html/rhv4.3/image

4- Copy the Kickstart file to the target location:

# mkdir /var/www/html/kickstarts/

# vi /var/www/html/kickstarts/ks.cfg 🡪 Create a new file called ks.cfg

5- Enable and start the httpd service:

#systemctl enable httpd; systemctl start httpd

Kickstart file

Here is sample Kickstart file which can be used to install the RHVH OS. Red Hat strongly recommends using the "Automatically configure portioning" option in the installation destination.

lang en_US
keyboard us
eula --agreed
rootpw Redhat --plaintext

#platform x86, AMD64, or Intel EM64T
reboot
text
bootloader --location=mbr --append="rhgb quiet crashkernel=auto"
zerombr
liveimg --url=http:// 192.168.1.10/rhv4.3/image/redhat-virtualization-host-version_number_version.squashfs.img
clearpart --drives=sda --all
ignoredisk --only-use=sda
autopart --type=thinp

%post --erroronfail
nodectl init
%end

Remarks:

1- The URL below is provided by Red Hat and can be used to build a customized Kickstart file:

[https://access.redhat.com/labs/kickstartconfig/](https://access.redhat.com/labs/kickstartconfig/)

2- In the %post section of the Kickstart file, Linux commands can be added to set the static IPs for the ovirtmgmt as well as the hostname.

Automation options

This procedure is suitable to install multiple servers, but it requires manual changes. It can be fully automated in different ways, however.

Automation is out of the scope of this article, but two different options for automation will be mentioned here on a high level:

Using Ansible to automate powering on the servers

Most of HW vendors now have their Ansible modules to retrieve different details regarding the servers, as well as to power them on and off, so to automate the OS installation, we need to create an Ansible playbook to do the following:

1. Play 1: Create Jinja2 template for the Kickstart file where the static IP and hostname are properly replaced in the post section, using a variable file which maps the MAC address of the PXE interface to the static IP and the hostname for every server,  then push the dynamically generated Kickstart file to the relevant location in HTTP server.
2. Play 2: Power on the needed server by using the MAC address of the PXE interface.
3. Play 3: Probe the newly installed server using the static IP prior to moving to another server to install.

Pros:

1. No manual activity is needed other than updating the variable file in case of new servers.

Cons:

1. This option requires integration with the hardware management API.
2. This option can install an OS on one server at a time (You can have only one kickstart file hosted in the http server at a time). 

Using Ansible to post configure the hosts after OS is installed

Here in the dhcpd.conf we hard code the IP address which will be assigned for every MAC address of a new server in the Provisioning Network. Next, we create an inventory file including these IPs, as well as creating a variable file containing the hostname and static IP address for every server which is mapped to its IP address in the provisioning network. Finally, create one Ansible play to post configure the server after the OS installed.

Pros:

1. No dependency on the Ansible modules from the hardware vendor.
2. Multiple servers can be configured in parallel.

Cons:

1. Not fully automated as the dhcpd.conf needs to be modified, as well as the variables for new servers, must be added. Additionally, the servers need to be powered on manually.

[ Need more on Ansible? Take a free technical overview course from Red Hat. Ansible Essentials: Simplicity in Automation Technical Overview ]

Firewalld and network setup

If there are firewalls between the PXE servers and Provisioning Networks, incoming TFTP and HTTP traffic from the Provisioning Networks towards the PXE servers must be permitted. ICMP traffic from the PXE servers towards the Provisioning Networks (new servers must be pingable from the DHCP server) must also be permitted.

Incoming SSH traffic from the Provisioning Networks towards the PXE servers has to be enabled for the sake of troubleshooting, as will be explained in the following section.

Furthermore, bonding the PXE interface on the new servers complicates the setup, and it is not needed.

From the PXE servers DHCP, TFTP, and HTTP services must be enabled by firewalld. Assuming that DHCP, TFTP, and HTTP services are all located on the same server, the following command must be executed:

# firewall-cmd --permanent --add-service=http --add-service=dhcp --add-service=tftp

# firewall-cmd –reload

Troubleshooting

PXE setups require some effort in the beginning, and troubleshooting might be an issue because we cannot do much on the new servers because they have no OS yet. Here are some hints which might help in troubleshooting:

1. Check that the server is reachable via the DHCP IP. This can be done by finding the relevant DHCP IP address in /var/lib/dhcpd/dhcpd.leases (or by using any of the Python scripts available in public to parse this file) and then ping it.
2. Use tcpdump on the PXE server(s) to help in troubleshooting.
3. Collecting /run/initramfs/rdsosreport.txt log in case the server installation fails.

This is the most tricky part in troubleshooting. If the installation fails you won't have any network to transfer this file, and Red Hat support won't be able to help you without this file.

There are many ways to collect this file, but the easiest way is as follows:

1. Boot the new server from RHEL 7 iso.
2. Choose the Rescue option.
3. Press Shift+F2 to get the shell prompt.
4. Use fdisk to create a new partition on /dev/sda of Linux type.
5. Unmount the ISO.
6. Reboot the server.
7. Once the installation fails and you are on the Dracut command line prompt, do the following:

#mkdir /mnt

#mount /dev/sda1 /mnt

#cp /run/initramfs/rdsosreport.txt /dev/sda1

#umount /dev/sda1

1. Reboot the server again and repeat steps from 1 to 3.
2. Mount /dev/sda1

#mkdir /mnt

#mount /dev/sda1 /mnt

1. Here you have two options:

• Get a DHCP IP address by using:

#dhclient

#scp /mnt/rdsosreport.txt root@192.168.1.10:/root

• Mount a directory on your laptop as a partition on your new server, and copy the rdsosreport.txt to this partition.

Wrapping up

Congratulations on reaching this point. You now have a functioning PXE system that can deploy new RHV hosts. Remember to refer to the troubleshooting section, if everything isn't working as expected. You also need to complete each step as written to successfully get to this point. Check your work because there are many detailed steps. Good luck to you with your new environment.

References

• Installation Hosts for Red Hat Virtualization
• Preparing For A Network Installation