Infrastructure Migration Solution, an open source technical tale

Red Hat announced its infrastructure migration solution in late August, with information about what it is and the value it can offer. What I’d like to talk about is how it came to be, and some of the behind-the-scenes work to make it happen.

A bit of history (only a bit)

The history of open source is full of examples on how to replace proprietary software with an open counterpart that, over the time, can not only match the expectations, but can exceed them and even transform the landscape.

One of those examples, that I happen to like, comes from the era of UNIX systems when open source was considered suitable for academic environments but not for production. Back in those public sector sysadmin days, UNIX admins would make fun of me for advocating for x86 platforms with Linux and Apache HTTP as the way to distribute content. Nowadays? It’s a de facto standard.

It was difficult for me to explain the benefits of open source to my managers, not so much tactically, but strategically. Thankfully, some years later, Red Hat released this open source Triple Play video by Michael Tiemann. In it, Tiemann explains how open source can lower cost, improve collaboration, and drive innovation.

In the same spirit, Red Hat has offered tailored consulting services, for many years, to customers interested in migrating from proprietary virtualization. These engagements have migrated many virtual machines, which underscores the interest in open source virtualization.

Solution origin, design and definition

The Red Hat Infrastructure Migration Solution has been designed and built as a prescribed combination of Red Hat components, including product subscriptions, new migration tooling, learning subscriptions, and a predefined consulting engagement, all designed to reduce cost, risk, and time to value.

In addition, through mentoring and formal training, the solution can empower administrators to migrate workloads on their own with Red Hat’s award-winning support, helping to limit the need for onsite consultants throughout the duration of the project. This can not only increase a team’s domain expertise, but more importantly, it can help maximize short and long term cost savings that can be shifted to broader digital transformation efforts like automation and/or containerization.

To work on the toolset we started engaging the consulting teams performing the migrations in the field to help make the most of the lessons learned. We also invited systems engineering to the effort to help build tools that are maintainable, tested and reliable, and of course, user experience and design to help make it frictionless and easy to use.

One of the coolest features added to the tooling of the solution is a point and click migration experience, that allows for wizard-style workload movement from proprietary environments to new target open source alternatives. Built by a cross-functional Red Hat team, this open source toolset has been tested for its reliability, functionality, and usability.

Solution tooling

In the initial design there are 2 pieces interoperating to perform the job:

• Cloud Management Platform: to gather the data from the Virtualization Platforms (hosts, datastores, networks …) and the VMs (ID, disk, memory, cores …), as well as to orchestrate the migration
• Conversion Tools: Low level conversion tools to perform VM conversion.

To improve user experience and increase the control over the migration we added the following:

• Migration Interface: to define Infrastructure Maps and Migration Plans to execute the migration
• Conversion Control: a conversion wrapper to improve control over the conversion tools and be able to receive status data on running processes
• Added automation: Pre and post migration execution of tasks to be able to more easily and consistently customize the workflow

Now let’s review the workflow of the solution to understand how it works. The best way is through the official documentation, so let’s reproduce an extract of it here:

1. The Infrastructure Admin creates an infrastructure mapping and a virtual machine migration plan in CloudForms and runs the migration plan.
2. CloudForms locates the virtual machines to be migrated based on the infrastructure mapping.
3. The ESXi host fingerprint is captured for authentication during the conversion process if the VDDK transport method is used. If SSH is used, a shared SSH key is used to connect to the ESX host where the virtual machine resides.
4. Using the RHV attributes for the target environment, CloudForms initiates communication with the RHV conversion host.
5. The RHV conversion host connects to the source datastore through the ESX host, using virt-v2v-wrapper.py, and streams the disk to be converted to the target data domain chosen in the infrastructure mapping using virt-v2v.
6. After the disk is converted, the target virtual machine is created in RHV. During creation, the target virtual machine uses the source virtual machine’s metadata to maintain the virtual machine’s attributes (tags, power state, MAC address, CPU count, memory, disks, and virtual machine name) after migration.
7. After the virtual machine is created, the disk is attached to the target virtual machine.

Finally the virtual machine migration is complete and its status is displayed in CloudForms.

Solution implementation

To implement the solution this "team of teams" consisting of systems engineering and user experience & design, supported by consulting, horizontal solutions, and marketing, has done some cool work to productize and implement the solution.

It’s been great to be able to participate and follow the whole process in the different open source upstream communities. You may enjoy it yourself with all the code written: a specific UI, Patternfly based, to manage the mapping and planning data (upstream code here), the orchestration and data gathering done with an Automate Domain for CloudForms (upstream code here - design here).

The tooling to convert the VMs is the virt-v2v tool included in RHEL since 2009-09-04 and updated for this project (upstream code here), the Ansible playbooks to install conversion hosts (upstream code here) and a wrapper to manage the conversion and communicate with the management platform (upstream code here). Open source in action.

Now it’s time have fun unlocking your VMs!