This article was originally published on the Red Hat Customer Portal. The information may no longer be current.

Overview

Product certificates are an important component of the subscription-manager toolkit. Understanding their usage makes working with subscription-manager significantly easier.

What is a product certificate?

Product certificates, which are x.509 certificates found in either the /etc/pki/product or /etc/pki/product-default directories are a means for the subscription tools to accurately track which Red Hat products are installed on a system. In Subscription Manager for the Former Red Hat Network User Part 1 , we covered the basics on how Red Hat Subscriptions are designed, but let's cover them again.

Introduction (Again) to Red Hat Subscriptions

Red Hat Subscriptions are built based upon a simple hierarchy.

  • Subscription Name. (This is the canonical name of the subscription. Example: 'Red Hat Satellite' or 'Red Hat Enterprise Linux Server, Premium (Physical or Virtual Nodes)'). This is the name of the subscription as you see it in the Customer Portal. It has a Stock Keeping Unit (SKU) and a price associated with it.
    • Products. These are the various basic components such as Red Hat Enterprise Linux, Red Hat Software Collections, etc that are part of the Subscription that is purchased.
      • Content Sets. This is where the software comes from. These are usually yum repositories, as we know and love, but there are other types too. Each Product provides 1 or more Content Sets.

This hierarchy is very important, as it is fundamental to understanding how the Subscription Management tools work.

  • 1 Subscription [Name] provides:
    • 1 or more Products, which each provide:
      • 1 or more Content Sets

To build Subscriptions (again, the things that are sold), we bundle various Products into a cohesive unit. This allows the ability to define "RHEL" in one place, and leverage it in any component that needs it. (RHEL, Satellite, Openshift, etc). This also provides the flexibility to add/remove Components without having to change the SKU.

For example, many Red Hat Enterprise Linux Subscriptions with Premium SLA in the 2013 subscription model include Extended Update Support (EUS). We can see this with via subscription-manager:

# subscription-manager list --all --available
+-------------------------------------------+
    Available Subscriptions
+-------------------------------------------+
# Name of the Subscription
Subscription Name:   Red Hat Enterprise Linux Server, Premium (Physical or Virtual Nodes)
# Products that are provided by this subscription.
Provides:            Red Hat Enterprise Linux Resilient Storage (for RHEL Server) - Extended Update Support
                     Red Hat Software Collections (for RHEL Server)
                     Red Hat Enterprise Linux Atomic Host Beta
                     Red Hat Enterprise Linux High Availability (for RHEL Server) - Extended Update Support
                     Red Hat EUCJP Support (for RHEL Server) - Extended Update Support
                     Red Hat Container Images Beta
                     Red Hat Enterprise Linux Server - Extended Update Support
                     Red Hat Beta
                     Oracle Java (for RHEL Server) - Extended Update Support
                     Red Hat Enterprise Linux High Performance Networking (for RHEL Server) - Extended Update Support
                     Red Hat Enterprise Linux Scalable File System (for RHEL Server) - Extended Update Support
                     Oracle Java (for RHEL Server)
                     Red Hat Enterprise Linux Load Balancer (for RHEL Server) - Extended Update Support
                     Red Hat Container Images
                     Red Hat Enterprise Linux Server
                     Red Hat Enterprise Linux Atomic Host
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat S-JIS Support (for RHEL Server) - Extended Update Support
                     Red Hat Developer Toolset (for RHEL Server)
SKU:                 RH00003
Contract:            11223344
Pool ID:             809d1a82-ffb0-4a28-b686-3fcc1f55fd41
Provides Management: No
Available:           2
Suggested:           2
Service Level:       Premium
Service Type:        L1-L3
Subscription Type:   Instance Based
Ends:                10/07/2016
System Type:         Physical

The information above tells me that SKU RH00003 is named
'Red Hat Enterprise Linux Server, Premium (Physical or Virtual Nodes)' and it
provides the following Products:

  • Red Hat Enterprise Linux Server
  • Red Hat Enterprise Linux Server - Extended Update Support

Fundamentally, a product, such as Red Hat Enterprise Linux Server is a collection of well-defined repositories that are grouped together and given a unique identifier. That unique identifier (and other data) is included in a product certificate so that our subscription management tools can correctly identify if/when those components are installed. This is one of the many heuristics that are used when attaching subscriptions to a system.

Why are product certificates important?

Product certificates are importing because without the correct product certificates:

  • A system will not properly report what products it has installed, thus making subscription accounting inaccurate.
  • (more importantly): A system will not have access to the proper content repositories.

Inspecting product certificates

Product certificates can be inspected leveraging the rct utility, which allows us to 'Prints the information that is contained in the certificate itself, such as the certificate headers, serial numbers, products, and content sets.'

# rct cat-cert /etc/pki/product/69.pem

+-------------------------------------------+
    Product Certificate
+-------------------------------------------+

Certificate:
    Path: /etc/pki/product/69.pem
    Version: 1.0
    Serial: 12750047592154747089
    Start Date: 2015-03-19 13:52:28+00:00
    End Date: 2035-03-14 13:52:28+00:00

Subject:
    CN: Red Hat Product ID [2cd2961c-261f-4b17-8e0b-d8fe2dce6654]

Issuer:
    C: US
    CN: Red Hat Entitlement Product Authority
    O: Red Hat, Inc.
    OU: Red Hat Network
    ST: North Carolina
    emailAddress: ca-support@redhat.com

Product:
    ID: 69
    Name: Red Hat Enterprise Linux Server
    Version: 6.7
    Arch: x86_64
    Tags: rhel-6,rhel-6-server
    Brand Type:
    Brand Name:

As you can see this product certificate is for 'Red Hat Enterprise Linux Server' identified by ID: 69. It also lists the version of RHEL, architecture, and Tags (which are used to determine which repositories to enable by default). Product certificates are version specific. Copying a RHEL 6.7 to a RHEL 7.2 system will have undesired effects.

Where are product certificates stored?

Product certificates are stored in one of two directories on the system:

  • /etc/pki/product - Dynamically populated directory of product certificates based upon repository usage.
  • /etc/pki/product-default - Static directory of product certificates, used primarily to have sane defaults.

In the event of conflicting certificates (example: there is a certificate in both directories that cover the same product), the certificate in the /etc/pki/product directory takes precedence.

How do I get a product certificate?

Product certificates come from one of 5 places:

  • They are provided at operating system installation time.
  • They are provided via the redhat-release-* package.
  • They are provided via the rhn-migrate-classic-to-rhsm utility
  • They are provided via bootstrap.py script
  • They are provided via the productid yum plugin.

Product certs at operating system installation time

Product certificates have been included since subscription-manager has been a shipping component of RHEL. (since RHEL 5.7, 6.1, and 7.0)

In older versions of Red Hat Enterprise Linux (prior to 6.7 & 7.2), product certificates were included in /etc/pki/product at installation time. Depending on how your system was provisioned, there was cases where the product certificate was not created/deployed.

This posed a challenge when registering or migrating a system to Red Hat Subscription Management or Red Hat Satellite 6: If I don't have a product certificate, I can't access content. And since they were only provided at installation time, this basically necessitated opening a support ticket to get the correct product certificate. Red Hat loves speaking with its customers, but we'd rather you not call us for this.

Product certificates via the redhat-release-* package

To alleviate the concerns above, more recent versions of Red Hat Enterprise Linux (versions >= 6.7 and 7.2) ship with the product certificate for the Operating system as part of the redhat-release-* package. This allows the product certificate (for the OS) to be:

  • owned by a package so that it can be upgraded/updated via the normal package & errata release cycles.
  • Easily (re)installable for systems that do not have them.

It also still ensures that a product certificate is installed at operating system installation time (as the redhat-release package is always installed)

Let's take a look:

First, make sure we have a system running RHEL 7.2 or newer

# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.3 (Maipo)

And take a look at the files that are provided via the redhat-release-server package.

# rpm -ql redhat-release-server
/etc/issue
/etc/issue.net
/etc/os-release
/etc/pki/product-default
/etc/pki/product-default/69.pem
/etc/pki/rpm-gpg
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-legacy-former
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-legacy-release
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-legacy-rhx
/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
/etc/redhat-release
/etc/rpm/macros.dist
/etc/system-release
/etc/system-release-cpe
/usr/lib/systemd/system-preset/85-display-manager.preset
/usr/lib/systemd/system-preset/90-default.preset
/usr/share/doc/redhat-release/GPL
/usr/share/redhat-release/EULA

As you can see, the redhat-release RPM contains provides both the /etc/pki/product-default directory and the /etc/pki/product-default/69.pem file.

Product certificates provided via rhn-migrate-classic-to-rhsm

Included in the subscription-manager migration package is the rhn-migrate-classic-to-rhsm script, which looks at the channels that the system was previously using via Red Hat Network Classic or Satellite 5, and installs the correct product certificates. If you are migrating a system from RHN Classic or Satellite 5 to Red Hat Subscription Management (RHSM), it is recommended that you use rhn-migrate-classic-to-rhsm. This utility can be used on its own to migrate systems from RHN Classic or Satellite 5 to Satellite 6, but there are more comprehensive ways to do this.

Product certificates provided via bootstrap.py

The Satellite 6 bootstrap script, which is the preferred means to register a system with Satellite 6, will ensure that a system has a product certificate prior to registering it with Satellite 6 (primarily by invoking rhn-migrate-classic-to-rhsm), and also takes care of other registration tasks needed with Satellite 6 clients, such as (but not limited to)

  • installing the Katello Agent
  • properly configuring Puppet
  • updating the system.
  • enabling additional content repositories

Product certificates provided via the productid yum plugin.

Lastly, for a running system, additional product certificates are provided via the productid yum plugin.

Each repository in the CDN (or Satellite 6) containing Red Hat Products contains, within its repodata directory a file named productid or productid.gz, which contains the product certificate for that Product. This ensures that when I install bits from the Satellite repository, subscription-manager is informed that Satellite product bits are actually installed on the running system.

Thus, it is imperative that the productid plugin is enabled when systems are registered to RHSM or Satellite 6. This can be done by changing enabled=0 to enabled=1 in the /etc/yum/pluginconf.d/product-id.conf file. NOTE: bootstrap.py does that for you in the Satellite 6 use-case.

Product certificates in action.

Let's register a system and install some bits to see all of the above theory in action, using the installation of Red Hat Satellite 6 as an example.

# subscription-manager register
Registering to: subscription.rhsm.redhat.com:443/subscription
Username: <redacted>
Password:
The system has been registered with ID: 112cd775-b4b2-43b5-8c8d-0328003db20a

And see which product certificates that we have installed:

# ls -l /etc/pki/product
total 0
# ls -l /etc/pki/product-default/
total 4
-rw-r--r--. 1 root root 2159 Sep 27 05:42 69.pem

Let's see which Satellite subscriptions I have available:

# subscription-manager list --all --available --matches '*Red Hat Satellite*'
Subscription Name:   Red Hat Satellite
Provides:            Red Hat Beta
                     Red Hat Satellite Capsule Beta
                     Red Hat Software Collections (for RHEL Server)
                     Red Hat Satellite Capsule
                     Red Hat Satellite with Embedded Oracle
                     Red Hat Satellite Beta
                     Red Hat Satellite 6 Beta
                     Red Hat Enterprise Linux Server
                     Red Hat Enterprise Linux High Availability (for RHEL Server)
                     Red Hat Satellite
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat Enterprise Linux Load Balancer (for RHEL Server)
SKU:                 MCT0370
Contract:            [REDACTED]
Pool ID:             8a85f98152d000770152d1d330140fee
Provides Management: Yes
Available:           11
Suggested:           1
Service Level:       Premium
Service Type:        L1-L3
Subscription Type:   Standard
Ends:                02/10/2017
System Type:         Physical

And attach the subscription

# subscription-manager attach --pool 8a85f98152d000770152d1d330140fee
Successfully attached a subscription for: Red Hat Satellite

Let's take a look at /etc/pki/product again

# ls -l /etc/pki/product
total 0

Still nothing! Even though I've attached a Satellite subscription, I haven't actually installed any Satellite RPMs, thus no product certificate for Satellite. I am not actually using Satellite bits. Well, not yet :)

Setup my repositories needed for installation (as per the Install Guide)

# subscription-manager repos --disable "*"

# subscription-manager repos --enable=rhel-7-server-rpms \
--enable=rhel-server-rhscl-7-rpms \
--enable=rhel-7-server-satellite-6.2-rpms

And install Satellite.

# yum install -y satellite

As per the Install Guide, Satellite requires repositories from Red Hat Enterprise Linux (RHEL), Red Hat Software Collections (RHSCL) and Red Hat Satellite for a successful installation. And now that we've installed bits from the RHSCL & Satellite repositories, we can see this.

# ls -l /etc/pki/product
total 8
-rw-r--r--. 1 root root 2171 Nov 15 07:03 201.pem
-rw-r--r--. 1 root root 2151 Nov 15 07:03 250.pem

And let's inspect those two product certificates:

# rct cat-cert /etc/pki/product/201.pem

+-------------------------------------------+
 Product Certificate
+-------------------------------------------+

Certificate:
 Path: /etc/pki/product/201.pem
 Version: 1.0
 Serial: 12750047592154746889
 Start Date: 2015-01-12 11:56:53+00:00
 End Date: 2035-01-07 11:56:53+00:00

Subject:
 CN: Red Hat Product ID [b6d76fbd-2da7-4167-97d4-5e6f3155e8d0]

Issuer:
 C: US
 CN: Red Hat Entitlement Product Authority
 O: Red Hat, Inc.
 OU: Red Hat Network
 ST: North Carolina
 emailAddress: ca-support@redhat.com

Product:
 ID: 201
 Name: Red Hat Software Collections (for RHEL Server)
 Version: 2
 Arch: x86_64
 Tags: rhscl,rhscl-2
 Brand Type:
 Brand Name:

# rct cat-cert /etc/pki/product/250.pem

 +-------------------------------------------+
    Product Certificate
 +-------------------------------------------+

 Certificate:
    Path: /etc/pki/product/250.pem
    Version: 1.0
    Serial: 12750047592154746893
    Start Date: 2016-01-20 16:05:50+00:00
    End Date: 2036-01-15 16:05:50+00:00

 Subject:
    CN: Red Hat Product ID [f4fb43a9-defc-4fb0-a8a9-0c1722aa44e7]

 Issuer:
    C: US
    CN: Red Hat Entitlement Product Authority
    O: Red Hat, Inc.
    OU: Red Hat Network
    ST: North Carolina
    emailAddress: ca-support@redhat.com

 Product:
    ID: 250
    Name: Red Hat Satellite
    Version: 6.2
    Arch: x86_64
    Tags: satellite-6.2,satellite
    Brand Type:
    Brand Name:

Product certificates and auto-attach

Now that I have installed Satellite on this system, subscription-manager knows it, and this data, along with other system facts (Sockets,cores,am I a virtual machine, etc) are used to auto-attach subscriptions to the system.

Let's see this in action.

Firstly, let's remove all the subscriptions from the system and see our subscription status.

subscription-manager remove --all && subscription-manager status
1 subscription removed at the server.
1 local certificate has been deleted.
+-------------------------------------------+
   System Status Details
+-------------------------------------------+
Overall Status: Invalid

Red Hat Software Collections (for RHEL Server):
- Not supported by a valid subscription.

Red Hat Enterprise Linux Server:
- Not supported by a valid subscription.

Red Hat Satellite:
- Not supported by a valid subscription.

And now let's allow the auto-attach algorithm to attach a subscription

# subscription-manager attach --auto


Installed Product Current Status:
Product Name: Red Hat Software Collections (for RHEL Server)
Status:       Subscribed

Product Name: Red Hat Enterprise Linux Server
Status:       Subscribed

Product Name: Red Hat Satellite
Status:       Subscribed

And take a look at the attached subscription:

# subscription-manager list --consumed
+-------------------------------------------+
   Consumed Subscriptions
+-------------------------------------------+
Subscription Name:   Red Hat Satellite
Provides:            Red Hat Satellite
                     Red Hat Enterprise Linux Server
                     Red Hat Beta
                     Red Hat Software Collections (for RHEL Server)
                     Red Hat Satellite Beta
                     Red Hat Software Collections Beta (for RHEL Server)
                     Red Hat Satellite 6 Beta
                     Red Hat Satellite Capsule Beta
                     Red Hat Enterprise Linux Load Balancer (for RHEL Server)
                     Red Hat Satellite with Embedded Oracle
                     Red Hat Satellite Capsule
                     Red Hat Enterprise Linux High Availability (for RHEL Server)
SKU:                 MCT0370
Contract:            [REDACTED]
Account:             [REDACTED]
Serial:              8076015858992714092
Pool ID:             8a85f98152d000770152d1d330140fee
Provides Management: Yes
Active:              True
Quantity Used:       1
Service Level:       Premium
Service Type:        L1-L3
Status Details:      Subscription is current
Subscription Type:   Standard
Starts:              02/11/2016
Ends:                02/10/2017
System Type:         Physical

The auto-attach algorithm attempts to properly cover the systems installed products (as seen in /etc/pki/product*) with a few subscriptions as possible. As this system has RHEL (69.pem), RHSCL (201.pem) and Satellite (250.pem) installed AND I have subscriptions that cover all three (such as the Satellite sub above), it is preferred.

Summary

Product certificates are very important and are a key component of the subscription toolkit. In future articles we'll cover activation keys, which build upon product certificates.

Further Reading


Sobre o autor

Rich Jerrido is the Senior Principal Product Manager for Hybrid Cloud Business Services.
 

Read full bio