[Spacewalk-list] spacewalk 2.6 - ubuntu integration trouble
Paul-Andre Panon
paul-andre.panon at avigilon.com
Wed Apr 12 20:13:58 UTC 2017
On March 16, 2017, <Christian Schilling o-s de> wrote about getting Spacewalk and Ubuntu 16.04 working.
He also asked about getting some Redhat Bugzilla patches applied to the repository. Those patches appear to be taken from Phil's DevOps blog post http://www.devops-blog.net/spacewalk/gpg-signing-apt-repository-in-spacewalk
The problem with Phil's suggested patches is that they are two years old and the spacewalk server patch (id=999612) is no longer sufficient. Whereas spacewalk (<2.3?) used to pull the files from disk by name, with 2.6 or later /usr/lib/python2.7/site-packages/spacewalk/server/rhnRepository.py has now abstracted those file fetches as method calls in a class, and there are no methods for the Release files. I think I would also need the client side patch https://bugzilla.redhat.com/attachment.cgi?id=999612 to be updated - I seem to remember it doesn't think it works well with current client builds either.
One thing I didn't see Christian mention is adding Acquire::PDiffs "false"; to /etc/apt/apt.conf per James M.'s comment in http://www.devops-blog.net/spacewalk/registering-ubuntu-and-debian-servers-with-spacewalk
We are starting to use Spacewalk for a mix of distributions+ versions: CentOS 7, Ubuntu 14.04, and Ubuntu 16.04. I wrote a single installation script that detects the installed O/S and runs commands appropriate to it, install_spacewalk_client.sh (below). I would up taking the files from the precise ppa and using the diffs as a starting point to build custom spacewalk 2.6 packages for Ubuntu 14.04 (in case you're wondering where those 14.04 packages came from) because the stock 14.04 packages are too out of date. In the case of the Ubuntu systems, I still need to manually add secondary repos from the Spacewalk console and then do an apt-get update on the clients to re-initialize the repo caches.
#!/bin/bash
#
# Install the SpaceWalk client on three possible O/S targets, CentOS 7, Ubuntu 14.04, and Ubuntu 16.04
# Register the client with the Avigilon Spacewalk server
# Author: Paul-Andre Panon
#
install_centos7_client() {
echo Installing Spacewalk client for CentOS 7.X
rpm -Uvh http://yum.spacewalkproject.org/2.6-client/RHEL/7/x86_64/spacewalk-client-repo-2.6-0.el7.noarch.rpm
wget https://muug.ca/mirror/fedora-epel/RPM-GPG-KEY-EPEL-7
rpm --import ./RPM-GPG-KEY-EPEL-7
BASEARCH=$(uname -i)
rpm -Uvh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum -y install rhn-client-tools rhn-check rhn-setup rhnsd m2crypto yum-rhn-plugin
rpm -Uvh http://myspacewalkserver.mydomain/pub/rhn-org-trusted-ssl-cert-1.0-2.noarch.rpm
rhnreg_ks --serverUrl=https://myspacewalkserver.mydomain/XMLRPC --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=1-centos7-x86_64
# enable osad
yum install osad
/bin/perl -pi -e "s/osa_ssl_cert =/osa_ssl_cert = \/usr\/share\/rhn\/RHN-ORG-TRUSTED-SSL-CERT/" /etc/sysconfig/rhn/osad.conf
/bin/systemctl enable osad
/bin/systemctl osad start
# enable openscap
yum install openscap-scanner spacewalk-oscap
wget https://copr-be.cloud.fedoraproject.org/results/openscapmaint/openscap-latest/epel-7-x86_64/00482175-scap-security-guide/scap-security-guide-0.1.31-2.el7.centos.noarch.rpm
rpm -ivh scap-security-guide-0.1.31-2.el7.centos.noarch.rpm
yum history sync
}
install_ubuntu14.04_client() {
echo Installing Spacewalk client for Ubuntu 14.04
if [ -n "`which rhnreg_ks`" ] ; then echo rhnreg_ks is already installed. Aborting; exit; fi
# Install some pre-requisites before our packages
apt-get -y install python-dbus python-newt python-dmidecode python-gudev python-dbus-dev python-support python-libxml2 python-gobject python-gi python-gobject-2 python-ethtool python-jabber
apt-get -y -f install
# Download and install our spacewalk packages, which are newer than in the default 14.04 repo
mkdir /tmp/trusty-debs
pushd /tmp/trusty-debs
wget https://myspacewalkserver.mydomain/pub/trusty-debs/apt-transport-spacewalk_1.0.6-4.1_all.deb
wget https://myspacewalkserver.mydomain/pub/trusty-debs/python-rhn_2.6.3-1_all.deb
wget https://myspacewalkserver.mydomain/pub/trusty-debs/rhn-client-tools_2.6.8-1_amd64.deb
wget https://myspacewalkserver.mydomain/pub/trusty-debs/rhnsd_5.0.25-1_amd64.deb
wget https://myspacewalkserver.mydomain/pub/trusty-debs/osad_5.11.74-2_all.deb
wget https://myspacewalkserver.mydomain/pub/trusty-debs/rhncfg_5.10.99-1_all.deb
dpkg -i *.deb
apt-get -y -f install
wget "https://myspacewalkserver.mydomain/pub/spacewalk.gpg.pubkey"
apt-key add spacewalk.gpg.pubkey
# Comment out lines in /usr/lib/python2.7/xmlrpclib.py, per http://www.devops-blog.net/spacewalk/registering-ubuntu-and-debian-servers-with-spacewalk
patch -p4 -d /usr/lib/python2.7 <<EOF
--- /usr/lib/python2.7/xmlrpclib.py 2017-01-30 18:35:09.579475863 -0800
+++ /usr/lib/python2.7/xmlrpclib.py.new 2017-01-30 18:34:33.971475863 -0800
@@ -655,7 +655,7 @@
f(self, value, write)
def dump_nil (self, value, write):
- if not self.allow_none:
- raise TypeError, "cannot marshal None unless allow_none is enabled"
+ #if not self.allow_none:
+ # raise TypeError, "cannot marshal None unless allow_none is enabled"
write("<value><nil/></value>")
dispatch[NoneType] = dump_nil
EOF
# Now set up and register with our Spacewalk server
mkdir /var/lock/subsys
wget http://myspacewalkserver.mydomain/pub/RHN-ORG-TRUSTED-SSL-CERT -O /usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT
rhnreg_ks --force --serverUrl=https://myspacewalkserver.mydomain/XMLRPC --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=1-avo-ubuntu-trusty-key
# Take updates and security patches for main and universe from spacewalk
echo 'deb spacewalk://myspacewalkserver.mydomain/XMLRPC channels: main trusty-updates trusty-backports trusty-security' > /etc/apt/sources.list.d/spacewalk.list
cp /etc/apt/sources.list /etc/apt/sources.list.bak
# and not from the standard repositories
perl -pi -e 's/^deb([-src]*) (.*) trusty main/#deb$1 $2 trusty main/' /etc/apt/sources.list
perl -pi -e 's/^deb([-src]*) (.*) trusty-updates ([m|u][^u])/#deb$1 $2 trusty-updates $3/' /etc/apt/sources.list
perl -pi -e 's/^deb([-src]*) (.*) trusty-security ([m|u][^u])/#deb$1 $2 trusty-security $3/' /etc/apt/sources.list
perl -pi -e 's/^deb([-src]*) (.*) trusty-backports/#deb$1 $2 trusty-backports/' /etc/apt/sources.list
# Work around https://bugzilla.redhat.com/show_bug.cgi?id=1187189
#wget http://myspacewalkserver.mydomain/pub/49spacewalk-workaround -O /etc/apt/apt-conf.d/49spacewalk-workaround
# clean up and test
popd
rm -Rf /tmp/trusty-debs
apt-get update
}
install_ubuntu16.04_client() {
echo Installing Spacewalk client for Ubuntu 16.04
apt-get -y install apt-transport-spacewalk python-rhn python-ethtool rhnsd rhn-client-tools python-jabber
wget http://myspacewalkserver.mydomain/pub/RHN-ORG-TRUSTED-SSL-CERT -O /usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT
rhnreg_ks --force --serverUrl=https://myspacewalkserver.mydomain/XMLRPC --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=1-avo-ubuntu-xenial-key
wget "https://myspacewalkserver.mydomain/pub/spacewalk.gpg.pubkey"
apt-key add spacewalk.gpg.pubkey
# Take updates and security patches for main and universe from spacewalk
echo 'deb spacewalk://myspacewalkserver.mydomain/XMLRPC channels: main xenial-updates xenial-backports xenial-security' > /etc/apt/sources.list.d/spacewalk.list
cp /etc/apt/sources.list /etc/apt/sources.list.bak
# and not from the standard repositories
perl -pi -e 's/^deb([-src]*) (.*) xenial main/#deb$1 $2 xenial main/' /etc/apt/sources.list
perl -pi -e 's/^deb([-src]*) (.*) xenial-updates ([m|u][^u])/#deb $2 xenial-updates $3/' /etc/apt/sources.list
perl -pi -e 's/^deb([-src]*) (.*) xenial-security ([m|u][^u])/#deb$1 $2 xenial-security $3/' /etc/apt/sources.list
perl -pi -e 's/^deb([-src]*) (.*) xenial-backports/#deb$1 $2 xenial-backports/' /etc/apt/sources.list
if [ ! -d /var/lib/up2date ]
then
mkdir /var/lib/up2date
fi
# Work around https://bugzilla.redhat.com/show_bug.cgi?id=1187189
wget http://myspacewalkserver.mydomain/pub/49spacewalk-workaround -O /etc/apt/apt-conf.d/49spacewalk-workaround
# and test
apt-get update
}
if [ ! -d /usr/local/share/ca-certificates/york.lan ]
then
mkdir /usr/local/share/ca-certificates/york.lan
wget -P /usr/local/share/ca-certificates/york.lan "https://myspacewalkserver.mydomain/pub/cacerts/AVO-IssCA1.crt" --no-check-certificate
wget -P /usr/local/share/ca-certificates/york.lan "https://myspacewalkserver.mydomain/pub/cacerts/AVO-IssCA2.crt" --no-check-certificate
wget -P /usr/local/share/ca-certificates/york.lan "https://myspacewalkserver.mydomain/pub/cacerts/AVO-rootca.crt" --no-check-certificate
sudo update-ca-certificates
fi
if [ -n "`which rhnreg_ks`" ]
then echo rhnreg_ks is already installed. Aborting
exit
fi
if [ -e /etc/centos-release ]
then
CENTOS_RELEASE=`cat /etc/centos-release`
if [ "`expr substr \"$CENTOS_RELEASE\" 1 23`" = "CentOS Linux release 7." ]
then install_centos7_client
fi
elif [ -e `which lsb_release` ]
then
OTHER_DISTRIB=`lsb_release -a | fgrep "Description:"`
if [ `expr match "$OTHER_DISTRIB" ".*Ubuntu.*"` -ge 6 ]
then
if [ `expr match "$OTHER_DISTRIB" ".*14\.04.*"` -ge 5 ]
then
install_ubuntu14.04_client
elif [ `expr match "$OTHER_DISTRIB" ".*16\.04.*"` -ge 5 ]
then
install_ubuntu16.04_client
fi
fi
fi
----------------------------------------------------------------------------------------------
I also have a cron job to sync the repos, running the file below.
#!/bin/bash
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64 --type yum
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64-addons --type yum
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64-epel --type yum
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64-extras --type yum
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64-jpackage --type yum
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64-pbis --type yum
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64-spacewalk --type yum
/usr/bin/spacewalk-repo-sync --channel centos7-x86_64-updates --type yum
/usr/bin/spacewalk-repo-sync --channel trusty --type deb
/usr/bin/spacewalk-repo-sync --channel trusty-backports --type deb
/usr/bin/spacewalk-repo-sync --channel trusty-security --type deb
/usr/bin/spacewalk-repo-sync --channel trusty-updates --type deb
/usr/bin/spacewalk-repo-sync --channel xenial --type deb
/usr/bin/spacewalk-repo-sync --channel xenial-backports --type deb
/usr/bin/spacewalk-repo-sync --channel xenial-security --type deb
/usr/bin/spacewalk-repo-sync --channel xenial-updates --type deb
pushd /var/cache/rhn/repodata/trusty
bash /opt/spacewalk-errata/secureApt.sh trusty main 14.04
cd ../trusty-security/
bash /opt/spacewalk-errata/secureApt.sh trusty security 14.04
cd ../trusty-backports/
bash /opt/spacewalk-errata/secureApt.sh trusty backports 14.04
cd ../trusty-updates/
bash /opt/spacewalk-errata/secureApt.sh trusty updates 14.04
cd ../trusty-spacewalk-client/
bash /opt/spacewalk-errata/secureApt.sh trusty spacewalk-client 14.04
cd ../xenial/
bash /opt/spacewalk-errata/secureApt.sh xenial main 16.04
cd ../xenial-security/
bash /opt/spacewalk-errata/secureApt.sh xenial security 16.04
cd ../xenial-backports/
bash /opt/spacewalk-errata/secureApt.sh xenial backports 16.04
cd ../xenial-updates/
bash /opt/spacewalk-errata/secureApt.sh xenial updates 16.04
popd
SPACEWALK_USER=xxxxxx SPACEWALK_PASS=yyyyyyyyyyy /bin/bash /opt/spacewalk-errata/spacewalk-errata.sh
# drop privileges when doing external communications
su spaceuser -c "/bin/curl https://raw.githubusercontent.com/stevemeier/cefs/master/errata.latest.xml" > /var/spacewalk/errata/centos-errata.xml
su spaceuser -c "/bin/curl https://www.redhat.com/security/data/oval/com.redhat.rhsa-all.xml" > /var/spacewalk/errata/com.redhat.rhsa-all.xml
SPACEWALK_USER=xxxxxx SPACEWALK_PASS=yyyyyyyyyyy /opt/spacewalk-errata/errata-import.pl --server myserver.mydomain --rhsa-oval /var/spacewalk/errata/com.redhat.rhsa-all.xml --publish --errata /var/spacewalk/errata/centos-errata.xml
----------------------------------------------------------------------------------------------
I modified Phil's secureApt.sh so I could also specify the O/S release. The errata-import.pl is from Steve Meier, and spacewalk-errata.sh is a wrapper script around the scripts mentioned by Phil in http://www.devops-blog.net/spacewalk/configuring-errata-for-ubuntu-with-spacewalk
#!/bin/bash
# Processes Ubuntu Errata and imports them to Spacewalk
update_monthly_errata() {
# Fetches the errata data from ubuntu.com.
rm -rf /var/spacewalk/errata/$DATE.txt
rm -rf /var/spacewalk/errata/ubuntu-errata.xml
# drop privileges when doing external communications
su spaceuser -c "curl https://lists.ubuntu.com/archives/ubuntu-security-announce/$DATE.txt.gz" > /var/spacewalk/errata/$DATE.txt.gz
gunzip -f /var/spacewalk/errata/$DATE.txt.gz
# Processes and imports the errata.
cd /var/spacewalk/errata/ && \
/opt/spacewalk-errata/parseUbuntu.py $DATE.txt
/opt/spacewalk-errata/errata-import.py 2>&1 | tee -a /var/log/ubuntu-errata.log
}
#make sure we have english locale
export LC_TIME="en_US.utf8"
# First figure out the previous month because it could have new errata since this was last run
DATE=` date --date='last month' +"%Y-%B"`
update_monthly_errata
# Obtains the current date and year.
DATE=`date +"%Y-%B"`
update_monthly_errata
----------------------------------------------------------
Despite all that work, some things are still broken:
a) while I'm building the Release* files I don't yet have a way (patch) for the server to provide them to the client, so the client still complains that the packages aren't signed. The clients have the GPG key though, thanks to the install script, so if I ever get the server and clients patched for those files, the clients may be happy.
b) Package version collisions - Spacewalk 2.6 handles Ubuntu package version better than before, but the internal era comparison is still broken. If you have libevent-2.0-5-2.0.21-stable-1ubuntu1.14.04.1.amd64-deb installed on a client and libevent-2.0-5-2.0.21-stable-1ubuntu1.14.04.2.amd64-deb has been synced from the Ubuntu repository, Spacewalk won't identify it as a package that's out of date, even though it could involve an errata. However do an
# apt-get update && apt-get upgrade
from the Ubuntu client and those older era packages will show as being out of date.
Paul-Andre Panon
Senior systems administrator
More information about the Spacewalk-list
mailing list