[Spacewalk-list] Client updates failing

Dimitri Yioulos dyioulos at netatlantic.com
Tue Feb 16 19:44:27 UTC 2016


-----Original Message-----
From: Dimitri Yioulos 
Sent: Tuesday, February 16, 2016 1:23 PM
To: spacewalk-list at redhat.com
Cc: 'Jan Hutař' <jhutar at redhat.com>; Wesley Owen <wes at netatlantic.com>
Subject: RE: [Spacewalk-list] Client updates failing



-----Original Message-----
From: Jan Hutař [mailto:jhutar at redhat.com]
Sent: Tuesday, February 16, 2016 12:08 PM
To: Dimitri Yioulos <dyioulos at netatlantic.com>
Cc: spacewalk-list at redhat.com; Wesley Owen <wes at netatlantic.com>
Subject: Re: [Spacewalk-list] Client updates failing

On 2016-02-16 14:41 +0000, Dimitri Yioulos wrote:
>
>
>-----Original Message-----
>From: Jan Hutař [mailto:jhutar at redhat.com]
>Sent: Tuesday, February 16, 2016 12:31 AM
>To: Dimitri Yioulos <dyioulos at netatlantic.com>
>Cc: spacewalk-list at redhat.com; Wesley Owen <wes at netatlantic.com>
>Subject: Re: [Spacewalk-list] Client updates failing
>
>On 2016-02-12 15:06 +0000, Dimitri Yioulos wrote:
>>
>>
>>-----Original Message-----
>>From: Jan Hutař [mailto:jhutar at redhat.com]
>>Sent: Friday, February 12, 2016 12:41 AM
>>To: spacewalk-list at redhat.com
>>Cc: Dimitri Yioulos <dyioulos at netatlantic.com>
>>Subject: Re: [Spacewalk-list] Client updates failing
>>
>>On Thu, 11 Feb 2016 14:22:18 +0000 Dimitri Yioulos <dyioulos at netatlantic.com> wrote:
>>
>>>
>>>
>>> -----Original Message-----
>>> From: Jan Hutař [mailto:jhutar at redhat.com]
>>> Sent: Thursday, February 11, 2016 1:31 AM
>>> To: spacewalk-list at redhat.com
>>> Cc: Dimitri Yioulos <dyioulos at netatlantic.com>
>>> Subject: Re: [Spacewalk-list] Client updates failing
>>>
>>> On Mon, 8 Feb 2016 17:38:04 +0000 Dimitri Yioulos 
>>> <dyioulos at netatlantic.com> wrote:
>>>
>>> > Hello, all.
>>> >
>>> > I have version 2.4 installed on a CentOS 6 box.  It was working 
>>> > great ... until now.  When I try to run an update, it ultimately 
>>> > fails.  The following appears in the client's Spacewalk Events
>>> > History:  Client execution returned
>>> > "Requested packages already installed" (code 0.   I've tried
>>> > dropping the jabber db's, removing osad-auth.conf on the clients, 
>>> > etc., all with no joy.  Extensive research has turned up no 
>>> > solution.
>>> >
>>> > Can anyone help?
>>> >
>>> > Many thanks.
>>>
>>> Hello
>>>
>>> Please could you change if you are hitting:
>>>
>>>   Bug 1302198 - remote package upgrade saying newer package is 
>>> installed, but it is not
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1302198
>>>
>>> ?
>>>
>>> > Dimitri
>>> >
>>>
>>> Regards,
>>> Jan
>>>
>>>
>>>
>>> --
>>> Jan Hutar     Systems Management QA
>>> jhutar at redhat.com     Red Hat, Inc.
>>>
>>>
>>> Hi, Jan.
>>>
>>> Sorry, I don't understand what you're asking me to do.  Have I hit a 
>>> bug?  A fix in the works?  A work-around?  Because, right now, I 
>>> have a non-working Spacewalk.
>>>
>>> Dimitri
>>
>>I'm sorry, I have wrote "Please could you change if you are hitting:" but meant "Please could you check if you are hitting:". If I understood you correctly, you have issues with updating your clients right? You might be hitting linked bug.
>>
>>Is your "non-working Spacewalk" a different issue?
>>
>>Please keep me in CC.
>>
>>Regards,
>>Jan
>>
>>
>>
>>--
>>Jan Hutar     Systems Management QA
>>jhutar at redhat.com     Red Hat, Inc.
>>
>>
>>Jan,
>>
>>I appear to be hitting at least a similar bug.  When I run "rhn_check -vv, I get a slightly different result:
>>
>> [root at bofi ~]# /usr/sbin/rhn_check -vv
>>D: do_call packages.checkNeedUpdate('rhnsd=1',){}
>>Loaded plugins: fastestmirror, rhnplugin
>>D: login(forceUpdate=False) invoked
>>D: readCachedLogin invoked
>>D: Checking pickled loginInfo, currentTime=1455286056.83, 
>>createTime=1455275977.21, expire-offset=3600.0
>>D: Pickled loginInfo has expired, created = 1455275977.21, expire = 1455279577.21.
>>logging into up2date server
>>D: rpcServer: Calling XMLRPC up2date.login
>>D: writeCachedLogin() invoked
>>D: Wrote pickled loginInfo at 1455286056.85 with expiration of 1455289656.85 seconds.
>>successfully retrieved authentication token from up2date server
>>D: logininfo:{'X-RHN-Server-Id': 1000010098, 'X-RHN-Auth-Server-Time':
>>'1455286058.04', 'X-RHN-Auth':
>>'WPcBJp95xQnGWkJegZDD9RnvXc0i4k9eC9d3IxzdCqk=', 'X-RHN-Auth-Channels':
>>[['centos5-x86_64', '20150219163429', '1', '1'], 
>>['rpmforge-el5-x86_64', '20160211020000', '0', '1'], 
>>['puppet-el5-x86_64', '20160211100000', '0', '1'], 
>>['spacewalk-client-el5-x86_64', '20160211110000', '0', '1'], 
>>['centos5-updates-x86_64', '20160211070000', '0', '1'], 
>>['centos5-base-x86_64', '20160211040000', '0', '1'], ['epel5-x86_64', 
>>'20160211000000', '0', '1']], 'X-RHN-Auth-User-Id': '',
>>'X-RHN-Auth-Expire-Offset': '3600.0'}
>>D: rpcServer: Calling XMLRPC up2date.listChannels This system is 
>>receiving updates from RHN Classic or Red Hat Satellite.
>>Loading mirror speeds from cached hostfile
>> * base: cosmos.illinois.edu
>> * epel: ftp.cse.buffalo.edu
>> * extras: mirror.lug.udel.edu
>> * updates: mirror.atlantic.net
>>Checking for new repos for mirrors
>>D: local action status: (0, 'rpm database not modified since last update (or package list recently updated)', {})                 <-------- NOTE THIS LINE
>>D: rpcServer: Calling XMLRPC registration.welcome_message
>>
>>I've noted the line of interest.
>>
>>From the Spacewalk UI:
>>
>> Details: This action will be executed after 2/11/16 9:11:00 AM EST 
>>This action's status is: Completed.
>>The client picked up this action on 2/11/16 9:57 AM The client 
>>completed this action on 2/11/16 9:58 AM
>>Client execution returned "Requested packages already installed" (code 0)           <--------- SAME AS REPORTED IN BUG
>>
>>Packages Scheduled:
>>tzdata-2016a-1.el5.x86_64
>>
>>Of course, a previous version of tzdata is installed on this machine.
>>
>>Is our Spacewalk database out-of-whack?  I must confess that I run this script to clean up obsolete packages: https://www.redhat.com/archives/spacewalk-list/2012-August/msg00249.html  Might that be the cause?  If so, how do I remediate?
>>
>>Dimitri
>
>Hello,
>I'm not sure if the outputs you have sent matches. According to timestampt in `rhn_check -vv` output, it was executed at 09:07:36, but text you have copy&pasted from Spacewalk webUI says 9:57.
>
>Cold you please try with osad stopped?:
>
>  1. client# service osad stop
>  2. on Spacewalk schedule package upgrade  3. client# rhn_check -vv
>
>And if output seems reasonable similar to whats in refferenced bug, try with change from:
>
>  https://bugzilla.redhat.com/show_bug.cgi?id=1302198#c1
>
>  1. client# service osad stop
>  2. on Spacewalk schedule package upgrade  3. client# cp 
> /usr/share/rhn/actions/packages.py{,.ORIG}
>  4. client# vim /usr/share/rhn/actions/packages.py   # add these 3
>               lines, make sure to preserve indentation  5. client# 
> rhn_check -vv  6. client# cp 
> /usr/share/rhn/actions/packages.py{.ORIG,}
>
>Also please provide info about currently installed tzdata package:
>
>  client# rpm -q tzdata --qf "%{NAME}-%{EPOCH}:%{VERSION}-%{RELEASE}.%{ARCH}\n"
>
>and package you are trying to update to:
>
>  spacewalk# rpm -qp /where/is/tzdata-...rpm --qf 
> "%{NAME}-%{EPOCH}:%{VERSION}-%{RELEASE}.%{ARCH}\n"
>
>Thank you in advance,
>Jan
>
>
>
>-- 
>Jan Hutar     Systems Management QA
>jhutar at redhat.com     Red Hat, Inc.
>
>
>Hi, Jan.
>
>This bug is similar, and I posted to it:  
>https://bugzilla.redhat.com/show_bug.cgi?id=1248713
>
>Here's the output of rhn_check after stopping the osad client, and scheduling a package update in Spacewalk:
>
>[root at server ~]# /sbin/service osad stop
>Shutting down osad:                                        [  OK  ]
>
>[root at server ~]# /usr/sbin/rhn_check -vv
>D: check_action{'action': "<?xml
>version='1.0'?>\n<methodCall>\n<methodName>packages.update</methodName>
>\n<params>\n<param>\n<value><array><data>\n<value><array><data>\n<value
>><string>tzdata</string></value>\n<value><string>2016a</string></value>
>\n<value><string>1.el5</string></value>\n<value><string></string></valu
>e>\n<value><string>x86_64</string></value>\n</data></array></value>\n</
>data></array></value>\n</param>\n</params>\n</methodCall>\n",
>'version': 2, 'id': 2738}
>updateLoginInfo() login info
>D: login(forceUpdate=True) invoked
>logging into up2date server
>D: rpcServer: Calling XMLRPC up2date.login
>D: writeCachedLogin() invoked
>D: Wrote pickled loginInfo at 1455630131.72 with expiration of 1455633731.72 seconds.
>successfully retrieved authentication token from up2date server
>D: logininfo:{'X-RHN-Server-Id': 1000010162, 'X-RHN-Auth-Server-Time': 
>'1455630132.38', 'X-RHN-Auth': 
>'jSQKwscXgB12m2VTlF90h0xFgqQCz0BBNULqmVMYLaE=', 'X-RHN-Auth-Channels': 
>[['centos5-x86_64', '20150219163429', '1', '1'], 
>['centos5-base-x86_64', '20160215172513', '0', '1'], 
>['centos5-updates-x86_64', '20160215181106', '0', '1'], 
>['epel5-x86_64', '20160215181856', '0', '1'], ['puppet-el5-x86_64', 
>'20160213002732', '0', '1'], ['rpmforge-el5-x86_64', '20160215131156', 
>'0', '1'], ['spacewalk-client-el5-x86_64', '20160215175220', '0', 
>'1']], 'X-RHN-Auth-User-Id': '', 'X-RHN-Auth-Expire-Offset': '3600.0'}
>D: handle_action{'action': "<?xml
>version='1.0'?>\n<methodCall>\n<methodName>packages.update</methodName>
>\n<params>\n<param>\n<value><array><data>\n<value><array><data>\n<value
>><string>tzdata</string></value>\n<value><string>2016a</string></value>
>\n<value><string>1.el5</string></value>\n<value><string></string></valu
>e>\n<value><string>x86_64</string></value>\n</data></array></value>\n</
>data></array></value>\n</param>\n</params>\n</methodCall>\n",
>'version': 2, 'id': 2738}
>D: handle_action actionid = 2738, version = 2
>D: do_call packages.update([['tzdata', '2016a', '1.el5', '',
>'x86_64']],){'cache_only': None} Loaded plugins: fastestmirror, 
>rhnplugin
>D: rpcServer: Calling XMLRPC up2date.listChannels This system is 
>receiving updates from RHN Classic or Red Hat Satellite.
>Loading mirror speeds from cached hostfile
> * base: mirror.atlanticmetro.net
> * epel: mirrors.rit.edu
> * extras: centos.mirror.nac.net
> * updates: mirror.trouble-free.net
>Reducing Extra Packages for Enterprise Linux 5 - x86_64 to included 
>packages only Finished Checking for new repos for mirrors
>D: Called update[['tzdata', '2016a', '1.el5', '', 'x86_64']]
>D: More recent version of package tzdata-0:2016a-1.el5.x86_64 is 
>already installed
>D: Sending back response(0, 'Requested packages already installed', {})
>D: do_call packages.checkNeedUpdate('rhnsd=1',){}
>D: local action status: (0, 'rpm database not modified since last 
>update (or package list recently updated)', {})
>D: rpcServer: Calling XMLRPC registration.welcome_message
>
>And the output from checking on which tzdata package is installed, and which I'm trying to update to:
>
>[root at server ~]# rpm -q tzdata --qf "%{NAME}-%{EPOCH}:%{VERSION}-%{RELEASE}.%{ARCH}\n"
>tzdata-(none):2015g-1.el5.x86_64
>
>[root at server ~]# rpm -qa|grep tzdata
>tzdata-2015g-1.el5
>
>[root at server ~]# rpm -qp http://mirror.centos.org/centos/5/updates/i386/RPMS/tzdata-java-2016a-1.el5.i386.rpm --qf "%{NAME}-%{EPOCH}:%{VERSION}-%{RELEASE}.%{ARCH}\n"
>tzdata-java-(none):2016a-1.el5.i386

Hello.

I guess this is a typo and you meant:

$ rpm -qp http://mirror.centos.org/centos/5/updates/i386/RPMS/tzdata-2016a-1.el5.i386.rpm --qf "%{NAME}-%{EPOCH}:%{VERSION}-%{RELEASE}.%{ARCH}\n"
tzdata-(none):2016a-1.el5.i386

(BTW: huh, did not knew rpm can handle packages on http://... for queries as well, thanks)

>When I do a manual update via yum on a similar system:
>
>Updated:
>  tzdata.x86_64 0:2016a-1.el5
>
>And the output of rpm -qa|grep tzdata is the same.
>
>From you previous post, I'm not sure which three lines you're referring to in:
>
>  4. client# vim /usr/share/rhn/actions/packages.py   # add these 3
>               lines, make sure to preserve indentation
>
>Dimitri

Err, you are right, there are 4 lines:

  https://bugzilla.redhat.com/show_bug.cgi?id=1302198#c1

(these prefixed with "+" - add them to the location of /usr/share/rhn/actions/packages.py as you can see there (without that leading plus sign).

Regards,
Jan



-- 
Jan Hutar     Systems Management QA
jhutar at redhat.com     Red Hat, Inc.


Jan,

After backing up/modifying packages.py as follows:

        found = False
        for pkg in pkgs:
            print "Current:", current.epoch, current.version, current.release
            print "Candidate:", evr.epoch, evr.version, evr.release
            print "Comparasion:", current.compare(evr)
            if pkg.returnEVR().compare(evr) == 0:
                log.log_debug('Package %s already installed' \
                    % _yum_package_tup(package))
           ~

I ran rhn_check, with the following output:

[root at server actions]# /usr/sbin/rhn_check -vv
D: check_action{'action': "<?xml version='1.0'?>\n<methodCall>\n<methodName>packages.update</methodName>\n<params>\n<param>\n<value><array><data>\n<value><array><data>\n<value><string>tzdata</string></value>\n<value><string>2016a</string></value>\n<value><string>1.el5</string></value>\n<value><string></string></value>\n<value><string>x86_64</string></value>\n</data></array></value>\n</data></array></value>\n</param>\n</params>\n</methodCall>\n", 'version': 2, 'id': 2739}
updateLoginInfo() login info
D: login(forceUpdate=True) invoked
logging into up2date server
D: rpcServer: Calling XMLRPC up2date.login
D: writeCachedLogin() invoked
D: Wrote pickled loginInfo at 1455646702.53 with expiration of 1455650302.53 seconds.
successfully retrieved authentication token from up2date server
D: logininfo:{'X-RHN-Server-Id': 1000010162, 'X-RHN-Auth-Server-Time': '1455646703.57', 'X-RHN-Auth': 'sZm51YnvCfQzwXsCTshBPfpBxhhT9xkRb9NubPX098Q=', 'X-RHN-Auth-Channels': [['centos5-x86_64', '20150219163429', '1', '1'], ['centos5-base-x86_64', '20160215172513', '0', '1'], ['centos5-updates-x86_64', '20160215181106', '0', '1'], ['epel5-x86_64', '20160215181856', '0', '1'], ['puppet-el5-x86_64', '20160213002732', '0', '1'], ['rpmforge-el5-x86_64', '20160215131156', '0', '1'], ['spacewalk-client-el5-x86_64', '20160215175220', '0', '1']], 'X-RHN-Auth-User-Id': '', 'X-RHN-Auth-Expire-Offset': '3600.0'}
D: handle_action{'action': "<?xml version='1.0'?>\n<methodCall>\n<methodName>packages.update</methodName>\n<params>\n<param>\n<value><array><data>\n<value><array><data>\n<value><string>tzdata</string></value>\n<value><string>2016a</string></value>\n<value><string>1.el5</string></value>\n<value><string></string></value>\n<value><string>x86_64</string></value>\n</data></array></value>\n</data></array></value>\n</param>\n</params>\n</methodCall>\n", 'version': 2, 'id': 2739}
D: handle_action actionid = 2739, version = 2
D: do_call packages.update([['tzdata', '2016a', '1.el5', '', 'x86_64']],){'cache_only': None} Loaded plugins: fastestmirror, rhnplugin
D: rpcServer: Calling XMLRPC up2date.listChannels This system is receiving updates from RHN Classic or Red Hat Satellite.
Loading mirror speeds from cached hostfile
 * base: mirror.atlanticmetro.net
 * epel: mirror.vcu.edu
 * extras: mirror.trouble-free.net
 * updates: mirror.netdepot.com
Reducing Extra Packages for Enterprise Linux 5 - x86_64 to included packages only Finished Checking for new repos for mirrors
D: Called update[['tzdata', '2016a', '1.el5', '', 'x86_64']]
Current: D: Sending back response((6,), 'Fatal error in Python code occurred', {})
D: do_call packages.checkNeedUpdate('rhnsd=1',){}
D: local action status: (0, 'rpm database not modified since last update (or package list recently updated)', {})
D: rpcServer: Calling XMLRPC registration.welcome_message

Dimitri


Since adding the lines in packages.py, I see the following in the Spacewalk UI:

Details: This action will be executed after 2/16/16 1:13:00 PM EST
This action's status is: Failed.
The client picked up this action on 2/16/16 1:18 PM
The client completed this action on 2/16/16 1:18 PM
Client execution returned "Fatal error in Python code occurred [[6]]" (code -1)

Packages Scheduled:
tzdata-2016a-1.el5.x86_64

Dimitri




More information about the Spacewalk-list mailing list