[Spacewalk-list] Clients not picking up requested changes, period
Chris Swingler
chris at chrisswingler.com
Tue Jan 27 16:36:42 UTC 2015
Yeah, osad/rhnsd don't seem to have any bearing on this problem, turning
them on/off and trying again was one of the first things I tried. No
change; so it's not like osad is somehow half-picking up the task and then
not finishing it.
Worth mentioning, if it's not clear: Spacewalk never sees the task even get
acknowledged by the remote system. It just stays at:
This action will be executed after 1/27/15 10:28:00 AM CST
This action's status is: Queued.
This action has not yet been picked up.
Of course, brand new VMs I set up to try to replicate this issue work fine.
Paulo: That's an idea, but not necessarily a *good* one... :)
On Tue, Jan 27, 2015 at 10:29 AM, Paulo Silva <paulojjs at gmail.com> wrote:
> I've found an ugly fix for this, all I had to do was to use rhnreg_ks to
> re-register the system and now events on the new profile are being picked
> up by rhn_check.
>
> 2015-01-27 16:07 GMT+00:00 Paulo Silva <paulojjs at gmail.com>:
>
>> Hi Glen,
>>
>> Can't try that since I'm using rhnsd and not osad.
>>
>> Thanks
>>
>> 2015-01-27 15:28 GMT+00:00 Glen Collins <glenc2004 at comcast.net>:
>>
>>> Hi Paulo,
>>>
>>> Why don't you try this on the client....
>>>
>>> service osad stop
>>> rm -rf /etc/sysconfig/rhn/osad-auth.conf
>>> service osad start
>>> (Wait for 2 minutes) and run (On the client)
>>> /usr/sbin/rhn_check
>>>
>>> I've had the same issues and just restarting OSAD didn't do it for me.
>>> But when I removed the auth file, that seemed to fix things. Not elegant in
>>> any sort of means but it seems to work. Have not tried it long term since
>>> I'm putting this "fix" into place right now. I guess this is what happens
>>> when you cobble a product together and "make it work". I have found jabber
>>> to be a PITA!
>>>
>>> Anyways, hope this helps!
>>>
>>> Glen Collins
>>>
>>> ------------------------------
>>> It's not permissions:
>>>
>>> [root at zbx-prx-elk-04 ~]# rhn-actions-control --report
>>> deploy is enabled
>>> diff is enabled
>>> upload is enabled
>>> mtime_upload is enabled
>>> run is enabled
>>>
>>> It's not SSL either, we have our systems configured to speak in the
>>> clear while troubleshooting this issue. Flipping over to HTTPS doesn't
>>> change anything.
>>>
>>> This particular server has three tasks pending now: two config file
>>> pushes and one remote command (just to run "date"). Here's the output of
>>> rhn_check -vvvv:
>>>
>>> [root at zbx-prx-elk-04 ~]# rhn_check -vvvv
>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>> D: locked db index /var/lib/rpm/Packages
>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>> D: loading keyring from rpmdb
>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>> D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring
>>> D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring
>>> D: added key gpg-pubkey-66fd4949-4803fe57 to keyring
>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>> D: added key gpg-pubkey-217521f6-45e8a532 to keyring
>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>> D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring
>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>> D: do_call packages.checkNeedUpdate('rhnsd=1',){}
>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>> D: loading keyring from rpmdb
>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>> D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring
>>> D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring
>>> D: added key gpg-pubkey-66fd4949-4803fe57 to keyring
>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>> D: added key gpg-pubkey-217521f6-45e8a532 to keyring
>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>> D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring
>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>> D: closed db index /var/lib/rpm/Providename
>>> D: closed db index /var/lib/rpm/Name
>>> D: closed db index /var/lib/rpm/Packages
>>> D: closed db environment /var/lib/rpm
>>> Loaded plugins: fastestmirror, rhnplugin
>>> Config time: 0.040
>>> D: login(forceUpdate=False) invoked
>>> D: readCachedLogin invoked
>>> D: Checking pickled loginInfo, currentTime=1422370609.99,
>>> createTime=1422370587.99, expire-offset=3600.0
>>> D: readCachedLogin(): using pickled loginInfo set to expire at
>>> 1422374187.99
>>> D: rpcServer: Calling XMLRPC up2date.listChannels
>>> Setting up Package Sacks
>>> Loading mirror speeds from cached hostfile
>>> pkgsack time: 0.071
>>> rpmdb time: 0.000
>>> Loading mirror speeds from cached hostfile
>>> repo time: 0.000
>>> D: local action status: (0, 'rpm database not modified since last update
>>> (or package list recently updated)', {})
>>> D: rpcServer: Calling XMLRPC registration.welcome_message
>>> D: closed db index /var/lib/rpm/Providename
>>> D: closed db index /var/lib/rpm/Name
>>> D: closed db index /var/lib/rpm/Packages
>>> D: closed db environment /var/lib/rpm
>>>
>>> Paulo: To answer your question about the API, spacecmd calls it
>>> extensively; and you can look up pending tasks by ID.
>>>
>>> spacecmd {SSM:0}> schedule_details 578593
>>> ID: 578593
>>> Action: Remote Command on zbx-prx-elk-04.company.com.
>>> User: admincswingler
>>> Date: 20150126T16:36:00
>>>
>>> Completed: 0
>>> Failed: 0
>>> Pending: 1
>>>
>>> Pending Systems
>>> ---------------
>>> zbx-prx-elk-04.company.com
>>> spacecmd {SSM:0}> schedule_details 578588
>>> ID: 578588
>>> Action: Deploy config files to system
>>> User: admincswingler
>>> Date: 20150126T16:22:00
>>>
>>> Completed: 0
>>> Failed: 0
>>> Pending: 1
>>>
>>> Pending Systems
>>> ---------------
>>> zbx-prx-elk-04.company.com
>>> spacecmd {SSM:0}> schedule_details 578586
>>> ID: 578586
>>> Action: Deploy config files to system
>>> User: admincswingler
>>> Date: 20150126T16:18:00
>>>
>>> Completed: 0
>>> Failed: 0
>>> Pending: 1
>>>
>>> Pending Systems
>>> ---------------
>>> zbx-prx-elk-04.company.com
>>> spacecmd {SSM:0}>
>>>
>>> Still scratching my head on this one, and getting rather frustrated :)
>>>
>>> On Tue, Jan 27, 2015 at 7:02 AM, Paulo Silva <paulojjs at gmail.com> wrote:
>>>
>>>> I'm checking pending events using the web interface (URL
>>>> /rhn/systems/details/history/Pending.do?sid=1000010296) and it shows 2
>>>> pending events: and Hardware Refresh and an Errata Update.
>>>>
>>>> The sid seams to be correct with the server I'm running my commands:
>>>>
>>>> # grep 1000010296 /etc/sysconfig/rhn/systemid
>>>> <value><string>ID-1000010296</string></value>
>>>>
>>>> Is there a way to check for pending events using the API?
>>>>
>>>> 2015-01-27 12:45 GMT+00:00 Alexander Innes <senni at necurity.co.uk>:
>>>>
>>>>> Just to make sure im not thinking of the wrong feture are you trying
>>>>> to use to remote commands?
>>>>> Systems -> System -> somethin -> remote command (or a different one so
>>>>> i can test in our environment :) )
>>>>>
>>>>> from that output its not seeing anything as you said :o
>>>>>
>>>>> On 27 January 2015 at 12:16, Paulo Silva <paulojjs at gmail.com> wrote:
>>>>>
>>>>>> I'm having the same issue, running rhn_check does this:
>>>>>>
>>>>>> # rhn_check -vvv
>>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>>>>> D: locked db index /var/lib/rpm/Packages
>>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>>>>> D: loading keyring from rpmdb
>>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>>>>> D: added key gpg-pubkey-b5c61460-41667588 to keyring
>>>>>> D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring
>>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>>>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>>>>> D: do_call packages.checkNeedUpdate('rhnsd=1',){}
>>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>>>>> D: loading keyring from rpmdb
>>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>>>>> D: added key gpg-pubkey-b5c61460-41667588 to keyring
>>>>>> D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring
>>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>>>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>>>>> D: closed db index /var/lib/rpm/Providename
>>>>>> D: closed db index /var/lib/rpm/Name
>>>>>> D: closed db index /var/lib/rpm/Packages
>>>>>> D: closed db environment /var/lib/rpm
>>>>>> Loaded plugins: etckeeper, fastestmirror, rhnplugin
>>>>>> Config time: 0.073
>>>>>> D: login(forceUpdate=False) invoked
>>>>>> D: readCachedLogin invoked
>>>>>> D: Checking pickled loginInfo, currentTime=1422360920.07,
>>>>>> createTime=1422360336.74, expire-offset=3600.0
>>>>>> D: readCachedLogin(): using pickled loginInfo set to expire at
>>>>>> 1422363936.74
>>>>>> D: rpcServer: Calling XMLRPC up2date.listChannels
>>>>>> This system is receiving updates from RHN Classic or Red Hat
>>>>>> Satellite.
>>>>>> rpmdb time: 0.000
>>>>>> Loading mirror speeds from cached hostfile
>>>>>> repo time: 0.002
>>>>>> Setting up Package Sacks
>>>>>> pkgsack time: 0.253
>>>>>> D: local action status: (0, 'rpm database not modified since last
>>>>>> update (or package list recently updated)', {})
>>>>>> D: rpcServer: Calling XMLRPC registration.welcome_message
>>>>>> D: closed db index /var/lib/rpm/Providename
>>>>>> D: closed db index /var/lib/rpm/Name
>>>>>> D: closed db index /var/lib/rpm/Packages
>>>>>> D: closed db environment /var/lib/rpm
>>>>>>
>>>>>>
>>>>>> 2015-01-27 10:53 GMT+00:00 Alexander Innes <senni at necurity.co.uk>:
>>>>>>
>>>>>>> Schedual the action then on the server do rhn-chck -vvv. It will
>>>>>>> tell you what the client's doing, my gut feeling is either SSL errors OR
>>>>>>> permsions (rhn-actions-control is it?)
>>>>>>>
>>>>>>> On 26 January 2015 at 23:16, Chris Swingler <chris at chrisswingler.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Hey Spacewalk-list!
>>>>>>>>
>>>>>>>> I have quite a few systems (I haven't dug into seeing if there is
>>>>>>>> any consistency in CentOS versions or the RHN tools) that just plain do not
>>>>>>>> seem to get tasks back from the Spacewalk server. We're running Spacewalk
>>>>>>>> 2.2 on CentOS release 6.5 (Final).
>>>>>>>>
>>>>>>>> I can create a simple task in Spacewalk, like just print the output
>>>>>>>> of "date", and an rhn_check never seems to execute it. It shows up under
>>>>>>>> Events > Pending, the SystemIDs match, but it never moves.
>>>>>>>>
>>>>>>>> Running a packet capture to watch the transaction, it looks like
>>>>>>>> Spacewalk itself isn't even sending the task to the remote system.
>>>>>>>> Spacewalk happily replies with an HTTP 200 and an empty payload.
>>>>>>>>
>>>>>>>> HTTP/1.1 200 OK
>>>>>>>> Date: Mon, 26 Jan 2015 22:47:55 GMT
>>>>>>>> Server: Apache
>>>>>>>> Content-Length: 126
>>>>>>>> X-RHN-Server-Capability: registration.finish_message(1)=1
>>>>>>>> X-RHN-Server-Capability: registration.remaining_subscriptions(1)=1
>>>>>>>> X-RHN-Server-Capability: abrt(1)=1
>>>>>>>> X-RHN-Server-Capability: registration.update_contact_info(1)=1
>>>>>>>> X-RHN-Server-Capability: staging_content(1)=1
>>>>>>>> X-RHN-Server-Capability: applet.has_base_channel(1)=1
>>>>>>>> X-RHN-Server-Capability: registration.smbios(1)=1
>>>>>>>> X-RHN-Server-Capability: registration.extended_update_support(1)=1
>>>>>>>> X-RHN-Server-Capability: rhncfg.filetype.directory(1)=1
>>>>>>>> X-RHN-Server-Capability: registration.update_systemid(1)=1
>>>>>>>> X-RHN-Server-Capability: registration.register_osad(1)=1
>>>>>>>> X-RHN-Server-Capability: registration.delta_packages(1)=1
>>>>>>>> X-RHN-Server-Capability: cpu_sockets(1)=1
>>>>>>>> X-RHN-Server-Capability: ipv6(1)=1
>>>>>>>> X-RHN-Server-Capability: rhncfg.content.base64_decode(1)=1
>>>>>>>> X-RHN-Server-Capability: xmlrpc.packages.extended_profile(1-2)=1
>>>>>>>> X-RHN-Server-Capability: xmlrpc.errata.patch_names(1)=1
>>>>>>>> X-RHN-Server-Capability: xmlrpc.packages.checksums(1)=1
>>>>>>>> X-RHN-Server-Capability: xmlrpc.login.extra_data(1)=1
>>>>>>>> X-RHN-Proxy-Version: 0
>>>>>>>> X-Transport-Info: Extended Capabilities Transport (C) Red Hat, Inc (version 2.5.72-1.el6)
>>>>>>>> X-RHN-Client-Version: 1
>>>>>>>> Connection: close
>>>>>>>> Content-Type: text/xml
>>>>>>>>
>>>>>>>> <?xml version='1.0'?>
>>>>>>>> <methodResponse>
>>>>>>>> <params>
>>>>>>>> <param>
>>>>>>>> <value><string></string></value>
>>>>>>>> </param>
>>>>>>>> </params>
>>>>>>>> </methodResponse>
>>>>>>>>
>>>>>>>>
>>>>>>>> Full transaction at
>>>>>>>>
>>>>>>>> https://gist.github.com/cswingler/f718abcb9ce290adec29
>>>>>>>>
>>>>>>>> rhn_server_xmlrpc.log simply shows:
>>>>>>>>
>>>>>>>> 2015/01/26 16:52:07 -05:00 30997 172.29.7.14:
>>>>>>>> xmlrpc/queue.get(1000015056, 2, 'checkins enabled')
>>>>>>>> 2015/01/26 16:52:07 -05:00 30998 172.29.7.14:
>>>>>>>> xmlrpc/up2date.listChannels(1000015056,)
>>>>>>>> 2015/01/26 16:52:08 -05:00 30995 172.29.7.14:
>>>>>>>> xmlrpc/registration.welcome_message('lang: None',)
>>>>>>>>
>>>>>>>> Any basic things I should be checking? I've tried updating the
>>>>>>>> spacewalk/rhn tools on the agent, restarting Spacewalk itself, and nothing
>>>>>>>> seems to make a change. This issue, as far as I can recall, did not seem to
>>>>>>>> appear until after we upgraded to 2.2.
>>>>>>>>
>>>>>>>> Some systems seem just fine, and I can't seem to find a consistent
>>>>>>>> reason why some succeed and others fail.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Spacewalk-list mailing list
>>>>>>>> Spacewalk-list at redhat.com
>>>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Spacewalk-list mailing list
>>>>>>> Spacewalk-list at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Paulo Silva <paulojjs at gmail.com>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Spacewalk-list mailing list
>>>>>> Spacewalk-list at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Spacewalk-list mailing list
>>>>> Spacewalk-list at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Paulo Silva <paulojjs at gmail.com>
>>>>
>>>> _______________________________________________
>>>> Spacewalk-list mailing list
>>>> Spacewalk-list at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>
>>>
>>>
>>> _______________________________________________
>>> Spacewalk-list mailing list
>>> Spacewalk-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>
>>>
>>> _______________________________________________
>>> Spacewalk-list mailing list
>>> Spacewalk-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>
>>
>>
>>
>> --
>> Paulo Silva <paulojjs at gmail.com>
>>
>
>
>
> --
> Paulo Silva <paulojjs at gmail.com>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20150127/6d9c5681/attachment.htm>
More information about the Spacewalk-list
mailing list