[Spacewalk-list] osad not picking up actions

[Nicki Clear]

ok -  I know that there have been a lot of messages about osad status and
it not working lately, but all of my troubleshooting steps that usually
work are not working and I am at a complete loss. I am running spacewalk v
2.4 on centos 6.7

on the webui - my nodes say osad status offline - and only 10 out of 70
nodes are picking up actions.  those 10 say status is online

When I run
(I replaced  the actual fqdn of spacewalk with <spacewalk fqdn> )

osad -N -vvvv


2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed to
{'rhn-dispatcher-sat@<spacewalk fqdn>': {'jid':
'rhn-dispatcher-sat@<spacewalk fqdn>', 'subscription': u'to'}}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed both {}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed none {}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed from {}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed to
2016-05-19 14:39:35 jabber_lib.send_presence: None None
2016-05-19 14:39:35 jabber_lib.process_forever:
2016-05-19 14:39:35 jabber_lib.process: 180

eventually it will fail with

2016-05-19 15:54:54 jabber_lib.process: 180
Error caught:
Traceback (most recent call last):
  File "/usr/share/rhn/osad/jabber_lib.py", line 120, in main
    self.process_forever(c)
  File "/usr/share/rhn/osad/jabber_lib.py", line 178, in process_forever
    self.process_once(client)
  File "/usr/share/rhn/osad/osad.py", line 249, in process_once
    client.process(timeout=180)
  File "/usr/share/rhn/osad/jabber_lib.py", line 1040, in process
    data = self._read(self.BLOCK_SIZE)
SSLError: ('OpenSSL error; will retry', "(-1, 'Unexpected EOF')")

2016-05-19 15:54:54 jabber_lib.main: Sleeping 115 seconds


so it looks like it is subscribed to the spacewalk server,  but it still
doesnt work.

to troubleshoot -

on the spacewalk server I ran

spacewalk-service stop
rm -rf /var/lib/jabberd/db/*
spacewalk-service start


and on the clients I ran

rhn-profile-sync
rhn-actions-control --enable-all
rm -rf /etc/sysconfig/rhn/osad-auth.conf
/etc/init.d/osad restart

and then ran
osad -N -vvvv

to check if it connects or picks up the action... which it doesnt

There arent a whole lot of messages in /var/log/rhn/osa-dispatcher.log

2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/jabber_lib.__init__
2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/jabber_lib.setup_connection
('Connected to jabber server', '<spacewalk fqdn>')
2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/osa_dispatcher.fix_connection
('Upstream notification server started on port', 1290)
2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/jabber_lib.process_forever
2016/05/19 14:37:03 -04:00 9177 0.0.0.0:
osad/dispatcher_client._message_callback('Ping response',)
2016/05/19 14:42:29 -04:00 10363 0.0.0.0: osad/jabber_lib.__init__
2016/05/19 14:42:29 -04:00 10363 0.0.0.0: osad/jabber_lib.setup_connection
('Connected to jabber server', '<spacewalk fqdn>')
2016/05/19 14:42:29 -04:00 10363 0.0.0.0:
osad/osa_dispatcher.fix_connection('Upstream notification server started on
port', 1290)
2016/05/19 14:42:30 -04:00 10363 0.0.0.0: osad/jabber_lib.process_forever
2016/05/19 14:48:28 -04:00 10363 0.0.0.0:
osad/dispatcher_client._message_callback('Ping response',)


Does anyone have any ideas?

Thanks!

**********************************************************************
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager.

This footnote also confirms that this email message has been scanned by Proofpoint PPS and F-Secure for the presence of computer viruses.