[Spacewalk-list] osa-dispatcher dies shortly after starting

[Jonathan Payne]

Hoping for some assistance here, as I’ve been banging my head against the wall on this issue.

I just upgraded from Spacewalk 2.2 to 2.6, and now osa-dispatcher is dying with no information on the error, and the status just saying the process is dead but the pid file still exists.

LOG at Debug level 7:
                2017/09/18 11:33:25 -06:00 12686 0.0.0.0: rhnSQL/driver_postgresql.convert_named_query_params('Converting query for PostgreSQL: select * from rhnPushClient where jabber_id = :p1',)
2017/09/18 11:33:25 -06:00 12686 0.0.0.0: rhnSQL/driver_postgresql.convert_named_query_params('New query: select * from rhnPushClient where jabber_id = %(p1)s',)
2017/09/18 11:33:25 -06:00 12686 0.0.0.0: rhnSQL/driver_postgresql._execute_wrapper('Executing SQL: "select * from rhnPushClient where jabber_id = %(p1)s" with bind params: {p1: osad-6d4dfeb426 at SPACEWALKSERVERNAME/osad}',)
2017/09/18 11:33:25 -06:00 12686 0.0.0.0: osad/jabber_lib.main('ERROR', 'Error caught:')


                service osa-dispatcher status
osa-dispatcher dead but pid file exists



I have checked my jabberd xml files, and they appear to be setup correctly. One thing I found odd is that the server name shows up twice in the sm.xml. I changed the <id> under <sm> back to the default of sm, but that didnt' fix the issue.

                /etc/jabberd/sm.xml
<sm>
  <!-- Our ID on the network (default: sm) -->
  <id>SPACEWALKSERVERNAME</id>

<!-- Local network configuration -->
  <local>
    <id>SPACEWALKSERVERNAME </id>
    <!-- Who we identify ourselves as.
         Users will have this as the domain part of their JID.
         If you want your server to be accessible from other
         Jabber servers, this IDs must be FQDN resolvable by DNSes.
         If not set, the SM id is used. -->
    <id>localhost.localdomain</id>
    <!--
    <id>vhost1.localdomain</id>
    <id>vhost2.localdomain</id>
    -->

                grep jabber /etc/rhn/rhn.conf
                                server.jabber_server = SPACEWALKSERVERNAME
                                osa-dispatcher.jabber_server = SPACEWALKSERVERNAME

                ll /etc/jabberd/*.xml
-rw-r-----. 1 jabber jabber  25K Sep 17 20:20 /etc/jabberd/c2s.xml
-rw-r-----. 1 jabber jabber 2.0K Jul 14 02:10 /etc/jabberd/router-filter.xml
-rw-r-----. 1 jabber jabber  258 Jul 14 02:10 /etc/jabberd/router-users.xml
-rw-r-----. 1 jabber jabber 6.9K Sep 17 20:20 /etc/jabberd/router.xml
-rw-r-----. 1 jabber jabber  12K Sep 17 20:20 /etc/jabberd/s2s.xml
-rw-r-----. 1 jabber jabber  31K Sep 18 11:20 /etc/jabberd/sm.xml




I also checked the SSL certs to verify that they are correct and that the certificate matches the hostname in /etc/jabberd/c2s.xml. I have verified permissions, as well. Hostname is in the hosts file.

md5sum /etc/pki/spacewalk/jabberd/server.pem /root/ssl-build/ SPACEWALKSERVERNAME server.pem
eda16d799aefba2acc5a5b4b278227ac  /etc/pki/ SPACEWALKSERVERNAME jabberd/server.pem
eda16d799aefba2acc5a5b4b278227ac  /root/ssl-build/ SPACEWALKSERVERNAME server.pem

ll /etc/pki/spacewalk/jabberd/
-rw-------. 1 jabber jabber 7.2K Mar  4  2014 server.pem

md5sum /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT
4a3fa3d648a87aee70151dd6ee030c9b  /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT
4a3fa3d648a87aee70151dd6ee030c9b  /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT

grep CN= $(grep spacewalk.crt /etc/httpd/conf.d/ssl.conf | cut -f 2 -d' ') | grep Subject
                OU= SPACEWALKSERVERNAME, CN= SPACEWALKSERVERNAME


grep require-starttls /etc/jabberd/c2s.xml | grep pemfile
                <id require-starttls="false" pemfile="/etc/pki/spacewalk/jabberd/server.pem" realm="" register-enable="true">SPACEWALKSERVERNAME</id>

grep CN= /etc/pki/spacewalk/jabberd/server.pem | grep Subject
                OU= SPACEWALKSERVERNAME, CN= SPACEWALKSERVERNAME

grep "osa-dispatcher.osa_ssl_cert" /etc/rhn/rhn.conf
osa-dispatcher.osa_ssl_cert = /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT

                grep C2S_AS_ROOT /etc/sysconfig/jabberd
# set C2S_AS_ROOT to "yes" to start c2s as root
C2S_AS_ROOT="no"



I tried deleting the jabberd database and tables, but to no avail.

                rm -f /var/lib/jabberd/db/*

                spacewalk-sql -i
                delete from rhnPushDispatcher;
                delete from rhnPushClient;
                \quit


I also tried re-configuring the jabberd service, to see if that would fix it, but it didn’t:

                service osa-dispatcher stop
service jabberd stop

cp -arv /etc/jabberd /etc/jabberd.bak
cp -arv /var/lib/jabberd/ /var/lib/jabberd.bak
rm -rf /etc/jabberd/* /var/lib/jabberd/*

yum reinstall jabberd -y
spacewalk-setup-jabberd

grep $(hostname) /etc/jabberd/*xml

service jabberd start
service osa-dispatcher start



I verified that ports are open:

                nmap -sT -p 5200-5400 localhost

Nmap scan report for localhost (127.0.0.1)
PORT     STATE SERVICE
5222/tcp open  xmpp-client
5269/tcp open  xmpp-server
5347/tcp open  unknown



I’ve run out of things to look at at this point. Any ideas?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20170918/cf13480c/attachment.htm>