[Spacewalk-list] Building Spacewalk With a DNS Alias for Failover

Eric ericb at enrsystems.com
Sat Feb 18 01:28:00 UTC 2017


    
Holy cow!  What a great write up!  Lot to digest here.  Very much appreciated!


Happy Connecting. Sent from my Sprint Samsung Galaxy S® 5 Sport

-------- Original message --------
From: "William H. ten Bensel" <WHTENBEN at up.com> 
Date: 2/17/17  3:34 PM  (GMT-07:00) 
To: spacewalk-list at redhat.com 
Subject: Re: [Spacewalk-list] Building Spacewalk With a DNS Alias for	Failover 

Not crazy at ALL, that is what I do+.  
Hopefully this brain dump will help. 

GOTCHA's ..... If you are using ISS
to sync other Spacewalks... You have to put BOTH failoverhost.network.comand primaryhost.network.comin the list!  Spacewalk does not associate ALIAS
to connecting server.  It only allows one name for connecting server
and does not do a DNS look up to see if the ALIAS listed matches the connecting
server.

Background...  I have an SLA of
99.9+.  So, up time and recovery are critical.  I support multiple
spacewalks and use this logic even on the Spacewalk's respective proxies.
 The link  from 2016 is outdated (https://www.redhat.com/archives/spacewalk-list/2016-May/msg00129.html)
so here is some updated information.

NOTE:  Both Primary and failover have the SAME version of Spacewalk.
 That is a requirement.  If you upgrade one, you must upgrade
the other.   

First things First... Before installing
the Spacewalk, I create the logical volumes/filesystems that I know have
to be sync'd or replaced..  If you put these on SAN, NFS, Gluster,
RedHat Storage, DFS, etc.. or need to instruct someone how to restore..
It is easy to say.. Look at the filesystems.. restore to there or unmount
and remount.   No point of restoring the entire server or reinstalling
the application, if you have a hot standby already.

Updated paths and sizing.
These paths get replaced or restored
on the Failover server.

  "mount_path": "/etc/cobbler",   "size":
"1"
  "mount_path": "/var/lib/cobbler",
 "size": "1"
  "mount_path": "/var/log/cobbler",
 "size": "1"
  "mount_path": "/var/log/httpd",
 "size": "6"
  "mount_path": "/var/lib/jabberd/db",
 "size": "1"  --> PREVENT jabberdb CORRUPTION
and osad system re-registration
  "mount_path": "/etc/jabberd",
 "size": "1"
  "mount_path": "/var/cache/rhn",
 "size": "50"
  "mount_path": "/etc/rhn",
 "size": "1"
  "mount_path": "/etc/httpd",
 "size": "1"
  "mount_path": "/var/lib/rhn",
 "size": "1"
  "mount_path": "/var/log/rhn",
 "size": "3"
  "mount_path": "/var/log/tomcat6",
 "size": "1"
  "mount_path": "/var/www",
 "size": "1"
  "mount_path": "/usr/share/spacewalk",
 "size": "1"
  "mount_path": "/etc/pki/spacewalk",
 "size": "1"
  "mount_path": "/var/cache/rhn",
 "size": "40"
  "mount_path": "/var/www",
 "size": "5"
  "mount_path": "/var/satellite",
"size": "500"


Do the following in this order:
        Primary
(#'s below)
        Shutdown
Spacewalk
        Restore
/etc/hosts, /etc/sysconfig/network, hostname to the original   (Linux
7 /etc/hostname)
        Failover
(#'s below)
         
      READ COMMENTS ABOUT DB..
        Shutdown
Spacewalk 
        Restore
/etc/hosts, /etc/sysconfig/network, hostname to the original  (Linux
7 /etc/hostname)
        Update
/etc/hosts, /etc/sysconfig/network, hostname on the primary    (Linux
7 /etc/hostname)
        Startup
Spacewalk

# update /etc/hosts, 1st line, needs
to point IP address to  "  IP spacewalk.network.comspacewalk "
# update /etc/sysconfig/network with
HOSTNAME=spacewalk.network.com  (Linux 7 /etc/hostname)
# run hostname spacewalk.network.com
# log out and back in to pick up the
changes.
## Install the software  PAY ATTENTION IN THE NEXT COMMAND OR ELSE
YOU CAN WIPE THE DB!!!  ANSWER N if the DB already has schema!!!!!
## NOTE ---> I use external DB that
has a failover.  That is not covered in this write-up... If you are
going local DB.. Then ?? You will have to failover the DB section when
configuring the Failover server.
# spacewalk-setup --disconnected --external-oracle

# vi /etc/rhn/rhn.conf
        Add
the following and adjust accordingly for the Spacewalk
        server.satellite.http_proxy
= my_proxy:my_port
        disconnected=0
 <----- THIS IS TO SUPPORT ISS
        #
The following overrides the defaults in /usr/share/rhn/config-defaults/*.conf
        product_name
= THIS IS MINE
        java.custom_header
= THIS IS MINE
        java.config_file_edit_size
= 128
        web.maximum_config_file_size
= 1048576
        maximum_config_file_size
= 1048576
 
        web.default_mail_from
= MINE <root at spacewalk.network.com>
        #
this setting determines web session timeout, 
        web.session_cookie_lifetime
= 28800
        web.session_database_lifetime
= 28800

        #BUG
FIX: Increase rhn-search https://access.redhat.com/solutions/322093
        rhn-search.java.maxmemory
= 2048
 
        #BUG
FIX: Increase java heap size and logging: https://access.redhat.com/solutions/43122
        #
/usr/bin/taskomaticd, processes the config files and flags for taskomatic.
        taskomatic.java.maxmemory=12288
 #If you have this much RAM
        taskomatic.java.initmemory=1024
        taskomatic.logfile.maxsize=100m
        java.taskomatic_channel_repodata_workers=8
        #taskomatic.logfile.loglevel
= DEBUG# legal note to be displayed on the login page
        #
(join long lines with \, HTML is okay to use)
        java.legal_note
= YOU HAVE REACHED........
 
        #https://www.redhat.com/archives/spacewalk-list/2016-September/msg00016.html
        #
The number of days to wait to hear from a system before warning that it
is not checking in.
        #web.system_checkin_threshold
= 5

#vi /var/lib/rhn/rhn-satellite-prep/satellite-local-rules.conf
 (/var/lib/rhn is on a Filesystem)
        serverDOTsatelliteDOThttp_proxy=my_proxy:my_port
#vi /etc/sysconfig/tomcat6
        #Increase
java heap size and logging: https://access.redhat.com/solutions/43122
        Modify
Xmx to Xmx16384m  #If you have this much RAM
        Modify
Xms to Xms2048m 
#vi  /etc/httpd/conf.d/zz-mine.conf
        #Increase
http Timeout: https://access.redhat.com/solutions/109993
        Timeout
600 
#Move configs from /etc/sysconfig
and snippets to SAN, DFS, etc, to support failover
        mkdir
-p /somedirectory/spacewalk/configs (owned by root:root)
        mv
/etc/sysconfig/tomcat6  /somedirectory/spacewalk/configs
        mv
/etc/sysconfig/osa-dispatcher  /somedirectory/spacewalk/configs
        mv
/etc/rhn/rhn.conf  /somedirectory/spacewalk/configs
        ln
-s  /somedirectory/spacewalk/configs/tomcat6 /etc/sysconfig/tomcat6
        ln
-s  /somedirectory/spacewalk/configs/osa-dispatcher /etc/sysconfig/osa-dispatcher
        ln
-s  /somedirectory/spacewalk/configs/rhn.conf /etc/rhn/rhn.conf
        mkdir
-p /somedirectory/spacewalk/snippets/
        mv
/var/lib/rhn/kickstarts/snippets/1 /somedirectory/spacewalk/snippets/  -->
If this fails, mkdir -p /somedirectory/spacewalk/snippets/1
        mv
/var/lib/rhn/kickstarts/snippets/2 /somedirectory/spacewalk/snippets/ -->
If this fails, mkdir -p  /somedirectory/spacewalk/snippets/2
        ln
-s  /somedirectory/spacewalk/snippets/1 /var/lib/rhn/kickstarts/snippets/1
        ln
-s /somedirectory/spacewalk/snippets/2 /var/lib/rhn/kickstarts/snippets/2
#vi /etc/logrotate.d/httpd by adding
the following. 
        #FIX
ME
        rotate
4
        compress

        dateext

#Unable to share all of the code...
chkconfig --add ; chkconf on ... /systemctl enable ... 
#Lots of these snippets of code are functions.. I use the same init scripts
on Spacewalks and the proxies.
# vi /etc/int.d/MYSpacewalk

   #Start/restart
   #Disable the spacewalk
components... DO NOT want them to start up because this script takes care
of it ALL.
    if [[ "$rhn_product"
== "spacewalk" ]]; then
        for spaceservice
in $($rhn_cmd list |grep "4:" |awk '{print $1}'); do
           
$rhn_cmd disable $spaceservice
        done
    elif [[ "$rhn_product"
== "proxy" ]]; then
        for proxy_apps
in squid httpd jabberd; do
           
chkconfig $proxy_apps off
        done
    fi

  #Check if server has spacewalk.network.com
  serverinf_ip=$(ip route get 1
| awk '{print $NF;exit}' |tr -d '\n')
  for cname_ip in $(/usr/bin/host
$spacecname|grep address|awk -F" address " '{print $2}'); do
.......
  
 
 
 # If the server has the DNS alias
 spacewalk.network.com
      check_update_files -> which
does the following
        #
validate or update /etc/hosts, 1st line, needs to point IP address to  
" IP spacewalk.network.comspacewalk "
        sed and echo here
        #
validate or update /etc/sysconfig/network with HOSTNAME=spacewalk.network.com
  (Linux 7 /etc/hostname)
           
    sed -i "s/^HOSTNAME=.*/HOSTNAME=$what_I_AM_LOOKINGFOR/"
/etc/sysconfig/network
        #
run hostname spacewalk.network.com
    #check to see if it is already running
                spacewalk-service
restart
          or
           
     spacewalk-service start
 else
        check_update_files
-> which also does the following
        #
validate or update /etc/hosts, 1st line, needs to point IP address to  "
IP  primaryhost/failoverhost.network.comprimaryhost/failoverhost "
        sed and echo here
        #
validate or update /etc/sysconfig/network with HOSTNAME=primaryhost/failoverhost.network.com
           
    sed -i "s/^HOSTNAME=.*/HOSTNAME=$what_I_AM_LOOKINGFOR/"
/etc/sysconfig/network    (Linux 7 /etc/hostname)
        #
run hostname primaryhost/failoverhost.network.com
   
 #If not already stopped
                spacewalk-service
stop

  #Stop
        check_update_files
(from above)
         #If
not already stopped
                spacewalk-service
stop
       
        
- Thanks and good luck




From:      
 ericb at enrsystems.com
To:      
 spacewalk-list at redhat.com
Date:      
 02/17/2017 02:15 PM
Subject:    
   [Spacewalk-list]
Building Spacewalk With a DNS Alias for Failover
Sent by:    
   spacewalk-list-bounces at redhat.com



This email originated from outside of the company.
 Please use discretion if opening attachments or clicking on links.

I have a current running Spacewalk Server, and a 2nd Failover server.  The

current configuration is quite painful to failover.  

I'm currently standing up a new server on 2.6.  The server's real
fqdn is 
primaryhost.network.com, and the failover is failoverhost.network.com.
 I have 
a DNS alias of spacewalk.network.com that is currently pointing at the
same IP 
as primaryhost.network.com.  I want to install it using the DNS alias.
 I 
can't change the "real" hostname.

How can I run this with dual hostnames, and have Spacewalk use the the
Alias, 
especially for cert creation, etc.

The logic being that bringing the failover online should be much easier,

restore the backups to the failover and swing DNS to have the Alias point
at 
the IP of the failover server.

Thoughts?  Am I crazy?  I feel like this should work.

_______________________________________________
Spacewalk-list mailing list
Spacewalk-list at redhat.com
https://www.redhat.com/mailman/listinfo/spacewalk-list








**







This email and any attachments may contain information that is confidential and/or privileged for the sole use of the intended recipient.  Any use, review, disclosure, copying, distribution or reliance by others, and any forwarding of this email or its contents, without the express permission of the sender is strictly prohibited by law.  If you are not the intended recipient, please contact the sender immediately, delete the e-mail and destroy all copies.



**
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20170217/713bdb40/attachment.htm>


More information about the Spacewalk-list mailing list