[Spacewalk-list] Mr. 500 here... more info

Thu Feb 5 21:18:34 UTC 2009

Mairin,

>Date: Thu, 05 Feb 2009 15:29:00 -0500
>From: Máirín Duffy <duffy at redhat.com>  
>m.roth2006 at rcn.com wrote:
>> Fascinating.
>> 
>> I found 
>> <http://www.mail-archive.com/spacewalk-list@redhat.com/msg01001.html>.
>>  In that, someone with the *same* problem as me, he suggests trying
>> to browse to 
>> <https://spacewalk.example.com/network/software/channels/manage/index.pxt>
>> 
>> So, on my system, I tried, and got my 500 error. Then I backed up... 
>> and I can see a directory of 
>> <mymachine>/network/software/channels/manage, and all the way down to
>>  <mymachine>... and at that point, I get a *different* sign-in page, 
>> at <mymachine>/rhn/Login.do
>> 
>> I just went back to com/network/software/channels/manage. If I point 
>> my browser at delete_confirm.pxt, I get that. Every other .pxt there 
>> gives me a 500 error.
>
>Parts of the UI are JSPs (Java stack), parts are PXTs (perl stack). It

Right, I got that from working through some debugging (including putting print to a file statements in Access.pm).

>sounds like you're able to navigate the JSP application pages but only
>one of the PXT pages. I have never gotten this problem on stable version
>of Spacewalk (or Satellite for that matter) but running my 

Well, I forget if it was last Thurs or Fri that I pulled down the 64-bit version with yum; the 32-bit version I did yesterday morning (4 Feb).

>own devel
>version out of a repo checkout I have run into this PXTs == 500 problem 
>sometimes when I'm running the app and then a change gets checked into 
>one of the rhn.conf files and I pull it down. (without having it in 
>front of me now, I think they are in rhn conf files in /etc/rhn or 
>/etc/satellite or /etc/sysconfig/rhn, there's various PXT config 
>settings to tweak in them) Usually my issue was that there is a DB 
>location config line somewhere just for the PXT stack, and it'd be 
>pointing to the wrong DB so I'd have to fix it. However, again, this is 
>running a devel version out of checkout - never ever had an issue like 
>that running a stable released version.
>
>I seem to have missed from reading through your messages to this list 
>the state the VM was in when you started installing Spacewalk onto it. 
>Was it a clean install? Did it used to have some other app installed on 
>it? Was it installed with @Base (I believe still the recommended 
>starting point) or was more installed on it by default?

These were clean installs, both the 64-bit CentOS and the 32-bit. Nothing else is running on them; they were intended solely for spacewalk.
>
>The reason I ask is because you should *never*, ever have to manually
>muck around with file permissions, etc. when installing Spacewalk. Has
>anyone else on the list here ever had to do this? (My guess is no.) The

Well, I was reading error log files, and the permission on /etc/rhn/rhn.conf was 600, so apache couldn't read it.

Then there were cobbler error messages, so with help from folks here, I fixed *them*. I even added a wrapper.conf, since *it* was giving error messages, and that included putting the 
WRAPPER_CMD="/usr/sbin/tanukiwrapper"
WRAPPER_CONF="/etc/tomcat5/wrapper.conf"
 in /etc/init.d/tomcat, after tomcat gets its own configuration files.

That got rid of every other error message in the tomcat log, the cobbler log, and /var/log/messages... except for this one.

>installer should really handle that for you and if it didn't then

I agree wholeheartedly.

>something must have gone wrong during installation or something on the
>machine was not set up correctly. Whatever went wrong that caused those
>permissions to not be set correctly is likely the problem, and changing
>the permissions manually is really just treating the symptoms and sort
>of pulling on the string sticking out of the rug and unravelling it
>rather than re-weaving and fixing it ;-)

I don't know what else to do. This install yesterday was on a machine my manager built from the std. template they use here for VMs. I followed that by following the directions for installed Oracle xe and spacewalk. Dunno how much cleaner I can go.

>
>Do you remember if there were any errors or problems that occurred
>during installation? E.g. maybe something took too long so you hit
>Ctrl+C then came back later and restarted it? If not, since 

Hmmmm... during the spacewalk install, after everything else is done, and it says everything is started, it just sits there. After minutes (2? 3? 5?) I did finally hit <ctrl-C>, and even then it takes a while to end.

>I'm not 
>seeing any posts by anyone else running into this, I'm 

Actually, in my last post, there was a thread about just this on 23 Jan on this list.

>thinking there's 
>some variable unqiue to your environment that is triggering this problem.

I would *love* to know what.
>
>Can you post somewhere the full log that's written out during
>installation? The RHN app logs (/var/log/rhn I think), the httpd logs,
>and any install or other logs that cobbler may spit out? Any logs that
>Oracle might spit out during the Spacewalk install? Also the first
>snippet of the log you posted in your first post, mentioned it sent a
>traceback to root at localhost - can you post that full traceback? Do you

Sure. Took me a while, and I finally realized it needed the sendmail daemon on.

The following exception occurred while executing this request:
 GET /network/software/channels/manage/edit.pxt HTTP/1.1 (from browser)  /network/software/channels/manage/edit.pxt (from Apache)

Date:
  Thu Feb  5 12:04:07 2009

Headers:
  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
  Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
  Accept-Encoding: gzip,deflate
  Accept-Language: en-us
  Connection: keep-alive
  Cookie: __utma=118906060.4410793078631395000.1231965409.1232030643.1233178612.3; __utmz=118906060.1231965409.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); pxt-session-cookie=92xe21f41f8fc58aa4194c88212ed3ad8ef
  Host: <FQDN>
  Keep-Alive: 300
  Referer: https://<FQDN>/network/software/channels/manage/
  User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5

Form variables:

User Information:
(not logged in)

Error notes:
  (none)

Initial Request:
  Yes

Error message:
  Odd number of parameters in call to Sniglets::HTML::render_help_link when named parameters were expected  at /usr/lib/perl5/site_perl/5.8.8/Sniglets/HTML.pm line 179
	Sniglets::HTML::render_help_link('-user', '-guide', 'channel-mgmt', '-href', 'channel-mgmt-Custom_Channel_and_Package_Management-Managed_So...', '-block', '<img src="/img/rhn-icon-help.gif" alt="Help Icon" />', '-satellite', 'undef', ...) called at /usr/lib/perl5/site_perl/5.8.8/Sniglets/HTML.pm line 156
	Sniglets::HTML::rhn_help('PXT::Request=HASH(0x983d62c)', 'href', 'channel-mgmt-Custom_Channel_and_Package_Management-Managed_So...', 'guide', 'channel-mgmt') called at /usr/lib/perl5/site_perl/5.8.8/Sniglets/HTML.pm line 95
	Sniglets::HTML::toolbar('PXT::Request=HASH(0x983d62c)', 'base', 'h1', 'help-guide', 'channel-mgmt', 'deletion-url="/network/software/channels/manage/delete_confir...', '{channel_id}', 'deletion-acl', 'user_role(channel_admin); formvar_exists(cid)', ...) called at /usr/lib/perl5/site_perl/5.8.8/PXT/Parser.pm line 171
	PXT::Parser::expand_tag('PXT::Parser=HASH(0x9844b1c)', 'rhn-toolbar', 'CODE(0x9ec98cc)', 'SCALAR(0x980d5e4)', 'PXT::Request=HASH(0x983d62c)') called at /usr/lib/perl5/site_perl/5.8.8/PXT/Parser.pm line 83
	PXT::Parser::expand_tags('PXT::Parser=HASH(0x9844b1c)', 'SCALAR(0x980d5e4)', 'PXT::Request=HASH(0x983d62c)') called at /usr/lib/perl5/site_perl/5.8.8/PXT/ApacheHandler.pm line 632
	PXT::ApacheHandler::pxt_parse_data('PXT::ApacheHandler', 'PXT::Request=HASH(0x983d62c)', 'SCALAR(0x980d5e4)') called at /usr/lib/perl5/site_perl/5.8.8/PXT/ApacheHandler.pm line 124
	eval {...} called at /usr/lib/perl5/site_perl/5.8.8/PXT/ApacheHandler.pm line 124
	PXT::ApacheHandler::handler('Apache2::RequestRec=SCALAR(0x8648a98)') called at -e line 0
	eval {...} called at -e line 0

>see any errors that indicate perhaps the PXT stack can't correctly
>contact the DB or can you see in the Oracle logs that it's receiving 
>requests from the PXT stack?
>
Oracle logs I really don't know a lot about. I do find some errors, but I don't know what they mean:

>From 
2009-02-05 11:39:25.364: [  OCROSD][3069982400]utgdv:2:ocr loc file  cannot  be opened
2009-02-05 11:39:25.364: [  OCROSD][3069982400]utopen:1: Couldnt find ocr,[ocrmirror] location in config file
[  OCRUTL][3069982400]u_set_gbl_comp_error: Parameter was NULL
2009-02-05 11:39:25.364: [  OCRRAW][3069982400]proprinit: Could not open raw device
2009-02-05 11:39:25.364: [ default][3069982400]a_init:7!: Backend init unsuccessful : [33]
[  OCRUTL][3069982400]u_set_ocr_error: Parameter was NULL
2009-02-05 11:39:25.364: [ CSSCLNT][3069982400]clsssinit: error(33 ) in OCR initialization

[  OCRUTL][3069773504]u_set_ocr_error: Parameter was NULL
2009-02-05 11:39:25.517: [  OCROSD][3069773504]utgdv:2:ocr loc file  cannot  be opened
2009-02-05 11:39:25.518: [  OCROSD][3069773504]utopen:1: Couldnt find ocr,[ocrmirror] location in config file
[  OCRUTL][3069773504]u_set_gbl_comp_error: Parameter was NULL
2009-02-05 11:39:25.518: [  OCRRAW][3069773504]proprinit: Could not open raw device
2009-02-05 11:39:25.518: [ default][3069773504]a_init:7!: Backend init unsuccessful : [33]
[  OCRUTL][3069773504]u_set_ocr_error: Parameter was NULL
2009-02-05 11:39:25.518: [ CSSCLNT][3069773504]clsssinit: error(33 ) in OCR initialization

LEM stack 1: error [clsdfopen1] [

LEM stack 2: error [clsdfgenfullname] [sclsdgcwd]
********************
*No* idea what this, above, is.

And from ./app/oracle/product/10.2.0/server/network/log/listener.log, I get

05-FEB-2009 14:41:17 * 12502
TNS-12502: TNS:listener received no CONNECT_DATA from client
05-FEB-2009 14:42:37 * (CONNECT_DATA=(SID=xe)(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))) * (ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=48408)) * establish * xe * 0
**************
This seems to indicate the same thing as the $pxt->user error.

On more thing: the spacewalk install set the default value for rhn_command_parameter to /opt/oracle where param_name = 'ORACLE_HOME'. Now, that *is* the std. location for a real Oracle install, or even for 9i, but xe installs under /usr/lib/oracle, and ORACLE_HOME points to /usr/lib/oracle/xe/app/oracle/product/10.2.0/server. HOWEVER, that's for the Oracle environment, and I don't know whether spacewalk needs the client as home. Any ideas?

>DISCLAIMER: I am not a Spacewalk developer, I'm a UI designer so I
>really don't know what's going on at a very deep level at all. I 
>apologize if I'm belaboring the obvious. But hopefully maybe something 
>I've written here might trigger something that leads to a solution.

Oh, no, but I really appreciate this. I've also posted my tale-o'-woe, with the web traceback log, to the spacewalk-devel list, and asked if I need to file it as a bug report.

       mark