[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Spacewalk-list] Postgresql Fails to Start After Datastore Failure



I celebrated too early… although the service starts up now and I can access spacewalk with all of the information intact, if I try to add any new items or make changes to a lot of existing stuff, I end up receiving internal errors on subsequent attempts to access those items.  For instance, adding a new machine to SW will generate an internal server error, deleting a machine will not remove the machine from SW and I will see internal server errors when trying to click on it from the interface. 

 

I tried to upgrade to 2.2 but I get errors on the schema upgrade step.  I think at this point I will just work on building a replacement and then use the old server to verify all of my settings, unless anyone has some last ditch attempt ideas?   

 

From: spacewalk-list-bounces redhat com [mailto:spacewalk-list-bounces redhat com] On Behalf Of Paul Robert Marino
Sent: Monday, November 24, 2014 5:57 PM
To: spacewalk-list redhat com
Subject: Re: [Spacewalk-list] Postgresql Fails to Start After Datastore Failure

 

Wow that's a very unusual problem. PostgreSQL usually recovers from that kind of thing easily. The only scenarios where I can picture this happening is if there was a power loss and a battery in a RAID got depleted or the same scenario with consumer grade SATA drives because they often lie about the cache state when an fsync is called.

-- Sent from my HP Pre3

 


On Nov 24, 2014 12:58 PM, Glennie, Jonathan - 0443 - MITLL <jrglenni ll mit edu> wrote:

Phew… blowing away the pgsql commit log allowed it to start up successfully.  There may have been items in there that were lost, but I’ll take that any day over having to recreate the entire server from scratch.  Thanks for your help!  

 

From: spacewalk-list-bounces redhat com [mailto:spacewalk-list-bounces redhat com] On Behalf Of Glennie, Jonathan - 0443 - MITLL
Sent: Monday, November 24, 2014 11:32 AM
To: spacewalk-list redhat com
Subject: Re: [Spacewalk-list] Postgresql Fails to Start After Datastore Failure

 

I found the log file… looks like it is complaining that it cannot complete the autorecovery and the database needs to be restores from backup.  Unfortunately all of my VM snapshots got hosed and that was my backup….

 

I’m still hoping there is a way I can attempt to repair the schema while preserving the data… in the process of trying to re-run the spacewalk-setup. 

 

From: spacewalk-list-bounces redhat com [mailto:spacewalk-list-bounces redhat com] On Behalf Of Glennie, Jonathan - 0443 - MITLL
Sent: Monday, November 24, 2014 10:13 AM
To: spacewalk-list redhat com
Subject: Re: [Spacewalk-list] Postgresql Fails to Start After Datastore Failure

 

I had received the lock error message before so I did start by removing that file.  It removed that error message but it was still saying Failed whenever I tried to start the service. 

 

Are there any commands that I can use to stat the DB manually in a more verbose way so I can get some additional info on what’s failing?   Or is there any “repair” command I can use from spacewalk to re-install the service but not overwrite the database?  

 

From: spacewalk-list-bounces redhat com [mailto:spacewalk-list-bounces redhat com] On Behalf Of Waldirio Manhães Pinheiro
Sent: Friday, November 21, 2014 8:05 PM
To: spacewalk-list redhat com
Subject: Re: [Spacewalk-list] Postgresql Fails to Start After Datastore Failure

 

Hello Jon,

 

You can check in /var/lib/pgsql/data, check if you have a file named postmaster.pid, if yes, you can remove (rm postmaster.pid) and restart or start the postgresql daemon.

 

[root spacewalk ~]# service postgresql start

 

if you check, will be recreated the file

 

[root spacewalk data]# ll postmaster.pid

-rw-------. 1 postgres postgres 45 Nov 21 23:03 postmaster.pid

[root spacewalk data]#

 

The problem normally is, if your db are running and for any reason you lost the connection with the directory, the file will still there and when you try to restart, the file will not be rewrite and the daemon will not start.

 

Try it and tell me if fix your problem.

 

Take Care and have a great weekend.

 

 


 

On Fri, Nov 21, 2014 at 6:36 PM, Glennie, Jonathan - 0443 - MITLL <jrglenni ll mit edu> wrote:

Hello-

 

Our spacewalk server failed badly the other day when the underlying datastore went offline.  The system boots but when spacewalk tries to start up, the postgresql service fails to start.  I have checked /var/log/messages for any indication as to why, but there is nothing I can find.  I’ve also checked in /var/lib/pqsql/pgstatrtup.log but the entries in there seem old and not relevant to the current situation. 

 

Does anyone know where or how I might go about starting the service in some kind of debug mode to try and get a better idea of what’s going on, or if there are logs that get saved somewhere that I haven’t found yet?  I’m definitely not very well versed in postgrest DB maintenance or troubleshooting, so any help would be greatly appreciated.  Thanks in advance.

 

-Jon

 

 


_______________________________________________
Spacewalk-list mailing list
Spacewalk-list redhat com
https://www.redhat.com/mailman/listinfo/spacewalk-list

 

Attachment: smime.p7s
Description: S/MIME cryptographic signature


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]