ext3 file system becoming read only

[Swapana Ghosh]

Hi 

In our office environment few servers mostly  database servers and yesterday it
happened
for one application server(first time) the partion is getting "read only".

I was checking the archives, found may be similar kind of issues in the
2007-July archives. 
But how it has been solved if someone describes me that will be really helpful.

In our case, just at the problem started found the line in log file as follows:

     EXT3-fs error (device dm-12): edxt3_find_entry: reading directory #2015496
offset 2

Then one blank line
Then the line is

    Aborting journal on device dm-12.
    ext3_abort called

    Ext3-fs error (device dm-12): ext3_journal_start_sb: Detected aborted
journal
    Remounting filesysem read-only

Then the continuous line as follows:


    EXT3-fs error (device dm-12) in start_transaction: Journal has aborted


  
The above message is continuous  until we remount the filesystem and partion
becomes
'read-write'.

We could not figure it out what is the root cause of the system.

We are using individual EMC luns and are configured with LVM volume groups and
then mounted on logical
volumes.

Here i am giving the server description:

____________________________________________________________

[root at server ~]# lsmod |grep -i qla
qla2300               130304  0 
qla2xxx_conf          305924  0 
qla2xxx               307448  21 qla2300
scsi_mod              117709  5 sg,emcp,qla2xxx,cciss,sd_mod

____________________________________________________________
[root at server ~]# cat /etc/modprobe.conf
alias eth0 tg3
alias eth1 tg3
alias eth2 e1000
alias eth3 e1000
alias eth4 e1000
alias eth5 e1000
alias bond0 bonding
alias scsi_hostadapter cciss
options bond0 max_bonds=2 miimon=100 mode=1
alias scsi_hostadapter1 qla2xxx
alias scsi_hostadapter2 qla2xxx_conf
#alias scsi_hostadapter3 qla6312
options qla2xxx  ql2xmaxqdepth=16 qlport_down_retry=64
ql2xloginretrycount=30 ql2xfailover=0 ql2xlbType=0
install qla2xxx /sbin/modprobe qla2xxx_conf; /sbin/modprobe
--ignore-install qla2xxx
remove qla2xxx /sbin/modprobe -r --first-time --ignore-remove qla2xxx && {
/sbin/modprobe -r --ignore-remove qla2xxx_conf; }
###BEGINPP
include /etc/modprobe.conf.pp
###ENDPP
###BEGINPP
include /etc/modprobe.conf.pp
###ENDPP
###BEGINPP
include /etc/modprobe.conf.pp
###ENDPP

________________________________________________
[root at server ~]# rpm -qa |grep -i EMC
EMCpower.LINUX-4.5.1-022

________________________________________________
[root at server ~]# rpm -qa|grep -i scli
scli-1.06.16-57

________________________________________________
[root at server ~]# rpm -qa|grep -i nav
naviagentcli-6.19.1.3.0-1

________________________________________________
 product: QLA2312 Fibre Channel Adapter

________________________________________________
[root at server ~]# rpm -qa|grep -i lvm
lvm2-2.02.06-6.0.RHEL4
system-config-lvm-1.0.19-1.0

________________________________________________

If I missed any info, pl. let me know. 

It would be really appreciated if I get some hints to solve the issues

Thanks in advance
-swapana



       
____________________________________________________________________________________
Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games.
http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow