From takeshi.uematsu at gmail.com Fri Feb 4 03:57:59 2011 From: takeshi.uematsu at gmail.com (Uematsu Takeshi) Date: Fri, 4 Feb 2011 12:57:59 +0900 Subject: minus disk usage Message-ID: Hi,This is Takeshi Uematsu. I have a trouble on the ext3 filesystem. The df command indicates minus disk usage. Dose anyone know why this happned ? Any ideas be appreciated. # df -h Filesystem Size Used Avail Use% Mounted on /dev/sda2 97G -26G 118G - / /dev/sda1 99M 15M 80M 16% /boot tmpfs 2.0G 0 2.0G 0% /dev/shm /dev/sda3 803G 3.9G 758G 1% /home fdisk /dev/sda Device Boot Start End Blocks Id System /dev/sda1 * 1 13 104391 83 Linux /dev/sda2 14 13067 104856255 83 Linux /dev/sda3 13068 121193 868522095 83 Linux /dev/sda4 121194 121454 2096482+ 82 Linux swap / Solaris From samuel at bcgreen.com Sat Feb 5 04:16:30 2011 From: samuel at bcgreen.com (Stephen Samuel) Date: Fri, 4 Feb 2011 20:16:30 -0800 Subject: minus disk usage In-Reply-To: References: Message-ID: Sooner or later, the system WILL come to a stop -- either on purpose, or in a crash ... the only question is whether or not it will be recoverable. Then again, how big is the filesystem? .. Nevermind... I looked back at your original post. A couple of ideas: 1) make a backup AS SHORT AS POSSIBLE before shutting down the system... If possible, after all services have been shut down. (if you can't do that, then backup everything, then backup everything that's changed since the first backup started, then shut down services and backup everything since the incremental backup started. (for minimal down time). If you've backed up to a second system that can replace the running (and broken system), then use the second system in place of the corrupt system, and you're now relatively happy. You can work on cleaning up the filesystem and/or figuring out what broke it. otherwise, you can either (1) wipe the filesystem and restore it from backup, or (2) FSCK the system in hopes of cleaning things up ((( or, ultimately, both, if #2 fails))). A couple of more options: 1: install / build smartmontools for your system and make sure that the drive, itself, isn't failing. 2: the problem is with your root filesystem, and I'm presuming that /home is where most of your 'live' data is. This means that, if you follow the process above, your final incremental backup should be TINY. If the hard drive, itself, isn't failing, then you can build yourself a new root filesystem (on an external usb/esata drive, if you don't have hot-swap capability) from/as your backup , and boot off of that. That would mean a minimal investment in new hardware (if you have a limited budget and/or esoteric hardware on your system). and minimal downtime (a few minutes) . If you have a second machine with the same/similar mother board, then you can practice booting off of the spare drive there. Worst case would be to boot off of a live CD and do an FSCK that way, but it could take quite a while, and there is no guarantee that the filesystem will have all of the parts left to be bootable after that, if FSCK removes a bunch of critical files -- or the files are already corrupted. On Fri, Feb 4, 2011 at 9:04 AM, Uematsu Takeshi wrote: > Stephen, > Thank you for your adviece. > > I have not done an FSCK yet. > /dev/sda2 is /(root) partition. > I know that it shouldn't execute fsck with the mounted filesystem. > and cannot umount root file system. > > This system cannot be stopped. > Do you know how to check the filesystem without stop ? > > > 2011/2/4 Stephen Samuel : > > 1) do a backup (NOW!) if you haven't already done TWO. > > 2) Have you done an FSCK with the filesystem unmounted? > > (given that you pretty clearly have some corruption, you might want > to > > do an 'fsck -n', and just see what FSCK is proposing to do to the system. > > > > On Thu, Feb 3, 2011 at 7:57 PM, Uematsu Takeshi < > takeshi.uematsu at gmail.com> > > wrote: > >> > >> Hi,This is Takeshi Uematsu. > >> > >> I have a trouble on the ext3 filesystem. > >> The df command indicates minus disk usage. > >> Dose anyone know why this happned ? > >> Any ideas be appreciated. > >> > >> # df -h > >> Filesystem Size Used Avail Use% Mounted on > >> /dev/sda2 97G -26G 118G - / > >> /dev/sda1 99M 15M 80M 16% /boot > >> tmpfs 2.0G 0 2.0G 0% /dev/shm > >> /dev/sda3 803G 3.9G 758G 1% /home > >> > >> fdisk /dev/sda > >> > >> Device Boot Start End Blocks Id System > >> /dev/sda1 * 1 13 104391 83 Linux > >> /dev/sda2 14 13067 104856255 83 Linux > >> /dev/sda3 13068 121193 868522095 83 Linux > >> /dev/sda4 121194 121454 2096482+ 82 Linux swap / > >> Solaris > >> > >> _______________________________________________ > >> Ext3-users mailing list > >> Ext3-users at redhat.com > >> https://www.redhat.com/mailman/listinfo/ext3-users > > > > > > > > -- > > Stephen Samuel http://www.bcgreen.com Software, like love, > > 778-861-7641 grows when you give it away > > > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From keith at karsites.net Sat Feb 5 09:59:49 2011 From: keith at karsites.net (Keith Roberts) Date: Sat, 5 Feb 2011 09:59:49 +0000 (GMT) Subject: minus disk usage In-Reply-To: References: Message-ID: On Fri, 4 Feb 2011, Stephen Samuel wrote: > Worst case would be to boot off of a live CD and do an > FSCK that way, but it could take quite a while, and there > is no guarantee that the filesystem will have all of the > parts left to be bootable after that, if FSCK removes a > bunch of critical files -- or the files are already > corrupted. I'd recommend using another machine (friends or someone else's) to download and burn a copy of the Ultimate Boot CD: http://www.ultimatebootcd.com/ There are *lots* of diagnostic and repair utilities on it - all together on one CD. If you need to boot a Live linux, then load up Parted Magic from the UBCD disk(I use the failsafe VGA mode). This is a complete linux system that runs entirely from memory. Parted Magic also has Gparted included with it, and a GUI interface for mounting file systems. If You connected a USB drive to your system (before running Parted Magic) you could mount your filesystems and the USB drive, then copy over any important data files from the possibly corrupt drive to the USB drive. Make sure any partitions you create on the USB drive do not have the *same* partition name as what's on your faulty system. This could cause a conflict with the fsck program. For example, if you have a partition on your system with an e2label called , then on the USB drive, call it something else, like - obviously without the <> delimiters. Kind Regards, Keith Roberts ----------------------------------------------------------------- Websites: http://www.karsites.net http://www.php-debuggers.net http://www.raised-from-the-dead.org.uk All email addresses are challenge-response protected with TMDA [http://tmda.net] ----------------------------------------------------------------- From alheidgj at upmc.edu Sat Feb 5 16:20:11 2011 From: alheidgj at upmc.edu (Alheid, Gregory) Date: Sat, 5 Feb 2011 16:20:11 +0000 Subject: minus disk usage In-Reply-To: References: Message-ID: > From: Uematsu Takeshi > To: ext3-users at redhat.com > Subject: minus disk usage > > Hi,This is Takeshi Uematsu. > > I have a trouble on the ext3 filesystem. > The df command indicates minus disk usage. > Dose anyone know why this happned ? > Any ideas be appreciated. > > # df -h > Filesystem Size Used Avail Use% Mounted on > /dev/sda2 97G -26G 118G - / > /dev/sda1 99M 15M 80M 16% /boot > tmpfs 2.0G 0 2.0G 0% /dev/shm > /dev/sda3 803G 3.9G 758G 1% /home > > fdisk /dev/sda > > Device Boot Start End Blocks Id System > /dev/sda1 * 1 13 104391 83 Linux > /dev/sda2 14 13067 104856255 83 Linux > /dev/sda3 13068 121193 868522095 83 Linux > /dev/sda4 121194 121454 2096482+ 82 Linux swap / Solaris You should really do a fsck of the "/" FS. You should have a backup before doing a fsck because if there are major errors, running a fsck can make a FS unusable. A couple of other things you can try before doing a fsck are: Use dumpe2fs and see if there are any issues; dumpe2fs /dev/sda2 Use find cmd to list any "unusually" large files, adjust the value +100000 as needed; find / -mount -size +100000 -exec ls -l {} \; Check the sizes of the directories and files within, here is a variation of a script I use to do this to list the 10 largest directories of the "/" FS; for d in $( ls -1 / ) ; do if [[ "x$d" = "xboot" || "x$d" = "xhome" ]] ; then : else ls -d /$d du -sk /$d fi done | sort -n | tail You can then check each of the listed directories with the following; du -sk /usr/* | sort -n | tail and do this on the directories listed until you find the issue. Lastly re-boot the system with a CD installed it the system with, or later, enter "linux rescue" and run fsck on the root FS. If there are any errors, answering yes will try to correct the FS errors. Greg From Florian.Weber at pfaffenhofen.de Sun Feb 6 17:23:57 2011 From: Florian.Weber at pfaffenhofen.de (Florian Weber) Date: Sun, 6 Feb 2011 18:23:57 +0100 Subject: [solved] Overwritten beginning of ext3 filesystem. Recovery? In-Reply-To: <20101227173751.qmvcc2ec8cgwwk0k@mail.bn-paf.de> References: <20101227173751.qmvcc2ec8cgwwk0k@mail.bn-paf.de> Message-ID: <201102061824.02169.Florian.Weber@pfaffenhofen.de> Hello list For the benefit of those searching the archives, here's how I got out of the mess described in my first mail on27dec2010. 0a. Be extra careful and use your brain before even touching the keyboard. Go to extreme measures to prevent typos. 0b. Keep backups: make a dd copy of your disk/partition and put the original hardware into a safe. Do not work on the master image. Make it read-only, create _another_ working copy and use _that_ for recovery. You will make mistakes and you do not want to touch the hardware. 0c. Be prepared to learn a lot of things about filesystems you never wanted to know. 0c. Whatevery you do, check the units your tools are using. Each time. They might be filesystem blocks, disk blocks, inodes, bytes, kB, kiB, .... 0d. Have a look at the thread "recovery recommendations" started by "m.p." on 21jan2011 1. I overwrote the stuff I definitely knew to be faulty with zeroes, i.e. the first 10GB. I erred on the safe side, rather keeping bogus stuff than deleting good data. 2. This action killed my partition table. I had to restore it manually but was prepared for that. 3. I ran e2fsck on the broken partition. After the first run, it prompted me to be run again, which I did. 4. The filesystem now contained lost+found/ as it's only toplevel directory. Below that were many files and directories 5. I sorted those files/dirs and gave them meaningful names instead of block numbers. About 70% were recognizable from their content, among it *all* my personal data! The rest was unrecognizable binary and ASCII fragments which I discarded, there's a high likelyhood that most of it was actually deleted in the old filesystem, and it couldn't be restored anyway. 6. I compared against very old backups which showed no data loss. 7. I'm still doing lots of random samples to check for damaged files and loss of newer files, but found none so far. Conclusions: I got all my data back, but that was pure luck. To account not only for hardware but also for software and human failures, I have bought a USB harddisk which I use for weekly backups. I'm still evaluating which backup tools best fit my needs. Hope this helps someone. With best regards, Florian Weber From scerveau at awox.com Mon Feb 7 14:45:17 2011 From: scerveau at awox.com (Stephane Cerveau) Date: Mon, 7 Feb 2011 15:45:17 +0100 Subject: Compute the real total size of a partition formated in EXT3 Message-ID: Dear All, In order to have a real percentage of freespace for a user interface, I'm trying to compute the size available on a 4GB USB key formatted in Ext3. Indeed after format, when I ask df to give a summary of size, it tells that there is 75MB already used. I would like to know the meaning of this 75MB ( is it the journal??) and especially how I can compute this when I want, whithout parsing the partition and the size of the file(s). /dev/sda1 3.7G 71.5MB 3.4G 2% /mnt/internal Best regards. St?phane Cerveau Software Engineer scerveau at awox.com Phone: +33 4 99 53 27 39 93, Pierre Duhem 34000 Montpellier FRANCE Phone: +33 4 67 47 10 00 Fax: +33 4 67 47 10 15 cid:069332510 at 17112008-1528 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 16426 bytes Desc: image001.jpg URL: From adilger at dilger.ca Mon Feb 7 16:30:59 2011 From: adilger at dilger.ca (Andreas Dilger) Date: Mon, 7 Feb 2011 08:30:59 -0800 Subject: Compute the real total size of a partition formated in EXT3 In-Reply-To: References: Message-ID: On 2011-02-07, at 06:45, Stephane Cerveau wrote: > In order to have a real percentage of freespace for a user interface, I?m trying to compute the size available on a 4GB USB key formatted in Ext3. Indeed after format, when I ask df to give a summary of size, it tells that there is 75MB already used. > I would like to know the meaning of this 75MB ( is it the journal??) and especially how I can compute this when I want, whithout parsing the partition and the size of the file(s). > /dev/sda1 3.7G 71.5MB 3.4G 2% /mnt/internal There are several different things that add up to this overhead. The journal is a significant factor for smaller filesystems, but there are also inode tables, allocation bitmaps, reserved space, and a few other things. If you are using a very small embedded filesystem that doesn't need a lot of performance, you can reduce the size of the journal at format time with options like "-J size=4", and disable resizing with "-O ^resize_inode", which also removes some overhead. The amount of reserved space can be reduced with "-m " (default 5%), though this can lead to significant file fragmentation and permanent performance impact. Finally, depending on your workload/usage pattern, the number of the inodes in the filesystem can be reduced using "-i ". As for computing the available size, you can't really do better than what statfs() returns. Cheers, Andreas From adilger at dilger.ca Mon Feb 7 16:31:58 2011 From: adilger at dilger.ca (Andreas Dilger) Date: Mon, 7 Feb 2011 08:31:58 -0800 Subject: mount problems with ext3 In-Reply-To: <30660579.post@talk.nabble.com> References: <30660579.post@talk.nabble.com> Message-ID: On 2011-01-13, at 01:26, shivraj wrote: > i have created ext3 partition on NetBSD system > i am sending data to that ext3 partition using iscsi protocol. > i have mounted the same partition on another linux machine . > The problem is that to view new upcomming data on ext3 partition i need to > first umount and then need to mount the previously mounted partition. > can any one tell me whether i can view new data on the ext3 partition > without unmounting and mounting that partition? Ext3 is not a shared filesystem, and mounting it from two different systems will lead to data corruption and loss. You need to use something like NFS to do this. Cheers, Andreas From scerveau at awox.com Mon Feb 7 17:17:48 2011 From: scerveau at awox.com (Stephane Cerveau) Date: Mon, 7 Feb 2011 18:17:48 +0100 Subject: Compute the real total size of a partition formated in EXT3 In-Reply-To: References: Message-ID: Thank you for your answer. And how can I calculate all this things (inode tables, allocation bitmaps, reserved space, and a few other things). There is no way to know this system used space independently from the files stored on the FS ? Because I'm doing a user interface where the user could be disturb to see that after even a format the size available is not equal to the total size ;) Best regards. Stephane. -----Original Message----- From: Andreas Dilger [mailto:adilger at dilger.ca] Sent: lundi 7 f?vrier 2011 17:31 To: Stephane Cerveau Cc: ext3-users at redhat.com Subject: Re: Compute the real total size of a partition formated in EXT3 On 2011-02-07, at 06:45, Stephane Cerveau wrote: > In order to have a real percentage of freespace for a user interface, I'm trying to compute the size available on a 4GB USB key formatted in Ext3. Indeed after format, when I ask df to give a summary of size, it tells that there is 75MB already used. > I would like to know the meaning of this 75MB ( is it the journal??) and especially how I can compute this when I want, whithout parsing the partition and the size of the file(s). > /dev/sda1 3.7G 71.5MB 3.4G 2% /mnt/internal There are several different things that add up to this overhead. The journal is a significant factor for smaller filesystems, but there are also inode tables, allocation bitmaps, reserved space, and a few other things. If you are using a very small embedded filesystem that doesn't need a lot of performance, you can reduce the size of the journal at format time with options like "-J size=4", and disable resizing with "-O ^resize_inode", which also removes some overhead. The amount of reserved space can be reduced with "-m " (default 5%), though this can lead to significant file fragmentation and permanent performance impact. Finally, depending on your workload/usage pattern, the number of the inodes in the filesystem can be reduced using "-i ". As for computing the available size, you can't really do better than what statfs() returns. Cheers, Andreas __________ Information from ESET NOD32 Antivirus, version of virus signature database 5853 (20110207) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 5853 (20110207) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com From scerveau at awox.com Tue Feb 8 14:16:20 2011 From: scerveau at awox.com (Stephane Cerveau) Date: Tue, 8 Feb 2011 15:16:20 +0100 Subject: Compute the real total size of a partition formated in EXT3 In-Reply-To: References: Message-ID: Dear all, I understood better now how to compute a real available size and the total size on an Ext3 Fs Indeed in statfs or df, you have three field: f_blocks f_bfree f_bavail The difference between f_blocks and f_bfree is approximatively the size of the journal with some more information. The difference between f_bavail and f_bfree is the reserved space used to maintain the filesystem ( -m options approx 5%by default). So if you want to be safe and know the size available, use f_bavail and if you want to know the max size, add a "du -sh folder" result. Best regards. Stephane -----Original Message----- From: Andreas Dilger [mailto:adilger at dilger.ca] Sent: lundi 7 f?vrier 2011 17:31 To: Stephane Cerveau Cc: ext3-users at redhat.com Subject: Re: Compute the real total size of a partition formated in EXT3 On 2011-02-07, at 06:45, Stephane Cerveau wrote: > In order to have a real percentage of freespace for a user interface, I'm trying to compute the size available on a 4GB USB key formatted in Ext3. Indeed after format, when I ask df to give a summary of size, it tells that there is 75MB already used. > I would like to know the meaning of this 75MB ( is it the journal??) and especially how I can compute this when I want, whithout parsing the partition and the size of the file(s). > /dev/sda1 3.7G 71.5MB 3.4G 2% /mnt/internal There are several different things that add up to this overhead. The journal is a significant factor for smaller filesystems, but there are also inode tables, allocation bitmaps, reserved space, and a few other things. If you are using a very small embedded filesystem that doesn't need a lot of performance, you can reduce the size of the journal at format time with options like "-J size=4", and disable resizing with "-O ^resize_inode", which also removes some overhead. The amount of reserved space can be reduced with "-m " (default 5%), though this can lead to significant file fragmentation and permanent performance impact. Finally, depending on your workload/usage pattern, the number of the inodes in the filesystem can be reduced using "-i ". As for computing the available size, you can't really do better than what statfs() returns. Cheers, Andreas __________ Information from ESET NOD32 Antivirus, version of virus signature database 5853 (20110207) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 5855 (20110208) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com From neilb at suse.de Fri Feb 18 01:51:25 2011 From: neilb at suse.de (NeilBrown) Date: Fri, 18 Feb 2011 12:51:25 +1100 Subject: physical size of the device inconsistent with superblock, after RAID problems In-Reply-To: <122234.22513.qm@web65103.mail.ac2.yahoo.com> References: <122234.22513.qm@web65103.mail.ac2.yahoo.com> Message-ID: <20110218125125.22037092@notabene.brown> On Thu, 17 Feb 2011 15:53:11 -0800 (PST) Gavin Flower wrote: > Hi Neil, > > My attempted post to ext3-users at redhat.com, had not been published there (even though I had emailed it 4 days ago!), as at a minute ago. > > I finally bit the bullet and went ahead. > > I accepted the fixes put forward by fsck associated with bitmap differences, and rebooted. > > Still problems. > > Still had the discrepancy in the file size.? So I ran the command: > > resize2fs -p /dev/md1 76799616 > > I used the smaller of the 2 block counts, as: > (a) I needed to reduce the file system size, because I had already reduced the RAID size (I _SHOULD_ have done this first, before resizing the RAID), and > (b) it is reported as the 'physical' size of the device, so it is likely to be the correct value IMHO > > The system the came up successfully after a reboot, and I was able to log in as normal. > > There appeared to be no apparent loss of data, not that I did an exhaustive systematic check. However, several users have logged on successfully, and it is playing its part as gateway to the Internet, and squid appears to be providing its normal functionality. > > Neil, your help and encouragement was/is greatly appreciated! > Excellent! I'm glad you found a way through. As you didn't really trim very much from your device it is certainly possible that no critical data was there. Quite possibly resize2fs would have told you if there was (I certainly hope it would have done). NeilBrown From gavinflower at yahoo.com Tue Feb 15 03:14:34 2011 From: gavinflower at yahoo.com (Gavin Flower) Date: Mon, 14 Feb 2011 19:14:34 -0800 (PST) Subject: physical size of the device inconsistent with superblock, after RAID problems Message-ID: <1581.38921.qm@web65110.mail.ac2.yahoo.com> Hi, I would appreciate advice recovering from the following situation, after an aborted mdadm resizing operation and subsequent recovery actions: /dev/md1: The filing system size (according to the superblock) is 76799952 blocks The physical size of the device is 76799616 Either the superblock or the partition table is likely to be corrupt! /dev/md1: UNEXPECTED INCONSISTENCY: RUN fsck manually (i.e. without -a or -p options) fsck.ext4 -f -n /dev/md1 output: e2fsck 1.41.12 (17-May-2010) The filesystem size (according to the superblock) is 76799952 blocks The physical size of the device is 76799616 blocks Either the superblock or the partition table is likely to be corrupt! Abort? no Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information Block bitmap differences: -9626 -(9728--9752) +(405344--405369) Fix? no /dev/md1: ********** WARNING: Filesystem still has errors ********** /dev/md1: 1693644/19202048 files (0.3% non-contiguous), 54273929/76799952 blocks Note that original size, according mdadm, was not a multiple of 512KB, so I reshaped it to be the largest multiple or 512KB less than the original size using the -size option of mdadm. So my second attempt to reshape, using the 512 chunk size, started okay. The previous chunk size was 64KB. Note I am using Fedora 14, up-to-date as of Friday February 11th, and that there are 5X500KB drives, with 3 RAID-6 arrays: /dev/md0 swap /dev/md1 mostly user data (the problematic one) /dev/md2 distribution & O/S files plus /boot on a non-RAID ext4 partition Sequence of events: Reshaped /dev/md1 using mdadm, without first reducing size of the ext4 filesystem. The process of reshaping /dev/md1 was about 20% through when I killed it. System appeared okay. I rebooted a few minute later, but shortly after I selected the kernel, it stopped, and I dropped into a shell. With the help of Neil Brown, I made some progress and /dev/md1 reshaping appeared to have completed without error. However, on the next reboot I got the INCONSISTENCY message. Will it be safe to simply accept fsck's offer to fix, or are there other things I should do? Thanks, Gavin -- All Adults share the Responsibility to help Raise Today's Children, for they are Tomorrow's Society! From gavinflower at yahoo.com Thu Feb 17 23:53:11 2011 From: gavinflower at yahoo.com (Gavin Flower) Date: Thu, 17 Feb 2011 15:53:11 -0800 (PST) Subject: physical size of the device inconsistent with superblock, after RAID problems Message-ID: <122234.22513.qm@web65103.mail.ac2.yahoo.com> Hi Neil, My attempted post to ext3-users at redhat.com, had not been published there (even though I had emailed it 4 days ago!), as at a minute ago. I finally bit the bullet and went ahead. I accepted the fixes put forward by fsck associated with bitmap differences, and rebooted. Still problems. Still had the discrepancy in the file size.? So I ran the command: resize2fs -p /dev/md1 76799616 I used the smaller of the 2 block counts, as: (a) I needed to reduce the file system size, because I had already reduced the RAID size (I _SHOULD_ have done this first, before resizing the RAID), and (b) it is reported as the 'physical' size of the device, so it is likely to be the correct value IMHO The system the came up successfully after a reboot, and I was able to log in as normal. There appeared to be no apparent loss of data, not that I did an exhaustive systematic check. However, several users have logged on successfully, and it is playing its part as gateway to the Internet, and squid appears to be providing its normal functionality. Neil, your help and encouragement was/is greatly appreciated! Thanks, Gavin -- All Adults share the Responsibility to help Raise Today's Children, for they are Tomorrow's Society! --- On Tue, 15/2/11, Gavin Flower wrote: From: Gavin Flower Subject: physical size of the device inconsistent with superblock, after RAID problems To: ext3-users at redhat.com Cc: neilb at suse.de, linux-raid at vger.kernel.org Date: Tuesday, 15 February, 2011, 16:14 Hi, I would appreciate advice recovering from the following situation, after an aborted mdadm resizing operation and subsequent recovery actions: /dev/md1: The filing system size (according to the superblock) is 76799952 blocks The physical size of the device is 76799616 Either the superblock or the partition table is likely to be corrupt! /dev/md1: UNEXPECTED INCONSISTENCY: RUN fsck manually (i.e. without -a or -p options) fsck.ext4 -f -n /dev/md1 output: e2fsck 1.41.12 (17-May-2010) The filesystem size (according to the superblock) is 76799952 blocks The physical size of the device is 76799616 blocks Either the superblock or the partition table is likely to be corrupt! Abort? no Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information Block bitmap differences:? -9626 -(9728--9752) +(405344--405369) Fix? no /dev/md1: ********** WARNING: Filesystem still has errors ********** /dev/md1: 1693644/19202048 files (0.3% non-contiguous), 54273929/76799952 blocks Note that original size, according mdadm, was not a multiple of 512KB, so I reshaped it to be the largest multiple or 512KB less than the original size using the -size option of mdadm.? So my second attempt to reshape, using the 512 chunk size, started okay.? The previous chunk size was 64KB. Note I am using Fedora 14, up-to-date as of Friday February 11th, and that there are 5X500KB drives, with 3 RAID-6 arrays: /dev/md0 swap /dev/md1 mostly user data (the problematic one) /dev/md2 distribution & O/S files plus /boot on a non-RAID ext4 partition Sequence of events: Reshaped /dev/md1 using mdadm, without first reducing size of the ext4 filesystem. The process of reshaping /dev/md1 was about 20% through when I killed it. System appeared okay. I rebooted a few minute later, but shortly after I selected the kernel, it stopped, and I dropped into a shell. With the help of Neil Brown, I made some progress and /dev/md1 reshaping appeared to have completed without error. However, on the next reboot I got the INCONSISTENCY message. Will it be safe to simply accept fsck's offer to fix, or are there other things I should do? Thanks, Gavin -- All Adults share the Responsibility to help Raise Today's Children, for they are Tomorrow's Society! From gavinflower at yahoo.com Fri Feb 18 03:50:48 2011 From: gavinflower at yahoo.com (Gavin Flower) Date: Thu, 17 Feb 2011 19:50:48 -0800 (PST) Subject: physical size of the device inconsistent with superblock, after RAID problems Message-ID: <546229.64645.qm@web65111.mail.ac2.yahoo.com> --- On Fri, 18/2/11, NeilBrown wrote: > From: NeilBrown > Subject: Re: physical size of the device inconsistent with superblock, after RAID problems > To: "Gavin Flower" > Cc: ext3-users at redhat.com, linux-raid at vger.kernel.org > Date: Friday, 18 February, 2011, 14:51 > On Thu, 17 Feb 2011 15:53:11 -0800 > (PST) Gavin Flower > wrote: > > > Hi Neil, > > > > My attempted post to ext3-users at redhat.com, > had not been published there (even though I had emailed it 4 > days ago!), as at a minute ago. > > > > I finally bit the bullet and went ahead. > > > > I accepted the fixes put forward by fsck associated > with bitmap differences, and rebooted. > > > > Still problems. > > > > Still had the discrepancy in the file size. So I ran > the command: > > > > resize2fs -p /dev/md1 76799616 > > > > I used the smaller of the 2 block counts, as: > > (a) I needed to reduce the file system size, because I > had already reduced the RAID size (I _SHOULD_ have done this > first, before resizing the RAID), and > > (b) it is reported as the 'physical' size of the > device, so it is likely to be the correct value IMHO > > > > The system the came up successfully after a reboot, > and I was able to log in as normal. > > > > There appeared to be no apparent loss of data, not > that I did an exhaustive systematic check. However, several > users have logged on successfully, and it is playing its > part as gateway to the Internet, and squid appears to be > providing its normal functionality. > > > > Neil, your help and encouragement was/is greatly > appreciated! > > > > Excellent! I'm glad you found a way through. > As you didn't really trim very much from your device it is > certainly possible > that no critical data was there. Quite possibly > resize2fs would have told > you if there was (I certainly hope it would have done). > > NeilBrown > Hi Neil, Having about 26% spare capacity (see output of the df) md1 (the problematic RAID 6), probably (?) meant that nothing was likely to be lost by trimming a tiny fraction of a percent from the end. However, since the md1 device actually resides on 5 real physical drives, reality is almost certainly more complicated! - possibly, hence the bit map discrepancies (now I'm firmly outside my area of expertise!). # df Filesystem 1K-blocks Used Available Use% Mounted on /dev/md2 1097254408 27547660 1013969456 3% / tmpfs 4097108 772 4096336 1% /dev/shm /dev/sda1 1032088 129800 849860 14% /boot /dev/md1 302377920 212244524 74773476 74% /data # mdadm --detail /dev/md1 /dev/md1: Version : 0.90 Creation Time : Thu Dec 3 13:05:02 2009 Raid Level : raid6 Array Size : 307198464 (292.97 GiB 314.57 GB) Used Dev Size : 102399488 (97.66 GiB 104.86 GB) Raid Devices : 5 Total Devices : 5 Preferred Minor : 1 Persistence : Superblock is persistent Update Time : Fri Feb 18 15:09:50 2011 State : clean Active Devices : 5 Working Devices : 5 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 512K UUID : 6f1176ae:a0ad6cac:bfe78010:bc810f04 Events : 0.3389728 Number Major Minor RaidDevice State 0 8 2 0 active sync /dev/sda2 1 8 18 1 active sync /dev/sdb2 2 8 66 2 active sync /dev/sde2 3 8 50 3 active sync /dev/sdd2 4 8 34 4 active sync /dev/sdc2 # Cheers, Gavin From scerveau at awox.com Mon Feb 28 11:18:59 2011 From: scerveau at awox.com (Stephane Cerveau) Date: Mon, 28 Feb 2011 12:18:59 +0100 Subject: Increase EXT3 format speed. Message-ID: Dear all, On certain USB MSC dongle, the mkfs.ext3 can take around 3 min for 4GB. It seems that the process is locked in sync function. Do you know how I could increase the speed of this process? With mkfs.ext4 it's about 1 min to format it ... St?phane Cerveau Software Engineer scerveau at awox.com Phone: +33 4 99 53 27 39 93, Pierre Duhem 34000 Montpellier FRANCE Phone: +33 4 67 47 10 00 Fax: +33 4 67 47 10 15 cid:069332510 at 17112008-1528 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 16426 bytes Desc: image001.jpg URL: From adilger at dilger.ca Mon Feb 28 15:25:03 2011 From: adilger at dilger.ca (Andreas Dilger) Date: Mon, 28 Feb 2011 08:25:03 -0700 Subject: Increase EXT3 format speed. In-Reply-To: References: Message-ID: You can speed up mke2fs by reducing the inode count (-i or -N) if your average file size is over 8kB, reduce the journal size (-J size=4) and/or use the lazy_journal_init patch I posted recently, and/or use the lazy_itable_init option. I assume since format performance is important that you do it often and the risk of an uninitialized inode table is low. Or, you could use ext4, which is also faster at runtime, not just format time. Cheers, Andreas On 2011-02-28, at 4:18, Stephane Cerveau wrote: > Dear all, > > > > On certain USB MSC dongle, the mkfs.ext3 can take around 3 min for 4GB. It seems that the process is locked in sync function. Do you know how I could increase the speed of this process? > > With mkfs.ext4 it?s about 1 min to format it ? > > > > > > St?phane Cerveau > Software Engineer > > scerveau at awox.com > > Phone: +33 4 99 53 27 39 > > > 93, Pierre Duhem > 34000 Montpellier > FRANCE > Phone: +33 4 67 47 10 00 > Fax: +33 4 67 47 10 15 > > > > > > > > __________ Information from ESET NOD32 Antivirus, version of virus signature database 5913 (20110228) __________ > > The message was checked by ESET NOD32 Antivirus. > > http://www.eset.com > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex at alex.org.uk Mon Feb 28 18:33:57 2011 From: alex at alex.org.uk (Alex Bligh) Date: Mon, 28 Feb 2011 18:33:57 +0000 Subject: Increase EXT3 format speed. In-Reply-To: References: Message-ID: <02DDC7932A9E7C0FDA4B122A@Ximines.local> --On 28 February 2011 08:25:03 -0700 Andreas Dilger wrote: > > You can speed up mke2fs by reducing the inode count (-i or -N) if your > average file size is over 8kB, reduce the journal size (-J size=4) and/or > use the lazy_journal_init patch I posted recently, and/or use the > lazy_itable_init option. I assume since format performance is important > that you do it often and the risk of an uninitialized inode table is low. > > > Or, you could use ext4, which is also faster at runtime, not just format > time. If you are doing this a lot, another alternative is to format a sparse file, keep this between formats, and copy the sparse file on. There are plenty of utilities to do that, including one here: http://blog.alex.org.uk/2010/12/02/copying-sparse-files/ If you are quite sure your USB device is completely blank (i.e. all sectors zero), run with -n, in which case it will only write the non-zero sectors and read nothing. If you are not completely sure your USB device is blank, don't run with -n, and discover that this will be probably be slower than a straight format as it will have to read every sector. -- Alex Bligh From tytso at mit.edu Mon Feb 28 18:54:50 2011 From: tytso at mit.edu (Ted Ts'o) Date: Mon, 28 Feb 2011 13:54:50 -0500 Subject: Increase EXT3 format speed. In-Reply-To: References: Message-ID: <20110228185450.GE28617@thunk.org> On Mon, Feb 28, 2011 at 12:18:59PM +0100, Stephane Cerveau wrote: > > On certain USB MSC dongle, the mkfs.ext3 can take around 3 min for > 4GB. It seems that the process is locked in sync function. Do you > know how I could increase the speed of this process? > > With mkfs.ext4 it's about 1 min to format it ... What is the high-level problem that you are trying to solve? Is this for testing purposes? Are you trying to do something for production? Are these dongles new, or do they contain a previously generated ext2/3/4 file system? Depending on the answers to these questions, there are some short-cuts you can take that will speed up mke2fs for ext3, but there are downsides and they aren't safe in all situations (which is why they are not the default). Ext4 has a way to initialize the inode table lazily, after the file system is mounted, which is why it's faster to use mke2fs. - Ted