[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: ext3 + fs > 2Tbyte



On Nov 04, 2005  12:17 +1100, Vincent McIntyre wrote:
> * boot with xraid device plugged in, kernel 2.6.7-1-686-smp
>     (packaged as 2.6.7-1.backports.org.1)
> * install a gpt disklabel with parted (-1.6.24 rather than 1.6.19)
> * make an ext2 filesystem as big as the disk with parted
> * mount - it mounts ok
> * umount
> * tune2fs -j (-1.38)
> * mount - it mounts ok (-2.12)
> * umount (-2.12)
> * reboot
> * try to mount - it fails.
>     (the filesystem is not mentioned in /etc/fstab, the system should
>      not be attempting to mount it of fsck it at boot time)
> 
> No files were written to the filesystem during the test sequence.

Hmm, I would expect at least the need to write something to the filesystem,
unless you are unlucky enough that the last group(s) aliases exactly over
the first superblock on disk, but is kept in the cache enough to remount
it before you reboot.

If you just to the mke2fs + reboot + mount does that fail?  Same with
just the tune2fs -j + reboot + remount?  Do you only use the parted
"mkfs" or do you actually use the mke2fs from e2fsprogs?

> I have not yet tried filesystems smaller than 2Tb across reboots.
> I expect it will work, but I will try that shortly to check.
> 
> 
> findsuper tells me there are superblocks, but fs_blk_sz changes (!?)

These are remnants of previous filesystems on the device, each with
slightly different offsets (maybe with and without a partition table,
or with different partition types).  In one case there was a small
1kB block filesystem on the disk in the past.

> # /root/e2fsprogs-1.38/misc/findsuper /dev/sdb1
> starting at 0, with 512 byte increments
>        thisoff     block fs_blk_sz  blksz grp last_mount
>          17920        17 586057719  4096    0 Thu Jan  1 10:00:00 1970

What is missing is the superblock at offset "1024".  What this tool
_should_ also print out is part of the superblock UUID so it is possible
to say which superblocks belong to a single filesystem.

With an ext3 filesystem you will also find copies of the superblock in
the journal, they will all be marked "grp 0" and are not valid backups.

>      134234624    131088 586057719  4096    1 Thu Jan  1 10:00:00 1970
>      134235648    131089 586057719  4096    1 Thu Jan  1 10:00:00 1970
>      209733120    204817   1023983  1024   25 Thu Jan  1 10:00:00 1970
>      226510336    221201   1023983  1024   27 Thu Jan  1 10:00:00 1970
>      402670080    393232 586057719  4096    3 Thu Jan  1 10:00:00 1970
>      402671104    393233 586057719  4096    3 Thu Jan  1 10:00:00 1970
>      411059712    401425   1023983  1024   49 Thu Jan  1 10:00:00 1970
>      671105536    655376 586057719  4096    5 Thu Jan  1 10:00:00 1970
>      671106560    655377 586057719  4096    5 Thu Jan  1 10:00:00 1970
>      679495168    663569   1023983  1024   81 Thu Jan  1 10:00:00 1970
>      939540992    917520 586057719  4096    7 Thu Jan  1 10:00:00 1970
>      939542016    917521 586057719  4096    7 Thu Jan  1 10:00:00 1970
>     1207976448   1179664 586057719  4096    9 Thu Jan  1 10:00:00 1970
>     1207977472   1179665 586057719  4096    9 Thu Jan  1 10:00:00 1970
>     3355460096   3276816 586057719  4096   25 Thu Jan  1 10:00:00 1970
>     3355461120   3276817 586057719  4096   25 Thu Jan  1 10:00:00 1970
>     3623895552   3538960 586057719  4096   27 Thu Jan  1 10:00:00 1970
>     3623896576   3538961 586057719  4096   27 Thu Jan  1 10:00:00 1970
>     6576685568   6422544 586057719  4096   49 Thu Jan  1 10:00:00 1970
>     6576686592   6422545 586057719  4096   49 Thu Jan  1 10:00:00 1970
>    10871652864  10616848 586057719  4096   81 Thu Jan  1 10:00:00 1970
>    10871653888  10616849 586057719  4096   81 Thu Jan  1 10:00:00 1970
>    16777232896  16384016 586057719  4096  125 Thu Jan  1 10:00:00 1970
>    16777233920  16384017 586057719  4096  125 Thu Jan  1 10:00:00 1970
> ^C
> This is not looking good...

There appear to be 2 filesystems of interest.  One has offset 0x4200 = 16896,
but is missing the primary superblock.  The other has offset 0x4600 = 17920.
Neither of these would allow you to mount the filesystem as-is, because the
superblock is not aligned at 1024 bytes from the start of the device.

I would suspect something wacky with the partitioning and/or the way that
parted is making the filesystem.

> Your nice od trick tells me slightly different locations for the
> superblock signatures -
> # od -Ax -tx4 /dev/sdb1 | \
>   grep "^[0-9a-f]*30 [0-9a-f]* [0-9a-f]* 000[1-3]ef53 "
> 004630 436a93dd 001e0000 0001ef53 00000001
> 8004630 00000000 001e0000 0001ef53 00000001
> c804630 00000000 001e0000 0001ef53 00000001
> d804630 00000000 001e0000 0001ef53 00000001
> 18004630 00000000 001e0000 0001ef53 00000001
> ^C
> 
> 0x004630 corresponds to byte offset 17968, 48 bytes away.
> Is this explainable by the position of the superblock signature within
> the disk block?

Yes, this hack is only looking for the ext[23] magic number, which is not
at the start of the superblock (0x30 = 48 bytes offset).

> So I tried a few e2fsck runs. I know I'm probably being dense but none
> of these worked:
> e2fsck -n -b 16        -B 4096 /dev/sdb1
> e2fsck -n -b 17        -B 4096 /dev/sdb1
> e2fsck -n -b 18        -B 4096 /dev/sdb1
> e2fsck -n -b 204816    -B 1024 /dev/sdb1
> e2fsck -n -b 204817    -B 1024 /dev/sdb1
> e2fsck -n -b 204818    -B 1024 /dev/sdb1
> e2fsck -n -b 221200    -B 1024 /dev/sdb1
> e2fsck -n -b 221201    -B 1024 /dev/sdb1
> e2fsck -n -b 221202    -B 1024 /dev/sdb1
> e2fsck -n -b 1179664   -B 4096 /dev/sdb1
> e2fsck -n -b 1179665   -B 4096 /dev/sdb1
> e2fsck -n -b 6422544   -B 4096 /dev/sdb1
> e2fsck -n -b 6422545   -B 4096 /dev/sdb1
> e2fsck -n -b 10616848  -B 4096 /dev/sdb1
> e2fsck -n -b 10616849  -B 4096 /dev/sdb1

No, I'd expect you need to do something with the device partitioning
to get the filesystem aligned properly.  They aren't even aligned on
a block boundary, there is a 512-byte offset.

> (The e2fsck manpage could be a tiny bit clearer in that - I think -
>  it means you to use -b <blocknumber>, not -b <offset_to_superblock>)

Send a patch to Ted.

I would recommend to do the following:
- make a partition
- reboot the system
- use mke2fs -j to make the filesystem
- test mount, unmount, reboot at this point

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]