NFS stale-filehandles
T. Horsnell
tsh at mrc-lmb.cam.ac.uk
Fri Dec 15 18:17:46 UTC 2006
[Charset ISO-8859-1 unsupported, filtering to ASCII...]
>T. Horsnell wrote:
>> FC6, NFS V3
>> -----------
>>
>> I'm trying to get to the state where
>>
>> 1. I can have a bunch of NFS clients which have mounted
>> filesystems exported by a fileserver,
>>
>> 2. The fileserver is stopped, the (SCSI) disks containing
>> the filesystems are moved to different positions on the SCSI bus,
>> and the fileserver is rebooted.
>>
>> 3. The clients can still continue to access the correct filesystems,
>> without having to umount/re-mount anything, even though their position
>> on the server's SCSI bus has changed.
>>
>> The clients are members of a compute farm. They mount the NFS
>> filesystems with the 'hard' option so that their NFS requests
>> stall if the server is off for any reason, and so far this works
>> well. However, if the server disks are moved, there are problems.
>>
>>
>> I started off mounting the server filesystems by label, with lines
>> in /etc/fstab like:
>>
>> LABEL=/filesys1 /filesystem1 ext3 defaults 0 2
>>
>> This gave me pseudo-persistence in that wherever the disks were on
>> the SCSI bus, they were always mounted on the correct mountpoint.
>> With this setup, the underlying device (/dev/sda /dev/sdb etc)
>> as shown by 'mount -t ext3' changed when the disk position changed.
>> I then discovered that if I swapped two disks over on the server
>> while those filesystems were still NFS-mounted by a client, the
>> client didnt notice the swap, but continued to access the disks
>> in the unswapped position, and hence access the wrong filesystem.
>> There were no NFS complaints, just complaints from users.
>>
>>
>> I'm now using a bunch of udev rules to give me device-name persistence
>> instead of relying on the partition label, and I have lines in fstab like:
>>
>> /dev/dsk0_1 /filesystem1 ext3 defaults 0 2
>>
>> Now, wherever I shift a disk to on the server SCSI bus, the underlying
>> device-name stays constant, but the client objects with a 'stale NFS filehandle'
>> error when the disk-position is changed, and I have to umount and remount at
>> the client. Its a slight improvement in that user-processes on the client
>> cant inadvertently use the wrong filesystem, but I would much prefer it to
>> be transparent. Is this possible with NFS
>>
>> Cheers,
>> Terry
>>
>
>Yeap.
>
>A client knows a filesystem by server ip address, and by fsid (see
>man exports), the default generated fsid is based on the underlying
>device id, so if the device id changes (moved on scsi bus)
>then the fsid changes, and from the clients point of view the
>filesystem is no longer there. And you can get odd failures,
>if someones "new" fsid matches someone elses "old" fsid you
>will get interesting bad results, ie /home will say it is /home
>on a client by the directories will look like /opt. If you have
>large fsids it is harder to get, but I have seen it happen.
>
>You can in the exports file set the fsid to something, and note
>that you can get into this same issue using LVM, and just adjusting
>the order of turning on the VG's (first gets id 1, second 2, changing
>order changes this). The fsids need to be unique across a
>given server.
>
Aha! Many thanks for this pointer. I never noticed that option in the
exports man page. I'll do some experiments straight away.
Cheers,
Terry.
> Roger
>
>--
>fedora-list mailing list
>fedora-list at redhat.com
>To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
>
More information about the fedora-list
mailing list