[linux-lvm] mirrored LV + cmirror problem

Jonathan Brassow jbrassow at redhat.com
Fri Feb 15 21:06:31 UTC 2008


If the problem is reproducible, we should be able to track it down.

When a failure happens, the kernel sends an event to userspace that  
signals 'dmeventd' to take action.  If we take dmeventd out of the  
picture, we can run the commands ourselves with higher verbose settings.

When you activate the volume, you can 'lvchange -ay --monitor n <vg>/ 
<lv>' - this will prevent dmeventd from monitoring the mirror.  Then  
kill the log device.  Finally, run 'vgreduce --removemissing <VG> - 
vvvv' to perform the recovery.  (redirecting all the output to a file  
will give us something to look at if the failure is reproduced.)

We may need to grab debugging output from clvmd too, but that can get  
messy, so we'll start with this.

  brassow

P.S.  It looks like you must have *.debug; in your /etc/syslog.conf,  
yes?

On Feb 15, 2008, at 10:17 AM, Lajkó Attila wrote:

>
>
> On Feb 15, 2008, at 4:41 PM, Jonathan Brassow wrote:
>
>> Are all the packages rhel4.6 as well, or have you compiled pkgs  
>> yourself?
>
> All the packages are binaries from rhel4.6:
> lvm2-cluster-2.02.27-2.el4_6.1
> cmirror-1.0.1-1
> cmirror-kernel-xenU-2.6.9-38.5
>
>>
>>
>> What was the load you had on the system?
>
> Very low, approx. 0.
>
>> The messages I see from dm-cmirror suggest that it is properly  
>> shutting down in the face of the failure... However, before it has  
>> finished, we can see "Failed to remove faulty devices in vgtest- 
>> lvtest".  This suggests to me that clvmd is not waiting long enough  
>> for the shutdown to complete, but I only see 3 seconds of the log.   
>> When was the device failure initiated?
>>
>> brassow
>>
>>
>
> The failure was initiated at the beginning of the log. I put the  
> complete messages files from both nodes (el42 and el4) to FTP.
>
> ftp://ftp.ulx.hu/upload/clvmd/messages.el4
> ftp://ftp.ulx.hu/upload/clvmd/messages.el42
>
> Attila
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/





More information about the linux-lvm mailing list