[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Linux-cluster] openais issue



On Mon, Sep 28, 2009 at 6:41 PM, brem belguebli
<brem belguebli gmail com> wrote:
> you mean it stopped successfully on all the nodes but it is failing to
> start only on node cvtst1 ?

Yes it has failed to start on cvtst1 only

Paras.

>
> look at the following page  to make rgmanager more verbose. It 'll
> help debug....
>
> http://sources.redhat.com/cluster/wiki/RGManager
>
> at Logging Configuration section
>
>
>
>
> 2009/9/29 Paras pradhan <pradhanparas gmail com>:
>> Brem,
>>
>> When I try to restart rgmanager on all the nodes, this time i do not
>> see rgmanager running on the first node. But I do see on other 2
>> nodes.
>>
>> Log on the first node:
>>
>> Sep 28 18:13:58 cvtst1 clurgmgrd[24099]: <notice> Resource Group
>> Manager Starting
>> Sep 28 18:17:29 cvtst1 rgmanager: [24627]: <notice> Shutting down
>> Cluster Service Manager...
>> Sep 28 18:17:29 cvtst1 clurgmgrd[24099]: <notice> Shutting down
>> Sep 28 18:17:39 cvtst1 clurgmgrd[24099]: <notice> Shutdown complete, exiting
>> Sep 28 18:17:39 cvtst1 rgmanager: [24627]: <notice> Cluster Service
>> Manager is stopped.
>> Sep 28 18:17:40 cvtst1 clurgmgrd[24679]: <notice> Resource Group
>> Manager Starting
>>
>> -
>> It seems service is running ,  but I do not see rgmanger running using clustat
>>
>>
>> Don't know what is going on.
>>
>> Thanks
>> Paras.
>>
>>
>> On Mon, Sep 28, 2009 at 5:46 PM, brem belguebli
>> <brem belguebli gmail com> wrote:
>>> Paras,
>>>
>>> Another thing, it would have been more interesting to have a start
>>> DEBUG not a stop.
>>>
>>> That's why I was asking you to first stop the vm manually on all your
>>> nodes, stop eventually rgmanager on all the nodes to reset the
>>> potential wrong states you may have, restart rgmanager.
>>>
>>> If your VM is configured to autostart, this will make it start.
>>>
>>> It should normally fail (as it does now). Send out your newly created
>>> DEBUG file.
>>>
>>> 2009/9/29 brem belguebli <brem belguebli gmail com>:
>>>> Hi Paras,
>>>>
>>>>
>>>> I don't know the xen/cluster combination well, but if I do remember
>>>> well, I think I've read somewhere that when using xen you have to
>>>> declare the use_virsh=0 key in the VM definition in the cluster.conf.
>>>>
>>>> This would make rgmanager use xm commands instead of virsh
>>>> The DEBUG output shows clearly that you are using virsh to manage your
>>>> VM instead of xm commands.
>>>> Check out the RH docs about virtualization
>>>>
>>>> I'm not a 100% sure about that, I may be completely wrong.
>>>>
>>>> Brem
>>>>
>>>> 2009/9/28 Paras pradhan <pradhanparas gmail com>:
>>>>> The only thing I noticed is the message after stopping the vm using xm
>>>>> in all nodes and starting using clusvcadm is
>>>>>
>>>>> "Virtual machine guest1 is blocked"
>>>>>
>>>>> The whole DEBUG file is attached.
>>>>>
>>>>>
>>>>> Thanks
>>>>> Paras.
>>>>>
>>>>> On Fri, Sep 25, 2009 at 5:53 PM, brem belguebli
>>>>> <brem belguebli gmail com> wrote:
>>>>>> There's a problem with the script that is called by rgmanager to start
>>>>>> the VM, I don't know what causes it
>>>>>>
>>>>>> May be you should try something like :
>>>>>>
>>>>>> 1) stop the VM on all nodes with xm commands
>>>>>> 2) edit the /usr/share/cluster/vm.sh script and add the following
>>>>>> lines (after the #!/bin/bash ):
>>>>>>   exec >/tmp/DEBUG 2>&1
>>>>>>   set -x
>>>>>> 3) start the VM with clusvcadm -e vm:guest1
>>>>>>
>>>>>> It should fail as it did before.
>>>>>>
>>>>>> edit the the /tmp/DEBUG file and you will be able to see where it
>>>>>> fails (it may generate a lot of debug)
>>>>>>
>>>>>> 4) remove the debug lines from /usr/share/cluster/vm.sh
>>>>>>
>>>>>> Post the DEBUG file if you're not able to see where it fails.
>>>>>>
>>>>>> Brem
>>>>>>
>>>>>> 2009/9/26 Paras pradhan <pradhanparas gmail com>:
>>>>>>> No I am not manually starting not using automatic init scripts.
>>>>>>>
>>>>>>> I started the vm using: clusvcadm -e vm:guest1
>>>>>>>
>>>>>>> I have just stopped using clusvcadm -s vm:guest1. For few seconds it
>>>>>>> says guest1 started . But after a while I can see the guest1 on all
>>>>>>> three nodes.
>>>>>>>
>>>>>>> clustat says:
>>>>>>>
>>>>>>>  Service Name                                            Owner (Last)
>>>>>>>                                          State
>>>>>>>  ------- ----                                            ----- ------
>>>>>>>                                          -----
>>>>>>>  vm:guest1                                               (none)
>>>>>>>                                          stopped
>>>>>>>
>>>>>>> But I can see the vm from xm li.
>>>>>>>
>>>>>>> This is what I can see from the log:
>>>>>>>
>>>>>>>
>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <notice> start on vm "guest1"
>>>>>>> returned 1 (generic error)
>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <warning> #68: Failed to start
>>>>>>> vm:guest1; return value: 1
>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <notice> Stopping service vm:guest1
>>>>>>> Sep 25 17:19:02 cvtst1 clurgmgrd[4298]: <notice> Service vm:guest1 is
>>>>>>> recovering
>>>>>>> Sep 25 17:19:15 cvtst1 clurgmgrd[4298]: <notice> Recovering failed
>>>>>>> service vm:guest1
>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <notice> start on vm "guest1"
>>>>>>> returned 1 (generic error)
>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <warning> #68: Failed to start
>>>>>>> vm:guest1; return value: 1
>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <notice> Stopping service vm:guest1
>>>>>>> Sep 25 17:19:17 cvtst1 clurgmgrd[4298]: <notice> Service vm:guest1 is
>>>>>>> recovering
>>>>>>>
>>>>>>>
>>>>>>> Paras.
>>>>>>>
>>>>>>> On Fri, Sep 25, 2009 at 5:07 PM, brem belguebli
>>>>>>> <brem belguebli gmail com> wrote:
>>>>>>>> Have you started  your VM via rgmanager (clusvcadm -e vm:guest1) or
>>>>>>>> using xm commands out of cluster control  (or maybe a thru an
>>>>>>>> automatic init script ?)
>>>>>>>>
>>>>>>>> When clustered, you should never be starting services (manually or
>>>>>>>> thru automatic init script) out of cluster control
>>>>>>>>
>>>>>>>> The thing would be to stop your vm on all the nodes with the adequate
>>>>>>>> xm command (not using xen myself) and try to start it with clusvcadm.
>>>>>>>>
>>>>>>>> Then see if it is started on all nodes (send clustat output)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2009/9/25 Paras pradhan <pradhanparas gmail com>:
>>>>>>>>> Ok. Please see below. my vm is running on all nodes though clustat
>>>>>>>>> says it is stopped.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> [root cvtst1 ~]# clustat
>>>>>>>>> Cluster Status for test @ Fri Sep 25 16:52:34 2009
>>>>>>>>> Member Status: Quorate
>>>>>>>>>
>>>>>>>>>  Member Name                                                     ID   Status
>>>>>>>>>  ------ ----                                                     ---- ------
>>>>>>>>>  cvtst2                                                    1 Online, rgmanager
>>>>>>>>>  cvtst1                                                     2 Online,
>>>>>>>>> Local, rgmanager
>>>>>>>>>  cvtst3                                                     3 Online, rgmanager
>>>>>>>>>
>>>>>>>>>  Service Name                                            Owner (Last)
>>>>>>>>>                                          State
>>>>>>>>>  ------- ----                                            ----- ------
>>>>>>>>>                                          -----
>>>>>>>>>  vm:guest1                                               (none)
>>>>>>>>>                                          stopped
>>>>>>>>> [root cvtst1 ~]#
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>> o/p of xm li on cvtst1
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> [root cvtst1 ~]# xm li
>>>>>>>>> Name                                      ID Mem(MiB) VCPUs State   Time(s)
>>>>>>>>> Domain-0                                   0     3470     2 r-----  28939.4
>>>>>>>>> guest1                                     7      511     1 -b----   7727.8
>>>>>>>>>
>>>>>>>>> o/p of xm li on cvtst2
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> [root cvtst2 ~]# xm li
>>>>>>>>> Name                                      ID Mem(MiB) VCPUs State   Time(s)
>>>>>>>>> Domain-0                                   0     3470     2 r-----  31558.9
>>>>>>>>> guest1                                    21      511     1 -b----   7558.2
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Paras.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 25, 2009 at 4:22 PM, brem belguebli
>>>>>>>>> <brem belguebli gmail com> wrote:
>>>>>>>>>> It looks like no.
>>>>>>>>>>
>>>>>>>>>> can you send an output of clustat  of when the VM is running on
>>>>>>>>>> multiple nodes at the same time?
>>>>>>>>>>
>>>>>>>>>> And by the way, another one after having stopped (clusvcadm -s vm:guest1) ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2009/9/25 Paras pradhan <pradhanparas gmail com>:
>>>>>>>>>>> Anyone having issue as mine? Virtual machine service is not being
>>>>>>>>>>> properly handled by the cluster.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> Paras.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Sep 21, 2009 at 9:55 AM, Paras pradhan <pradhanparas gmail com> wrote:
>>>>>>>>>>>> Ok.. here is my cluster.conf file
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> [root cvtst1 cluster]# more cluster.conf
>>>>>>>>>>>> <?xml version="1.0"?>
>>>>>>>>>>>> <cluster alias="test" config_version="9" name="test">
>>>>>>>>>>>>        <fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="3"/>
>>>>>>>>>>>>        <clusternodes>
>>>>>>>>>>>>                <clusternode name="cvtst2" nodeid="1" votes="1">
>>>>>>>>>>>>                        <fence/>
>>>>>>>>>>>>                </clusternode>
>>>>>>>>>>>>                <clusternode name="cvtst1" nodeid="2" votes="1">
>>>>>>>>>>>>                        <fence/>
>>>>>>>>>>>>                </clusternode>
>>>>>>>>>>>>                <clusternode name="cvtst3" nodeid="3" votes="1">
>>>>>>>>>>>>                        <fence/>
>>>>>>>>>>>>                </clusternode>
>>>>>>>>>>>>        </clusternodes>
>>>>>>>>>>>>        <cman/>
>>>>>>>>>>>>        <fencedevices/>
>>>>>>>>>>>>        <rm>
>>>>>>>>>>>>                <failoverdomains>
>>>>>>>>>>>>                        <failoverdomain name="myfd1" nofailback="0" ordered="1" restricted="0">
>>>>>>>>>>>>                                <failoverdomainnode name="cvtst2" priority="3"/>
>>>>>>>>>>>>                                <failoverdomainnode name="cvtst1" priority="1"/>
>>>>>>>>>>>>                                <failoverdomainnode name="cvtst3" priority="2"/>
>>>>>>>>>>>>                        </failoverdomain>
>>>>>>>>>>>>                </failoverdomains>
>>>>>>>>>>>>                <resources/>
>>>>>>>>>>>>                <vm autostart="1" domain="myfd1" exclusive="0" max_restarts="0"
>>>>>>>>>>>> name="guest1" path="/vms" recovery="r
>>>>>>>>>>>> estart" restart_expire_time="0"/>
>>>>>>>>>>>>        </rm>
>>>>>>>>>>>> </cluster>
>>>>>>>>>>>> [root cvtst1 cluster]#
>>>>>>>>>>>> ------
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>> Paras.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Sep 20, 2009 at 9:44 AM, Volker Dormeyer <volker ixolution de> wrote:
>>>>>>>>>>>>> On Fri, Sep 18, 2009 at 05:08:57PM -0500,
>>>>>>>>>>>>> Paras pradhan <pradhanparas gmail com> wrote:
>>>>>>>>>>>>>> I am using cluster suite for HA of xen virtual machines. Now I am
>>>>>>>>>>>>>> having another problem. When I start the my xen vm in one node, it
>>>>>>>>>>>>>> also starts on other nodes. Which daemon controls  this?
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is usually done bei clurgmgrd (which is part of the rgmanager
>>>>>>>>>>>>> package). To me, this sounds like a configuration problem. Maybe,
>>>>>>>>>>>>> you can post your cluster.conf?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Volker
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>> Linux-cluster redhat com
>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>> Linux-cluster redhat com
>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>> Linux-cluster redhat com
>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Linux-cluster mailing list
>>>>>>>>> Linux-cluster redhat com
>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Linux-cluster mailing list
>>>>>>>> Linux-cluster redhat com
>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Linux-cluster mailing list
>>>>>>> Linux-cluster redhat com
>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Linux-cluster mailing list
>>>>>> Linux-cluster redhat com
>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>
>>>>>
>>>>> --
>>>>> Linux-cluster mailing list
>>>>> Linux-cluster redhat com
>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>
>>>>
>>>
>>> --
>>> Linux-cluster mailing list
>>> Linux-cluster redhat com
>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>
>>
>> --
>> Linux-cluster mailing list
>> Linux-cluster redhat com
>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>
>
> --
> Linux-cluster mailing list
> Linux-cluster redhat com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]