[Linux-cluster] failover domain and service start
emmanuel segura
emi2fast at gmail.com
Thu Dec 19 13:52:28 UTC 2013
?
2013/12/19 YB Tan Sri Dato Sri' Adli a.k.a Dell <white.heron at yahoo.com>
> please call me 60173623661 for linux system configuration.
>
> Sent from Yahoo Mail for iPhone <http://overview.mail.yahoo.com?.src=iOS>
>
> ------------------------------
> * From: * emmanuel segura <emi2fast at gmail.com>;
> * To: * linux clustering <linux-cluster at redhat.com>;
> * Subject: * Re: [Linux-cluster] failover domain and service start
> * Sent: * Thu, Dec 19, 2013 9:05:18 AM
>
> other thing, change the rgmanager loglevel, for more information you
> can use this link https://fedorahosted.org/cluster/wiki/RGManager
>
>
> 2013/12/19 emmanuel segura <emi2fast at gmail.com>
>
>> grep "/vms_c" /proc/mounts in every node
>>
>>
>> 2013/12/18 Paras pradhan <pradhanparas at gmail.com>
>>
>>> I see this:
>>>
>>> Dec 18 16:17:18 cvtst3 clurgmgrd[13935]: <notice> Starting stopped
>>> service vm:guest1
>>> Dec 18 16:17:19 cvtst3 clurgmgrd[13935]: <notice> start on vm "guest1"
>>> returned 1 (generic error)
>>> Dec 18 16:17:19 cvtst3 clurgmgrd[13935]: <warning> #68: Failed to start
>>> vm:guest1; return value: 1
>>> Dec 18 16:17:19 cvtst3 clurgmgrd[13935]: <notice> Stopping service
>>> vm:guest1
>>>
>>>
>>>
>>> On Wed, Dec 18, 2013 at 4:39 PM, emmanuel segura <emi2fast at gmail.com>wrote:
>>>
>>>> can you show the log?
>>>>
>>>>
>>>> 2013/12/18 Paras pradhan <pradhanparas at gmail.com>
>>>>
>>>>> Added but same problem. vm does not start.
>>>>>
>>>>> -Paras.
>>>>>
>>>>>
>>>>> On Wed, Dec 18, 2013 at 1:13 PM, emmanuel segura <emi2fast at gmail.com>wrote:
>>>>>
>>>>>> from script vm.sh i saw it try to discovery the hypervisor you are
>>>>>> using, with hypervisor="xen" you force the script to use xen
>>>>>>
>>>>>>
>>>>>> 2013/12/18 Paras pradhan <pradhanparas at gmail.com>
>>>>>>
>>>>>>> The only parameter I don't have is: hypervisor="xen"
>>>>>>>
>>>>>>> Does it matter?
>>>>>>>
>>>>>>> This is what i have:
>>>>>>>
>>>>>>> <vm autostart="1" domain="myfd1" exclusive="0" max_restarts="0"
>>>>>>> name="guest1" path="/vms_c" recovery="restart" restart_expire_time="0"
>>>>>>> use_virsh="0"/>
>>>>>>>
>>>>>>> -Paras.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 18, 2013 at 12:24 PM, emmanuel segura <
>>>>>>> emi2fast at gmail.com> wrote:
>>>>>>>
>>>>>>>> <vm name="guest1" hypervisor="xen" path="/vms_c" use_virsh="0">
>>>>>>>>
>>>>>>>> Incriment the config version of cluster.conf and ccs_tool update
>>>>>>>> /etc/cluster/cluster.conf
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/12/18 Paras pradhan <pradhanparas at gmail.com>
>>>>>>>>
>>>>>>>>> Emmauel,
>>>>>>>>>
>>>>>>>>> With export OCF_RESKEY_name="guest1" ; export
>>>>>>>>> OCF_RESKEY_path="/vms_c" ; export OCF_RESKEY_use_virsh=0
>>>>>>>>>
>>>>>>>>> , I can start the vm using : /usr/share/cluster/vm.sh
>>>>>>>>>
>>>>>>>>> I am wondering how to make the changes to cluser.conf or other
>>>>>>>>> files so that we can start the vm using clucvcsadm.
>>>>>>>>>
>>>>>>>>> -Thanks
>>>>>>>>> sorry for the delay.
>>>>>>>>>
>>>>>>>>> Paras.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Dec 5, 2013 at 12:36 PM, Paras pradhan <
>>>>>>>>> pradhanparas at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Emmanue no.l I was busy on some other things I will test and let
>>>>>>>>>> you know asap !
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Dec 5, 2013 at 12:26 PM, emmanuel segura <
>>>>>>>>>> emi2fast at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Paras
>>>>>>>>>>>
>>>>>>>>>>> did you solved the problem?
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> Emmanuel
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/11/25 emmanuel segura <emi2fast at gmail.com>
>>>>>>>>>>>
>>>>>>>>>>>> Hello Paras
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe i found the solution, in function validate_all we got
>>>>>>>>>>>>
>>>>>>>>>>>> if [ -z "$OCF_RESKEY_hypervisor" ] ||
>>>>>>>>>>>> [ "$OCF_RESKEY_hypervisor" = "auto" ]; then
>>>>>>>>>>>> export OCF_RESKEY_hypervisor="`virsh version |
>>>>>>>>>>>> grep \"Running hypervisor:\" | awk '{print $3}' | tr A-Z a-z`"
>>>>>>>>>>>> if [ -z "$OCF_RESKEY_hypervisor" ]; then
>>>>>>>>>>>> ocf_log err "Could not determine
>>>>>>>>>>>> Hypervisor"
>>>>>>>>>>>> return $OCF_ERR_ARGS
>>>>>>>>>>>> fi
>>>>>>>>>>>> echo Hypervisor: $OCF_RESKEY_hypervisor
>>>>>>>>>>>> fi
>>>>>>>>>>>>
>>>>>>>>>>>> #
>>>>>>>>>>>> # Xen hypervisor only for when use_virsh = 0.
>>>>>>>>>>>> #
>>>>>>>>>>>> if [ "$OCF_RESKEY_use_virsh" = "0" ]; then
>>>>>>>>>>>> if [ "$OCF_RESKEY_hypervisor" != "xen" ]; then
>>>>>>>>>>>> ocf_log err "Cannot use
>>>>>>>>>>>> $OCF_RESKEY_hypervisor hypervisor without using virsh"
>>>>>>>>>>>> return $OCF_ERR_ARGS
>>>>>>>>>>>> fi
>>>>>>>>>>>>
>>>>>>>>>>>> with this following enviroment variables, when i tested by hand
>>>>>>>>>>>> the angent uses xm commands
>>>>>>>>>>>>
>>>>>>>>>>>> env | grep OCF
>>>>>>>>>>>> OCF_RESKEY_hypervisor=xen
>>>>>>>>>>>> OCF_RESKEY_path=/vms_c
>>>>>>>>>>>> OCF_RESKEY_name=guest1
>>>>>>>>>>>> OCF_RESKEY_use_virsh=0
>>>>>>>>>>>>
>>>>>>>>>>>> [root at client ~]# /usr/share/cluster/vm.sh status
>>>>>>>>>>>> Management tool: xm
>>>>>>>>>>>> <err> Cannot find 'xm'; is it installed?
>>>>>>>>>>>> [vm.sh] Cannot find 'xm'; is it installed?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I don't have xen installed to test it
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> if [ -n "$OCF_RESKEY_xmlfile" ]; then
>>>>>>>>>>>> ocf_log err "Cannot use xmlfile if
>>>>>>>>>>>> use_virsh is set to 0"
>>>>>>>>>>>> return $OCF_ERR_ARGS
>>>>>>>>>>>> fi
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2013/11/25 emmanuel segura <emi2fast at gmail.com>
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello paras
>>>>>>>>>>>>>
>>>>>>>>>>>>> missing the export command in front of variables, the correct
>>>>>>>>>>>>> way is this
>>>>>>>>>>>>>
>>>>>>>>>>>>> export OCF_RESKEY_name="guest1" ; export
>>>>>>>>>>>>> OCF_RESKEY_path="/vms_c" ; export OCF_RESKEY_use_virsh=0
>>>>>>>>>>>>> [root at client ~]# env | grep OCF
>>>>>>>>>>>>> OCF_RESKEY_path=/vms_c
>>>>>>>>>>>>> OCF_RESKEY_name=guest1
>>>>>>>>>>>>> OCF_RESKEY_use_virsh=0
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/11/25 emmanuel segura <emi2fast at gmail.com>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello Paras
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have a centos 6, i don't know if it is different on redhat
>>>>>>>>>>>>>> 5, but i saw in the script vm.sh calls do_start function when start
>>>>>>>>>>>>>> parameter is given
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> do_start()
>>>>>>>>>>>>>> {
>>>>>>>>>>>>>> if [ "$OCF_RESKEY_use_virsh" = "1" ]; then
>>>>>>>>>>>>>> do_virsh_start $*
>>>>>>>>>>>>>> return $?
>>>>>>>>>>>>>> fi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> do_xm_start $*
>>>>>>>>>>>>>> return $?
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> i don't know why because the vm.sh uses virsh when you launch
>>>>>>>>>>>>>> the script by hand :(
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/11/25 Paras pradhan <pradhanparas at gmail.com>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Looks like use_virsh=0 has no effect.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> [root at cvtst3 ~]# export OCF_RESKEY_name="guest1" ;
>>>>>>>>>>>>>>> OCF_RESKEY_path="/vms_c" ; OCF_RESKEY_use_virsh=0
>>>>>>>>>>>>>>> [root at cvtst3 ~]# set -x
>>>>>>>>>>>>>>> ++ printf '\033]0;%s@%s:%s\007' root cvtst3 '~'
>>>>>>>>>>>>>>> [root at cvtst3 ~]# /usr/share/cluster/vm.sh start
>>>>>>>>>>>>>>> + /usr/share/cluster/vm.sh start
>>>>>>>>>>>>>>> Hypervisor: xen
>>>>>>>>>>>>>>> Management tool: virsh
>>>>>>>>>>>>>>> Hypervisor URI: xen:///
>>>>>>>>>>>>>>> Migration URI format: xenmigr://target_host/
>>>>>>>>>>>>>>> Virtual machine guest1 is error: failed to get domain
>>>>>>>>>>>>>>> 'guest1'
>>>>>>>>>>>>>>> error: Domain not found: xenUnifiedDomainLookupByName
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> <debug> virsh -c xen:/// start guest1
>>>>>>>>>>>>>>> error: failed to get domain 'guest1'
>>>>>>>>>>>>>>> error: Domain not found: xenUnifiedDomainLookupByName
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ++ printf '\033]0;%s@%s:%s\007' root cvtst3 '~'
>>>>>>>>>>>>>>> [root at cvtst3 ~]# set +x
>>>>>>>>>>>>>>> + set +x
>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Paras.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Nov 22, 2013 at 5:22 PM, emmanuel segura <
>>>>>>>>>>>>>>> emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hellos Paras
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Stop the vm and retry to start the vm with following
>>>>>>>>>>>>>>>> commands and if you got some error show it
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> export OCF_RESKEY_name="guest1" ; OCF_RESKEY_path="/vms_c"; OCF_RESKEY_use_virsh=0
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> set -x
>>>>>>>>>>>>>>>> /usr/share/cluster/vm.sh start
>>>>>>>>>>>>>>>> set +x
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2013/11/22 Paras pradhan <pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I found the workaround to my issue. What i did is:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> run the vm using xm and then start using cluvscadm. This
>>>>>>>>>>>>>>>>> works for me for the time being but I am not sure what is causing this.
>>>>>>>>>>>>>>>>> This is what I did
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> xm create /vms_c/guest1
>>>>>>>>>>>>>>>>> clusvcadm -e vm: guest1 ( This detects that guest1 is up
>>>>>>>>>>>>>>>>> and quickly changes its status to success)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Although i used virt-install, it also create a xem format
>>>>>>>>>>>>>>>>> configuration file and since use_virsh=0 it should be able to use this xen
>>>>>>>>>>>>>>>>> format config file. This is my vm configuration:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>> name = "guest1"
>>>>>>>>>>>>>>>>> maxmem = 2048
>>>>>>>>>>>>>>>>> memory = 512
>>>>>>>>>>>>>>>>> vcpus = 1
>>>>>>>>>>>>>>>>> #cpus="1-2"
>>>>>>>>>>>>>>>>> bootloader = "/usr/bin/pygrub"
>>>>>>>>>>>>>>>>> on_poweroff = "destroy"
>>>>>>>>>>>>>>>>> on_reboot = "restart"
>>>>>>>>>>>>>>>>> on_crash = "restart"
>>>>>>>>>>>>>>>>> vfb = [ ]
>>>>>>>>>>>>>>>>> disk = [ "tap:aio:/vms_c/guest1.img,xvda,w",
>>>>>>>>>>>>>>>>> "tap:aio:/vms_c/guest1-disk.img,xvdb,w" ]
>>>>>>>>>>>>>>>>> vif = [ "rate=10MB/s,mac=00:16:3e:6b:be:71,bridge=xenbr0" ]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks for you help Emmanuel ! Really appreciate it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -Paras.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Nov 22, 2013 at 11:10 AM, emmanuel segura <
>>>>>>>>>>>>>>>>> emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ok, but your vm doesn't start on others nodes, i think,
>>>>>>>>>>>>>>>>>> for configuration problems
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ================================================================
>>>>>>>>>>>>>>>>>> Nov 21 15:40:29 vtst3 clurgmgrd[13911]: <notice> start on
>>>>>>>>>>>>>>>>>> vm "guest1" returned 1 (generic error)
>>>>>>>>>>>>>>>>>> Nov 21 15:40:29 vtst3 clurgmgrd[13911]: <warning> #68:
>>>>>>>>>>>>>>>>>> Failed to start vm:guest1; return value: 1
>>>>>>>>>>>>>>>>>> Nov 21 15:40:29 vtst3 clurgmgrd[13911]: <notice> Stopping
>>>>>>>>>>>>>>>>>> service vm:guest1
>>>>>>>>>>>>>>>>>> Nov 21 15:40:35 vtst3 clurgmgrd[13911]: <notice> Service
>>>>>>>>>>>>>>>>>> vm:guest1 is recovering
>>>>>>>>>>>>>>>>>> Nov 21 15:40:35 vtst3 clurgmgrd[13911]: <warning> #71:
>>>>>>>>>>>>>>>>>> Relocating failed service vm:guest1
>>>>>>>>>>>>>>>>>> Nov 21 15:40:35 vtst3 clurgmgrd[13911]: <notice> Service
>>>>>>>>>>>>>>>>>> vm:guest1 is stopped
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ================================================================
>>>>>>>>>>>>>>>>>> in few words, try in every cluster node
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> export OCF_RESKEY_name="guest1" ; OCF_RESKEY_path="/vms_c"
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> set -x
>>>>>>>>>>>>>>>>>> /usr/share/cluster/vm.sh start
>>>>>>>>>>>>>>>>>> /usr/share/cluster/vm.sh stop
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> after you check if your vm can start and stop on every
>>>>>>>>>>>>>>>>>> cluster node,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> /usr/share/cluster/vm.sh start
>>>>>>>>>>>>>>>>>> /usr/share/cluster/vm.sh migrate name_of_a_cluster_node
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> can you show me your vm configuration under /vms_c?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>> Emmanuel
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2013/11/22 Paras pradhan <pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> And also to test I made use_virsh=1 , same problem. The
>>>>>>>>>>>>>>>>>>> vm does not start up if the FD domains are offline.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -Paras.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Fri, Nov 22, 2013 at 10:37 AM, Paras pradhan <
>>>>>>>>>>>>>>>>>>> pradhanparas at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Well thats seems to theoretically correct. But right
>>>>>>>>>>>>>>>>>>>> now my cluser has use_virsh=0 and I don't have any issue untill my mebmers
>>>>>>>>>>>>>>>>>>>> on the failover domains are offline. So wondering what is it that clusvcadm
>>>>>>>>>>>>>>>>>>>> -e is looking when I don't use virsh .
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Fri, Nov 22, 2013 at 10:05 AM, emmanuel segura <
>>>>>>>>>>>>>>>>>>>> emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> If you used virt-install, i think you need use virsh,
>>>>>>>>>>>>>>>>>>>>> the cluster uses xm xen command if you got use_virsh=0 and virsh if you got
>>>>>>>>>>>>>>>>>>>>> use_virsh=1 in your cluster config
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2013/11/22 Paras pradhan <pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I use virt-install to create virtual machines. Is
>>>>>>>>>>>>>>>>>>>>>> there a way to debug why clusvcadm -e vm:guest1 is failing? vm.sh seems to
>>>>>>>>>>>>>>>>>>>>>> use virsh and my cluster.conf has use_virsh=0
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Paras.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Nov 21, 2013 5:53 PM, "emmanuel segura" <
>>>>>>>>>>>>>>>>>>>>>> emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> but did you configure your vm with xen tools or
>>>>>>>>>>>>>>>>>>>>>>> using virt-manager?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2013/11/22 Paras pradhan <pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Well no i don't want to use virsh. But as we are
>>>>>>>>>>>>>>>>>>>>>>>> debugging with virsh now i found a strange issue.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I exported an xml file and imported to all nodes .
>>>>>>>>>>>>>>>>>>>>>>>> Ran
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>>>>>> name="guest1" path="/vms_c"
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> export OCF_RESKEY_name="guest1" ;
>>>>>>>>>>>>>>>>>>>>>>>> OCF_RESKEY_path="/vms_c"
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> set -x
>>>>>>>>>>>>>>>>>>>>>>>> /usr/share/cluster/vm.sh start
>>>>>>>>>>>>>>>>>>>>>>>> set +x
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> vm starts now. BUT from a cluster service :
>>>>>>>>>>>>>>>>>>>>>>>> cluvscam -e vm:guest1 , same error.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> So if i populate all my domains' config files to
>>>>>>>>>>>>>>>>>>>>>>>> all my cluser nodes and make use_virsh=1, then the issue is resolved. But
>>>>>>>>>>>>>>>>>>>>>>>> this is a lot of work for those who have hundreds of vm.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> vm.start uses virsh . Is there a way to tell him
>>>>>>>>>>>>>>>>>>>>>>>> not use virsh?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>> Paras.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Nov 21, 2013 at 5:19 PM, emmanuel segura <
>>>>>>>>>>>>>>>>>>>>>>>> emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> if you are using virsh for manage your vms, change
>>>>>>>>>>>>>>>>>>>>>>>>> this in your cluster.conf
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>>>>>>>>>> use_virsh="0"
>>>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>>>> use_virsh="1"
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2013/11/22 Paras pradhan <pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I think i found the problem.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> virsh list --all does not show my vm . This is
>>>>>>>>>>>>>>>>>>>>>>>>>> because it was created on another node. and another node has it. Now I want
>>>>>>>>>>>>>>>>>>>>>>>>>> to start the service on a different node in which it was not created or
>>>>>>>>>>>>>>>>>>>>>>>>>> where virsh list --all does not have an entry. Is it possible to create
>>>>>>>>>>>>>>>>>>>>>>>>>> this entry using a xen config file?Looks like this is now a Xen issue
>>>>>>>>>>>>>>>>>>>>>>>>>> rather than a linux-cluster issue . :)
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Paras.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Nov 21, 2013 at 4:58 PM, emmanuel segura
>>>>>>>>>>>>>>>>>>>>>>>>>> <emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 1:did you verify your xen livemigration
>>>>>>>>>>>>>>>>>>>>>>>>>>> configuration?
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2: where you vm disk reside?
>>>>>>>>>>>>>>>>>>>>>>>>>>> 3: can you see your vm defined on every cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>> node with xm list?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/11/21 Paras pradhan <pradhanparas at gmail.com
>>>>>>>>>>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is what I get
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor: xen
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Management tool: virsh
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hypervisor URI: xen:///
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Migration URI format: xenmigr://target_host/
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Virtual machine guest1 is error: failed to get
>>>>>>>>>>>>>>>>>>>>>>>>>>>> domain 'guest1'
>>>>>>>>>>>>>>>>>>>>>>>>>>>> error: Domain not found:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> xenUnifiedDomainLookupByName
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> <debug> virsh -c xen:/// start guest1
>>>>>>>>>>>>>>>>>>>>>>>>>>>> error: failed to get domain 'guest1'
>>>>>>>>>>>>>>>>>>>>>>>>>>>> error: Domain not found:
>>>>>>>>>>>>>>>>>>>>>>>>>>>> xenUnifiedDomainLookupByName
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> ++ printf '\033]0;%s@%s:%s\007' root vtst3 '~'
>>>>>>>>>>>>>>>>>>>>>>>>>>>> [root at cvtst3 ~]# set +x
>>>>>>>>>>>>>>>>>>>>>>>>>>>> + set +x
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I am wondering why it failed to get domain .
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Paras.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Nov 21, 2013 at 4:43 PM, emmanuel
>>>>>>>>>>>>>>>>>>>>>>>>>>>> segura <emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> yes
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/11/21 Paras pradhan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Well it is guest1. Isn't it?.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <vm autostart="1" domain="myfd1"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> exclusive="0" max_restarts="0" name="guest1" path="/vms_c"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> recovery="restart" restart_expire_time="0" use_virsh="0"/>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It is a vm service if it matters.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Paras.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Nov 21, 2013 at 4:22 PM, emmanuel
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> segura <emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> use the servicename you defined in your
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> cluster.conf
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/11/21 Paras pradhan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Says:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Running in test mode.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> No resource guest1 of type service found
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -Paras.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Nov 21, 2013 at 4:07 PM, emmanuel
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> segura <emi2fast at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> rg_test test /etc/cluster/cluster.conf
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> start service guest1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/11/21 Paras pradhan <
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> pradhanparas at gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> My failover domain looks like this:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomain name="myfd1"
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> nofailback="1" ordered="1" restricted="0">
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomainnode name="vtst1" priority="1"/>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomainnode name="vtst3" priority="2"/>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomainnode name="vtst2" priority="3"/>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> </failoverdomain>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have vm service that uses this failover
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> domain. If my node vtst1 is offline, the service doesnot start on vtst3
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> which is 2nd in the priority.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I tried to start it with: clusvcadm -e
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> vm:guest1 and even with -F and -m option.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> All i see is this error:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nov 21 15:40:29 vtst3 clurgmgrd[13911]:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <notice> start on vm "guest1" returned 1 (generic error)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nov 21 15:40:29 vtst3 clurgmgrd[13911]:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <warning> #68: Failed to start vm:guest1; return value: 1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nov 21 15:40:29 vtst3 clurgmgrd[13911]:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <notice> Stopping service vm:guest1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nov 21 15:40:35 vtst3 clurgmgrd[13911]:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <notice> Service vm:guest1 is recovering
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nov 21 15:40:35 vtst3 clurgmgrd[13911]:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <warning> #71: Relocating failed service vm:guest1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Nov 21 15:40:35 vtst3 clurgmgrd[13911]:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <notice> Service vm:guest1 is stopped
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> How do I debug?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Paras.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> dios quiera
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quiera
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> quiera
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios
>>>>>>>>>>>>>>>>>>>>>>>>>>> quiera
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Linux-cluster mailing list
>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>>>
>>>>>>>> --
>>>>>>>> Linux-cluster mailing list
>>>>>>>> Linux-cluster at redhat.com
>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Linux-cluster mailing list
>>>>>>> Linux-cluster at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>>>
>>>>>> --
>>>>>> Linux-cluster mailing list
>>>>>> Linux-cluster at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Linux-cluster mailing list
>>>>> Linux-cluster at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> esta es mi vida e me la vivo hasta que dios quiera
>>>>
>>>> --
>>>> Linux-cluster mailing list
>>>> Linux-cluster at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>
>>>
>>>
>>> --
>>> Linux-cluster mailing list
>>> Linux-cluster at redhat.com
>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>
>>
>>
>>
>> --
>> esta es mi vida e me la vivo hasta que dios quiera
>>
>
>
>
> --
> esta es mi vida e me la vivo hasta que dios quiera
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>
--
esta es mi vida e me la vivo hasta que dios quiera
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20131219/980c0cb5/attachment.htm>
More information about the Linux-cluster
mailing list