[Fedora-xen] Multihomed networking

Tue Apr 25 01:27:45 UTC 2006

Is anyone using FC5 Xen with an FC5 guest, where the Dom0 has two ethernet devices (succesfully)?

My system has eth0 as a 192.168.2.x network, eth1 as my static Internet address, and iptables NAT between them. All real machines on my home network can get out just fine.

My guest VM, also a 192.168.2.x address, is able to reach other machines on my internal network without problem, but trying to access the internet doesn't work. At first I thought it was an iptables configuration issue on the Dom0 machine, but I don't think so anymore. As a test, I inserted at the top of every table a rule to emit a log message when any machine on my net tried to access a specific remote host; performing a wget command in the guest VM shows my outbound SYN, shows an incoming SYN+ACK from the remote host (so outbound masquarading must be working?) and an outbound ACK (so incoming demasquarading must be working?)... I never receive another packet from the remote host (until quite a while later, when I receive FIN). If I try the same exercise with a real host on my network, I get a normal data stream in/out.

Also, traceroute from the guest VM doesn't work exactly as I would expect. I would expect the results of one to be quite similar to those from my Dom0 machine, but consistantly I am missing responses along the path. For example:

>From dom0:
[root at heilpern ~]# traceroute www.yahoo.com
traceroute to www.yahoo.com (216.109.117.106), 30 hops max, 40 byte packets
 1  [my cable modem]
 2  10.106.32.1 (10.106.32.1)  8.279 ms   11.786 ms *
 3  * * *
 4  * * *
 5  * * *
 6  * * *
 7  ae-0-0.bbr2.Washington1.Level3.net (4.68.128.210)  45.855 ms * *
 8  ae-21-54.car1.Washington1.Level3.net (4.68.121.114)  35.593 ms ae-21-56.car1.Washington1.Level3.net (4.68.121.178)  35.382 ms   41.539 ms
 9  4.79.228.2 (4.79.228.2)  40.073 ms   39.931 ms *
10  * * *
11  * * *
12  * p21.www.dcn.yahoo.com (216.109.117.106)  35.227 ms *

>From the guest VM:
[root at testvm ~]# traceroute www.yahoo.com
traceroute to www.yahoo.com (216.109.117.106), 30 hops max, 40 byte packets
 1  192.168.2.1 (192.168.2.1)  0.000 ms   0.000 ms   0.000 ms
 2  [my cable modem]
 3  10.106.32.1 (10.106.32.1)  9.910 ms   13.919 ms *
 4  * * *
 5  * * *
 6  * * *
 7  * * *
 8  ae-0-0.bbr2.Washington1.Level3.net (4.68.128.210)  48.307 ms * *
 9  ae-11-51.car1.Washington1.Level3.net (4.68.121.18)  37.938 ms   42.377 ms ae-21-54.car1.Washington1.Level3.net (4.68.121.114)  43.048 ms
10  4.79.228.2 (4.79.228.2)  45.568 ms   43.918 ms *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *
[root at testvm ~]#

This is pretty repeatable.

Any thoughts?

Here are my Dom0 configurations:
[root at heilpern ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.feffffffffff       no              peth0
                                                        vif0.0
                                                        vif2.0
[root at heilpern ~]# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:10:DC:F2:70:17
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::210:dcff:fef2:7017/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:733780 errors:0 dropped:0 overruns:0 frame:0
          TX packets:822036 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:87799449 (83.7 MiB)  TX bytes:207088442 (197.4 MiB)

[root at heilpern ~]# ifconfig eth1
eth1      Link encap:Ethernet  HWaddr 00:90:47:03:B8:63
          inet addr:[my IP]  Bcast:[my BCAST IP]  Mask:255.255.255.252
          inet6 addr: fe80::290:47ff:fe03:b863/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:829216 errors:0 dropped:0 overruns:0 frame:0
          TX packets:556902 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:169446616 (161.5 MiB)  TX bytes:85050877 (81.1 MiB)
          Interrupt:18

[root at heilpern ~]# iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

(temporary wide-open rules):
[root at heilpern ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain INBOUND (0 references)
target     prot opt source               destination

Chain LOG_FILTER (0 references)
target     prot opt source               destination

Chain LSI (0 references)
target     prot opt source               destination

Chain LSO (0 references)
target     prot opt source               destination

Chain OUTBOUND (0 references)
target     prot opt source               destination

Thanks in advance...
Mark