[rhos-list] Nova-network woes

[Kashif Mohammad]

Hi Guys

I am using rhel6.4 and RedHat Openstack Folsom rpms. I have  one controller node which is running glance, cinder, scheduler, api server, consoleauth and other node is running compute, network and api. Both nodes have one public and one private IP and running in multi_host mode.

I am not running nova-network and nova-compute on controller node.  Network bit is like this on compute node

public_interface = em1
flat_interface = em2
fixed_range = 10.0.1.0/24
flat_network_dhcp_start = 10.0.1.5
connection_type = libvirt
flat_injected = False
multi_host = True
flat_network_bridge = br100

created network like this
nova-manage network create private 10.0.1.0/24 1 256 --bridge=br100 --multi_host=True


I can create VM through dashboard and initially it couldn't get ip address from dhcp server running on compute node but after adding 

iptables -A POSTROUTING -t mangle -p udp --dport 68 -j CHECKSUM --checksum-fill

it is getting ip address and I can log in into vm from compute node. It can reach metadata server and grab public keys.
My problem is that VM's can not access outside word. I think that issue is in this line of iptable

47  3247 nova-network-float-snat  all  --  any    any     anywhere             anywhere
    2   168 SNAT       all  --  any    em1     10.0.1.0/24          anywhere            to:192.168.9.3

It is changing source IP address to local ip rather than external ip. If insert a rule

iptables -t nat -I nova-network-snat 2 -o em1 -j SNAT --to-source 163.1.5.40 

Then it vm can access interner but the moment I create a new VM this line disappear from iptable.

My n/w setting is like this

2: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:22:19:6a:bc:09 brd ff:ff:ff:ff:ff:ff
    inet 163.1.5.40/24 brd 163.1.5.255 scope global em1
    inet6 fe80::222:19ff:fe6a:bc09/64 scope link
       valid_lft forever preferred_lft forever
3: em2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff
    inet6 fe80::222:19ff:fe6a:bc0b/64 scope link
       valid_lft forever preferred_lft forever
5:  br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
    link/ether 00:22:19:6a:bc:0b brd ff:ff:ff:ff:ff:ff
    inet 10.0.1.1/24 brd 10.0.1.255 scope global br100
    inet 192.168.9.3/24 brd 192.168.9.255 scope global br100
    inet6 fe80::80cd:6fff:fef7:d955/64 scope link
       valid_lft forever preferred_lft forever

I can not reach vnc as well but I first fix this n/w issue and then look into vnc.

Thanks
Kashif