[libvirt] [PATCH v3 00/14] Network filtering (ACL) extensions for libvirt
Daniel P. Berrange
berrange at redhat.com
Thu Mar 25 11:37:14 UTC 2010
On Tue, Mar 23, 2010 at 10:54:06AM -0400, stefanb at us.ibm.com wrote:
> One comment about the instantiation of the rules: Since the XML allows
> to create nearly any possible combination of parameters to ebtables or
> iptables commands, I haven't used the ebtables or iptables wrappers.
> Instead, I am writing ebtables/iptables command into a buffer, add
> command line options to each one of them as described in the rule's XML,
> write the buffer into a file and run it as a script. For those commands
> that are not allowed to fail I am using the following format to run
> them:
>
> cmd="ebtables <some options>"
> r=`${cmd}`
> if [ $? -ne 0 ]; then
> echo "Failure in command ${cmd}."
> exit 1
> fi
>
> cmd="..."
> [...]
>
> If one of the command fails in such a batch, the libvirt code is going
> pick up the error code '1', tear down anything previously established
> and report an error back. The actual error message shown above is
> currently not reported back, but can be later on with some changes to
> the commands running external programs that need to read the script's
> stdout.
I've tried out this patch series, but not entirely sure whether it is
working correctly, since there are lots of errors in the libvirt debug
logs.
I have a guest
<interface type='network'>
<mac address='52:54:00:4d:c8:5f'/>
<source network='default'/>
<filterref filter='demofilter4'>
<parameter name='IP' value='9.59.241.151'/>
</filterref>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</interface>
And demofilter4 is setup as:
<filter name='demofilter4' chain='root'>
<uuid>d33d17bf-0198-6b89-0b03-55abba1eae7f</uuid>
<filterref filter='no-mac-spoofing'/>
<filterref filter='no-mac-broadcast'/>
<filterref filter='no-arp-spoofing'/>
<filterref filter='allow-dhcp'>
<parameter name='DHCPSERVER' value='10.0.0.1'/>
</filterref>
</filter>
The filter it references here are the XML examples you sent out previously
The guest starts without error, and I get a lot of things in the ebtables
'nat' table, but nothing in the 'filter' table - is that expected ?
# ebtables -t nat -L
Bridge table: nat
Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j I-vnet0
Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
-o vnet0 -j O-vnet0
Bridge chain: I-vnet0, entries: 2, policy: ACCEPT
-p IPv4 -j I-vnet0-ipv4
-p ARP -j I-vnet0-arp
Bridge chain: O-vnet0, entries: 2, policy: ACCEPT
-p IPv4 -j O-vnet0-ipv4
-p ARP -j O-vnet0-arp
Bridge chain: I-vnet0-ipv4, entries: 3, policy: ACCEPT
-s ! 52:54:0:4d:c8:5f -j DROP
-p IPv4 --ip-src 0.0.0.0 --ip-dst 255.255.255.255 --ip-proto udp --ip-sport 68 --ip-dport 67 -j ACCEPT
-d Broadcast -j DROP
Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT
-p IPv4 --ip-src 10.0.0.1 --ip-proto udp --ip-sport 67 --ip-dport 68 -j ACCEPT
Bridge chain: I-vnet0-arp, entries: 5, policy: ACCEPT
-p ARP --arp-mac-src ! 52:54:0:4d:c8:5f -j DROP
-p ARP --arp-ip-src ! 9.59.241.151 -j DROP
-p ARP --arp-op Request -j ACCEPT
-p ARP --arp-op Reply -j ACCEPT
-j DROP
Bridge chain: O-vnet0-arp, entries: 5, policy: ACCEPT
-p ARP --arp-op Reply --arp-mac-dst ! 52:54:0:4d:c8:5f -j DROP
-p ARP --arp-ip-dst ! 9.59.241.151 -j DROP
-p ARP --arp-op Request -j ACCEPT
-p ARP --arp-op Reply -j ACCEPT
-j DROP
The libvirt logs show a bunch of shell erorrs - see the attachment to this
mail. Are these errors to be expected, or are they actual errors ?
Also, I'm wondering if it is possible to include 'libvirt-' in the custom
ebtables/iptables chains that we create ? I think that would be useful
for sysadmins who might be surprised at where all these rules are coming
from. From the code it looks like we have a fairly low size limit on the
possible chain names, but I think at least for the first two chains we
ought to be able to get 'libvirt-' prefix in there. Or perhaps instead
of adding the per-VIF chains straight to PREROUTING/POSTROUTING, have
an unconditional jump to a libvirt specific chain.
So either have
Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j libvirt-I-vnet0
Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
-o vnet0 -j libvirt-O-vnet0
Or the slightly more complex
Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-j libvirt-I
Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
-j libvirt-O
Bridge chain: libvirt-I, entries: 1, policy: ACCEPT
-i vnet0 -j I-vnet0
Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
Bridge chain: libvirt-O, entries: 1, policy: ACCEPT
-o vnet0 -j O-vnet0
Regards,
Daniel
--
|: Red Hat, Engineering, London -o- http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|
-------------- next part --------------
11:29:43.570: debug : qemudStartVMDaemon:2855 : Building emulator command line
11:29:43.583: debug : _virNWFilterInstantiateFilter:526 : filter name: demofilter4
11:29:43.583: debug : _virNWFilterInstantiateRec:321 : Instantiating filter no-mac-spoofing
11:29:43.583: debug : _virNWFilterInstantiateRec:321 : Instantiating filter no-mac-broadcast
11:29:43.583: debug : _virNWFilterInstantiateRec:321 : Instantiating filter no-arp-spoofing
11:29:43.583: debug : _virNWFilterInstantiateRec:321 : Instantiating filter allow-dhcp
11:29:43.583: debug : ebiptablesExecCLI:998 : /sbin/ebtables -t nat -D PREROUTING -i vnet0 -j J-vnet0
/sbin/ebtables -t nat -D POSTROUTING -o vnet0 -j P-vnet0
/sbin/ebtables -t nat -D J-vnet0 -p ipv4 -j J-vnet0-ipv4
/sbin/ebtables -t nat -F J-vnet0-ipv4
/sbin/ebtables -t nat -X J-vnet0-ipv4
/sbin/ebtables -t nat -D P-vnet0 -p ipv4 -j P-vnet0-ipv4
/sbin/ebtables -t nat -F P-vnet0-ipv4
/sbin/ebtables -t nat -X P-vnet0-ipv4
/sbin/ebtables -t nat -D J-vnet0 -p ipv6 -j J-vnet0-ipv6
/sbin/ebtables -t nat -F J-vnet0-ipv6
/sbin/ebtables -t nat -X J-vnet0-ipv6
/sbin/ebtables -t nat -D P-vnet0 -p ipv6 -j P-vnet0-ipv6
/sbin/ebtables -t nat -F P-vnet0-ipv6
/sbin/ebtables -t nat -X P-vnet0-ipv6
/sbin/ebtables -t nat -D J-vnet0 -p arp -j J-vnet0-arp
/sbin/ebtables -t nat -F J-vnet0-arp
/sbin/ebtables -t nat -X J-vnet0-arp
/sbin/ebtables -t nat -D P-vnet0 -p arp -j P-vnet0-arp
/sbin/ebtables -t nat -F P-vnet0-arp
/sbin/ebtables -t nat -X P-vnet0-arp
/sbin/ebtables -t nat -F J-vnet0
/sbin/ebtables -t nat -X J-vnet0
/sbin/ebtables -t nat -F P-vnet0
/sbin/ebtables -t nat -X P-vnet0
11:29:43.584: debug : virRunWithHook:900 : /tmp/virtd8nBVpY
11:29:43.695: debug : virRunWithHook:918 : Command stderr: Illegal target name 'J-vnet0'.
Illegal target name 'P-vnet0'.
Chain 'J-vnet0' doesn't exist.
Chain 'J-vnet0-ipv4' doesn't exist.
Chain 'J-vnet0-ipv4' doesn't exist.
Chain 'P-vnet0' doesn't exist.
Chain 'P-vnet0-ipv4' doesn't exist.
Chain 'P-vnet0-ipv4' doesn't exist.
Chain 'J-vnet0' doesn't exist.
Chain 'J-vnet0-ipv6' doesn't exist.
Chain 'J-vnet0-ipv6' doesn't exist.
Chain 'P-vnet0' doesn't exist.
Chain 'P-vnet0-ipv6' doesn't exist.
Chain 'P-vnet0-ipv6' doesn't exist.
Chain 'J-vnet0' doesn't exist.
Chain 'J-vnet0-arp' doesn't exist.
Chain 'J-vnet0-arp' doesn't exist.
Chain 'P-vnet0' doesn't exist.
Chain 'P-vnet0-arp' doesn't exist.
Chain 'P-vnet0-arp' doesn't exist.
Chain 'J-vnet0' doesn't exist.
Chain 'J-vnet0' doesn't exist.
Chain 'P-vnet0' doesn't exist.
Chain 'P-vnet0' doesn't exist.
11:29:43.695: debug : ebiptablesExecCLI:1014 : rc = 0, status = 255
11:29:43.695: debug : ebiptablesExecCLI:998 : cmd="/sbin/ebtables -t nat -N J-vnet0"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -N P-vnet0"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -N J-vnet0-ipv4"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0 -p ipv4 -j J-vnet0-ipv4"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -N P-vnet0-ipv4"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0 -p ipv4 -j P-vnet0-ipv4"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -N J-vnet0-arp"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0 -p arp -j J-vnet0-arp"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -N P-vnet0-arp"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0 -p arp -j P-vnet0-arp"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
11:29:43.695: debug : virRunWithHook:900 : /tmp/virtdbspnp8
11:29:43.764: debug : ebiptablesExecCLI:1014 : rc = 0, status = 0
11:29:43.764: debug : ebiptablesExecCLI:998 : cmd="/sbin/ebtables -t nat -A J-vnet0-ipv4 -s ! 52:54:00:4D:C8:5F -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0-ipv4 -p ipv4 --ip-source 0.0.0.0 --ip-destination 255.255.255.255 --ip-protocol 17 --ip-source-port 68 --ip-destination-port 67 -j ACCEPT"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0-ipv4 -p ipv4 --ip-source 10.0.0.1 --ip-protocol 17 --ip-source-port 67 --ip-destination-port 68 -j ACCEPT"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0-arp -p arp --arp-mac-src ! 52:54:00:4D:C8:5F -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0-arp -p arp --arp-ip-src ! 9.59.241.151 -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0-arp -p arp --arp-opcode 2 --arp-mac-dst ! 52:54:00:4D:C8:5F -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0-arp -p arp --arp-ip-dst ! 9.59.241.151 -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0-ipv4 -d FF:FF:FF:FF:FF:FF -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0-arp -p arp --arp-opcode 1 -j ACCEPT"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0-arp -p arp --arp-opcode 1 -j ACCEPT"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0-arp -p arp --arp-opcode 2 -j ACCEPT"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0-arp -p arp --arp-opcode 2 -j ACCEPT"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A J-vnet0-arp -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A P-vnet0-arp -j DROP"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
11:29:43.765: debug : virRunWithHook:900 : /tmp/virtdXNecyi
11:29:43.863: debug : ebiptablesExecCLI:1014 : rc = 0, status = 0
11:29:43.863: debug : ebiptablesExecCLI:998 : cmd="/sbin/ebtables -t nat -A PREROUTING -i vnet0 -j J-vnet0"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
cmd="/sbin/ebtables -t nat -A POSTROUTING -o vnet0 -j P-vnet0"
res=`${cmd}`
if [ $? -ne 0 ]; then echo "Failure to execute command '${cmd}'."; exit 1;fi
11:29:43.863: debug : virRunWithHook:900 : /tmp/virtdHfjnUs
11:29:43.888: debug : ebiptablesExecCLI:1014 : rc = 0, status = 0
11:29:43.888: debug : ebiptablesExecCLI:998 : /sbin/iptables -D virt-out -m physdev --physdev-out vnet0 -g FO-vnet0
/sbin/iptables -D virt-in -m physdev --physdev-in vnet0 -g FI-vnet0
/sbin/iptables -D host-in -m physdev --physdev-in vnet0 -g HI-vnet0
/sbin/iptables -F FO-vnet0
/sbin/iptables -X FO-vnet0
/sbin/iptables -F FI-vnet0
/sbin/iptables -X FI-vnet0
/sbin/iptables -F HI-vnet0
/sbin/iptables -X HI-vnet0
/sbin/ebtables -t nat -D PREROUTING -i vnet0 -j I-vnet0
/sbin/ebtables -t nat -D POSTROUTING -o vnet0 -j O-vnet0
/sbin/ebtables -t nat -D I-vnet0 -p ipv4 -j I-vnet0-ipv4
/sbin/ebtables -t nat -F I-vnet0-ipv4
/sbin/ebtables -t nat -X I-vnet0-ipv4
/sbin/ebtables -t nat -D O-vnet0 -p ipv4 -j O-vnet0-ipv4
/sbin/ebtables -t nat -F O-vnet0-ipv4
/sbin/ebtables -t nat -X O-vnet0-ipv4
/sbin/ebtables -t nat -D I-vnet0 -p ipv6 -j I-vnet0-ipv6
/sbin/ebtables -t nat -F I-vnet0-ipv6
/sbin/ebtables -t nat -X I-vnet0-ipv6
/sbin/ebtables -t nat -D O-vnet0 -p ipv6 -j O-vnet0-ipv6
/sbin/ebtables -t nat -F O-vnet0-ipv6
/sbin/ebtables -t nat -X O-vnet0-ipv6
/sbin/ebtables -t nat -D I-vnet0 -p arp -j I-vnet0-arp
/sbin/ebtables -t nat -F I-vnet0-arp
/sbin/ebtables -t nat -X I-vnet0-arp
/sbin/ebtables -t nat -D O-vnet0 -p arp -j O-vnet0-arp
/sbin/ebtables -t nat -F O-vnet0-arp
/sbin/ebtables -t nat -X O-vnet0-arp
/sbin/ebtables -t nat -F I-vnet0
/sbin/ebtables -t nat -X I-vnet0
/sbin/ebtables -t nat -F O-vnet0
/sbin/ebtables -t nat -X O-vnet0
/sbin/ebtables -t nat -E J-vnet0-ipv4 I-vnet0-ipv4
/sbin/ebtables -t nat -E P-vnet0-ipv4 O-vnet0-ipv4
/sbin/ebtables -t nat -E J-vnet0-ipv6 I-vnet0-ipv6
/sbin/ebtables -t nat -E P-vnet0-ipv6 O-vnet0-ipv6
/sbin/ebtables -t nat -E J-vnet0-arp I-vnet0-arp
/sbin/ebtables -t nat -E P-vnet0-arp O-vnet0-arp
/sbin/ebtables -t nat -E J-vnet0 I-vnet0
/sbin/ebtables -t nat -E P-vnet0 O-vnet0
11:29:43.888: debug : virRunWithHook:900 : /tmp/virtd1KpUjD
11:29:44.076: debug : virRunWithHook:918 : Command stderr: iptables v1.4.7: goto 'FO-vnet0' is not a chain
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.4.7: goto 'FI-vnet0' is not a chain
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.4.7: goto 'HI-vnet0' is not a chain
Try `iptables -h' or 'iptables --help' for more information.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
Illegal target name 'I-vnet0'.
Illegal target name 'O-vnet0'.
Chain 'I-vnet0' doesn't exist.
Chain 'I-vnet0-ipv4' doesn't exist.
Chain 'I-vnet0-ipv4' doesn't exist.
Chain 'O-vnet0' doesn't exist.
Chain 'O-vnet0-ipv4' doesn't exist.
Chain 'O-vnet0-ipv4' doesn't exist.
Chain 'I-vnet0' doesn't exist.
Chain 'I-vnet0-ipv6' doesn't exist.
Chain 'I-vnet0-ipv6' doesn't exist.
Chain 'O-vnet0' doesn't exist.
Chain 'O-vnet0-ipv6' doesn't exist.
Chain 'O-vnet0-ipv6' doesn't exist.
Chain 'I-vnet0' doesn't exist.
Chain 'I-vnet0-arp' doesn't exist.
Chain 'I-vnet0-arp' doesn't exist.
Chain 'O-vnet0' doesn't exist.
Chain 'O-vnet0-arp' doesn't exist.
Chain 'O-vnet0-arp' doesn't exist.
Chain 'I-vnet0' doesn't exist.
Chain 'I-vnet0' doesn't exist.
Chain 'O-vnet0' doesn't exist.
Chain 'O-vnet0' doesn't exist.
Chain 'J-vnet0-ipv6' doesn't exist.
Chain 'P-vnet0-ipv6' doesn't exist.
11:29:44.076: debug : ebiptablesExecCLI:1014 : rc = 0, status = 0
11:29:44.077: debug : virExecWithHook:709 : LC_ALL=C PATH=/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin HOME=/root USER=root LOGNAME=root QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M pc-0.11 -cpu qemu32 -no-kvm -m 62 -smp 1,sockets=1,cores=1,threads=1 -name plain -uuid c7a3edbd-edaf-9455-926a-d65c16db1809 -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/plain.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -no-acpi -boot c -kernel /var/lib/libvirt/boot/vmlinuz -initrd /var/lib/libvirt/boot/initrd.img -drive file=/var/lib/libvirt/images/plain.img,if=none,id=drive-ide0-0-0,boot=on,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -device rtl8139,vlan=0,id=net0,mac=52:54:00:4d:c8:5f,bus=pci.0,addr=0x4 -net tap,fd=19,vlan=0,name=hostnet0 -usb -vnc 127.0.0.1:0,password -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3
More information about the libvir-list
mailing list