[libvirt] [PATCH v3 00/14] Network filtering (ACL) extensions for libvirt
Stefan Berger
stefanb at us.ibm.com
Thu Mar 25 12:26:42 UTC 2010
"Daniel P. Berrange" <berrange at redhat.com> wrote on 03/25/2010 07:37:14
AM:
>
> Please respond to "Daniel P. Berrange"
>
> On Tue, Mar 23, 2010 at 10:54:06AM -0400, stefanb at us.ibm.com wrote:
> > One comment about the instantiation of the rules: Since the XML allows
> > to create nearly any possible combination of parameters to ebtables or
> > iptables commands, I haven't used the ebtables or iptables wrappers.
> > Instead, I am writing ebtables/iptables command into a buffer, add
> > command line options to each one of them as described in the rule's
XML,
> > write the buffer into a file and run it as a script. For those
commands
> > that are not allowed to fail I am using the following format to run
> > them:
> >
> > cmd="ebtables <some options>"
> > r=`${cmd}`
> > if [ $? -ne 0 ]; then
> > echo "Failure in command ${cmd}."
> > exit 1
> > fi
> >
> > cmd="..."
> > [...]
> >
> > If one of the command fails in such a batch, the libvirt code is going
> > pick up the error code '1', tear down anything previously established
> > and report an error back. The actual error message shown above is
> > currently not reported back, but can be later on with some changes to
> > the commands running external programs that need to read the script's
> > stdout.
>
> I've tried out this patch series, but not entirely sure whether it is
> working correctly, since there are lots of errors in the libvirt debug
> logs.
>
The errors in the debugging log are due to attempts to cleanup
user-defined tables
that do not exist. Though attempting to clean them up gets the system into
a well
known start state. The errors could be avoided with a sequence of checking
for
existence of the user-defined chain first and then try to flush and delete
them,
but it's faster not to do so.
> I have a guest
>
> <interface type='network'>
> <mac address='52:54:00:4d:c8:5f'/>
> <source network='default'/>
> <filterref filter='demofilter4'>
> <parameter name='IP' value='9.59.241.151'/>
> </filterref>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </interface>
>
> And demofilter4 is setup as:
>
> <filter name='demofilter4' chain='root'>
> <uuid>d33d17bf-0198-6b89-0b03-55abba1eae7f</uuid>
> <filterref filter='no-mac-spoofing'/>
> <filterref filter='no-mac-broadcast'/>
> <filterref filter='no-arp-spoofing'/>
> <filterref filter='allow-dhcp'>
> <parameter name='DHCPSERVER' value='10.0.0.1'/>
> </filterref>
> </filter>
>
> The filter it references here are the XML examples you sent out
previously
>
> The guest starts without error, and I get a lot of things in the
ebtables
> 'nat' table, but nothing in the 'filter' table - is that expected ?
Yes, this is correct. The nat table handles the filtering correctly and
with the
PREROUTING chain we can filter for traffic coming from the VM (that
becomes
incoming to the host - prefix letter 'I') and in the POSTROUTING chain
traffic going to the VM (outgoing from the host - prefix letter 'O').
>
> # ebtables -t nat -L
> Bridge table: nat
>
> Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
> -i vnet0 -j I-vnet0
>
> Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
>
> Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
> -o vnet0 -j O-vnet0
>
> Bridge chain: I-vnet0, entries: 2, policy: ACCEPT
> -p IPv4 -j I-vnet0-ipv4
> -p ARP -j I-vnet0-arp
>
> Bridge chain: O-vnet0, entries: 2, policy: ACCEPT
> -p IPv4 -j O-vnet0-ipv4
> -p ARP -j O-vnet0-arp
>
> Bridge chain: I-vnet0-ipv4, entries: 3, policy: ACCEPT
> -s ! 52:54:0:4d:c8:5f -j DROP
> -p IPv4 --ip-src 0.0.0.0 --ip-dst 255.255.255.255 --ip-proto udp --
> ip-sport 68 --ip-dport 67 -j ACCEPT
> -d Broadcast -j DROP
>
> Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT
> -p IPv4 --ip-src 10.0.0.1 --ip-proto udp --ip-sport 67 --ip-dport 68-j
ACCEPT
>
> Bridge chain: I-vnet0-arp, entries: 5, policy: ACCEPT
> -p ARP --arp-mac-src ! 52:54:0:4d:c8:5f -j DROP
> -p ARP --arp-ip-src ! 9.59.241.151 -j DROP
> -p ARP --arp-op Request -j ACCEPT
> -p ARP --arp-op Reply -j ACCEPT
> -j DROP
>
> Bridge chain: O-vnet0-arp, entries: 5, policy: ACCEPT
> -p ARP --arp-op Reply --arp-mac-dst ! 52:54:0:4d:c8:5f -j DROP
> -p ARP --arp-ip-dst ! 9.59.241.151 -j DROP
> -p ARP --arp-op Request -j ACCEPT
> -p ARP --arp-op Reply -j ACCEPT
> -j DROP
>
>
>
> The libvirt logs show a bunch of shell erorrs - see the attachment to
this
> mail. Are these errors to be expected, or are they actual errors ?
They are due to attempted cleanups that try to cleanup user-defined tables
that did not exist on your system,
however. As said, this gets the system into a well known start-out state
so that those user-defined ebtables
tables can be established with rules. As seen above, every interface gets
a 'tree' of user-defined tables.
Following the chain identifier in the above XML
<filter name='demofilter4' chain='root'>
<uuid>d33d17bf-0198-6b89-0b03-55abba1eae7f</uuid>
[...]
which can be either left out entirely for meaning of 'root', or be 'root'
explicitly or have the values 'arp' or 'ipv4'.
The patterns of the user-defined ebtables tables is due to 32 byte size
restrictions (including trailing 0) as follows:
<one letter prefix>-<name of the interface>-<filter chain identifier>:
This then leads to
I-vnet0 and
I-vnet0-ipv4 etc.
as shown above.
However, the tricky part are updates of filters. You can, using
virsh-define, update any filter while it is being used by running
VMs. For updates I decided not to compare the current filter tree against
the new one, calculating 'diffs'
for the rules and remove rules that don't exist in the new one along with
user defined tables that may not exist anymore
etc., which would be much more complicated. Instead, I am completely
rebuilding the filter tree. So, a temporary filter
tree is established with user-defined names following the above pattern
while the current filter tree is still active.
However, the prefixes are different. So the temporary filter tree would
have user-defined tables named
J-vnet0 and
J-vnet0-ipv4 etc.
where all the rules are established. Then, while the 'old' filter tree is
active, the new one is anchored, leading to
entries like this one:
Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j I-vnet0
-i vnet0 -j J-vnet0
Yes, temporarily you would have two parallel filter trees for an
interface, but the old one is still active. Then
comes the switch-over to the new tree where the old filter tree is
un-anchored, leading to this situation:
Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j J-vnet0
Now all possible chains starting with prefix I-vnet0-<XYZ> are flushed and
deleted.
In the subsequent step I rename the filter tree starting with
J-<vnet0>-<XYZ> to I-<vnet0>-<XYZ> leading to
Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
-i vnet0 -j I-vnet0
which is where we want to be so that the next update can do the same
procedure again.
I have made some changes to the core patch (part 11) that split up the
application of new filter rules into
3 different (interface) functions, for application of new rules, switch
over + tear down of old, or
tear down of new in case an error happened.
>
>
> Also, I'm wondering if it is possible to include 'libvirt-' in the
custom
> ebtables/iptables chains that we create ? I think that would be useful
> for sysadmins who might be surprised at where all these rules are coming
> from. From the code it looks like we have a fairly low size limit on the
> possible chain names, but I think at least for the first two chains we
> ought to be able to get 'libvirt-' prefix in there. Or perhaps instead
> of adding the per-VIF chains straight to PREROUTING/POSTROUTING, have
> an unconditional jump to a libvirt specific chain.
>
> So either have
>
>
> Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
> -i vnet0 -j libvirt-I-vnet0
>
> Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
>
> Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
> -o vnet0 -j libvirt-O-vnet0
>
>
>
> Or the slightly more complex
>
> Bridge chain: PREROUTING, entries: 1, policy: ACCEPT
> -j libvirt-I
>
> Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
>
> Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT
> -j libvirt-O
>
> Bridge chain: libvirt-I, entries: 1, policy: ACCEPT
> -i vnet0 -j I-vnet0
>
> Bridge chain: OUTPUT, entries: 0, policy: ACCEPT
>
> Bridge chain: libvirt-O, entries: 1, policy: ACCEPT
> -o vnet0 -j O-vnet0
Both is possible with the first one resulting in being more efficient.
I'll try to make that change.
Regards,
Stefan
>
>
> Regards,
> Daniel
> --
> |: Red Hat, Engineering, London -o-
http://people.redhat.com/berrange/:|
> |: http://libvirt.org -o- http://virt-manager.org -o-
http://deltacloud.org:|
> |: http://autobuild.org -o-
http://search.cpan.org/~danberr/:|
> |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B
9505 :|
> [attachment "filter.log" deleted by Stefan Berger/Watson/IBM]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20100325/e4ae006a/attachment-0001.htm>
More information about the libvir-list
mailing list