<tt>"Daniel P. Berrange" <berrange@redhat.com> wrote on 03/25/2010 07:37:14 AM: </tt> <tt>> > Please respond to "Daniel P. Berrange"</tt> <tt>> > On Tue, Mar 23, 2010 at 10:54:06AM -0400, stefanb@us.ibm.com wrote: > > One comment about the instantiation of the rules: Since the XML allows > > to create nearly any possible combination of parameters to ebtables or > > iptables commands, I haven't used the ebtables or iptables wrappers. > > Instead, I am writing ebtables/iptables command into a buffer, add > > command line options to each one of them as described in the rule's XML, > > write the buffer into a file and run it as a script. For those commands > > that are not allowed to fail I am using the following format to run > > them: > > > > cmd="ebtables <some options>" > > r=`${cmd}` > > if [ $? -ne 0 ]; then > > echo "Failure in command ${cmd}." > > exit 1 > > fi > > > > cmd="..." > > [...] > > > > If one of the command fails in such a batch, the libvirt code is going > > pick up the error code '1', tear down anything previously established > > and report an error back. The actual error message shown above is > > currently not reported back, but can be later on with some changes to > > the commands running external programs that need to read the script's > > stdout. > > I've tried out this patch series, but not entirely sure whether it is > working correctly, since there are lots of errors in the libvirt debug > logs. > </tt> <tt>The errors in the debugging log are due to attempts to cleanup user-defined tables</tt> <tt>that do not exist. Though attempting to clean them up gets the system into a well</tt> <tt>known start state. The errors could be avoided with a sequence of checking for</tt> <tt>existence of the user-defined chain first and then try to flush and delete them,</tt> <tt>but it's faster not to do so.</tt> <tt> > I have a guest > > <interface type='network'> > <mac address='52:54:00:4d:c8:5f'/> > <source network='default'/> > <filterref filter='demofilter4'> > <parameter name='IP' value='9.59.241.151'/> > </filterref> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x0'/> > </interface> > > And demofilter4 is setup as: > > <filter name='demofilter4' chain='root'> > <uuid>d33d17bf-0198-6b89-0b03-55abba1eae7f</uuid> > <filterref filter='no-mac-spoofing'/> > <filterref filter='no-mac-broadcast'/> > <filterref filter='no-arp-spoofing'/> > <filterref filter='allow-dhcp'> > <parameter name='DHCPSERVER' value='10.0.0.1'/> > </filterref> > </filter> > > The filter it references here are the XML examples you sent out previously > > The guest starts without error, and I get a lot of things in the ebtables > 'nat' table, but nothing in the 'filter' table - is that expected ?</tt> <tt>Yes, this is correct. The nat table handles the filtering correctly and with the</tt> <tt>PREROUTING chain we can filter for traffic coming from the VM (that becomes</tt> <tt>incoming to the host - prefix letter 'I') and in the POSTROUTING chain</tt> <tt>traffic going to the VM (outgoing from the host - prefix letter 'O').</tt> <tt> > > # ebtables -t nat -L > Bridge table: nat > > Bridge chain: PREROUTING, entries: 1, policy: ACCEPT > -i vnet0 -j I-vnet0 > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT > -o vnet0 -j O-vnet0 > > Bridge chain: I-vnet0, entries: 2, policy: ACCEPT > -p IPv4 -j I-vnet0-ipv4 > -p ARP -j I-vnet0-arp > > Bridge chain: O-vnet0, entries: 2, policy: ACCEPT > -p IPv4 -j O-vnet0-ipv4 > -p ARP -j O-vnet0-arp > > Bridge chain: I-vnet0-ipv4, entries: 3, policy: ACCEPT > -s ! 52:54:0:4d:c8:5f -j DROP > -p IPv4 --ip-src 0.0.0.0 --ip-dst 255.255.255.255 --ip-proto udp -- > ip-sport 68 --ip-dport 67 -j ACCEPT > -d Broadcast -j DROP > > Bridge chain: O-vnet0-ipv4, entries: 1, policy: ACCEPT > -p IPv4 --ip-src 10.0.0.1 --ip-proto udp --ip-sport 67 --ip-dport 68-j ACCEPT > > Bridge chain: I-vnet0-arp, entries: 5, policy: ACCEPT > -p ARP --arp-mac-src ! 52:54:0:4d:c8:5f -j DROP > -p ARP --arp-ip-src ! 9.59.241.151 -j DROP > -p ARP --arp-op Request -j ACCEPT > -p ARP --arp-op Reply -j ACCEPT > -j DROP > > Bridge chain: O-vnet0-arp, entries: 5, policy: ACCEPT > -p ARP --arp-op Reply --arp-mac-dst ! 52:54:0:4d:c8:5f -j DROP > -p ARP --arp-ip-dst ! 9.59.241.151 -j DROP > -p ARP --arp-op Request -j ACCEPT > -p ARP --arp-op Reply -j ACCEPT > -j DROP > > > > The libvirt logs show a bunch of shell erorrs - see the attachment to this > mail. Are these errors to be expected, or are they actual errors ?</tt> <tt>They are due to attempted cleanups that try to cleanup user-defined tables that did not exist on your system, </tt> <tt>however. As said, this gets the system into a well known start-out state so that those user-defined ebtables</tt> <tt>tables can be established with rules. As seen above, every interface gets a 'tree' of user-defined tables.</tt> <tt>Following the chain identifier in the above XML</tt> <tt> <filter name='demofilter4' chain='root'> <uuid>d33d17bf-0198-6b89-0b03-55abba1eae7f</uuid> [...]</tt> <tt>which can be either left out entirely for meaning of 'root', or be 'root' explicitly or have the values 'arp' or 'ipv4'.</tt> <tt>The patterns of the user-defined ebtables tables is due to 32 byte size restrictions (including trailing 0) as follows:</tt> <tt><one letter prefix>-<name of the interface>-<filter chain identifier>:</tt> <tt>This then leads to</tt> <tt>I-vnet0 and</tt> <tt>I-vnet0-ipv4 etc.</tt> <tt>as shown above.</tt> <tt>However, the tricky part are updates of filters. You can, using virsh-define, update any filter while it is being used by running</tt> <tt>VMs. For updates I decided not to compare the current filter tree against the new one, calculating 'diffs' </tt> <tt>for the rules and remove rules that don't exist in the new one along with user defined tables that may not exist anymore</tt> <tt>etc., which would be much more complicated. Instead, I am completely rebuilding the filter tree. So, a temporary filter</tt> <tt>tree is established with user-defined names following the above pattern while the current filter tree is still active.</tt> <tt>However, the prefixes are different. So the temporary filter tree would have user-defined tables named</tt> <tt>J-vnet0 and </tt> <tt>J-vnet0-ipv4 etc.</tt> <tt>where all the rules are established. Then, while the 'old' filter tree is active, the new one is anchored, leading to</tt> <tt>entries like this one:</tt> <tt>Bridge chain: PREROUTING, entries: 1, policy: ACCEPT -i vnet0 -j I-vnet0</tt> <tt>-i vnet0 -j J-vnet0</tt> <tt>Yes, temporarily you would have two parallel filter trees for an interface, but the old one is still active. Then</tt> <tt>comes the switch-over to the new tree where the old filter tree is un-anchored, leading to this situation:</tt> <tt>Bridge chain: PREROUTING, entries: 1, policy: ACCEPT -i vnet0 -j J-vnet0</tt> <tt> Now all possible chains starting with prefix I-vnet0-<XYZ> are flushed and deleted.</tt> <tt>In the subsequent step I rename the filter tree starting with J-<vnet0>-<XYZ> to I-<vnet0>-<XYZ> leading to</tt> <tt>Bridge chain: PREROUTING, entries: 1, policy: ACCEPT -i vnet0 -j I-vnet0</tt> <tt>which is where we want to be so that the next update can do the same procedure again.</tt> <tt>I have made some changes to the core patch (part 11) that split up the application of new filter rules into</tt> <tt>3 different (interface) functions, for application of new rules, switch over + tear down of old, or </tt> <tt>tear down of new in case an error happened. </tt> <tt> > > > Also, I'm wondering if it is possible to include 'libvirt-' in the custom > ebtables/iptables chains that we create ? I think that would be useful > for sysadmins who might be surprised at where all these rules are coming > from. From the code it looks like we have a fairly low size limit on the > possible chain names, but I think at least for the first two chains we > ought to be able to get 'libvirt-' prefix in there. Or perhaps instead > of adding the per-VIF chains straight to PREROUTING/POSTROUTING, have > an unconditional jump to a libvirt specific chain. > > So either have > > > Bridge chain: PREROUTING, entries: 1, policy: ACCEPT > -i vnet0 -j libvirt-I-vnet0 > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT > -o vnet0 -j libvirt-O-vnet0 > > > > Or the slightly more complex > > Bridge chain: PREROUTING, entries: 1, policy: ACCEPT > -j libvirt-I > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: POSTROUTING, entries: 1, policy: ACCEPT > -j libvirt-O > > Bridge chain: libvirt-I, entries: 1, policy: ACCEPT > -i vnet0 -j I-vnet0 > > Bridge chain: OUTPUT, entries: 0, policy: ACCEPT > > Bridge chain: libvirt-O, entries: 1, policy: ACCEPT > -o vnet0 -j O-vnet0</tt> <tt>Both is possible with the first one resulting in being more efficient.</tt> <tt>I'll try to make that change.</tt> <tt> Regards,</tt> <tt> Stefan</tt> <tt> > > > Regards, > Daniel > -- > |: Red Hat, Engineering, London -o- </tt><a href=http://people.redhat.com/berrange/:|><tt>http://people.redhat.com/berrange/:|</tt></a><tt> > |: </tt><a href=http://libvirt.org/><tt>http://libvirt.org</tt></a><tt> -o- </tt><a href="http://virt-manager.org/"><tt>http://virt-manager.org</tt></a><tt> -o- </tt><a href=http://deltacloud.org:|/><tt>http://deltacloud.org:|</tt></a><tt> > |: </tt><a href=http://autobuild.org/><tt>http://autobuild.org</tt></a><tt> -o- </tt><a href=http://search.cpan.org/~danberr/:|><tt>http://search.cpan.org/~danberr/:|</tt></a><tt> > |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| > [attachment "filter.log" deleted by Stefan Berger/Watson/IBM] </tt>