[libvirt PATCH 00/28] native support for nftables in virtual network driver

[Laine Stump]

On 5/4/23 6:47 AM, Daniel P. Berrangé wrote:
> On Sun, Apr 30, 2023 at 11:19:15PM -0400, Laine Stump wrote:
>> This patch series enables libvirt to use nftables rules rather than
>> iptables *when setting up virtual networks* (it does *not* add
>> nftables support to the nwfilter driver). It accomplishes this by
>> abstracting several iptables functions (from viriptables.[ch] called
>> by the virtual network driver into a rudimentary "virNetfilter API"
>> (in virnetfilter.[ch], having the virtual network driver call the
>> virNetFilter API rather than calling the existing iptables functions
>> directly, and then finally adding an equivalent virNftables backend
>> that can be used instead of iptables (selected manually via a
>> network.conf setting, or automatically if iptables isn't found on the
>> host).
>>
>> A first look at the result may have you thinking that it's filled with
>> a lot of bad decisions. While I would agree with that in many cases, I
>> think that overall they are the "least bad" decisions, or at least
>> "bad within acceptable limits / no worse than something else", and
>> point out that it's been done in a way that minimizes (actually
>> eliminates) the need for immediate changes to nwfilter (the other
>> consumer of iptables, which *also* needs to be updated to use native
>> nftables), and makes it much easier to change our mind about the
>> details in the future.
>>
>> When I first started on this (long, protracted, repeatedly interrupted
>> for extended periods - many of these patches are > a year old) task, I
>> considered doing an all-at-once complete replacement of iptables with
>> nftables, since all the Linux distros we support have had nftables for
>> several years, and I'm pretty sure nobody has it disabled (not even
>> sure if it's possible to disable nftables while still enabling
>> iptables, since they both use xtables in the kernel). But due to
>> libvirt's use of "-t mangle -j CHECKSUM --checksum-fill" (see commit
>> fd5b15ff all the way back in July 2010 for details) which has no
>> equivalent in nftables rules (and we don't *want* it to!!), and the
>> desire to be able to easily switch back to iptables in case of an
>> unforeseen regression, we decided that both iptables and nftables need
>> to be supported (for now), with the default (for now) remaining as
>> iptables.
>>
>> Just allowing for dual backends complicated matters, since it means
>> that we have to have a config file, a setting, detection of which
>> backends are available, and of course some sort of concept of an
>> abstracted frontend that can use either backend based on the config
>> setting (and/or auto-detection). Combining that with the fact that it
>> would just be "too big" of a project to switch over nwfilter's
>> iptables usage at the same time means that we have to keep around a
>> lot of existing code for compatibility's sake rather than just wiping
>> it all away and starting over.
>>
>> So, what I've ended up with is:
>>
>> 1) a network.conf file (didn't exist before) with a single setting
>> "firewall_backend". If unset, the network driver tries to use iptables
>> on the backend, and if that's missing, then tries to use nftables.
> 
> When testing your git branch active-nft-10 leavnig it unset didn't
> work:
> 
> Running './src/libvirtd'...
> 2023-05-04 10:16:11.447+0000: 115377: info : libvirt version: 9.3.0
> 2023-05-04 10:16:11.447+0000: 115377: info : hostname: localhost.localdomain
> 2023-05-04 10:16:11.447+0000: 115377: error : virFirewallNew:118 : internal error: firewall_backend wasn't set, and no usable setting could be auto-detected
> 2023-05-04 10:16:11.447+0000: 115377: error : virNetFilterBackendUnsetError:51 : internal error: firewall_backend wasn't set, and no usable setting could be auto-detected
> 2023-05-04 10:16:11.447+0000: 115377: error : virNetFilterBackendUnsetError:51 : internal error: firewall_backend wasn't set, and no usable setting could be auto-detected
> 2023-05-04 10:16:11.473+0000: 115377: error : virFirewallNew:118 : internal error: firewall_backend wasn't set, and no usable setting could be auto-detected

Ugh :-(

I guess I did brak something with all the pre-posting cleanups after my 
last round of testing, as that was working just fine as of the end of 
last week.. I'll see what I broke, and if it's simple I'll update the 
branch on gitlab so that you can try it out if you want (even though 
it's obvious I need to change some things)