Isolated bridge does not bridge

[Paul van der Vlis]

Op 09-09-2020 om 19:34 schreef Laine Stump:
> On 9/9/20 7:13 AM, Paul van der Vlis wrote:
>> Hello,
>>
>> I want to do some testing and I have removed two VM's from the bridge
>> what connects them to internet, and added them to another isolated
>> bridge what's not connected to internet. Problem is that I cannot reach
>> the other host in the isolated network.
>>
>> Something like this:
>>
>> virsh shutdown kvm66
>> virsh shutdown kvm68
>>
>> brctl delif br0 vnet10 vnet6  # the interfaces of kvm66 and kvm68
>> brctl addbr br1
>> brctl addif br1 vnet10 vnet6
> 
> The delif and addif commands won't do anything if the guests are not
> running (you've done "virsh shutdown", but that will either take some
> time, or never be honored (depending on how the guest OS deals with
> ACPI, I think)

When I do "brctl show br1" then I see everything nice connected:

root at kvms12:~# brctl show br1
bridge name	bridge id		STP enabled	interfaces
br1		8000.fe54000a90f3	no		vnet10
							vnet6
root at kvms12:~#

>> Then I've replaced br0 to br1 in the XML of both VM's with "virsh edit".
> 
> Just be certain that each guest is either completely inactive (doesn't
> show up in the output of "virsh list" when you edit, or at some point
> after you've edited it (i.e. there must be a complete "virtual
> powercycle" of the guest for the changes to take effect).

What I do then is stop the guest. And start it. Then the new
configuration is used.

>> Then I did start the VM's using the serial console (no network):
>> virsh start --console kvm66
>> virsh start --console kvm68
>>
>> I cannot ping from one machine to the other. Why??
> 
> I guess you're using <interface type='bridge'> ... right?

Yes.

> Since the bridge devices were created and are managed outside libvirt's
> control, you need to do more than just create a bridge to get the
> connected guests talking to each other. In particular, if the guests are
> getting their IP addresses from DHCP, then you need to assign an IP
> address to the bridge device, and run a DHCP server that is listening on
> the bridge. (I'm curious what you used as the argument of the ping
> command, if the guests didn't have an IP address...)

I am using a fixed network configuration. This are servers.

> (Aside from that, a bridge created with brctl will disappear when the
> host is rebooted, and not be recreated until you again enter the commands.)

This machine is not often rebooted, and I know how to make a bridge
permanent.

> If you want a simple way to create a bridge, start a dnmasq instance to
> serve DHCP, 

I don't want DHCP.

> and add iptables rules to prevent the guests from breaking
> out of the isolated bridge, *and* as a bonus *re*create all of that
> every time you reboot the host, you can create an isolated libvirt
> virtual network, with a config file like the one here:
> 
> 
> https://libvirt.org/formatnetwork.html#examplesPrivate

Interesting that "omission of the forward element".

Not sure, is a "virbr" the same as what's normal a "br" ?

> (editing to your taste for bridge name and IPv4 and IPv6 addresses). Put
> that in a file (e.g. net.xml) and run (as root) "virsh net-define
> net.xml; virsh net-start private; virsh net-autostart private".
> 
> Then define your guest interfaces with this:
> 
>    <interface type='network'>
>      <source network='private'/>
>      ...
>    </interface>

Thanks for your information, I will look at it.

But I still wonder why my setup with good-old brctl what I use for years
now in production does not work in an isolated network.

With regards,
Paul



-- 
Paul van der Vlis Linux systeembeheer Groningen
https://www.vandervlis.nl/