<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> Take a look at Madison Kellys' various blog postings at alteeve.com. She has a generally very good set of how to's on how to build a 2 node cluster. <div class="moz-cite-prefix">On 05/30/2018 10:32 AM, Cobin Bluth wrote: </div> <blockquote type="cite" cite="mid:CAGvEgFGBuOtdF6dWa-qA-PmVWVRkUE-665GqKY=474cgvWNnew@mail.gmail.com"> <div dir="ltr">Hello Libvirt Users, I would like to setup a two node bare-metal cluster. I need to guidance on the network configuration. I have attached a small diagram, the same diagram can be seen here: <a href="https://i.imgur.com/SOk6a6G.png" moz-do-not-send="true">https://i.imgur.com/SOk6a6G.png</a> I would like to configure the following details: - Each node has a DHCP enabled guest network where VMs will run. (eg, <a href="http://192.168.1.0/24" moz-do-not-send="true">192.168.1.0/24</a> for Host1, and <a href="http://192.168.2.0/24" moz-do-not-send="true">192.168.2.0/24</a> for Host2) - Any guest in Host1 should be able to ping guests in Host2, and vice versa. - All guests have routes to reach the open internet (so that 'yum update' will work "out-of-the-box") - Each node will be able to operate fully if the other physical node fails. (no central DHCP server, etc) - I would like to add more physical nodes later when I need the resources. This is what I have done so far: - Installed latest Ubuntu 18.04, with latest version of libvirt and supporting software from ubuntu's apt repo. - Each node can reach the other via its own eth0. - Each node has a working vxlan0, which can ping the other via its vxlan0, so it looks like the vxlan config is working. (I used ip link add vxlan0 type vxlan...) - Configured route on Host1 like so: ip route add <a href="http://192.168.2.0/24" moz-do-not-send="true">192.168.2.0/24</a> via 172.20.0.1 - Configured route on Host2 also: ip route add <a href="http://192.168.1.0/24" moz-do-not-send="true">192.168.1.0/24</a> via 172.20.0.2 <div>- All guests on Host1 (and Host1) can ping eth0 and vxlan0 on Host2, and vice versa, yay. - Guests on Host1 cannot ping guests on Host2, I suspect because the the default NAT config of the libvirt network. So, at this point I started to search for tutorials or more information/documentation, but I am a little overwhelmed by the sheer amount of information, as well as a lot of "stale" information on blogs etc. I have learned that I can virsh net-edit default, and then change it to an "open" network: <forward mode='open'/></div> <div>After doing this, the guests cannot reach outside their own network, nor reach the internet, so I assume that I would need to add some routes, or something else to get the network functioning like I want it. There is also <forward mode="route"/>, but I dont fully understand the scenarios where one would need an open or a route forward mode. I have also shied away from using openvswitch, and have opted for ifupdown2. (I have taken most of my inspiration from this blog post: <a href="https://joejulian.name/post/how-to-configure-linux-vxlans-with-multiple-unicast-endpoints/" moz-do-not-send="true">https://joejulian.name/post/how-to-configure-linux-vxlans-with-multiple-unicast-endpoints/</a> ) Some questions that I have for the mailing list, any help would be greatly appreciated: - Is my target configuration of a KVM cluster uncommon? Do you see drawbacks of this setup, or does it go against "typical convention"? - Would my scenario be better suited for an "open" network or a "route" network? - What would be the approach to complete this setup? </div> </div> <fieldset class="mimeAttachmentHeader"></fieldset> <pre wrap="">_______________________________________________ libvirt-users mailing list <a class="moz-txt-link-abbreviated" href="mailto:libvirt-users@redhat.com">libvirt-users@redhat.com</a> <a class="moz-txt-link-freetext" href="https://www.redhat.com/mailman/listinfo/libvirt-users">https://www.redhat.com/mailman/listinfo/libvirt-users</a></pre> </blockquote> <pre class="moz-signature" cols="72">-- Alvin Starr || land: (905)513-7688 Netvel Inc. || Cell: (416)806-0133 <a class="moz-txt-link-abbreviated" href="mailto:alvin@netvel.net">alvin@netvel.net</a> || </pre> </body> </html>