[Linux-cluster] OT? Odd Xen network issue

[Madison Kelly]

Edson Marquezani Filho wrote:
> On Mon, Oct 26, 2009 at 16:33, Madison Kelly <linux at alteeve.com> wrote:
>> Hi all,
>>
>>  I hope this isn't too off-topic being more a Xen than cluster experience,
>> but I am having trouble with the Xen mailing list and I gather many here use
>> Xen. If it is too off-topic, the please have a mod delete it and kick my
>> butt. :)
>>
>> I've got a simple two-node cluster, each with three NICs running CentOS 5.3
>> x86_64. Each node has this configuration:
>>
>> eth0 - Internal network and VM back channel (IPMI on this NIC)
>> eth1 - DRBD link
>> eth2 - Internet-facing NIC, dom0 has no IP
> 
> I have a very similar setup here.

So far, it seems like a pretty decent low-cost, high availability setup. :)

>>  I've built a simple VM that uses 'xenbr0' and 'xenbr2' as the VM's 'eth0'
>> and 'eth1', respectively. I've set a static, public IP to domU's eth1
>> (xenbr2) but I can't ping it from a workstation nor can the domU ping out. I
>> know that the network itself (outside the server) is setup right as I tested
>> the static IP from that cable earlier on a test machine.
> 
> This sounds to me like being related only to routing stuff, rather than Xen.
> I don't know if what I'm going to say is already implicit in what you
> said, or very obvious to you, but to reach that public IP address in
> question, the workstation from where you are trying to do this must to
> have a route that leads to this IP, usually a gateway address.
> Try thinking about the path the packet is following across the
> network, and if there is routing rules enough to reach its destiny.
> 
> Even from dom0, you are subjected to the same rules. For example: to
> ping some public IP address assigned to a domU's eth1, you should have
> either a IP of the same network assigned to your physical eth2/peth2
> (or a virtual ethX interface atached on xenbr2 bridge), or routes that
> leads your packet from your dom0 to that domU's IP.

Before I began work on Xen I had assigned the public IPs to eth2 and it 
had worked. For some reason, Xen + 64bit + D-Link DGE-560T = no luck. I 
took a cue from this email to check traffic on xenbr2 on dom0 and I 
could see traffic while debugging. As I mentioned to Rick, I could see 
the ICMP requests up to vif0.2, but not on peth2.

>>  Could someone hit me with a clue-stick as to what I might be doing wrong?
>> Let me know if any logs or config files are helpful.
> 
> I'm not sure if I understood your scenario correctly, and I don't know
> if I could be clear enough with this bad english, but at least, I
> tried. =)

Pfft, your English is awesome. :)

>> Thanks!
> 
> I hope this can help a bit. Good luck. =)

It did. Once I saw it hit vif0.2 I knew that bridging was working and 
that it was likely a hardware/driver bug.

Thanks!

Madi