[Fedora-xen] TCP checksum corruption

[Daniel P. Berrange]

On Tue, May 08, 2007 at 11:39:14AM -0500, Mike McGrath wrote:
> We're using xen heavily in Fedora's Infrastructure and presently a 
> number of the xen domU hosts are experiencing terrible checksum issues.  
> I've tried the ethtool -K eth0 tx off fix and it didn't work.

What sort of network config have you got with these ?  Briding straight
to physical device, or NAT'd ?

> dom0:  2.6.19-1.2911.fc6xen x86_64
> domU: 2.6.20-1.2933.fc6xen x86_64
> 
> Anyone have any ideas?

There are a couple issues at play:

 - There is a general bug in 2.6.20  that breaks checksum offload
   when used with NAT.
 - In 2.6.19 or later Dom0 will transmits to guests using checksum
   offload, so DHCP client in the guest will mistakenly thing it
   has a corrupt checksum.

To address the first bug requires disabling checksum offload in the eth0 in
the guest. ethtool -K eth0 tx off    in the guest should do it.

To address the 2nd is really difficult since the FC6 install images themsves
have a broken DHCP client for example, so we need to workaround it in the
kernel. This can be done by disabling checksums on the device in Dom0 - any
of vifN.0,  xenbr0, phet0 should have ethtook -K <dev> tx off done.

NB, ignore eth0 in Dom0, that's a fake device so turning off tx on that does
not fix things.

So in summary, to get it working in general case requires:

   ethtool -K eth0 tx off    in guest

And

   ethtool -K <dev> tx off   on whatever bridge device the guest is attached to

We have just fixed both these issues in the latest  rawhide kernel available
in Koji http://koji.fedoraproject.org/packages/kernel-xen-2.6/2.6.20/2925.7.fc7/
and intend to also apply them to FC5 & 6 asap.

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|