[Fedora-xen] slow disk performance in Xen0 FC5

[Thorsten von Eicken]

> On Wed, 2006-05-31 at 23:07 -0700, Ashe Canvar wrote:
> 
>> For instance my avg data read rate ( tested using hdparm -t /dev/hda)
>> is 55MB/s using the regular smp kernel but it falls to 2.6MB/s when I
>> boot into dom0. No other domains are runnign at this point. I have
>> looked for incrementing interrupts in ide0 , there are none.
> 
> There were some important fixes to Xen's irq delivery mechanism that
> have been added to FC5 since the 1.2080 kernel you are running.  Could
> you please try updating to a newer kernel and seeing if you can
> reproduce the problem?

Same problem here. Machine is dog slow whenever disk accesses are 
required. Running yum takes ages. This is a dual-cpu 2.4Ghz xeon w/ATA 
drive.

Running a stock FC5 kernel I get ~30MB/sec read or write bandwidth. With 
xen kernel I get 3-7MB/s in dom0 and it varies a lot. Typically I see 
the following in top (this was taken while running yum):

[root at medusa1 ~]# top

top - 16:19:22 up 11 min,  2 users,  load average: 0.69, 1.63, 1.43
Tasks:  72 total,   1 running,  71 sleeping,   0 stopped,   0 zombie
Cpu(s):  7.4% us,  1.5% sy,  0.0% ni, 16.9% id,  0.0% wa, 74.2% hi,  0.0% si
Mem:    262308k total,   142696k used,   119612k free,     9072k buffers
Swap:  1052248k total,        0k used,  1052248k free,    49168k cached

   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND 

  2748 root      15   0 52860  42m 5268 S 49.9 16.5   0:48.33 yum 

  2778 root      17   0  2140 1004  808 R 10.2  0.4   0:01.75 top 

   368 root      16   0     0    0    0 S  0.6  0.0   0:04.00 kjournald

Note the high %hi!
At almost the same time:

[root at medusa1 ~]# iostat -x 10
Linux 2.6.16-1.2122_FC5xen0 (medusa1)   06/11/2006

avg-cpu:  %user   %nice %system %iowait   %idle
            1.02    0.00   64.62    0.00   34.36

Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s    wkB/s 
avgrq-sz avgqu-sz   await  svctm  %util
hda          0.00   3.68  0.00  0.72    0.00   35.17     0.00    17.59 
   49.14     0.00    0.00   0.00   0.00

Almost no disk activity, but high system time. I'm pretty sure it was 
not doing network I/O.

[root at medusa1 ~]# dmesg | head
Linux version 2.6.16-1.2122_FC5xen0 
(bhcompile at hs20-bc1-3.build.redhat.com) (gcc version 4.1.0 20060304 (Red 
Hat 4.1.0-3)) #1 SMP Sun May 21 15:34:43 EDT 2006
[root at medusa1 ~]# dmesg | egrep ATA
hda: ST380011A, ATA DISK drive
[root at medusa1 ~]# dmesg | egrep IDE
PCI: Ignoring BAR0-3 of IDE controller 0000:00:1f.1
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ICH3: IDE controller at PCI slot 0000:00:1f.1

Well, the bottom line is that the machine is not usable like this. Grrr.
Thorsten