[rhelv6-list] ICH10R SATA bandwidth is unbalanced

[Kirby Zhou]

ICH10R SATA bandwidth is not unbalanced

If I start multiple dd process to read different SSD, the prior ones will be
faster than others.

For instance, 4 dd to read sdc sdd sde and sdf:

[root at djt_134_44 ~]# ( killall dd; sleep 1 ; trap "killall dd" EXIT; for d
in /dev/sd{c..f}; do dd if=$d of=/dev/null bs=1M iflag=direct & done; iostat
-kx 1 /dev/sd? /dev/cciss/c0d? ; );
The speed will be:

sdc               0.00     0.00  504.00    0.00 258048.00     0.00  1024.00
1.48    2.93   1.97  99.20
sdd               0.00     0.00  503.00    0.00 257536.00     0.00  1024.00
1.47    2.92   1.97  99.20
sde               0.00     0.00  403.00    0.00 206336.00     0.00  1024.00
1.47    3.64   2.46  99.20
sdf               0.00     0.00  156.00    0.00 79872.00     0.00  1024.00
1.49    9.52   6.39  99.70

for more, 2 disk test:
sdc               0.00     0.00  518.00    0.00 265216.00     0.00  1024.00
1.49    2.87   1.91  99.10
sde               0.00     0.00    0.00    0.00     0.00     0.00     0.00
0.00    0.00   0.00   0.00
sdd               0.00     0.00    0.00    0.00     0.00     0.00     0.00
0.00    0.00   0.00   0.00
sdf               0.00     0.00  536.00    0.00 274432.00     0.00  1024.00
1.50    2.80   1.85  99.40
for more, 3 disk test:
sdc               0.00     0.00  517.00    0.00 264704.00     0.00  1024.00
1.48    2.86   1.92  99.20
sde               0.00     0.00  533.00    0.00 272896.00     0.00  1024.00
1.48    2.77   1.86  98.90
sdd               0.00     0.00    0.00    0.00     0.00     0.00     0.00
0.00    0.00   0.00   0.00
sdf               0.00     0.00  478.00    0.00 244736.00     0.00  1024.00
1.48    3.10   2.07  99.00

It looks very strange. If connect the SSD with a RAID or SAS-HBA adapter, it
seems everything goes right. 
For example, with IBM-M5014:

sdc               0.00     0.00 728.00  0.00 186368.00     0.00   512.00
2.98    4.08   1.36  99.00
sdd               0.00     0.00 729.00  0.00 186688.00     0.00   512.18
2.91    3.99   1.36  98.90
sde               0.00     0.00 728.00  0.00 186368.00     0.00   512.00
3.09    4.24   1.36  99.30
sdf               0.00     0.00 728.00  0.00 186368.00     0.00   512.00
2.97    4.07   1.37  99.50

It is a slower than ICH10R, but is balanced.


Additional Info:
[root at djt_134_44 ~]# ls /dev/disk/by-path/ -l
total 0
lrwxrwxrwx 1 root root  9 Mar 15 15:50 pci-0000:00:1f.2-scsi-0:0:0:0 ->
../../sda
lrwxrwxrwx 1 root root 10 Mar 15 15:51 pci-0000:00:1f.2-scsi-0:0:0:0-part1
-> ../../sda1
lrwxrwxrwx 1 root root 10 Mar 15 15:51 pci-0000:00:1f.2-scsi-0:0:0:0-part2
-> ../../sda2
lrwxrwxrwx 1 root root  9 Mar 15 15:51 pci-0000:00:1f.2-scsi-1:0:0:0 ->
../../sdb
lrwxrwxrwx 1 root root  9 Mar 15 15:51 pci-0000:00:1f.2-scsi-2:0:0:0 ->
../../sdc
lrwxrwxrwx 1 root root  9 Mar 15 15:51 pci-0000:00:1f.2-scsi-3:0:0:0 ->
../../sdd
lrwxrwxrwx 1 root root  9 Mar 15 15:51 pci-0000:00:1f.2-scsi-4:0:0:0 ->
../../sde
lrwxrwxrwx 1 root root  9 Mar 15 15:51 pci-0000:00:1f.2-scsi-5:0:0:0 ->
../../sdf

[root at djt_134_44 ~]# lspci | fgrep 1f.2
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI
Controller

[root at djt_134_44 ~]# modinfo ahci
filename:       /lib/modules/2.6.32-71.el6.x86_64/kernel/drivers/ata/ahci.ko
version:        3.0
license:        GPL
description:    AHCI SATA low-level driver
author:         Jeff Garzik
srcversion:     9ADB15E0E68E68D1073D16F

[root at djt_134_44 ~]# uname -a
Linux djt_134_44 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010
x86_64 x86_64 x86_64 GNU/Linux

[root at djt_134_44 ~]# lsscsi 
[0:0:0:0]    disk    ATA      MM0500EANCR      HPG3  /dev/sda 
[1:0:0:0]    disk    ATA      MM0500EANCR      HPG3  /dev/sdb 
[2:0:0:0]    disk    ATA      INTEL SSDSA2M160 2CV1  /dev/sdc 
[3:0:0:0]    disk    ATA      INTEL SSDSA2M160 2CV1  /dev/sdd 
[4:0:0:0]    disk    ATA      INTEL SSDSA2M160 2CV1  /dev/sde 
[5:0:0:0]    disk    ATA      INTEL SSDSA2M160 2CV1  /dev/sdf 
Regards,
   Kirby Zhou    
   from   SOHU-RD   +86-10-6272-8261