[rhelv6-list] ICH10R SATA bandwidth is unbalanced
Kirby Zhou
kirbyzhou at sogou-inc.com
Tue Mar 15 08:10:10 UTC 2011
ICH10R SATA bandwidth is unbalanced
If I start multiple dd process to read different SSD, the prior ones will be
faster than others.
For instance, 4 dd to read sdc sdd sde and sdf:
[root at djt_134_44 ~]# ( killall dd; sleep 1 ; trap "killall dd" EXIT; for d
in /dev/sd{c..f}; do dd if=$d of=/dev/null bs=1M iflag=direct & done; iostat
-kx 1 /dev/sd? /dev/cciss/c0d? ; );
The speed will be:
sdc 0.00 0.00 504.00 0.00 258048.00 0.00 1024.00
1.48 2.93 1.97 99.20
sdd 0.00 0.00 503.00 0.00 257536.00 0.00 1024.00
1.47 2.92 1.97 99.20
sde 0.00 0.00 403.00 0.00 206336.00 0.00 1024.00
1.47 3.64 2.46 99.20
sdf 0.00 0.00 156.00 0.00 79872.00 0.00 1024.00
1.49 9.52 6.39 99.70
for more, 2 disk test:
sdc 0.00 0.00 518.00 0.00 265216.00 0.00 1024.00
1.49 2.87 1.91 99.10
sde 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
sdd 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
sdf 0.00 0.00 536.00 0.00 274432.00 0.00 1024.00
1.50 2.80 1.85 99.40
for more, 3 disk test:
sdc 0.00 0.00 517.00 0.00 264704.00 0.00 1024.00
1.48 2.86 1.92 99.20
sde 0.00 0.00 533.00 0.00 272896.00 0.00 1024.00
1.48 2.77 1.86 98.90
sdd 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
sdf 0.00 0.00 478.00 0.00 244736.00 0.00 1024.00
1.48 3.10 2.07 99.00
It looks very strange. If connect the SSD with a RAID or SAS-HBA adapter, it
seems everything goes right.
For example, with IBM-M5014:
sdc 0.00 0.00 728.00 0.00 186368.00 0.00 512.00
2.98 4.08 1.36 99.00
sdd 0.00 0.00 729.00 0.00 186688.00 0.00 512.18
2.91 3.99 1.36 98.90
sde 0.00 0.00 728.00 0.00 186368.00 0.00 512.00
3.09 4.24 1.36 99.30
sdf 0.00 0.00 728.00 0.00 186368.00 0.00 512.00
2.97 4.07 1.37 99.50
It is a slower than ICH10R, but is balanced.
Additional Info:
[root at djt_134_44 ~]# ls /dev/disk/by-path/ -l
total 0
lrwxrwxrwx 1 root root 9 Mar 15 15:50 pci-0000:00:1f.2-scsi-0:0:0:0 ->
../../sda
lrwxrwxrwx 1 root root 10 Mar 15 15:51 pci-0000:00:1f.2-scsi-0:0:0:0-part1
-> ../../sda1
lrwxrwxrwx 1 root root 10 Mar 15 15:51 pci-0000:00:1f.2-scsi-0:0:0:0-part2
-> ../../sda2
lrwxrwxrwx 1 root root 9 Mar 15 15:51 pci-0000:00:1f.2-scsi-1:0:0:0 ->
../../sdb
lrwxrwxrwx 1 root root 9 Mar 15 15:51 pci-0000:00:1f.2-scsi-2:0:0:0 ->
../../sdc
lrwxrwxrwx 1 root root 9 Mar 15 15:51 pci-0000:00:1f.2-scsi-3:0:0:0 ->
../../sdd
lrwxrwxrwx 1 root root 9 Mar 15 15:51 pci-0000:00:1f.2-scsi-4:0:0:0 ->
../../sde
lrwxrwxrwx 1 root root 9 Mar 15 15:51 pci-0000:00:1f.2-scsi-5:0:0:0 ->
../../sdf
[root at djt_134_44 ~]# lspci | fgrep 1f.2
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI
Controller
[root at djt_134_44 ~]# modinfo ahci
filename: /lib/modules/2.6.32-71.el6.x86_64/kernel/drivers/ata/ahci.ko
version: 3.0
license: GPL
description: AHCI SATA low-level driver
author: Jeff Garzik
srcversion: 9ADB15E0E68E68D1073D16F
[root at djt_134_44 ~]# uname -a
Linux djt_134_44 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010
x86_64 x86_64 x86_64 GNU/Linux
[root at djt_134_44 ~]# lsscsi
[0:0:0:0] disk ATA MM0500EANCR HPG3 /dev/sda
[1:0:0:0] disk ATA MM0500EANCR HPG3 /dev/sdb
[2:0:0:0] disk ATA INTEL SSDSA2M160 2CV1 /dev/sdc
[3:0:0:0] disk ATA INTEL SSDSA2M160 2CV1 /dev/sdd
[4:0:0:0] disk ATA INTEL SSDSA2M160 2CV1 /dev/sde
[5:0:0:0] disk ATA INTEL SSDSA2M160 2CV1 /dev/sdf
Regards,
Kirby Zhou
from SOHU-RD +86-10-6272-8261
More information about the rhelv6-list
mailing list