Hi, I think I found part of the problem. Our RAID array vendor explained to me that they have a problem of not getting enough data from ext3 to do full stripe writes when ext3 issues an cache flush command. Basically something to do with the journaling and cache flushing. To test this I replaced ext3 with ext2 and now I get the expected results: mkfs.ext2 -Tlargefile4 -b4096 /dev/dm-1 dd if=/dev/zero of=/mnt/test bs=1024K count=1024 conv=fsync 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 6.85545 seconds, 157 MB/s So now I think I will just try to just have ext3 journal to a different device. See if that helps any. Thanks Rene <div>On 8/8/07, GARDAIS Ionel <<a href="mailto:Ionel.Gardais@tech-advantage.com">Ionel.Gardais@tech-advantage.com</a>> wrote: <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <div> stride is the chunk size of your raid whatever the number of physical disks composing the RAID array. So for 256k chunks with a block size of 4k, stride should be 256/4 = 64 instead of 640. Maybe it will help. Ionel -------- Message d'origine-------- De: Rene Salmon [<a href="mailto:rsalmon74@gmail.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">mailto:rsalmon74@gmail.com</a>] Date: mer. 08/08/2007 22:53 À: GARDAIS Ionel Cc: <a href="mailto:ext3-users@redhat.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ext3-users@redhat.com</a> Objet : Re: RE : Poor ext3 performance on RAID array Hi, Thanks for the reply. I tried using the -E stride=X option as follows: mkfs.ext3 -b4096 -Tlargefile4 -E stride=640 /dev/dm-1 and got the same results around 50 MBytes/sec. Maybe I have the wrong number for stride so here is the math for that: stride=stripe-size Configure the filesystem for a RAID array with stripe-size filesystem blocks per stripe. Here is some more detail on the RAID array. RAID level : 5 (10 drives + 1 parity) Chunk Size : 256 KB Stripe Size : 2560 KB (10 drives * 256KB) stride=640 * 4096(byte blocks) = 2560KB I will try other stride options but they don't seem to change much. Thanks Rene On 8/8/07, GARDAIS Ionel <<a href="mailto:Ionel.Gardais@tech-advantage.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">Ionel.Gardais@tech-advantage.com</a>> wrote: > > Hi Rene, > > You should try to add the "-E stride=X" option to the mkfs command line. > Where X is expalined in the man page. > > This will basically map ext3 "blocks" on the RAID stripe size. > > Ionel > > > -------- Message d'origine-------- > De: <a href="mailto:ext3-users-bounces@redhat.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ext3-users-bounces@redhat.com</a> de la part de Rene Salmon > Date: mer. 08/08/2007 22:21 > À: <a href="mailto:ext3-users@redhat.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ext3-users@redhat.com</a> > Objet : Poor ext3 performance on RAID array > > Hi list, > > > I am having some strange performance issues with ext3 and I am hoping to > get some advice/hints on how to make this better. First some background > on the setup. > > We have a RAID 5 array 10+1+1 with one LUN. That is 10 SATA drives one > parity drive and one dedicated spare. The LUN is about 6.5TB. > > Using a 2Gbit/sec fiber channel card I can do some dd writes to the raw > device and get speeds close to 200Mbytes/sec which is more or less the > max the card can do. > > Next I create an xfs file system on the LUN and do a dd to xfs and get > speeds close to 150Mbytes/sec. > > I want to use ext3 not xfs so next I put ext3 on the lun. Now when I do > the dd to the ext3 lun I get 25-50Mbytes/sec depending on whether I have > the Raid controller cache turned on or off. Getting 50MBytes/sec with > the raid controller cache turned off. > > I know that ext3 should perform better so I must be doing something > wrong. Here is my mkfs.ext3 > > mkfs.ext3 -b4096 -Tlagefile4 /dev/dm-0 > > Thanks in advanced for any help on this. > > Rene > > </div> </blockquote></div>