[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: External Journal scenario - good idea?

Jeremy Rumpf wrote:

Yea, that's the biggest issue facing use of these new large capacity drives. Figuring out how to keep performance acceptable while maximizing utilized space. Of course, at least with IDE drives, the cost of large capacity drives is still relatively low (compared to SCSI). I also suspect it will be dropping off in the future even more so as Maxtor is readying a line of 320GB drives.

Goodness...  that's pretty huge.  I remember sort of salivating at the big 180GB SCSI
drives that came out a little while back.  Can't remember who made those, I think it
was WD or Seagate.

calculating parity data for a 4-5 drive array, and even a separate RAID1
array working behind the same RAID controller may suffer write
performance issues because the data has to be processed by the same RAID
controller to actually get written to the RAID1 drives.

I would hope that the controller design is such that it can handle enough operations per second to satisfy its host interface (the SCSI connection exported to the host machine). Some things to consider if I may.

Yep, hopefully. I know from corresponding with Promise on the newer Ultratrak-SX8000 (I think
that's what they're called), they totally re-did the electronics to up the interface up to U160 SCSI
and increased the cache on the controller to I think 32MB. I was wondering if it was just a matter
of replacing the controller itself (and docs for the new unit even show how to replace it), but they
stated that backplane, drive carriers, and more had to be replaced. They offered to upgrade my
Ultratrak100-TX8 unit for $1350, which I declined. For another $600 or so I could buy the whole
unit, and still have the Ultratrak100-TX8 too... ;)

Well I think we're getting ready to get a little better idea of the interface speed of this unit can deal
with. We have offloaded the majority of the files stored on the external RAID unit to other storage,
reducing the total storage of the unit in use to approx 14GB. I am almost finished creating a software
RAID pair of RAID1 drives, and partitioning it out (as /dev/mdp0), to copy the remaining filesystem to
it from the big unit (and also breaking the one large filesystem up into several partitions and mount points).

Once I get this RAID1 pair where I can boot and run from it, I will be taking the large unit offline, and
doing all kinds of experimenting with it. My first major curiousity is just how fast this server can write to
the unit, so I'm going to create a 4-drive RAID0 array. I suspect this will reveal whether the write bottleneck
is all due to being a RAID5 array, or that being only partly to blame. I'll also do some other testing and see
what works well.

My impression right now is to build an 8-drive RAID0+1 (some call it RAID10) array. While this will cut
our storage space nearly in half, the potential fault tolerance is better, and write performance should be better
without the parity overhead also. We'll use this "drive" for our mount point(s) requiring a lot of space.

I'm also tempted to experiment with a combination of software and hardware RAID -- making four RAID1
arrays on the unit, then create a RAID0 array in linux with those. I suspect letting the unit's RAID controller
handle both (RAID0 and RAID1) would be better though.

Across the board, this is less intensive than RAID5 operation. The offsetting factor here would be that the controller has to support one RAID5 set while it has to support multiple RAID1 sets. I would bet to say that the load on the controller itself would stay about the same using 4 RAID1 sets vs 1 huge RAID5 set. The number of write operations dispatched to the drives would be about the same aggregaed over time and you'd be saving the overhead of calculating the parity information.

Yep. I guess the main thing I was antsy about was losing the extra space. But it won't
seem so great to have so much space if I went with a RAID5 array, and ever have two
drives fail on the RAID5 array. At least with RAID0+1 I have a decent chance (6 out
of 7) of survival in that case.

But I am really not even sure that what we're seeing here is a problem
with the speed of the RAID controller.  From some other reading I have
done, it seems that grabbing up RAM to cache writes and combine it all
into one big write is something that the 2.4 kernel series is rather
notorious for.

Yes, the pagecache becomes a vaccumn cleaner during I/O intensive periods. This is being looked into in the development series (cachiness tuning). One thing maybe to try:


Specifically the section on tuning the I/O Elevator.


Also has some interesting notes in it.

Thanks for the URL's - good reading and good tips!

One other thing, if your cache controller is set to run in "write back mode", try disabling that. Write back caches on RAID controllers will aggregate/delay writes as well. Might be worth looking into (I know it tends to kill perfromance on my LSI MegaRaid at times).

Thanks - good idea, that was. The drives are already write-caching, with 8MB cache on each drive.
Ought to be enough... ;)

It ought to get pretty exciting around here in another few hours... ;)

Thanks again for all the advice Jeremy.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]