From linuxluck at 163.com Tue Mar 4 02:16:01 2014 From: linuxluck at 163.com (fsluck) Date: Tue, 4 Mar 2014 10:16:01 +0800 (CST) Subject: how to know ext cache hit rate? In-Reply-To: <20140226155040.GA11452@thunk.org> References: <1e217c12.2688.1432761fffb.Coremail.linuxluck@163.com> <20131225180043.GA10885@thunk.org> <1ab392b9.1b16e.1446d3602fb.Coremail.linuxluck@163.com> <20140226155040.GA11452@thunk.org> Message-ID: <677d4440.1a528.1448ae055d1.Coremail.linuxluck@163.com> virtualization is to make best use of resources,memory and disk over-providing. so i guess cache hit rate is very low in such case,because cache is very small and vm's disk is very large. im virtualization environment , my original idea is to cache part of backing file into memory(for example 1G as cache), all vms that use that backing file can access cache directly and performance improved. do you have a idea? At 2014-02-26 23:50:40,"Theodore Ts'o" wrote: >On Wed, Feb 26, 2014 at 04:01:00PM +0800, fsluck wrote: >> i think it is page cache. my case is in virualization environment, >> because vm's disk image is much larger than system's cache, i want >> to know when many vms are running if system cache hit rate is very >> low? if there is a need to seek for other way to suit >> virtualization. > >Why not simply measure the I/O rate on the disk? If there are cache >misses, they will show up as I/O requests, and it's the cache misses >that you care about. Or more to the point, what's important is >determining whether I/O is the bottleneck or whether something else >like memory pressure. > >Also note that with Linux, any available memory will be used for >caching. If the cache is very small, it's likely the real problem is >that your system doesn't have enough memory --- which is very often >the problem with virtualization environments, and also why Amazon >charges $$$$$ for VM's that have a large amount of memory. > > - Ted -------------- next part -------------- An HTML attachment was scrubbed... URL: From m4rtntns at gmail.com Thu Mar 6 20:46:01 2014 From: m4rtntns at gmail.com (Martin T) Date: Thu, 6 Mar 2014 20:46:01 +0000 Subject: questions regarding file-system optimization for sortware-RAID array Message-ID: Hi, I created a RAID1 array of two physical HDD's with chunk size of 64KiB under Debian "wheezy" using mdadm. As a next step, I would like to create an ext3(or ext4) file-system to this RAID1 array using mke2fs utility. According to RAID-related tutorials, I should create the file-system like this: # mkfs.ext3 -v -L myarray -m 0.5 -b 4096 -E stride=16,stripe-width=32 /dev/md0 Questions: 1) According to manual of mke2fs, value of the "stride" has to be the RAID chunk size in clusters. As I use chunk size of 64KiB, then I have to use "stride" value of 16(16*4096=65536). Why is it important for file-system to know the size of chunk used in RAID array? I know it improves the I/O performance, but why is this so? 2) If the "stride" size in my case is 16, then the "stripe_width=" is 32 because there are two drives in the array which contain the actual data. Manual page of the mke2fs explain this option as "This allows the block allocator to prevent read-modify-write of the parity in a RAID stripe if possible when the data is written.". How to understand this? What is this "read-modify-write" behavior? Could somebody explain this with an example? regards, Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From adilger at dilger.ca Fri Mar 7 17:57:49 2014 From: adilger at dilger.ca (Andreas Dilger) Date: Fri, 7 Mar 2014 10:57:49 -0700 Subject: questions regarding file-system optimization for sortware-RAID array In-Reply-To: References: Message-ID: <369D2359-D7EC-4C74-8EB1-2326A89BE2E8@dilger.ca> Note that stride and stripe width only make sense for RAI-5/6 arrays. For RAID-1 it doesn't really matter. Cheers, Andreas > On Mar 6, 2014, at 13:46, Martin T wrote: > > Hi, > > I created a RAID1 array of two physical HDD's with chunk size of 64KiB under Debian "wheezy" using mdadm. As a next step, I would like to create an ext3(or ext4) file-system to this RAID1 array using mke2fs utility. According to RAID-related tutorials, I should create the file-system like this: > > # mkfs.ext3 -v -L myarray -m 0.5 -b 4096 -E stride=16,stripe-width=32 /dev/md0 > > > Questions: > > 1) According to manual of mke2fs, value of the "stride" has to be the RAID chunk size in clusters. As I use chunk size of 64KiB, then I have to use "stride" value of 16(16*4096=65536). Why is it important for file-system to know the size of chunk used in RAID array? I know it improves the I/O performance, but why is this so? > > 2) If the "stride" size in my case is 16, then the "stripe_width=" is 32 because there are two drives in the array which contain the actual data. Manual page of the mke2fs explain this option as "This allows the block allocator to prevent read-modify-write of the parity in a RAID stripe if possible when the data is written.". How to understand this? What is this "read-modify-write" behavior? Could somebody explain this with an example? > > > regards, > Martin > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users From m4rtntns at gmail.com Sat Mar 8 02:23:42 2014 From: m4rtntns at gmail.com (Martin T) Date: Sat, 8 Mar 2014 02:23:42 +0000 Subject: questions regarding file-system optimization for sortware-RAID array In-Reply-To: <369D2359-D7EC-4C74-8EB1-2326A89BE2E8@dilger.ca> References: <369D2359-D7EC-4C74-8EB1-2326A89BE2E8@dilger.ca> Message-ID: Andreas, why is it relevant only in case of RAID5 or RAID6? regards, Martin On Fri, Mar 7, 2014 at 5:57 PM, Andreas Dilger wrote: > Note that stride and stripe width only make sense for RAI-5/6 arrays. > For RAID-1 it doesn't really matter. > > Cheers, Andreas > >> On Mar 6, 2014, at 13:46, Martin T wrote: >> >> Hi, >> >> I created a RAID1 array of two physical HDD's with chunk size of 64KiB under Debian "wheezy" using mdadm. As a next step, I would like to create an ext3(or ext4) file-system to this RAID1 array using mke2fs utility. According to RAID-related tutorials, I should create the file-system like this: >> >> # mkfs.ext3 -v -L myarray -m 0.5 -b 4096 -E stride=16,stripe-width=32 /dev/md0 >> >> >> Questions: >> >> 1) According to manual of mke2fs, value of the "stride" has to be the RAID chunk size in clusters. As I use chunk size of 64KiB, then I have to use "stride" value of 16(16*4096=65536). Why is it important for file-system to know the size of chunk used in RAID array? I know it improves the I/O performance, but why is this so? >> >> 2) If the "stride" size in my case is 16, then the "stripe_width=" is 32 because there are two drives in the array which contain the actual data. Manual page of the mke2fs explain this option as "This allows the block allocator to prevent read-modify-write of the parity in a RAID stripe if possible when the data is written.". How to understand this? What is this "read-modify-write" behavior? Could somebody explain this with an example? >> >> >> regards, >> Martin >> _______________________________________________ >> Ext3-users mailing list >> Ext3-users at redhat.com >> https://www.redhat.com/mailman/listinfo/ext3-users From adilger at dilger.ca Sat Mar 8 17:20:20 2014 From: adilger at dilger.ca (Andreas Dilger) Date: Sat, 8 Mar 2014 10:20:20 -0700 Subject: questions regarding file-system optimization for sortware-RAID array In-Reply-To: References: <369D2359-D7EC-4C74-8EB1-2326A89BE2E8@dilger.ca> Message-ID: The stripe and stride options do two things: - shift block and inode bitmaps in each group to be on different disks - align the block allocation to the stripe and stride boundaries to avoid read-modify-write in RAID The first one is irrelevant if the flex_bg option is used, since it already packs the bitmaps together and achieves the same effect. The second is meaningless for RAID-1 since writes go to every disk and there is no parity or read-modify-write for small or unaligned writes. Cheers, Andreas > On Mar 7, 2014, at 19:23, Martin T wrote: > > Andreas, > > why is it relevant only in case of RAID5 or RAID6? > > > regards, > Martin > >> On Fri, Mar 7, 2014 at 5:57 PM, Andreas Dilger wrote: >> Note that stride and stripe width only make sense for RAID-5/6 arrays. >> For RAID-1 it doesn't really matter. >> >> Cheers, Andreas >> >>> On Mar 6, 2014, at 13:46, Martin T wrote: >>> >>> Hi, >>> >>> I created a RAID1 array of two physical HDD's with chunk size of 64KiB under Debian "wheezy" using mdadm. As a next step, I would like to create an ext3(or ext4) file-system to this RAID1 array using mke2fs utility. According to RAID-related tutorials, I should create the file-system like this: >>> >>> # mkfs.ext3 -v -L myarray -m 0.5 -b 4096 -E stride=16,stripe-width=32 /dev/md0 >>> >>> >>> Questions: >>> >>> 1) According to manual of mke2fs, value of the "stride" has to be the RAID chunk size in clusters. As I use chunk size of 64KiB, then I have to use "stride" value of 16(16*4096=65536). Why is it important for file-system to know the size of chunk used in RAID array? I know it improves the I/O performance, but why is this so? >>> >>> 2) If the "stride" size in my case is 16, then the "stripe_width=" is 32 because there are two drives in the array which contain the actual data. Manual page of the mke2fs explain this option as "This allows the block allocator to prevent read-modify-write of the parity in a RAID stripe if possible when the data is written.". How to understand this? What is this "read-modify-write" behavior? Could somebody explain this with an example? >>> >>> >>> regards, >>> Martin >>> _______________________________________________ >>> Ext3-users mailing list >>> Ext3-users at redhat.com >>> https://www.redhat.com/mailman/listinfo/ext3-users From m4rtntns at gmail.com Wed Mar 12 14:30:31 2014 From: m4rtntns at gmail.com (Martin T) Date: Wed, 12 Mar 2014 16:30:31 +0200 Subject: questions regarding file-system optimization for sortware-RAID array In-Reply-To: References: <369D2359-D7EC-4C74-8EB1-2326A89BE2E8@dilger.ca> Message-ID: Stephen, thank you for the reply! I made a drawing based on your answer: http://s21.postimg.org/kcn4q92rr/fs_and_chunk_size.png As you can see, there are two scenario described- first one where I modify f1, f2, f3 and f4 files on a file-system which does not take the RAID chunk-size into account and second scenario where ext3/ext4 file-system is aware of the RAID chunk-size. Please confirm, that I understand you correctly. Andreas, what does this "read-modify-write" behavior mean? regards, Martin On 3/8/14, Andreas Dilger wrote: > The stripe and stride options do two things: > - shift block and inode bitmaps in each group to be on different disks > - align the block allocation to the stripe and stride boundaries to > avoid read-modify-write in RAID > > The first one is irrelevant if the flex_bg option is used, since it already > packs > the bitmaps together and achieves the same effect. > > The second is meaningless for RAID-1 since writes go to every disk and > there is no parity or read-modify-write for small or unaligned writes. > > Cheers, Andreas > >> On Mar 7, 2014, at 19:23, Martin T wrote: >> >> Andreas, >> >> why is it relevant only in case of RAID5 or RAID6? >> >> >> regards, >> Martin >> >>> On Fri, Mar 7, 2014 at 5:57 PM, Andreas Dilger >>> wrote: >>> Note that stride and stripe width only make sense for RAID-5/6 arrays. >>> For RAID-1 it doesn't really matter. >>> >>> Cheers, Andreas >>> >>>> On Mar 6, 2014, at 13:46, Martin T wrote: >>>> >>>> Hi, >>>> >>>> I created a RAID1 array of two physical HDD's with chunk size of 64KiB >>>> under Debian "wheezy" using mdadm. As a next step, I would like to >>>> create an ext3(or ext4) file-system to this RAID1 array using mke2fs >>>> utility. According to RAID-related tutorials, I should create the >>>> file-system like this: >>>> >>>> # mkfs.ext3 -v -L myarray -m 0.5 -b 4096 -E stride=16,stripe-width=32 >>>> /dev/md0 >>>> >>>> >>>> Questions: >>>> >>>> 1) According to manual of mke2fs, value of the "stride" has to be the >>>> RAID chunk size in clusters. As I use chunk size of 64KiB, then I have >>>> to use "stride" value of 16(16*4096=65536). Why is it important for >>>> file-system to know the size of chunk used in RAID array? I know it >>>> improves the I/O performance, but why is this so? >>>> >>>> 2) If the "stride" size in my case is 16, then the "stripe_width=" is 32 >>>> because there are two drives in the array which contain the actual data. >>>> Manual page of the mke2fs explain this option as "This allows the block >>>> allocator to prevent read-modify-write of the parity in a RAID stripe if >>>> possible when the data is written.". How to understand this? What is >>>> this "read-modify-write" behavior? Could somebody explain this with an >>>> example? >>>> >>>> >>>> regards, >>>> Martin >>>> _______________________________________________ >>>> Ext3-users mailing list >>>> Ext3-users at redhat.com >>>> https://www.redhat.com/mailman/listinfo/ext3-users >