<div dir="ltr"><div><div>Okay, I'm trying to reproduce the problem using cache_restore, so far with no luck. The hardware setup is the same as the original.<br><br></div>kernel 3.10.11-gentoo<br><br><br></div><div>Cache raid, 1 is meta, 2 data;<br> <br># parted /dev/sdc unit b print<br>Model: DELL PERC H700 (scsi)<br>Disk /dev/sdc: 2397799710720B<br>Sector size (logical/physical): 512B/512B<br>Partition Table: gpt<br>Disk Flags:<br><br>Number Start End Size File system Name Flags<br> 1 1048576B 1024458751B 1023410176B primary<br> 2 1024458752B 2397798662143B 2396774203392B primary<br><br></div><div>Origin:<br><br># parted /dev/sdb unit b print<br>Model: DELL PERC H700 (scsi)<br> Disk /dev/sdb: 20973392756736B<br>Sector size (logical/physical): 512B/512B<br>Partition Table: gpt<br>Disk Flags:<br><br>Number Start End Size File system Name Flags<br> 1 1048576B 20973391708159B 20973390659584B xfs primary<br> <br></div><div>According to /dev/sda2 size, number of blocks should be <br><br></div><div>571435, with block size 4MiB( 8192 sectors)<br><br>Generating fake meta, I leave 5 blocks free<br><span style="font:13px Arial"><br> cache_xml create --nr-cache-blocks 571430 --nr-mappings 571430 0 > /root/meta.xml<br><br></span></div><div><span style="font:13px Arial">I edit the xml to set block size 8192 since there is no option for this.<br></span></div> <div> <br><span style="font:13px Arial">cache_restore -i /root/meta.xml -o /dev/sdc1</span><br> <br><span style="font:13px Arial">dmsetup create storage --table '0 40963653632 cache /dev/sdc1 /dev/sdc2 /dev/sdb1 8192 1 writeback default 0’ </span><br> <br><br></div><div>mkfs.xfs /dev/mapper/storage<br><br></div><div>mount //dev/mapper/storage /mnt<br><br></div><div>Then I run fio with following setup:<br><br>[test1]<br>loops=10000<br>randrepeat=1<br>directory=/mnt/fio/data/<br> new_group<br>group_reporting=1<br>size=100g<br>rwmixread=70<br>rw=randrw<br>numjobs=12<br>ioengine=sync<br>#direct=1<br>bs=1M<br>nrfiles=3000<br></div><div><br></div><div>This is the status so far:<br><br># dmsetup status storage<br> 0 40963653632 cache 1719/249856 93161125 11990840 93713813 27533087 307339 307344 571435 306978 0 2 migration_threshold 10000000 4 random_threshold 4 sequential_threshold 10000000<br><br></div><div>There are ~300k demotions with no crash so far. <br> </div><div>So probably filling the cache/demotions are not the only factor here.<br><br>regards.<br><br><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 14, 2013 at 10:22 PM, Vladimir Smolensky <span dir="ltr"><<a href="mailto:arizal@gmail.com" target="_blank">arizal@gmail.com</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Okay, a little correction here. The problematic setup was with 2.3TB cache, not 3TB. The mistake comes from the fact that at the time of the writing I was dealing with another server which has 8x480GB ssd's in raid5, which is little over 3TB. Obviously the number was floating in my head.<br></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 14, 2013 at 5:21 PM, Vladimir Smolensky <span dir="ltr"><<a href="mailto:arizal@gmail.com" target="_blank">arizal@gmail.com</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">./cache_restore -V<br>0.2.8<br></div><div><div><div class="gmail_extra"><br><br> <div class="gmail_quote">On Thu, Nov 14, 2013 at 4:38 PM, Joe Thornber <span dir="ltr"><<a href="mailto:thornber@redhat.com" target="_blank">thornber@redhat.com</a>></span> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On Wed, Nov 13, 2013 at 06:39:37PM +0000, Mears, Morgan wrote:<br> > On Wed, Nov 13, 2013 at 11:29:26AM -0005, Vladimir Smolensky wrote:<br> > > Hello,<br> > > I'm compiling the for-linus branch, it shows 3.12.0-rc5, that's ok right?<br> > ><br> > > Where can I get cache_restore, cache_dump, etc. and info how to use them.<br> ><br> > Hi Vladimir,<br> ><br> > git clone <a href="https://github.com/jthornber/thin-provisioning-tools.git" target="_blank">https://github.com/jthornber/thin-provisioning-tools.git</a><br> > cd thin-provisioning-tools<br> > autoreconf<br> > ./configure --enable-testing<br> > make<br> > sudo cp cache_check cache_dump cache_restore cache_repair /usr/sbin<br> > cache_check --version<br> ><br> > This is mostly distilled from README.md in the repo; there are some<br> > additional dependencies in there that you might have to address.<br> ><br> > All the tools have a --help option; not sure about other documentation.<br> ><br> > --Morgan<br> <br> </div>'make install' should work, and I believe there are some man pages.<br> <span><font color="#888888"><br> - Joe<br> </font></span><div><div><br> --<br> dm-devel mailing list<br> <a href="mailto:dm-devel@redhat.com" target="_blank">dm-devel@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/dm-devel" target="_blank">https://www.redhat.com/mailman/listinfo/dm-devel</a><br> </div></div></blockquote></div><br></div> </div></div></blockquote></div><br></div> </div></div></blockquote></div><br></div>