[dm-devel] dm-integrity

Wed Jul 12 18:30:06 UTC 2017

Hi.*Please, give link to your blog.*

I succesfully create configuration for virtual machines (and it works 
fine), test force reboot many time (and machine have some problem with 
ACPI table, thats why SATA controller reset links for all disks (now it 
fixed), but only one disk is out from md-raid and system worked. when 
controller problem was it heavy test I/O). E.g. I think, that with 
journal and crc32 dm-integrity is stable. In one of tests I write 1Mb 
with dd (with skip), then read (cat > /dev/null) LVM-partition on top of 
md. dm-intergity succesfully detect corrution, generate i/o error and md 
fencing disk, but still works.

I use this configuration: 7 SATA disks => each with own dm-integrity => 
md raid6 6 disks + 1 spare disk -> LVM for virtual machines.

If any disk gives up buggy info, dm-integrity will generate i/o error 
and md fence disk from raid, but system continious working. You can say, 
that I must use ZFS for this case, but I don't have SSD-cache, and 
without SSD-cache ZFS is too slow, SLOWSLOWSLOW in future, when virtual 
machines disks will fragment, e.g. CoW. My solution gives integrity 
opportunity without fragmentation penalty.

Please, use this info in your blog, becase in production we want not 
only know, that info from disk is buggy, but read non-buggy info and 
continue working. If you will paste to your blog, please. change serial 
numbers of my disks. :)

*And one more think: after system boot and assemble script executed, it 
eates ~4Gb RAM (only dm-intergity). What parameter doing it?*

I use this sript on boot to assemle raid:

cd /root/local/integrity/src

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup 
--integrity=crc32 --buffer-sectors=414096 --journal-watermark=70 
--journal-commit-time=60000 open 
/dev/disk/by-id/ata-WDC_WD5000AADS-00S9B0_WD-WCAV90886975-part1 
integra-WCAV90886975

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup 
--integrity=crc32 --buffer-sectors=414096 --journal-watermark=70 
--journal-commit-time=60000 open 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F0SJX4T6-part1 
integra-WCC3F0SJX4T6

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup 
--integrity=crc32 --buffer-sectors=414096 --journal-watermark=70 
--journal-commit-time=60000 open 
/dev/disk/by-id/ata-WDC_WD5001AALS-00J7B0_WD-WMATV7467797-part1 
integra-WMATV7467797

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup 
--integrity=crc32 --buffer-sectors=414096 --journal-watermark=70 
--journal-commit-time=60000 open 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F0YY89KN-part1 
integra-WCC3F0YY89KN

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup 
--integrity=crc32 --buffer-sectors=414096 --journal-watermark=70 
--journal-commit-time=60000 open 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F5HF3NN0-part1 
integra-WCC3F5HF3NN0

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup 
--integrity=crc32 --buffer-sectors=414096 --journal-watermark=70 
--journal-commit-time=60000 open 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00K3CA0_WD-WCC6Y4XEA1RS-part1 
integra-WCC6Y4XEA1RS

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup 
--integrity=crc32 --buffer-sectors=414096 --journal-watermark=70 
--journal-commit-time=60000 open 
/dev/mapper/vg--super--blue-lv--super--spare integra-spare

cd /

mdadm --assemble --scan

sleep 6

echo 50000 > /proc/sys/dev/raid/speed_limit_min

echo 500000 > /proc/sys/dev/raid/speed_limit_max

echo 32768 > /sys/block/md0/md/stripe_cache_size

And this to create:

LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' ./integritysetup -v 
--sector-size=512 --integrity=crc32 --tag-size 32 
--buffer-sectors=414096 --journal-size=836870912 format 
/dev/disk/by-id/ata-WDC_WD5000AADS-00S9B0_WD-WCAV90886975-part1 ##-> 
../../sda1 LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' 
./integritysetup -v --sector-size=512 --integrity=crc32 --tag-size 32 
--buffer-sectors=414096 --journal-size=836870912 format 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F0SJX4T6-part1 ## -> 
../../sdb1 LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' 
./integritysetup -v --sector-size=512 --integrity=crc32 --tag-size 32 
--buffer-sectors=414096 --journal-size=836870912 format 
/dev/disk/by-id/ata-WDC_WD5001AALS-00J7B0_WD-WMATV7467797-part1 ##-> 
../../sdc1 LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' 
./integritysetup -v --sector-size=512 --integrity=crc32 --tag-size 32 
--buffer-sectors=414096 --journal-size=836870912 format 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F0YY89KN-part1 ##-> 
../../sdd1 LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' 
./integritysetup -v --sector-size=512 --integrity=crc32 --tag-size 32 
--buffer-sectors=414096 --journal-size=836870912 format 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00MK2A0_WD-WCC3F5HF3NN0-part1 ## -> 
../../sde1 LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' 
./integritysetup -v --sector-size=512 --integrity=crc32 --tag-size 32 
--buffer-sectors=414096 --journal-size=836870912 format 
/dev/disk/by-id/ata-WDC_WD5003AZEX-00K3CA0_WD-WCC6Y4XEA1RS-part1 ## -> 
../../sdf1 LD_PRELOAD='../lib/.libs/libcryptsetup.so.12' 
./integritysetup -v --sector-size=512 --integrity=crc32 --tag-size 32 
--buffer-sectors=414096 --journal-size=836870912 format 
/dev/mapper/vg--super--blue-lv--super--spare

mdadm --verbose --create /dev/md0  --bitmap=internal 
--bitmap-chunk=524288 -c 128  --level=6  --raid-devices=6 
/dev/mapper/integra-WCAV90886975 /dev/mapper/integra-WCC3F0SJX4T6 
/dev/mapper/integra-WMATV7467797 /dev/mapper/integra-WCC3F0YY89KN 
/dev/mapper/integra-WCC3F5HF3NN0 /dev/mapper/integra-WCC6Y4XEA1RS 
--spare-devices=1 /dev/mapper/integra-spare

11.07.2017 12:52, Milan Broz пишет:
> On 07/10/2017 05:53 PM, Renesanso wrote:
>> And I have other question: is it ready for production in crc32+journal
>> mode? Not as like IBM-like production, I mean, but it is not full> buggy, but stable now? :)
> Hi,
>
> well, there are some tests, we found and fixed some issues in 4.12-rcX
> (so be sure you are using code from released 4.12.0 or later!).
>
> So I hope it is stable, but it would definitely need more users to test.
> If you find any problem, crash or anything related to kernel code, please send
> a report to the dm-devel at redhat.com list.
>
> Thanks,
> Milan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20170712/f8a1c5fc/attachment.htm>