[dm-devel] SMP-aware kcryptd?

[Larry Dickson]

On 4/13/09, Milan Broz <mbroz at redhat.com> wrote:
>
> Tomasz Chmielewski wrote:
> > Currently, the trend with CPUs is to add more cores rather than increase
> > the speed of a single core.
> >
> > This does not scale very well with certain things in the Linux kernel.
> > One of them is kcryptd.
> >
> >
> > A system able to deliver data with a speed of ~200 MB/s from a RAID
> > array, will be only able to deliver a fraction of it (i.e. ~40 MB/s in
> > my case) when reads are being done from a dm-crypt device.
> >
> > This is because kcryptd is not SMP-aware: it performs all calculations
> > on a single logical CPU only.
> >
> > Are there any plans to change it?
>
> Yes, I think I already commented this in some thread,
> there are basically 2 possible approaches:
>
> 1) create several "encryption" threads in dm-crypt (e.g. per CPU core) or
> 2) parallelize async crypto requests processing (in crypto layer)
>
> I have some patches for 1) but the result is not what I expected
> - performance highly depends on write pattern for example.
> In short, the code need more thought.


This is a golden opportunity. So many times, code (e.g. barriers) that still
may "need more thought" has been rushed into production, leaving headaches
that apparently last forever. I think this is a chance to get everyone in at
the design stage. Can you publish a short sketch of the proposed design
approaches for 1) and 2), with their advantages and drawbacks?

Larry Dickson
Cutting Edge Networked Storage

(I tried to parallelize sector encryption _vs_ whole bio request encryption,
> and there are still some problem with both. Also barriers and reordering
> of requests can be problem here.)
>
> Another problem with 1) is what happens when there is hw acceleration -
> multiple threads are useful only if main cpu runs encryption,
> for hw & async crypto requests it can cause problems.
>
> The crypto driver is completely transparent - dm-crypt have no idea
> if encryption runs on main cpu or in an accelerator hw
> (=> solution 2) is more generic).
>
> Anyway, it is not highest priority currently... but still in slow progress.
> (but I guess high speed SSDs will increase priority for this too,
> currently it is real problem only for RAID arrays:-)
>
> Milan
> --
> mbroz at redhat.com
>
> --
> dm-devel mailing list
> dm-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20090413/10357eb9/attachment.htm>