[dm-devel] BIO_RW_SYNCIO

[Mikulas Patocka]

On Mon, 26 Jul 2010, Vivek Goyal wrote:

> On Mon, Jul 26, 2010 at 05:53:40PM -0400, Mikulas Patocka wrote:
> > Hi Jens
> > 
> [ Jens's mail id has changed. Ccing him on different mail id ]
> 
> > I found out that when I remove BIO_RW_SYNCIO from dm-kcopyd.c, performance 
> > when writing to the snapshot origin is increased twice. In this workload, 
> > snapshot driver reads a lot of 8k chunks from one place on the disk and 
> > writes them to the other place on the same disk. Without BIO_RW_SYNCIO, it 
> > is twice faster with CFQ scheduler.
> > 
> > What does BIO_RW_SYNCIO exactly do?  Does it attempt to decrease latency 
> > and decrease throughput? (so that disk head seeks more between the source 
> > and destination and it causes performance degradation) Is there some 
> > general guidance, when use BIO_RW_SYNCIO and when not?
> 
> BIO_RW_SYNC marks a request as SYNC. reads are by default sync. Writes
> can be marked as SYNC so that they get priority of ASYNC WRITES.
> 
> Marking writes as SYNC has advantage that it gets its own queue, it can
> preempt ASYNC WRITES and CFQ also idles on the cfq queue waiting for
> more requests.
> 
> I suspect if it is idling on SYNC WRITES which is a problem here.
> 
> - Is it the same thread which is submitting both read and write requests?

Yes.

> - Are you interleaving reads and writes. Like reading some data, then
>   writting it and reading again...

I issue write immediatelly when read finishes.

> I guess after writting some data, CFQ is idling on WRITE_SYNC and next
> READ does not get dispatched immediately to the disk.
> 
> Is it possible to provide some 30 second blktrace of the underlying device
> where CFQ is running. You can also try setting slice_idle to 0 and see if
> it helps.
> 
> If setting slice_idle=0 helps you, can you please also give a try to
> following patch.
> 
> https://patchwork.kernel.org/patch/113061/
> 
> Thanks
> Vivek

I took the traces and placed them at 
http://people.redhat.com/mpatocka/data/blktrace/

It shows that WRITE requests are merged without SYNCIO flags and are not 
merged if SYNCIO is used.

slice_idle=0 or the patch don't help.


BTW. blkparse ignores the input file, when I use "blkparse -i 
sda.blktrace.0.sync", blkparse still displays content of "sda.blktrace.0" 
and not "sda.blktrace.0.sync".

I have to use "cat sda.blktrace.0.sync|blkparse -i -" to read th trace. Is 
it a blkparse bug? I use blkparse from Debian Lenny.

Mikulas