[dm-devel] Shell Scripts or Arbitrary Priority Callouts?

[Pasi Kärkkäinen]

On Tue, Mar 24, 2009 at 11:41:00PM -0400, John A. Sullivan III wrote:
> > > Latency seems to be our key.  If I can add only 20 micro-seconds of
> > > latency from initiator and target each, that would be roughly 200 micro
> > > seconds.  That would almost triple the throughput from what we are
> > > currently seeing.
> > > 
> > 
> > Indeed :) 
> > 
> > > Unfortunately, I'm a bit ignorant of tweaking networks on opensolaris.
> > > I can certainly learn but am I headed in the right direction or is this
> > > direction of investigation misguided? Thanks - John
> > > 
> > 
> > Low latency is the key for good (iSCSI) SAN performance, as it directly
> > gives you more (possible) IOPS. 
> > 
> > Other option is to configure software/settings so that there are multiple
> > outstanding IO's on the fly.. then you're not limited with the latency (so much).
> > 
> > -- Pasi
> <snip>
> Ross has been of enormous help offline.  Indeed, disabling jumbo packets
> produced an almost 50% increase in single threaded throughput.  We are
> pretty well set although still a bit disappointed in the latency we are
> seeing in opensolaris and have escalated to the vendor about addressing
> it.
> 

Ok. That's pretty big increase. Did you figure out why that happens? 

> The once piece which is still a mystery is why using four targets on
> four separate interfaces striped with dmadm RAID0 does not produce an
> aggregate of slightly less than four times the IOPS of a single target
> on a single interface. This would not seem to be the out of order SCSI
> command problem of multipath.  One of life's great mysteries yet to be
> revealed.  Thanks again, all - John

Hmm.. maybe the out-of-order problem happens at the target? It gets IO
requests to nearby offsets from 4 different sessions and there's some kind
of locking or so going on? 

Just guessing. 

-- Pasi