parallel I/O on shared-memory multi-CPU machines

Andreas Dilger adilger at
Mon Apr 2 22:02:53 UTC 2007

On Apr 02, 2007  05:57 -0700, i.vlad at wrote:
> My question is -- can the filesystem actually read/write to files truly
> in parallel (more processors -- faster read/write), or if I write such
> a code the I/O commands from the CPUs will just queue after each other
> and it would be the same thing as using a single CPU? If parallel I/O
> is possible, can it be accomplished entirely transparently, or using
> special libraries, or only in special circumstances, like reading in
> parallel with N CPUs from N different physical disks? Or only on some
> types of hardware? Is there a max nr of threads/processes that can
> write to disk in parallel? If ext3 does not do this, which (stable)
> Linux filesystem does it?

You can do this with Lustre (, which is a GPL parallel,
distributed filesystem that runs on top of ext3 (essentially ext4 though).
Since Lustre does its own distributed extent locking on a file, it is
possible to do fully parallel IO to a single file from multiple nodes.

> I know that the vast majority of clusters use something else than ext3
> (NFS, Lustre, etc), but the question still stands because: (1) individual
> nodes in commodity clusters do have very often individual ext3 disks that
> are used for temporary files (intermediate computational results); (2)
> grid computers made of standalone user machines are likely to have the
> most common filesystem, ext3; (3) There are scientific data processing
> steps that need to be done on a single shared-memory machine because of
> intensive data exchange between CPUs. (4) Software development is easier
> to do on a single machine (i.e. powerful multi-core laptop).

We are looking to make even single-node file locking better than just
i_mutex (i.e. fine-grained extent locking in the VFS) but this is not
a high priority task for us, since we target large systems mostly.
If anyone is interested in doing such work we would definitely want
to help out with it.

Cheers, Andreas
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

More information about the Ext3-users mailing list