[Linux-cluster] architecture discussion -- NFS clustering with iSCSI

[Terry]

On Jan 27, 2008 8:28 PM, Rainer Duffner <rainer at ultra-secure.de> wrote:
>
> Am 28.01.2008 um 02:36 schrieb Terry:
>
> >
> >
> > Good questions:
> > 1) Do you have concurrent writes to the same file from different
> > nodes?
> >   1a) No
>
>
> Well, that's one of the things GFS is good at ;-)
>
>
> > 2) How many nodes do you have?
> >   2a) 3 to start, probably won't go beyond 12
> >
>
>
> OK, that's still in the range GFS can handle (AFAIK).
>
>
> > I appreciate alternative ideas to NFS.  NFS could possibly introduce
> > performance issues (comments here appreciated).
>
>
> One problem might be that NFS was never supposed to run on  GBit-
> networks.
> Thus there is overhead.
> But, OTOH, the vendors I mentioned have managed to squeeze a lot of
> performance out of NFS.
> It's also a question of optimizing/matching NFS clients and servers.
>
>
> >   The majority of the
> > system is write.  I would say 80%.
> >
>
> Do you have a lot of small files?
> Small files are usually what degrades GFS-performance.
>
> As I mentioned, if your requirements are tending to be more grid-
> computing related, best ask somebody with a grid-computing background.
> Though, they may think in dimensions where 12 nodes is what they have
> at home in a VMware-team.
> ;-)
>
> Personally, I would be very careful to consider GFS for a system with
> a lot of writes (concurrent to a file or not).
> The reason is that it is most times impossible to predict the
> behavior of GFS with a certain application-load-pattern at a given
> cluster-size - there are just too many variables.
> This is, of course, also true for NFS - but there's simply much more
> NFS out there "in the field" and you can *always* find someone who
> does nearly (or exactly) the same thing as you want to do and then
> make better educated guesses about how the system will perform.
>
> So, if you want to go with GFS: build your cluster, see if it
> performs, if it doesn't perform: work with your integrator and RedHat
> and see what they can do.
> If that doesn't help: scrap it and install an NFS server on some box
> (use Solaris+ZFS - though there are other issues with that, too, of
> course).
>
>
>
> cheers,
> Rainer
> --
> Rainer Duffner
> CISSP, LPI, MCSE
> rainer at ultra-secure.de


Good tips.  Yes, a ton of small files and many directories. 56,000,000
files roughly today.  I wonder how an active-active NFS would fare in
this type of environment.