[Cluster-devel] How can be metadata(e.g., inode) in the GFS2 file system shared between client nodes?

[Daegyu Han]

Thank you for your explanation.

ᐧ

2019년 8월 9일 (금) 오후 9:26, Andrew Price <anprice at redhat.com>님이 작성:

> On 09/08/2019 12:46, Daegyu Han wrote:
> > Thank you for the clarification.
> >
> > I have one more question.
> >
> > I've seen some web page by redhat and it says that gfs2 has a poor
> > filesystem performance (i.e. throughput) compared to xfs or ext4.
> > [image: image.png]
> >
> > In a high performance hardware environment (nvme over fabric, infiniband
> > (56G)), I ran a FIO benchmark, expecting GFS2 to be comparable to local
> > filesystems (ext4, xfs).
> >
> > Unexpectedly, however, GFS2 showed 25% lower IOPS or throughput than
> ext4,
> > as the web page results.
> >
> > Does GFS2 perform worse than EXT4 or XFS even on high-performance
> network +
> > storage?
>
> gfs2 has performance overheads that ext4 and xfs don't encounter due to
> the extra work it has to do to keep the fs consistent across the
> cluster, such as the extra cache invalidation we've discussed, journal
> flushing and updates to structures relating to quotas and statfs. Even
> in a single-node configuration, extra codepaths are still active (but
> gfs2 isn't meant to be used as a single-node fs, so fio is not a good
> demonstration of its strengths). It's also worth noting that gfs2 is not
> extent-based so you may see performance differences relating to that. We
> are continually working to minimise the overheads, of course.
>
> The size of the performance difference is highly dependent on the
> workload and access pattern. (Clustered) applications looking to get the
> best performance out of gfs2 will have each node processing its own
> working set - preferably in its own subdirectory - which will minimise
> the overheads.
>
> Cheers,
> Andy
>
> > Thank you,
> > Daegyu
> > ᐧ
> >
> > 2019년 8월 9일 (금) 오후 8:26, Andrew Price <anprice at redhat.com>님이 작성:
> >
> >> On 09/08/2019 12:01, Daegyu Han wrote:
> >>> Thank you for your reply.
> >>>
> >>> If what I understand is correct,
> >>> In a gfs2 file system shared by clients A and B, if A creates
> /foo/a.txt,
> >>> does B re-read the filesystem metadata area on storage to keep the data
> >>> consistent?
> >>
> >> Yes, that's correct, although 'clients' is inaccurate as there is no
> >> 'server'. Through the locking mechanism, B would know to re-read block
> >> allocation states and the contents of the /foo directory, so a path
> >> lookup on B would then find a.txt.
> >>
> >>> After all, what makes gfs2 different from local filesystems like ext4,
> >>> because of lock_dlm?
> >>
> >> Exactly.
> >>
> >>> In general, if we mount an ext4 file system on two different clients
> and
> >>> update the file system on each client, we know that the file system
> state
> >>> is not reflected in each other.
> >>
> >> Yes.
> >>
> >> Cheers,
> >> Andy
> >>
> >>> Thank you,
> >>> Daegyu
> >>> ᐧ
> >>>
> >>> 2019년 8월 9일 (금) 오후 7:50, Andrew Price <anprice at redhat.com>님이 작성:
> >>>
> >>>> Hi Daegyu,
> >>>>
> >>>> On 09/08/2019 09:10, 한대규 wrote:
> >>>>> Hi, I'm Daegyu from Sungkyunkwan University.
> >>>>>
> >>>>> I'm curious how GFS2's filesystem metadata is shared between nodes.
> >>>>
> >>>> The key thing to know about gfs2 is that it is a shared storage
> >>>> filesystem where each node mounts the same storage device. It is
> >>>> different from a distributed filesystem where each node has storage
> >>>> devices that only it accesses.
> >>>>
> >>>>> In detail, I wonder how the metadata in the memory of the node
> mounting
> >>>> GFS2
> >>>>> looks the consistent filesystem to other nodes.
> >>>>
> >>>> gfs2 uses dlm for locking of filesystem metadata among the nodes. The
> >>>> transfer of locks between nodes allows gfs2 to decide when its
> in-memory
> >>>> caches are invalid and require re-reading from the storage.
> >>>>
> >>>>> In addition, what role does corosync play in gfs2?
> >>>>
> >>>> gfs2 doesn't communicate with corosync directly but it operates on top
> >>>> of a high-availability cluster. corosync provides synchronization and
> >>>> coherency for the cluster. If a node stops responding, corosync will
> >>>> notice and trigger actions (fencing) to make sure that node is put
> back
> >>>> into a safe and consistent state. This is important in gfs2 to prevent
> >>>> "misbehaving" nodes from corrupting the filesystem.
> >>>>
> >>>> Hope this helps.
> >>>>
> >>>> Cheers,
> >>>> Andy
> >>>>
> >>>>
> >>>>
> >>>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20190809/4e83a043/attachment.htm>