From pasik at iki.fi Mon Dec 1 08:59:47 2008 From: pasik at iki.fi (Pasi =?iso-8859-1?Q?K=E4rkk=E4inen?=) Date: Mon, 1 Dec 2008 10:59:47 +0200 Subject: [Linux-cluster] GFS2 on CentOS v5.2 In-Reply-To: <2ca799770811300912y2813bb6cv339ae169588a5b7d@mail.gmail.com> References: <20081130163407.GA15616@jasmine.xos.nl> <2ca799770811300912y2813bb6cv339ae169588a5b7d@mail.gmail.com> Message-ID: <20081201085947.GF15052@edu.joroinen.fi> On Sun, Nov 30, 2008 at 07:12:07PM +0200, Mikko Partio wrote: > On Sun, Nov 30, 2008 at 6:34 PM, Jos Vos wrote: > > > On Sun, Nov 30, 2008 at 01:01:09PM +0100, Shaun Mccullagh wrote: > > > > > I'm setting a GFS cluster on CentOS v5.2 with latest rpms. > > > > > > I think gfs2 is still at the Technology Preview level. > > > > > > Does this mean I should use gfs in production at the moment? > > > > Yes, this is true for RHEL 5.2, but GFS2 is will be changed to > > production level in RHEL 5.3 (now in beta). > > > There has to be some drastic changes in GFS2 between 5.2 and 5.3 since I > still haven't seen a single post in this list stating that GFS2 in 5.2 (or > previous RHEL versions) is working for them. :) > Then you haven't been paying enough attention ;) http://www.mail-archive.com/linux-cluster at redhat.com/msg04706.html -- Pasi From jos at xos.nl Mon Dec 1 09:03:57 2008 From: jos at xos.nl (Jos Vos) Date: Mon, 1 Dec 2008 10:03:57 +0100 Subject: [Linux-cluster] GFS2 on CentOS v5.2 In-Reply-To: <20081201085947.GF15052@edu.joroinen.fi> References: <20081130163407.GA15616@jasmine.xos.nl> <2ca799770811300912y2813bb6cv339ae169588a5b7d@mail.gmail.com> <20081201085947.GF15052@edu.joroinen.fi> Message-ID: <20081201090357.GA23809@jasmine.xos.nl> On Mon, Dec 01, 2008 at 10:59:47AM +0200, Pasi K?rkk?inen wrote: > On Sun, Nov 30, 2008 at 07:12:07PM +0200, Mikko Partio wrote: > > > > There has to be some drastic changes in GFS2 between 5.2 and 5.3 since I > > still haven't seen a single post in this list stating that GFS2 in 5.2 (or > > previous RHEL versions) is working for them. :) > > > > Then you haven't been paying enough attention ;) > > http://www.mail-archive.com/linux-cluster at redhat.com/msg04706.html I don't see how this answer negates Mikko's statement... -- -- Jos Vos -- X/OS Experts in Open Systems BV | Phone: +31 20 6938364 -- Amsterdam, The Netherlands | Fax: +31 20 6948204 From geoff at galitz.org Mon Dec 1 09:29:59 2008 From: geoff at galitz.org (Geoff Galitz) Date: Mon, 1 Dec 2008 10:29:59 +0100 Subject: [Linux-cluster] GFS on Centos Message-ID: We are investigating deploying GFS across a small pool of servers: Centos 5.1 x86_64 GigE Networking The data will consist of approximately 400GB of small JPG files accessed by an inhouse java app. The entire cluster is 50 machines but only 7 will require access to this data repository. GFS2 is not ready, yet... but my main question is, is it worth it to wait for GFS2? We are also looking at glusterfs. Our goal is: - low administrative (sysadm) overhead - good performance when accessing lots of small files (<100Mb) Geoff Galitz Blankenheim NRW, Deutschland http://www.galitz.org From swhiteho at redhat.com Mon Dec 1 09:29:57 2008 From: swhiteho at redhat.com (Steven Whitehouse) Date: Mon, 01 Dec 2008 09:29:57 +0000 Subject: [Linux-cluster] GFS on Centos In-Reply-To: References: Message-ID: <1228123797.9571.240.camel@quoit> Hi, On Mon, 2008-12-01 at 10:29 +0100, Geoff Galitz wrote: > > > We are investigating deploying GFS across a small pool of servers: > > Centos 5.1 x86_64 > GigE Networking > > The data will consist of approximately 400GB of small JPG files accessed by > an inhouse java app. The entire cluster is 50 machines but only 7 will > require access to this data repository. > > GFS2 is not ready, yet... but my main question is, is it worth it to wait > for GFS2? We are also looking at glusterfs. > > Our goal is: > > - low administrative (sysadm) overhead > - good performance when accessing lots of small files (<100Mb) Depending on just how small the files are, then possilbly it might be worth your while. A few tens of Mb is big enough to be considered "large" I think, but if you are thinking of running something like an email server with maildir spools, then it would certainly be worth waiting for GFS2, Steve. > > > > > > > > Geoff Galitz > Blankenheim NRW, Deutschland > http://www.galitz.org > > > > -- > Linux-cluster mailing list > Linux-cluster at redhat.com > https://www.redhat.com/mailman/listinfo/linux-cluster From swhiteho at redhat.com Mon Dec 1 09:32:00 2008 From: swhiteho at redhat.com (Steven Whitehouse) Date: Mon, 01 Dec 2008 09:32:00 +0000 Subject: [Linux-cluster] GFS2 on CentOS v5.2 In-Reply-To: <20081201090357.GA23809@jasmine.xos.nl> References: <20081130163407.GA15616@jasmine.xos.nl> <2ca799770811300912y2813bb6cv339ae169588a5b7d@mail.gmail.com> <20081201085947.GF15052@edu.joroinen.fi> <20081201090357.GA23809@jasmine.xos.nl> Message-ID: <1228123920.9571.243.camel@quoit> Hi, On Mon, 2008-12-01 at 10:03 +0100, Jos Vos wrote: > On Mon, Dec 01, 2008 at 10:59:47AM +0200, Pasi K?rkk?inen wrote: > > > On Sun, Nov 30, 2008 at 07:12:07PM +0200, Mikko Partio wrote: > > > > > > There has to be some drastic changes in GFS2 between 5.2 and 5.3 since I > > > still haven't seen a single post in this list stating that GFS2 in 5.2 (or > > > previous RHEL versions) is working for them. :) > > > > > > > Then you haven't been paying enough attention ;) > > > > http://www.mail-archive.com/linux-cluster at redhat.com/msg04706.html > > I don't see how this answer negates Mikko's statement... > The reason that you've not seen reports from people running 5.2 and 5.1 (whether RHEL or CentOS) is that we've been discouraging people from using those versions. The 5.3 beta version of RHEL is the best one for eval purposes at the moment, also Fedora would be a good distro to test if you want something thats most uptodate, Steve. From Shaun.Mccullagh at espritxb.nl Mon Dec 1 09:59:48 2008 From: Shaun.Mccullagh at espritxb.nl (Shaun Mccullagh) Date: Mon, 1 Dec 2008 10:59:48 +0100 Subject: [Linux-cluster] GFS2 on CentOS v5.2 In-Reply-To: <1228123920.9571.243.camel@quoit> References: <20081130163407.GA15616@jasmine.xos.nl><2ca799770811300912y2813bb6cv339ae169588a5b7d@mail.gmail.com><20081201085947.GF15052@edu.joroinen.fi><20081201090357.GA23809@jasmine.xos.nl> <1228123920.9571.243.camel@quoit> Message-ID: Many thanks for the clear info. I've installed kmod-gfs and cman-2.0.84. I notice that kmod gfs2 is loaded when I start service cman. I see that gfs2.ko is part of kernel-2.6.18-92.1.18.el5, which is the kernel in use on the system. Is this expected behaviour? When I exec service gfs start should I expect gks.ko to be loaded? Thx Again.... Shaun -----Original Message----- From: linux-cluster-bounces at redhat.com [mailto:linux-cluster-bounces at redhat.com] On Behalf Of Steven Whitehouse Sent: maandag 1 december 2008 10:32 To: linux clustering Subject: Re: [Linux-cluster] GFS2 on CentOS v5.2 Hi, On Mon, 2008-12-01 at 10:03 +0100, Jos Vos wrote: > On Mon, Dec 01, 2008 at 10:59:47AM +0200, Pasi K?rkk?inen wrote: > > > On Sun, Nov 30, 2008 at 07:12:07PM +0200, Mikko Partio wrote: > > > > > > There has to be some drastic changes in GFS2 between 5.2 and 5.3 > > > since I still haven't seen a single post in this list stating that > > > GFS2 in 5.2 (or previous RHEL versions) is working for them. :) > > > > > > > Then you haven't been paying enough attention ;) > > > > http://www.mail-archive.com/linux-cluster at redhat.com/msg04706.html > > I don't see how this answer negates Mikko's statement... > The reason that you've not seen reports from people running 5.2 and 5.1 (whether RHEL or CentOS) is that we've been discouraging people from using those versions. The 5.3 beta version of RHEL is the best one for eval purposes at the moment, also Fedora would be a good distro to test if you want something thats most uptodate, Steve. -- Linux-cluster mailing list Linux-cluster at redhat.com https://www.redhat.com/mailman/listinfo/linux-cluster Op dit e-mailbericht is een disclaimer van toepassing, welke te vinden is op http://www.espritxb.nl/disclaimer From swhiteho at redhat.com Mon Dec 1 10:07:23 2008 From: swhiteho at redhat.com (Steven Whitehouse) Date: Mon, 01 Dec 2008 10:07:23 +0000 Subject: [Linux-cluster] GFS2 on CentOS v5.2 In-Reply-To: References: <20081130163407.GA15616@jasmine.xos.nl> <2ca799770811300912y2813bb6cv339ae169588a5b7d@mail.gmail.com> <20081201085947.GF15052@edu.joroinen.fi> <20081201090357.GA23809@jasmine.xos.nl> <1228123920.9571.243.camel@quoit> Message-ID: <1228126043.9571.252.camel@quoit> Hi, On Mon, 2008-12-01 at 10:59 +0100, Shaun Mccullagh wrote: > Many thanks for the clear info. > > I've installed kmod-gfs and cman-2.0.84. I notice that kmod gfs2 is loaded when I start service cman. > I see that gfs2.ko is part of kernel-2.6.18-92.1.18.el5, which is the kernel in use on the system. > > Is this expected behaviour? > > When I exec service gfs start should I expect gks.ko to be loaded? > > Thx Again.... > > Shaun > That all sounds about right. GFS2 was made into a kmod and then more recently its become part of the standard kernel again. So if you have an uptodate kernel then it should be just like any other fs. GFS is a kmod and will remain so. The init scripts do load some of the modules early, and also in older kernels there was a dependency between GFS and GFS2 since they both shared the same lock modules. In recent kernels the lock modules are no longer shared between GFS and GFS2, in fact in Fedora lock_nolock no longer exists for GFS2 since its been merged into GFS2 itself. There is a plan to merge lock_dlm into GFS2 as well which will solve a problem thats been flagged up with selinux and init scripts (mount shouldn't need to load modules directly, but GFS2's mount does have to do this while lock_dlm is a separate module). Watch the cluster-devel list if you want to track progress on merging of the lock_dlm module. I should also point out, just to remove any possible confusion, that by lock_dlm, I mean the module which GFS2 uses to interface to the DLM, and not the DLM itself which will obviously remain separate, Steve. From nico at altiva.fr Mon Dec 1 13:33:41 2008 From: nico at altiva.fr (NM) Date: Mon, 1 Dec 2008 13:33:41 +0000 (UTC) Subject: [Linux-cluster] #462910 -- how's postgres-8.sh even supposed to work? Message-ID: See https://bugzilla.redhat.com/show_bug.cgi?id=462910 Just wondering how that script is even supposed to work in the first place, and why it uses sudo instead of su. Any clues? I'm having a hard time with this trying to get it to work properly. From jeff.sturm at eprize.com Mon Dec 1 19:19:23 2008 From: jeff.sturm at eprize.com (Jeff Sturm) Date: Mon, 1 Dec 2008 14:19:23 -0500 Subject: [Linux-cluster] GFS on Centos In-Reply-To: References: Message-ID: <64D0546C5EBBD147B75DE133D798665F0180708A@hugo.eprize.local> 1) What is the ratio of file reads to writes/creates for your Java application? If this is very high (say 100:1 or more) GFS may work just fine. In our experience we have the most trouble with write contention, esp. on shared directories. 2) How much time elapses (statistically speaking) between consective reads of the same file on the same node? If this is low enough you may be able to tune demote_secs such that glocks can be reused for file accesses. If you have too many files to cache the inodes or glocks in memory, you may be better off tuning demote_secs and glock_purge to keep the numbers small, and accept the overhead that each file access is going to have to obtain a lock. 3) What does your directory layout look like? How many files are you placing in the same directory? You'll probably want to avoid very large directories. If e.g. all files are kept in a single directory, you'll get write contention that would effectively limit file creates to a single node at a time. For directories with a high percentage of file creates, we've had better luck establishing one directory per node, such that each node can read files created by others, but only write to their own directory. (And session affinity to reduce the frequency of cross-node reads.) Good luck. The above advice is based on empirical evidence from our own performance testing and other net wisdom, and the positive results we obtained from strategies we employed both within our application and via gfs tuning. (The experts can tell you if I got any of this right or wrong, since I lack an in-depth understanding of GFS/DLM internals. GFS2 may behave very differently; we haven't had a chance to try it yet.) Jeff -----Original Message----- From: linux-cluster-bounces at redhat.com [mailto:linux-cluster-bounces at redhat.com] On Behalf Of Geoff Galitz Sent: Monday, December 01, 2008 4:30 AM To: linux-cluster at redhat.com Subject: [Linux-cluster] GFS on Centos We are investigating deploying GFS across a small pool of servers: Centos 5.1 x86_64 GigE Networking The data will consist of approximately 400GB of small JPG files accessed by an inhouse java app. The entire cluster is 50 machines but only 7 will require access to this data repository. GFS2 is not ready, yet... but my main question is, is it worth it to wait for GFS2? We are also looking at glusterfs. Our goal is: - low administrative (sysadm) overhead - good performance when accessing lots of small files (<100Mb) Geoff Galitz Blankenheim NRW, Deutschland http://www.galitz.org -- Linux-cluster mailing list Linux-cluster at redhat.com https://www.redhat.com/mailman/listinfo/linux-cluster From tiagocruz at forumgdh.net Mon Dec 1 20:13:06 2008 From: tiagocruz at forumgdh.net (Tiago Cruz) Date: Mon, 01 Dec 2008 17:13:06 -0300 Subject: [Linux-cluster] fence_vmware on rhel 5.3 Message-ID: <1228162386.17621.0.camel@tuxkiller.ig.com.br> Hello guys, Someone can send me one use's example from fence_vmware? This can't show on conga and I'm confuse to use it on command line... It's not perl any more, python now :-) [root at dcrs6037 ~]# fence_vmware -h Usage: fence_vmware [options] Options: -o Action: status, reboot (default), off or on -a IP address or hostname of fencing device -l Login name -p Login password or passphrase -S