<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN"> <HTML> <HEAD> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8"> <META NAME="GENERATOR" CONTENT="GtkHTML/3.0.10"> </HEAD> <BODY> I have a GFS cluster set up for testing. The source code was pulled from CVS around 2 weeks ago, I believe the release number is 1095350377. I set up 3 logical volumes, lv1, lv2, and lv3. lv1 was mounted on a host named eightoften as /var/spool/mqueue, lv2 was mounted on host a host named nineoften as /var/spool/mqueue. Lv3 was mounted on both eightoften and nineoften as as /var/spool/mail. The goal is to have one mailbox for each user on either host and have a load balancer distribute the incoming between the 2 hosts. Under light loads, things work well, but under heavy loads sendmail has some locking problems: <PRE>Oct 1 15:57:00 nineoften sendmail[29243]: NOQUEUE: SYSERR(root): cannot lockf(qfi91LpkrM029041, fd=4, type=5, omode=0, euid=0): Operation not permitted Oct 1 15:57:00 nineoften sendmail[29243]: NOQUEUE: 4: fl=0x0, mode=100600: dev=253/3, ino=290, nlink=1, u/gid=0/55, size=1188 Oct 1 15:57:00 nineoften sendmail[29243]: NOQUEUE: SYSERR(root): cannot lockf(qfi91Lpl88029050, fd=4, type=5, omode=0, euid=0): Operation not permitted Oct 1 15:57:00 nineoften sendmail[29243]: NOQUEUE: 4: fl=0x0, mode=100600: dev=253/3, ino=193, nlink=1, u/gid=0/55, size=1187 Oct 1 15:57:00 nineoften sendmail[29243]: NOQUEUE: SYSERR(root): cannot lockf(qfi91Lpik7029006, fd=4, type=5, omode=0, euid=0): Operation not permitted Oct 1 15:57:00 nineoften sendmail[29243]: NOQUEUE: 4: fl=0x0, mode=100600: dev=253/3, ino=297, nlink=1, u/gid=0/55, size=1187</PRE> Other host: <PRE>Oct 1 15:56:52 eightoften sendmail[7288]: NOQUEUE: SYSERR(root): cannot lockf(qfi91LpiRJ006964, fd=4, type=5, omode=0, euid=0): Operation not permitted Oct 1 15:56:52 eightoften sendmail[7288]: NOQUEUE: 4: fl=0x0, mode=100600: dev=253/1, ino=156, nlink=1, u/gid=0/55, size=1188 Oct 1 15:56:52 eightoften sendmail[7288]: NOQUEUE: SYSERR(root): cannot lockf(qfi91Lpkw1006986, fd=4, type=5, omode=0, euid=0): Operation not permitted Oct 1 15:56:52 eightoften sendmail[7288]: NOQUEUE: 4: fl=0x0, mode=100600: dev=253/1, ino=203, nlink=1, u/gid=0/55, size=1203 Oct 1 15:56:52 eightoften sendmail[7288]: NOQUEUE: SYSERR(root): cannot lockf(qfi91Lpt0k007183, fd=4, type=5, omode=0, euid=0): Operation not permitted </PRE> I wrote a couple of test programs that call fcntl to get a read lock or a write lock. The test program work fine on ext2, or reiser file systems, but return an operation not permitted on a GFS file system under some cases: If you call fcntl with F_SETLKW, things appear to work, calling fnctl with F_SETLK will return with a -1 and set errno to 1. readlock and writelock were compiled to call fcntl with F_SETLK: First, I ran writelock on nineoften: <PRE>[mbrookov@nineoften locktest]$ ./writelock /var/spool/mail/test/afile Have write lock, hit return to free write lock on /var/spool/mail/test/afile and exit</PRE> Then ran readlock on eightoften: <PRE>[mbrookov@eightoften locktest]$ ./readlock /var/spool/mail/test/afile Could not get read lock on /var/spool/mail/test/afile errno=1:Operation not permitted [mbrookov@eightoften locktest]$</PRE> Similar problems happen if you run readlock and writelock on the same host. Out of 500 mail messages sent, only 216 arrived in the mail box. I have a load balancer set up to distribute the mail messages between the 2 hosts. Any ideas? thanks Matt Academic Computing and Networking Colorado School of Mines <A HREF="mailto:mbrookov@mines.edu">mbrookov@mines.edu</A> 303-273-3436 </BODY> </HTML>