<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi Abhi,<div> </div><div>I'll collect this info for you as soon as I have a chance and forward it to you. Is it your opinion that GFS2 should not be used in a production environment and we should stick with GFS v1 for now?</div><div> </div><div><div apple-content-edited="true"> <span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div> Steven Lee</div><div><a href="mailto:shl1@cac.cornell.edu">shl1@cac.cornell.edu</a></div><div><div>Center for Advanced Computing</div><div>Cornell University</div><div> </div></div></div></div> </div> <div><div>On Mar 11, 2008, at 3:18 PM, Abhijith Das wrote:</div> <blockquote type="cite">Hi Steven, Here are some things you could try. 1. comment out the gfs2 entry in /etc/fstab and mount only one gfs2 fs on only one node and perform the reset and init. 2. I wonder if this happens in a freshly created gfs2 fs. Just for sanity, try this: if you have a spare device, use that or create a file (and mount loopback) dd if=/dev/zero of=/tmp/foo bs=1024 count=1048576 Use the 'nolock' locking protocol. This will make your filesystem single-node mkfs.gfs2 -j 1 -p lock_nolock -t foo:bar /tmp/foo If you're not using a loopback device, you can remove 'loop' below. mount -t gfs2 -o loop,quota=on /tmp/foo /mnt/bar gfs2_quota reset -f /mnt/bar gfs2_quota init -f /mnt/bar If in both these cases you see the warning, we have a bug. You can go ahead and file one against me at bugzilla.redhat.com It would really help if you can dump the device containing the filesystem to a file, compress it and post it somewhere. If that's not possible, I'd like to take a look at the hidden quota file. 'mount -t gfs2meta /dev/gfs2-device /mnt/gfs2-metamount' will mount the gfs2 meta filesystem. (You should have the gfs2 filesystem already mounted before you attempt mounting gfs2meta). There should be a quota file under this mount that you can copy and send out. I should warn you though that the meta mount is only used by the gfs2_utils to manipulate gfs2 metadata and must not be used as a normal filesystem. You should unmount the metafs as soon as you're done with it. Thanks, --Abhi Steven Lee wrote: <blockquote type="cite">Hi Abhi, </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">Thanks for your response. I didn't do much to run into this problem. </blockquote><blockquote type="cite"> The 2 file servers in this RedHat cluster are running fully patched </blockquote><blockquote type="cite">RHEL 5.1. In addition to NFS, they are also serving to Windows </blockquote><blockquote type="cite">clients using samba. ACLs were enabled. Everything was running </blockquote><blockquote type="cite">smoothly until I tried enabling quotas: I simply added quota=on in </blockquote><blockquote type="cite">/etc/fstab and re-mounted the GFS partitions. </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">As you can see below, I simply issued "reset" and then "init" command </blockquote><blockquote type="cite">before getting the error message. The "get" command also returned </blockquote><blockquote type="cite">inconsistent result from the "list" command. This is a new file </blockquote><blockquote type="cite">server still under testing with only a small "prof" directory. </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">Did I do anything wrong? </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">Steven </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">--- </blockquote><blockquote type="cite">[root@cacfs02 ~]# *mount * </blockquote><blockquote type="cite">/dev/mapper/vg00-root on / type ext3 (rw) </blockquote><blockquote type="cite">proc on /proc type proc (rw) </blockquote><blockquote type="cite">sysfs on /sys type sysfs (rw) </blockquote><blockquote type="cite">devpts on /dev/pts type devpts (rw,gid=5,mode=620) </blockquote><blockquote type="cite">/dev/mapper/vg00-var on /var type ext3 (rw) </blockquote><blockquote type="cite">/dev/mapper/vg00-usr on /usr type ext3 (rw) </blockquote><blockquote type="cite">/dev/mapper/vg00-tmp on /tmp type ext3 (rw) </blockquote><blockquote type="cite">/dev/sda1 on /boot type ext3 (rw) </blockquote><blockquote type="cite">tmpfs on /dev/shm type tmpfs (rw) </blockquote><blockquote type="cite">none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) </blockquote><blockquote type="cite">sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) </blockquote><blockquote type="cite">none on /sys/kernel/config type configfs (rw) </blockquote><blockquote type="cite">/dev/mapper/st01vg01-lv01 on /export/gfs01 type gfs2 </blockquote><blockquote type="cite">(rw,hostdata=jid=0:id=65537:first=0,acl,quota=on) </blockquote><blockquote type="cite">/dev/mapper/st01vg02-lv01 on /export/gfs02 type gfs2 </blockquote><blockquote type="cite">(rw,hostdata=jid=1:id=393218:first=0,acl,quota=on) </blockquote><blockquote type="cite">/dev/mapper/st02vg01-lv01 on /export/gfs03 type gfs2 </blockquote><blockquote type="cite">(rw,hostdata=jid=1:id=524290:first=0,acl,quota=on) </blockquote><blockquote type="cite">/dev/mapper/st02vg02-lv01 on /export/gfs04 type gfs2 </blockquote><blockquote type="cite">(rw,hostdata=jid=1:id=655362:first=0,acl,quota=on) </blockquote><blockquote type="cite">[root@cacfs02 ~]# *gfs2_quota reset -f /export/gfs02* </blockquote><blockquote type="cite">This operation will permanently erase all quota information. You will </blockquote><blockquote type="cite">have to re-assign all quota limit/warn values. Proceed [y/N]? y </blockquote><blockquote type="cite">[root@cacfs02 ~]# *gfs2_quota init -f /export/gfs02* </blockquote><blockquote type="cite">warning: quota file size not a multiple of struct gfs2_quota </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">Warning: This filesystem doesn't seem to have the new quota list </blockquote><blockquote type="cite">format or the quota list is corrupt. list, check and init operation </blockquote><blockquote type="cite">performance will suffer due to this. It is recommended that you run </blockquote><blockquote type="cite">the 'gfs2_quota reset' operation to reset the quota file. All current </blockquote><blockquote type="cite">quota information will be lost and you will have to reassign all quota </blockquote><blockquote type="cite">limits and warnings </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">[root@cacfs02 ~]# *gfs2_quota list -f /export/gfs02* </blockquote><blockquote type="cite">warning: quota file size not a multiple of struct gfs2_quota </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">Warning: This filesystem doesn't seem to have the new quota list </blockquote><blockquote type="cite">format or the quota list is corrupt. list, check and init operation </blockquote><blockquote type="cite">performance will suffer due to this. It is recommended that you run </blockquote><blockquote type="cite">the 'gfs2_quota reset' operation to reset the quota file. All current </blockquote><blockquote type="cite">quota information will be lost and you will have to reassign all quota </blockquote><blockquote type="cite">limits and warnings </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">user root: limit: 0.0 warn: 0.0 value: 0.0 </blockquote><blockquote type="cite">user prof: limit: 0.0 warn: 0.0 value: 0.4 </blockquote><blockquote type="cite">group root: limit: 0.0 warn: 0.0 value: 0.0 </blockquote><blockquote type="cite">group Domain Users: limit: 0.0 warn: 0.0 value: 0.4 </blockquote><blockquote type="cite">[root@cacfs02 ~]# *gfs2_quota get -u prof -f /export/gfs02* </blockquote><blockquote type="cite">warning: quota file size not a multiple of struct gfs2_quota </blockquote><blockquote type="cite">user prof: limit: 0.0 warn: 0.0 value: 0.0 </blockquote><blockquote type="cite">[root@cacfs02 ~]# *ls -l /export/gfs02* </blockquote><blockquote type="cite">total 8 </blockquote><blockquote type="cite">drwxrwx---+ 12 prof Domain Users 3864 Mar 10 14:06 prof </blockquote><blockquote type="cite">[root@cacfs02 ~]# *uname -a* </blockquote><blockquote type="cite">Linux cacfs02.cac.cornell.edu 2.6.18-53.1.14.el5 #1 SMP Wed Mar 5 </blockquote><blockquote type="cite">11:37:38 EST 2008 x86_64 x86_64 x86_64 GNU/Linux </blockquote><blockquote type="cite">[root@cacfs02 ~]# *rpm -q gfs2-utils* </blockquote><blockquote type="cite">gfs2-utils-0.1.38-1.el5 </blockquote><blockquote type="cite">--- </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">Steven Lee </blockquote><blockquote type="cite">shl1@cac.cornell.edu <<a href="mailto:shl1@cac.cornell.edu">mailto:shl1@cac.cornell.edu</a>> </blockquote><blockquote type="cite">Center for Advanced Computing </blockquote><blockquote type="cite">Cornell University </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">On Mar 10, 2008, at 5:45 PM, Abhijith Das wrote: </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"><blockquote type="cite">Hi Steven, </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Does the 'gfs2_quota reset' operation fail for some reason? The reset </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">operation should truncate your quota file and you should be able to </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">start using quotas afresh. </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">I just tried resetting and initing the quota file on my gfs2 filesystem </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">(latest bits) and I don't see this warning. </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">What kernel and what gfs2-utils packages are you running? </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Also, any clue with respect to the quota operations you did on the fs </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">to get to this state would be helpful. </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Are you able to perform other quota operations like limit, warn, list, </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">get etc., without seeing this warning? </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Thanks, </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">--Abhi </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Steven Lee wrote: </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Hi, </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">I'm trying to turn on quota on a GFS2 file system. The cluster has 2 </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">nodes with 2 EMC SAN storage units. The nodes are running RHEL 5.1. </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">When I initialize the quota file using: </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">*gfs2_quota init -f /export/gfs02* </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">I would get the following warning: </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">*warning: quota file size not a multiple of struct gfs2_quota* </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">* </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">* </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">*Warning: This filesystem doesn't seem to have the new quota list </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">format or the quota list is corrupt. list, check and init operation </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">performance will suffer due to this. It is recommended that you run </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">the 'gfs2_quota reset' operation to reset the quota file. All current </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">quota information will be lost and you will have to reassign all quota </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">limits and warnings* </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">I've tried the "gfs2_quota reset" and then "gfs2_quota init" commands, </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">but still got the warning. I've looked through RedHat GFS </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">documentation and googled for gfs2. Very little information seems to </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">be available. </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Any help is appreciated. Thanks! </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Steven Lee </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">shl1@cac.cornell.edu <<a href="mailto:shl1@cac.cornell.edu">mailto:shl1@cac.cornell.edu</a>> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Center for Advanced Computing </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Cornell University </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">------------------------------------------------------------------------ </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">-- </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Linux-cluster mailing list </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Linux-cluster@redhat.com <<a href="mailto:Linux-cluster@redhat.com">mailto:Linux-cluster@redhat.com</a>> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><a href="https://www.redhat.com/mailman/listinfo/linux-cluster">https://www.redhat.com/mailman/listinfo/linux-cluster</a> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">-- </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Linux-cluster mailing list </blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Linux-cluster@redhat.com <<a href="mailto:Linux-cluster@redhat.com">mailto:Linux-cluster@redhat.com</a>> </blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><a href="https://www.redhat.com/mailman/listinfo/linux-cluster">https://www.redhat.com/mailman/listinfo/linux-cluster</a> </blockquote></blockquote><blockquote type="cite"> </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">------------------------------------------------------------------------ </blockquote><blockquote type="cite"> </blockquote><blockquote type="cite">-- </blockquote><blockquote type="cite">Linux-cluster mailing list </blockquote><blockquote type="cite"><a href="mailto:Linux-cluster@redhat.com">Linux-cluster@redhat.com</a> </blockquote><blockquote type="cite"><a href="https://www.redhat.com/mailman/listinfo/linux-cluster">https://www.redhat.com/mailman/listinfo/linux-cluster</a> </blockquote><blockquote type="cite"> </blockquote> -- Linux-cluster mailing list <a href="mailto:Linux-cluster@redhat.com">Linux-cluster@redhat.com</a> https://www.redhat.com/mailman/listinfo/linux-cluster </blockquote></div> </div></body></html>