[Linux-cluster] Only root can write on GFS volume...

[Dascalu Dragos]

We are currently experimenting w/ GFS and have run into a problems we 
can not seem to find an answer to. To set the stage: We have 3 machines 
connected through an optical switch to a SAN. For simplicity purposes 
we have created a LUN which can be seen by the 3 machines. 
GFS+modules+patches are successfully running. We are using LMV2 and 
created a volume group called "test" on the /dev/sdb5 which is what the 
machines see the LUN as. We then created a logical volume called "one" 
on this volume group.

web3:~# ls -la /dev/test/
total 28
dr-x------    2 root     root         4096 Jul 29 10:01 .
drwxr-xr-x   13 root     root        24576 Jul 29 10:01 ..
lrwx------    1 root     root           20 Jul 29 10:01 one -> 
/dev/mapper/test-one

web3:~# ls -la /dev/mapper/
total 28
drwxr-xr-x    2 tomcat   tomcat       4096 Jul 29 10:13 .
drwxr-xr-x   13 root     root        24576 Jul 29 10:01 ..
crw-------    1 root     root      10,  63 Jul 29 10:13 control
brw-------    1 root     root     254,   0 Jul 29 10:01 test-one

/dev/test/one was formatted using "gfs_mkfs -p lock_dlm -t 
webserver:one -j 4 /dev/test/one". (there will be 4 machines in the 
future)
System starts fine, all 3 machines are member nodes and can 
successfully mount /dev/test/one on "/test" (mount point on / we 
created for testing).
_______
Problem
------------
When /test is accessed and written to as root everything is fine, new 
data gets updated on the other nodes in real time. However if another 
user besides root attempts to write to /test the partition locks up 
(basically the shell we are in locks appearing to wait for the return 
of the "touch new_file" command). In the process tree we can see the 
touch command however it can not be killed nor can /test be unmounted. 
At this point on any of the 3 machines an "ls -la /test" has the same 
frozen behavior. Only a reboot gets things back to normal :(

We first thought that this may be an LMV2 problem but if /dev/test/one 
is formatted as ext3 not gfs then mounted on /test is can be written to 
fine by all users including root (we also gave 777 permissions to all 
objects in /dev/test and /dev/mapper). This also does not seem to be an 
obvious OS perm issue...

Scenario 1:
drwxr-xr-x    2 root     root         4096 Jul 29 10:02 test
Here only root can write and everyone read. If another user but root 
tries to write the get an "touch: creating `/test/tom': Permission 
denied". This does not cause a system freeze.

Scenario 2:
drwxrwxrwx    2 root     root         4096 Jul 29 10:02 test
Here everyone can do anything to this directory and if another user but 
root tries to write system freezes.

It appears that when it is formatted as gfs no one but root can write 
to it. Any thoughts?

Dede.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3399 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20040729/0a695c04/attachment.p7s>