[Linux-cluster] GFS2 directory hangs on one node CentOS 5.3

[Libor Tomsik]

Hi all,

I'm having a strange issue with a two nodes cluster based on xen
virtual hosts with shared disk on clvm. The servers are running apache
and one is considered as hot backup. On that node awstats are counted
from the apache custom logs stored on the shared device. Web data,
logs, configs and awstats results are in different directories withing
the same GFS2 volume.

Everything works fine, but sometimes (at production environment, damn)
the directory with logs get frozen for the spare node with awstats.
All commands like ls, cd, mc on that directory get status D. On the
second node all works fine. Other directories seems unaffected too.

I can not umount fs neither remout it ro and back rw since there are
"running" processes at D state.

Can someone give me some advice, how-to prevent this problem? And
how-to recovery from it? It is a production with SLA on :(  In next
time, I'll try to make lockdump on both nodes.

Kernel is 2.6.18-128.1.10.el5xen, gfs2-utils-0.1.53-1.el5_3.2,
kmod-gfs2-xen-1.92-1.1.el5_2.2

Regards

Libor