[Linux-cluster] umount failed - device is busy

[Herta Van den Eynde]

environment:
- Red Hat AS 3 (kernel-smp-2.4.21-37.EL - custom built to probe all LUNs 
on each SCSI device)
- clumanager 1.2.28

The cluster consists of 2 members running three services which simply 
nfs export a number of directories to five other systems.
The cluster has been operational since February.

Following the latest upgrade (from kernel-smp-2.4.21-32.0.1.EL custom 
built and clumanager-1.2.26.1-1), all services are running on one 
member.  When I try to locate the services, the operation fails, and the 
following message pops up:

	A Problem has occurred while changing ownership
	of this service.  Please check logs for details.

The cluster log reports the following:

==== begin log extract
Member arnebd trying to relocate lepustl to nihald...Oct 10 16:08:06 
arnebd clusvcmgrd: [13627]: <notice> service notice: Stopping service 
lepustl ...
Oct 10 16:08:06 arnebd clurmtabd[26429]: <debug> Signal 15 received; exiting
Oct 10 16:08:12 arnebd clusvcmgrd: [13627]: <err> service error: 'umount 
/dev/sdb2' failed (/usr/local/lepus-tl), error=1
Oct 10 16:08:12 arnebd clusvcmgrd: [13627]: <err> service error: umount: 
/usr/local/lepus-tl: device is busy
Oct 10 16:08:12 arnebd clusvcmgrd: [13627]: <err> service error: umount: 
/usr/local/lepus-tl: device is busy
Oct 10 16:08:12 arnebd clusvcmgrd: [13627]: <err> service error: Cannot 
stop filesystems for lepustl
Oct 10 16:08:12 arnebd clusvcmgrd[13626]: <notice> Starting stopped 
service lepustl
Oct 10 16:08:12 arnebd clusvcmgrd: [14083]: <notice> service notice: 
Starting service lepustl ...
Oct 10 16:08:12 arnebd clurmtabd[14194]: <debug> Log level is now 7
Oct 10 16:08:12 arnebd clurmtabd[14194]: <debug> Polling interval is now 
4 seconds
failed
Oct 10 16:08:12 arnebd clusvcmgrd: [14083]: <notice> service notice: 
Started service lepustl ...
Oct 10 16:08:14 arnebd clurmtabd[6533]: <debug> Detected modified 
/var/lib/nfs/rmtab
Oct 10 16:08:14 arnebd clurmtabd[9655]: <debug> Detected modified 
/var/lib/nfs/rmtab
==== end log extract

FWIIW, no one was logged in but me, and my current directory was not on 
this filesystem.
Neither fuser nor lsof returned any process using the filesystem.
I figured the clurmtabd process may be locking it, so I did verify that 
there is only one clurmtab process for that filesystem.

Any ideas/suggestions?

Kind regards,

Herta

-- 
Herta Van den Eynde              -=- Toledo system management
K.U. Leuven - Ludit              -=- phone: +32 (0)16 322 166
                                  -=- 50°51'27" N 004°40'39" E

"I wish I were two little cats.  Then I could play together."

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm