Ext3 - Frequent read-only FS issues

[Dushyanth]

Hi,

Theodore Tso <tytso <at> mit.edu> writes:
> 
> On Tue, Mar 24, 2009 at 03:37:48PM +0000, Dushyanth wrote:
> > > Do you lose power much?  Do these errors correspond to power loss
> > > events?
> > 
> > These errors come up suddenly on running systems. We do IPMI based 
> > reboots for power cycling the boxes when they hang.
> 
> I assume you do have a full fsck done on the filesystem after you see
> these errors?

Its quite possible that we might have not not run the suggested fsck. Ops/DC
staff sometimes skip fsck to get the server up & running quick.
 
> > Is it possible that some corruption that occurred during one such 
> > reboots to cause a read only FS later on with the errors i mentioned ? 
> > Iam guessing yes, cos force fsck during boot is not forced always cos 
> > sometimes a FS check is suggested by ext3 during mounting the disk.
> 
> If an fs check is getting suggested by ext3 during the mounting of the
> disk, then your boot scripts must not be set up correctly, or
> /etc/fstab has been set up to disable fsck getting run automatically
> on said filesystem.

Boot scripts are redhat's defaults and fstab has fsck check is enabled 
for the disks in question. 

If the boot scripts run a fsck on ext3 filesystems that are marked unclean 
then i was wrong in guessing that ext3 fs's get mounted in such cases with a
suggestion.
 
> I would strongly recommend running forced fsck to make sure all of
> your filesystems are clean first; if the errors have been around for a
> while, and for some reason the automatic fsck has been disabled so
> they aren't getting checked, that could be really bad.

Ok. I might as well do the preemptive maintenance just to be sure.

Thanks for the suggestions.

TIA
Dushyanth