yesterday I experienced following problem with my ext3 filesystem:
- I had ext3 filesystem of the size of a few TB with journal. I correctly unmounted it and it was marked clean.
- I then ran fsck.etx3 -f on it and it did not find any problem.
- After increasing size of its LVM volume by 1.5 TB I resized the filesystem by resize2fs lvm_volume and it finished without problem.
- But fsck.ext3 -f immediately after that showed "Inodes that were part of a corrupted orphan linked list found." and many thousands of "Inode XXX was part of the orphaned inode list." I did not accepted fix. According to debugfs all the inodes I check from these reported orphaned inodes (I checked only some from beginning of list of errors) have size 0.
- When I mount the fs read only the data I was able to check seem OK. (But I am unable to check everything.)
- I created LVM snapshot and repaired the fs on it with fsck.ext3. After that there we no files in lost+found. Does it mean that all that orphaned inodes have size 0? Or when the fsck does not create files in lost+found?
- I am checking the data against various backups but I will not be able to check everything and some less important data dont have backup. So I would like to know in what state the fs is and what are best next steps.
- Right now I am planning to use current LVM snapshot as test run and discard it after data check. Original fs is in the state just after resize2fs, fsck was run on it after that but I did not accepted any fix and cancelled the check. I then plan to create backup snapshot, fsck original fs / LVM volume, check once again against backups and go with it. But this will not tell me status of all my data and the fs and if it is secure to use it. Another problem is all operations take long hours.
- I have also some technical specific questions. Orphan inode is valid inode not found in any directory, right? What exactly is CORRUPTED orphan linked list? What can cause such problem? Is it known problem? How can orphaned inodes and corrupted orphan linked list can be created by resize2fs or why was it not detected by fsck.ext3 before that? Can it be serious and can it be symptom of some data loss? Can fixing it by fsck.ext3 corrupt other data which are OK now, when I mount the fs read-only?
- The platform used was latest stable Debian with kernel linux-image-3.2.0-4-amd64 version 3.2.46-1+deb7u1 and e2fsprogs 1.42.5-1.1. After the incident I started using linux-image-3.13-1-amd64 version 3.13.7-1 (from the point of snapshot's creation and running fsck for real on snapshot) and thinking about going to e2fsprogs 1.42.9 from sources.
Thank you very much.