From cxzheng at glocom-us.com Wed Feb 2 18:56:04 2005 From: cxzheng at glocom-us.com (Andy Zheng) Date: Wed, 2 Feb 2005 10:56:04 -0800 Subject: Help for Linux Message-ID: <000801c50958$e00dcce0$8600a8c0@glocomeng> Hi All,I need help with the following problems:The system might suffer a power loss when I run a program. After the reboot process, it has happened, this prompt is issued on the filesystem check failed. This message appears:*** An error occurred during the file system check*** Dropping you to a shell; the system will reboot*** when you leave the shellGive root password for maintenance(or type Control-D for normal startup) When the filesystem check failed, I will log in by specifying the root password and run the fsck program manually. When I have typed in the root password, I am presented with the following prompt:(Repair filesystem) # How can I repair filesytem? Best regards,AndySystem EngineerE-mail: cxzheng at glocom-us.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From d_baron at 012.net.il Thu Feb 3 20:35:31 2005 From: d_baron at 012.net.il (David Baron) Date: Thu, 03 Feb 2005 22:35:31 +0200 Subject: Ext3-users Digest, Vol 12, Issue 1 In-Reply-To: <20050203170004.A513373032@hormel.redhat.com> References: <20050203170004.A513373032@hormel.redhat.com> Message-ID: <200502032235.31792.d_baron@012.net.il> Often enough, even on the periodic fsck's that occur every so-many mounts, the fsck will fail and leave me in a shell. On the Debian boot, I am already logged in as root and the root filesystem is mounted read-only. I then need to run fsck -f [/mnt/hd....] manually -f forces. It will go through a series of checks. Usually, it simply ends and I do the control D and everything is clean. --------------------------- If something has occured like a hangup or power off, the journal recovery itself might give me warnings about delete inodes, etc. Thankfully, all is "clean" afterwards. Ext3 has been good to me so far :-) From daniel at rimspace.net Thu Feb 3 23:51:06 2005 From: daniel at rimspace.net (Daniel Pittman) Date: Fri, 04 Feb 2005 10:51:06 +1100 Subject: Help for Linux References: <000801c50958$e00dcce0$8600a8c0@glocomeng> Message-ID: <876519qk2d.fsf@enki.rimspace.net> On 3 Feb 2005, Andy Zheng wrote: > I need help with the following problems: > > The system might suffer a power loss when I run a program. After the reboot > process, it has happened, this prompt is issued on the filesystem > check failed. This message appears: > > *** An error occurred during the file system check > *** Dropping you to a shell; the system will reboot > *** when you leave the shell > Give root password for maintenance > (or type Control-D for normal startup) > > When the filesystem check failed, I will log in by specifying the root > password and run the fsck program manually. When I have typed in the > root password, I am presented with the following prompt: > > (Repair filesystem) # > > How can I repair filesytem? Run fsck manually on the drive. You may find reading the 'e2fsck' manual page helpful here. Regardless of that prompt, it is simply a standard shell. daniel -- A good critic is the sorcerer that makes some hidden spring gush forth unexpectedly under our feet. -- Fran?ois Mauriac From theman at josephdwagner.info Fri Feb 4 02:29:29 2005 From: theman at josephdwagner.info (Joseph D. Wagner) Date: Thu, 03 Feb 2005 21:29:29 -0500 Subject: Help for Linux Message-ID: You may want to try running e2fsck from the Rescue CD. From gene at czarc.net Sat Feb 5 17:36:39 2005 From: gene at czarc.net (Gene Czarcinski) Date: Sat, 5 Feb 2005 12:36:39 -0500 Subject: ext3 partition compatibility Message-ID: <200502051236.39884.gene@czarc.net> Recently, I had some experiences that raises the question as to how compatible (how safe) it is to mount and then read/write a partition using a system such as FC1 or debian where the partition was created under FC3. 1. I recently attempted to run e2fsck running on a FC1 system against a FC3 ext3 partition ... the program refused to run. When I took the e2fsprogs package from FC3 and rebuilt it on a FC1 system, I was able to run the rebuilt e2fsck no problem. 2. I have a systems (laptops) which is dual boot with win2k and FC3. I have another dual boot system with win2k and debian. I attempted to use Symantec's ghost to image (backup) the drives -- the whole disk with all partitions. Per advertising, ghost supports NTFS, fat16, fat32. ext2, ext3, and swap partitions. Running ghost on the win2k+debian system worked fine. However, when I ran it on the win2k+FC3 system it got errors. After finally finding the right google search, I located a mesage from a user with the same problem. That user had contacted Symantec and, apparent, ghost did not work on FC3 (or FC2) partitions. My suspicion is that the reason it does not work is SELinux. Although the FC3 system was installed with SELinux disabled, I suspect that SELinux must require some modifications/additions to the file system which may (does) cause problems when it is access from a non-SELinux system. Any help would be appreciated. I am not looking for the ability to access these partions, just to know if it should work or not. Gene From theman at josephdwagner.info Sun Feb 6 09:59:18 2005 From: theman at josephdwagner.info (Joseph D. Wagner) Date: Sun, 6 Feb 2005 03:59:18 -0600 Subject: ext3 partition compatibility In-Reply-To: <200502051236.39884.gene@czarc.net> Message-ID: <200502061129.j16BT9i4006529@mx1.redhat.com> In so far as I know, the difference is in the way a FC3 partition stores directories (a newer, faster, more efficient hash method). But don't take my word for it. I haven't thoroughly researched this issue; this is just off the top of my head. I hope it points you in the right direction. Joseph D. Wagner From silfreed at silfreed.net Mon Feb 7 01:08:22 2005 From: silfreed at silfreed.net (Douglas E. Warner) Date: Sun, 6 Feb 2005 20:08:22 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs Message-ID: <200502062008.23249.silfreed@silfreed.net> I found a thread that has almost the exact same symptoms as me, but didn't seem to come to a resolution: https://listman.redhat.com/archives/ext3-users/2004-December/msg00018.html I have an LVM(2) array that I've just lvextend'd and want to resize2fs, but I can't get through the e2fsck. I get these errors when fsck-ing: Group 3125's inode table at 102400545 conflicts with some other fs block. Relocate? yes Group 3125's inode table at 102400546 conflicts with some other fs block. Relocate? yes Group 3125's inode table at 102400547 conflicts with some other fs block. Relocate? yes Group 3125's block bitmap at 102400034 conflicts with some other fs block. Relocate? yes Group 3125's inode bitmap at 102400035 conflicts with some other fs block. Relocate? yes And contines to fsck, but then restarts the fsck around 66% and the exact same errors occur again. If I cancel the fsck, I can still mount the partition and everything seems fine, but I can't resize the partition without getting through the fsck. Any thoughts? Let me know if I need to provide any more information. -Doug -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From swarren at wwwdotorg.org Mon Feb 7 01:27:13 2005 From: swarren at wwwdotorg.org (Stephen Warren) Date: Sun, 06 Feb 2005 18:27:13 -0700 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <200502062008.23249.silfreed@silfreed.net> References: <200502062008.23249.silfreed@silfreed.net> Message-ID: <1107739418.21811.TMDA@tmda.severn.wwwdotorg.org> Douglas E. Warner wrote: > I found a thread that has almost the exact same symptoms as me, but didn't > seem to come to a resolution: > https://listman.redhat.com/archives/ext3-users/2004-December/msg00018.html This was me. My solution: mkreiserfs /dev/... Luckily, the only partition of mine that got corrupted was my backup partition, so it was trivial to re-create this simply by running my backup scripts again after formatting the partition as reiser. I then converted all my other partitions over to reiser, luckily having left enough unpartitioned space on my disk to create some temp partitions for use during the conversion. Whilst there were (I think?) some (potential?) solutions listed in that thread, since the issue seemed very complicated, I wasn't convinced that it would be solved with a simple update to the FC3 RPMs. I think that even if I had stuck with ext3, I would have needed to reformat my partitions anyway, so switching fs type was no extra work. I tested out reiserfs, and found that resizing worked out of the box without any workarounds, and figured that was safer than hopefully remembering to do the required workarounds/fixes. -- Stephen Warren, Software Engineer, NVIDIA, Fort Collins, CO swarren at wwwdotorg.org http://www.wwwdotorg.org/pgp.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 250 bytes Desc: OpenPGP digital signature URL: From silfreed at silfreed.net Mon Feb 7 01:32:29 2005 From: silfreed at silfreed.net (Douglas E. Warner) Date: Sun, 6 Feb 2005 20:32:29 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <1107739418.21811.TMDA@tmda.severn.wwwdotorg.org> References: <200502062008.23249.silfreed@silfreed.net> <1107739418.21811.TMDA@tmda.severn.wwwdotorg.org> Message-ID: <200502062032.29631.silfreed@silfreed.net> On Sunday 06 February 2005 20:27, Stephen Warren wrote: > This was me. My solution: > > mkreiserfs /dev/... > I've resized both reiserfs and ext3 partitions in the past without problems. Reiserfs partitions were on FC3, but I haven't resized any ext3 partitions in FC3 before. Since I wasn't expecting any problems resizing this (as I've done it before without problems) this is on some data I'd prefer not to loose (about 500G). -Doug -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From joerg at alea.gnuu.de Fri Feb 4 16:45:27 2005 From: joerg at alea.gnuu.de (Joerg Sommer) Date: Fri, 4 Feb 2005 17:45:27 +0100 Subject: Failures they e2fsck doesn't find Message-ID: <20050204164527.GA24817@alea.gnuu.de> Hi, I've run many time e2fsck, but in a special dir ls tells me: ls: r?cksendung-wlan.dvi: No such file or directory ls: baf?g_r?ckmeldung.latex: No such file or directory ls: finpr?f.pdf: No such file or directory $ cat finpr?f.pdf cat: finpr?f.pdf: Datei oder Verzeichnis nicht gefunden I don't know what to do? How can I find the failure? If I cat the files with debugfs, I see the their contents. What information do you need to help me? $ uname -a Linux ibook 2.6.9 #1 Wed Jan 26 10:21:54 CET 2005 ppc GNU/Linux $ e2fsck -V e2fsck 1.36-rc4 (26-Jan-2005) Benutze EXT2FS Library version 1.36-rc4, 26-Jan-2005 # tune2fs -l /dev/mapper/_dev_discs_disc0_part5 tune2fs 1.36-rc4 (26-Jan-2005) Filesystem volume name: Last mounted on: Filesystem UUID: 0a97dddc-ad56-4eb8-9a07-a43ddfc2e22b Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal dir_index needs_recovery large_file Default mount options: (none) Filesystem state: clean Errors behavior: Remount read-only Filesystem OS type: Linux Inode count: 958464 Block count: 18874368 Reserved block count: 0 Free blocks: 1846379 Free inodes: 910758 First block: 1 Block size: 1024 Fragment size: 1024 Blocks per group: 8192 Fragments per group: 8192 Inodes per group: 416 Inode blocks per group: 52 Filesystem created: Tue Jun 22 17:35:13 2004 Last mount time: Wed Feb 2 16:34:30 2005 Last write time: Wed Feb 2 16:34:30 2005 Mount count: 1 Maximum mount count: 35 Last checked: Wed Feb 2 16:31:01 2005 Check interval: 15552000 (6 months) Next check after: Mon Aug 1 17:31:01 2005 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 First orphan inode: 237957 Default directory hash: tea Directory Hash Seed: 12789f3d-cc34-4173-8c32-70513958750d Journal backup: inode blocks Joerg. -- Diskusion "Pascal vs. rest der Welt" : 30 Aug 2000 00:13:11 GMT, Adrian Knoth Und selbst wenn eure 100000 Zeilen-Programme noch so oft unter Windows verwendet werden: mit einem Handwagen f?hrt man nicht Formel-1. From silfreed at silfreed.net Mon Feb 7 13:45:46 2005 From: silfreed at silfreed.net (Douglas E. Warner) Date: Mon, 7 Feb 2005 08:45:46 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <200502062008.23249.silfreed@silfreed.net> References: <200502062008.23249.silfreed@silfreed.net> Message-ID: <200502070845.46703.silfreed@silfreed.net> I should also note that this happened on the second resize for the array. I started with a single disk, then was able to add a second disk fine (lvextend, e2fsck, resize2fs). Only after this next lvextend I cannot proceed past the e2fsck. I'll try to include some more details about my setup below. -Doug # cat /etc/fedora-release Fedora Core release 3 (Heidelberg) # uname -a Linux thor.home.silfreed.net 2.6.10-1.760_FC3smp #1 SMP Wed Feb 2 00:29:03 EST 2005 i686 i686 i386 GNU/Linux # pvdisplay --- Physical volume --- PV Name /dev/hde VG Name StorageArray PV Size 372.59 GB / not usable 0 Allocatable yes (but full) PE Size (KByte) 32768 Total PE 11923 Free PE 0 Allocated PE 11923 PV UUID GxQUjD-faST-c2aA-MgQg-Y78a-GwrD-gD3l1y --- Physical volume --- PV Name /dev/hdi VG Name StorageArray PV Size 152.66 GB / not usable 0 Allocatable yes (but full) PE Size (KByte) 32768 Total PE 4885 Free PE 0 Allocated PE 4885 PV UUID Vr7dmh-7wTx-RiqP-ndOq-SJa8-H0Ug-ssU0dN --- Physical volume --- PV Name /dev/hdk VG Name StorageArray PV Size 114.47 GB / not usable 0 Allocatable yes (but full) PE Size (KByte) 32768 Total PE 3663 Free PE 0 Allocated PE 3663 PV UUID NaNqVE-JxjU-lo52-TUrN-2k8m-YShA-mOYEiY --- Physical volume --- PV Name /dev/sda VG Name StorageArray PV Size 232.88 GB / not usable 0 Allocatable yes (but full) PE Size (KByte) 32768 Total PE 7452 Free PE 0 Allocated PE 7452 PV UUID 7b1GCZ-1dVx-93ho-f1D9-k0AK-LuNF-DchmiY # lvdisplay --- Logical volume --- LV Name /dev/StorageArray/lvol0 VG Name StorageArray LV UUID OXdAij-4vy0-o8B2-40bk-AhAc-tuRo-Ur5QIy LV Write Access read/write LV Status available # open 0 LV Size 872.59 GB Current LE 27923 Segments 4 Allocation inherit Read ahead sectors 0 Block device 253:0 # vgdisplay --- Volume group --- VG Name StorageArray System ID Format lvm2 Metadata Areas 4 Metadata Sequence No 7 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 4 Act PV 4 VG Size 872.59 GB PE Size 32.00 MB Total PE 27923 Alloc PE / Size 27923 / 872.59 GB Free PE / Size 0 / 0 VG UUID kSWgYl-4FN2-HC3O-XdIM-nSnZ-QM2V-dxQFlc # dumpe2fs /dev/StorageArray/lvol0 dumpe2fs 1.35 (28-Feb-2004) Filesystem volume name: StorageArray Last mounted on: Filesystem UUID: c7cebb3e-eea2-4905-9eb1-ce2ebd9b8f2f Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal resize_inode filetype sparse_super large_file Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 68845568 Block count: 137691136 Reserved block count: 0 Free blocks: 18826646 Free inodes: 68824841 First block: 0 Block size: 4096 Fragment size: 4096 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 16384 Inode blocks per group: 512 Filesystem created: Mon Dec 20 14:01:34 2004 Last mount time: Sun Feb 6 17:40:31 2005 Last write time: Sun Feb 6 18:50:24 2005 Mount count: 3 Maximum mount count: -1 Last checked: Sat Feb 5 18:47:06 2005 Check interval: 0 () Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 Default directory hash: tea Directory Hash Seed: 4054770a-3868-4699-82c4-6e7346ad18d2 Journal backup: inode blocks -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tytso at mit.edu Mon Feb 7 17:56:24 2005 From: tytso at mit.edu (Theodore Ts'o) Date: Mon, 7 Feb 2005 12:56:24 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <200502062008.23249.silfreed@silfreed.net> References: <200502062008.23249.silfreed@silfreed.net> Message-ID: <20050207175624.GB8014@thunk.org> On Sun, Feb 06, 2005 at 08:08:22PM -0500, Douglas E. Warner wrote: > I found a thread that has almost the exact same symptoms as me, but didn't > seem to come to a resolution: > https://listman.redhat.com/archives/ext3-users/2004-December/msg00018.html > > I have an LVM(2) array that I've just lvextend'd and want to resize2fs, but I > can't get through the e2fsck. I get these errors when fsck-ing: > > Group 3125's inode table at 102400545 conflicts with some other fs block. > Relocate? yes This is cuased by a bug in Fedora Core 3's e2fsprogs. It had patches to support the new on-line resizing, but those patches were incomplete, and off-line resize (i.e., resize2fs) was broken. The fixes have been out for a while, and e2fsprogs 1.36 has the fixes that will resolve this problem. Please see the RELEASE-NOTES file in e2fsprogs 1.36 for more details: All of the patches that were applied to Fedore Core 3's e2fsprogs-1.35-11.2 have been integrated, although sometimes with a lot of bug fixes first. Users of Fedora Core 3 are strongly encouraged to upgrade to e2fsprogs 1.36 as soon as possible. Add support for filesystem with the online resizing via resize inode feature. Fixed numerous bugs from the Fedora patches. The Fedora patches also didn't bother to do any consistency checking on the resize inode, or add any tests to the regression test suite. The "-R resize=4g" option to mke2fs was a no-op in the Fedora patches, despite being listed in mke2fs's usage message. All of these shortcomings have been corrected. E2fsck can also also fix filesystems trashed by Fedora's resize2fs program. In order to do this, the user must run the commands: debugfs -w /dev/hdXXX -R "features ^resize_inode e2fsck -f /dev/hdXXX Optionally, the ext2prepare command can be used to re-enable online resizing after the filesystem has been fixed. - Ted From jwbaker at acm.org Mon Feb 7 17:58:31 2005 From: jwbaker at acm.org (Jeffrey W. Baker) Date: Mon, 07 Feb 2005 09:58:31 -0800 Subject: mke2fs options for very large filesystems Message-ID: <1107799111.3307.3.camel@localhost.localdomain> Wow, it takes a really long time to make a 2TB ext2fs. Are there better-than-default options that could be used for a large filesystem? mke2fs 1.34 (25-Jul-2003) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 244203520 inodes, 488382016 blocks 24419100 blocks (5.00%) reserved for the super user First data block=0 14905 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000, 214990848 -jwb From silfreed at silfreed.net Mon Feb 7 18:06:27 2005 From: silfreed at silfreed.net (Douglas E. Warner) Date: Mon, 7 Feb 2005 13:06:27 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <20050207175624.GB8014@thunk.org> References: <200502062008.23249.silfreed@silfreed.net> <20050207175624.GB8014@thunk.org> Message-ID: <200502071306.27375.silfreed@silfreed.net> On Monday 07 February 2005 12:56, you wrote: > This is cuased by a bug in Fedora Core 3's e2fsprogs. ?It had patches > to support the new on-line resizing, but those patches were > incomplete, and off-line resize (i.e., resize2fs) was broken. ?The > fixes have been out for a while, and e2fsprogs 1.36 has the fixes that > will resolve this problem. ?Please see the RELEASE-NOTES file in > e2fsprogs 1.36 for more details: So, the recommendation is to upgrade to e2fsprogs 1.36, run the 'debugfs, e2fsck' commands, and ext2prepare to make everything happy again? Another way to put it is: "Don't run the debugfs using FC3's e2fsprogs 1.35"? Anyone know of e2fsprogs 1.36 RPMs for FC3? I don't see any in development, yet. -Doug -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From silfreed at silfreed.net Mon Feb 7 18:23:48 2005 From: silfreed at silfreed.net (Douglas E. Warner) Date: Mon, 7 Feb 2005 13:23:48 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <200502071306.27375.silfreed@silfreed.net> References: <200502062008.23249.silfreed@silfreed.net> <20050207175624.GB8014@thunk.org> <200502071306.27375.silfreed@silfreed.net> Message-ID: <200502071323.48845.silfreed@silfreed.net> On Monday 07 February 2005 13:06, Douglas E. Warner wrote: > Anyone know of e2fsprogs 1.36 RPMs for FC3? I don't see any in > development, yet. > I managed to find the Red Hat bug detailing this problem with numerous others exeperiencing it: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=144771 I guess I'll just download and compile the e2fsprogs 1.36 tarball and use that to repair my filesystem for now. Thanks for all the help! -Doug -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From silfreed at silfreed.net Mon Feb 7 23:35:00 2005 From: silfreed at silfreed.net (Douglas E. Warner) Date: Mon, 7 Feb 2005 18:35:00 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <20050207175624.GB8014@thunk.org> References: <200502062008.23249.silfreed@silfreed.net> <20050207175624.GB8014@thunk.org> Message-ID: <200502071835.00393.silfreed@silfreed.net> On Monday 07 February 2005 12:56, you wrote: > ? ? Optionally, the ext2prepare command can be used to re-enable online > ? ? resizing after the filesystem has been fixed. I can't seem to find any reference to the 'ext2prepare' command other than in the release notes for 1.36. Any other tips on how to set this back up? -Doug [root at thor e2fsprogs-1.36]# grep -ri ext2prepare * RELEASE-NOTES:Optionally, the ext2prepare command can be used to re-enable online -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tytso at mit.edu Tue Feb 8 05:52:38 2005 From: tytso at mit.edu (Theodore Ts'o) Date: Tue, 8 Feb 2005 00:52:38 -0500 Subject: e2fsck errors after lvextend when trying to resize2fs In-Reply-To: <200502071835.00393.silfreed@silfreed.net> References: <200502062008.23249.silfreed@silfreed.net> <20050207175624.GB8014@thunk.org> <200502071835.00393.silfreed@silfreed.net> Message-ID: <20050208055238.GB9879@thunk.org> On Mon, Feb 07, 2005 at 06:35:00PM -0500, Douglas E. Warner wrote: > On Monday 07 February 2005 12:56, you wrote: > > ? ? Optionally, the ext2prepare command can be used to re-enable online > > ? ? resizing after the filesystem has been fixed. > > I can't seem to find any reference to the 'ext2prepare' command other than in > the release notes for 1.36. Any other tips on how to set this back up? I believe Red Hat included ext2prepare in their hacked version of e2fsprogs. It originally came from the user space utilities that Andreas Dilger put together. You only need it if you plan to try to use the on-line resizing feature. I haven't included the ext2prepare program in the official e2fsprogs release because whenever I look at the code my fingers itch to rewrite large portions of it, and I haven't had time to do that yet. - Ted From adilger at clusterfs.com Tue Feb 8 09:26:25 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Tue, 8 Feb 2005 02:26:25 -0700 Subject: mke2fs options for very large filesystems In-Reply-To: <1107799111.3307.3.camel@localhost.localdomain> References: <1107799111.3307.3.camel@localhost.localdomain> Message-ID: <20050208092625.GF2635@schnapps.adilger.int> On Feb 07, 2005 09:58 -0800, Jeffrey W. Baker wrote: > Wow, it takes a really long time to make a 2TB ext2fs. Are there > better-than-default options that could be used for a large filesystem? > > mke2fs 1.34 (25-Jul-2003) > Filesystem label= > OS type: Linux > Block size=4096 (log=2) > Fragment size=4096 (log=2) > 244203520 inodes, 488382016 blocks > 24419100 blocks (5.00%) reserved for the super user > First data block=0 > 14905 block groups > 32768 blocks per group, 32768 fragments per group > 16384 inodes per group > Superblock backups stored on blocks: > 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, > 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, > 102400000, 214990848 Yes, if you are creating larger files. By default e2fsck assumes the average file size is 8kB and allocates a corresponding number of inodes there. If, for example, you are storing lots of larger files there (digital photos, MP3s, etc) that are in the MB range you can use "-t largefile" or "-t largefile4" to specify an average file size of 1MB or 4MB respectively. You can also use -i or -N (see man page) to override the default bytes-per-inode value. This will also speed up e2fsck noticably. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From kwijibo at zianet.com Tue Feb 8 15:00:16 2005 From: kwijibo at zianet.com (kwijibo at zianet.com) Date: Tue, 08 Feb 2005 08:00:16 -0700 Subject: mke2fs options for very large filesystems In-Reply-To: <20050208092625.GF2635@schnapps.adilger.int> References: <1107799111.3307.3.camel@localhost.localdomain> <20050208092625.GF2635@schnapps.adilger.int> Message-ID: <4208D400.5010808@zianet.com> Just as a side note but do you really want to reserve 5% of your 2TB partition for root? Another side question on this. Is it possible to make mkfs accept non-integer values for the reserved percentage? It is getting to the point these days where partitions are getting so big that even 1% is becoming quite a waste. However it would still be good to have some space reserved. Andreas Dilger wrote: > On Feb 07, 2005 09:58 -0800, Jeffrey W. Baker wrote: > >>Wow, it takes a really long time to make a 2TB ext2fs. Are there >>better-than-default options that could be used for a large filesystem? >> >>mke2fs 1.34 (25-Jul-2003) >>Filesystem label= >>OS type: Linux >>Block size=4096 (log=2) >>Fragment size=4096 (log=2) >>244203520 inodes, 488382016 blocks >>24419100 blocks (5.00%) reserved for the super user >>First data block=0 >>14905 block groups >>32768 blocks per group, 32768 fragments per group >>16384 inodes per group >>Superblock backups stored on blocks: >> 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, >> 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, >> 102400000, 214990848 > > > Yes, if you are creating larger files. By default e2fsck assumes the average > file size is 8kB and allocates a corresponding number of inodes there. If, > for example, you are storing lots of larger files there (digital photos, MP3s, > etc) that are in the MB range you can use "-t largefile" or "-t largefile4" > to specify an average file size of 1MB or 4MB respectively. You can also > use -i or -N (see man page) to override the default bytes-per-inode value. > This will also speed up e2fsck noticably. > > Cheers, Andreas > -- > Andreas Dilger > http://sourceforge.net/projects/ext2resize/ > http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ > > > > ------------------------------------------------------------------------ > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users From evilninja at gmx.net Tue Feb 8 17:47:52 2005 From: evilninja at gmx.net (_evil) Date: Tue, 08 Feb 2005 18:47:52 +0100 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050204164527.GA24817@alea.gnuu.de> References: <20050204164527.GA24817@alea.gnuu.de> Message-ID: <4208FB48.90300@gmx.net> Joerg Sommer wrote: > Hi, > > I've run many time e2fsck, but in a special dir ls tells me: > ls: r?cksendung-wlan.dvi: No such file or directory > ls: baf?g_r?ckmeldung.latex: No such file or directory > ls: finpr?f.pdf: No such file or directory fsck doesn't complain and what you see here are most likely some weird charset issues. > $ cat finpr?f.pdf > cat: finpr?f.pdf: Datei oder Verzeichnis nicht gefunden you're tools can't handle umlauts and thus fails to do anything with files containing umlauts. i'm not good [1] at charset issues either, but in debian the packages "locales" and "util-linux-locales" seems to be responsible for all that stuff. Christian. [1] an ugly putty session... evil at sheep:~$ touch ?ml?ute.txt evil at sheep:~$ ls ?ml?ute.txt ??ml??ute.txt From adilger at clusterfs.com Tue Feb 8 18:05:47 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Tue, 8 Feb 2005 11:05:47 -0700 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050204164527.GA24817@alea.gnuu.de> References: <20050204164527.GA24817@alea.gnuu.de> Message-ID: <20050208180547.GH2635@schnapps.adilger.int> On Feb 04, 2005 17:45 +0100, Joerg Sommer wrote: > I've run many time e2fsck, but in a special dir ls tells me: > ls: r?cksendung-wlan.dvi: No such file or directory > ls: baf?g_r?ckmeldung.latex: No such file or directory > ls: finpr?f.pdf: No such file or directory > > $ cat finpr?f.pdf > cat: finpr?f.pdf: Datei oder Verzeichnis nicht gefunden > > I don't know what to do? How can I find the failure? If I cat the files > with debugfs, I see the their contents. When you run e2fsck, you use "e2fsck -f"? Otherwise it will just check the superblock for ext3 filesystems and report it to be clean. Also, if you run "\ls {dir}" does it display the files? If yes, then in debugfs run "stat {file}" and show the output here. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From adilger at clusterfs.com Tue Feb 8 19:59:53 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Tue, 8 Feb 2005 12:59:53 -0700 Subject: mke2fs options for very large filesystems In-Reply-To: <4208D400.5010808@zianet.com> References: <1107799111.3307.3.camel@localhost.localdomain> <20050208092625.GF2635@schnapps.adilger.int> <4208D400.5010808@zianet.com> Message-ID: <20050208195953.GI2635@schnapps.adilger.int> On Feb 08, 2005 08:00 -0700, kwijibo at zianet.com wrote: > Just as a side note but do you really want to reserve 5% > of your 2TB partition for root? > > Another side question on this. Is it possible to make mkfs > accept non-integer values for the reserved percentage? > It is getting to the point these days where partitions > are getting so big that even 1% is becoming quite a waste. > However it would still be good to have some space reserved. The value is stored in the kernel as a blocks count, so it is possible to store any value there. It should be possible to change the mke2fs and tune2fs option parsing to accept a float there. I think you can set this manually via debugfs and "ssv" (in the latest e2fsprogs there is also a command to set a single superblock value). Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From greg.martin at ddos.com Tue Feb 8 20:38:52 2005 From: greg.martin at ddos.com (greg.martin at ddos.com) Date: Tue, 08 Feb 2005 21:38:52 +0100 Subject: Ext3 Journal corruption on hitachi deskstars Message-ID: I recently came across an enormous cluster of x86 clone machines running fedora core 1 (2.4.24) which have typically all intel or amd have VIA IDE chipsets. They frequently experience corrupted journals rendering the ext3 partition in read-only mode. More important than recovering the filesystem, I am interested in finding the root of the problem. The common hardware that all of these machines share is a Hitachi deskstar drive (New IBM drive)... Searching through the mailing list and google I have seen references to problems with this particular drive but for the life of me and (the broken redhat search tool) I cannot pin down specific information on the cause of these failures. A bug within the IDE bus driver and the harddrive, seems most sensible. Please help me find the right place to look. Thanks in advance, -Greg -- Greg Martin Sr. Information Security Engineer Melior Inc. www.ddos.com From evilninja at gmx.net Wed Feb 9 00:25:49 2005 From: evilninja at gmx.net (Christian) Date: Wed, 09 Feb 2005 01:25:49 +0100 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050208234732.GB8405@alea.gnuu.de> References: <20050204164527.GA24817@alea.gnuu.de> <4208FB48.90300@gmx.net> <20050208234732.GB8405@alea.gnuu.de> Message-ID: <4209588D.5080204@gmx.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 J?rg Sommer schrieb: > > But I can see other files with umlauts and can open them. And other > programs like firefox, openoffice, jed can't show them. hm, i still doubt there are fs problems here. other programs might be umlaut-aware, some others are not. try to cp to another name and try to open the file then: $ mkdir new $ cp ?ml?ut.txt new/umlaut.txt (use wildcards for the umlauts: *uml*ut.txt) $ diff *uml*ut.txt new/umlaut.txt you could try and move the files to another filesystem, then try to open it with the same tools (on the same machine!) Christian. - -- BOFH excuse #161: monitor VLF leakage -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCCViMC/PVm5+NVoYRAlnWAJ4yYLo81mzH8CovXMGFyoePOANSAwCgju4U TpdHkG+Tpr8dUbs6NScqJ54= =4UZD -----END PGP SIGNATURE----- From evilninja at gmx.net Wed Feb 9 00:17:48 2005 From: evilninja at gmx.net (Christian) Date: Wed, 09 Feb 2005 01:17:48 +0100 Subject: Ext3 Journal corruption on hitachi deskstars In-Reply-To: References: Message-ID: <420956AC.8010502@gmx.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 greg.martin at ddos.com schrieb: > They frequently experience corrupted journals rendering the ext3 > partition in read-only mode. More important than recovering the > filesystem, I am interested in finding the root of the problem. > The common hardware that all of these machines share is a Hitachi > deskstar drive (New IBM drive)... maybe you can elaborate a bit more on the "corrupted journals": what does "fsck" say, what's in the kernel log (during mount). if we know the symptoms, perhaps someone can find the root of the problem... Christian. - -- BOFH excuse #161: monitor VLF leakage -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCCVasC/PVm5+NVoYRAlDHAKDhp8Ep0S2J9NO8xfewnOMKdo1TEgCgsvhW 6r5zEZIJbznNz7+qd2X67VI= =GavO -----END PGP SIGNATURE----- From tytso at mit.edu Wed Feb 9 02:03:22 2005 From: tytso at mit.edu (Theodore Ts'o) Date: Tue, 8 Feb 2005 21:03:22 -0500 Subject: mke2fs options for very large filesystems In-Reply-To: <20050208195953.GI2635@schnapps.adilger.int> References: <1107799111.3307.3.camel@localhost.localdomain> <20050208092625.GF2635@schnapps.adilger.int> <4208D400.5010808@zianet.com> <20050208195953.GI2635@schnapps.adilger.int> Message-ID: <20050209020322.GA6663@thunk.org> On Tue, Feb 08, 2005 at 12:59:53PM -0700, Andreas Dilger wrote: > On Feb 08, 2005 08:00 -0700, kwijibo at zianet.com wrote: > > Just as a side note but do you really want to reserve 5% > > of your 2TB partition for root? The filesystem performance will degrade if you eat into the last 5%, but as long as you're willing to live with this tradeoff, and assuming you don't need to reserve space for things like log files, then sure, you can always drop the percentage. > > Another side question on this. Is it possible to make mkfs > > accept non-integer values for the reserved percentage? > > It is getting to the point these days where partitions > > are getting so big that even 1% is becoming quite a waste. > > However it would still be good to have some space reserved. > > The value is stored in the kernel as a blocks count, so it is possible > to store any value there. It should be possible to change the mke2fs > and tune2fs option parsing to accept a float there. I think you can > set this manually via debugfs and "ssv" (in the latest e2fsprogs there > is also a command to set a single superblock value). The officially supported to set a specific number of reserved blocks is tune2fs -r Since mke2fs sometimes is used in install floppies, etc., I decided there was no point to drag in floating point routines, especially when you can always set the number of reserved blocks directly using tune2fs -r after you create the filesystem. - Ted From joerg at alea.gnuu.de Tue Feb 8 23:44:49 2005 From: joerg at alea.gnuu.de (=?iso-8859-1?Q?J=F6rg?= Sommer) Date: Wed, 9 Feb 2005 00:44:49 +0100 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050208180547.GH2635@schnapps.adilger.int> References: <20050204164527.GA24817@alea.gnuu.de> <20050208180547.GH2635@schnapps.adilger.int> Message-ID: <20050208234449.GA8405@alea.gnuu.de> Andreas Dilger schrieb am Tue 08. Feb, 11:05 (-0700) : > On Feb 04, 2005 17:45 +0100, Joerg Sommer wrote: > > I've run many time e2fsck, but in a special dir ls tells me: > > ls: r?cksendung-wlan.dvi: No such file or directory > > ls: baf?g_r?ckmeldung.latex: No such file or directory > > ls: finpr?f.pdf: No such file or directory > > > > $ cat finpr?f.pdf > > cat: finpr?f.pdf: Datei oder Verzeichnis nicht gefunden > > > > I don't know what to do? How can I find the failure? If I cat the files > > with debugfs, I see the their contents. > > When you run e2fsck, you use "e2fsck -f"? Otherwise it will just check Yes. -C 0 -D -f -y > Also, if you run "\ls {dir}" does it display the files? If yes, then > in debugfs run "stat {file}" and show the output here. Where? At the command line this fails, but in debugfs ls prints all files and stat prints: debugfs: stat baf?g_r?ckmeldung.latex Inode: 734270 Type: regular Mode: 0644 Flags: 0x0 Generation: 210778105 User: 1000 Group: 100 Size: 715 File ACL: 0 Directory ACL: 0 Links: 1 Blockcount: 2 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x40d85922 -- Tue Jun 22 18:06:58 2004 atime: 0x41f8b83f -- Thu Jan 27 10:45:35 2005 mtime: 0x40644750 -- Fri Mar 26 16:08:00 2004 BLOCKS: (0):14465723 TOTAL: 1 debugfs: stat finpr?f.pdf Inode: 519716 Type: regular Mode: 0644 Flags: 0x0 Generation: 2062801645 User: 1000 Group: 100 Size: 28514 File ACL: 0 Directory ACL: 0 Links: 1 Blockcount: 58 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x41f8c231 -- Thu Jan 27 11:28:01 2005 atime: 0x41f8c16d -- Thu Jan 27 11:24:45 2005 mtime: 0x41f82aa0 -- Thu Jan 27 00:41:20 2005 BLOCKS: (0):10237961, (1-5):10240535-10240539, (6-11):10388563-10388568, (IND):10388569, (12-18):10388570-10388576, (19 TOTAL: 29 debugfs: stat r?cksendung-wlan.dvi Inode: 54501 Type: regular Mode: 0644 Flags: 0x0 Generation: 2068632333 User: 1000 Group: 100 Size: 1688 File ACL: 0 Directory ACL: 0 Links: 1 Blockcount: 4 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x41f6d9b4 -- Wed Jan 26 00:43:48 2005 atime: 0x41f8b83f -- Thu Jan 27 10:45:35 2005 mtime: 0x416af7ac -- Mon Oct 11 23:14:20 2004 BLOCKS: (0-1):1080841-1080842 TOTAL: 2 Here is stat of a file that I see at the command line: debugfs: stat baf?g_r?ckmeldung.dvi Inode: 734276 Type: regular Mode: 0644 Flags: 0x0 Generation: 210778111 User: 1000 Group: 100 Size: 1412 File ACL: 0 Directory ACL: 0 Links: 1 Blockcount: 4 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x40d85922 -- Tue Jun 22 18:06:58 2004 atime: 0x41f8b843 -- Thu Jan 27 10:45:39 2005 mtime: 0x40644750 -- Fri Mar 26 16:08:00 2004 BLOCKS: (0-1):14465734-14465735 TOTAL: 2 J?rg. -- at lilo press tab key | an Luftmatratzenpressetabulatorschl?ssel (?bersetzung von Personal Translator 2000) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From joerg at alea.gnuu.de Tue Feb 8 23:47:32 2005 From: joerg at alea.gnuu.de (=?iso-8859-1?Q?J=F6rg?= Sommer) Date: Wed, 9 Feb 2005 00:47:32 +0100 Subject: Failures they e2fsck doesn't find In-Reply-To: <4208FB48.90300@gmx.net> References: <20050204164527.GA24817@alea.gnuu.de> <4208FB48.90300@gmx.net> Message-ID: <20050208234732.GB8405@alea.gnuu.de> _evil schrieb am Tue 08. Feb, 18:47 (+0100) : > Joerg Sommer wrote: > >$ cat finpr?f.pdf > >cat: finpr?f.pdf: Datei oder Verzeichnis nicht gefunden > > you're tools can't handle umlauts and thus fails to do anything with > files containing umlauts. i'm not good [1] at charset issues either, but > in debian the packages "locales" and "util-linux-locales" seems to be > responsible for all that stuff. But I can see other files with umlauts and can open them. And other programs like firefox, openoffice, jed can't show them. J?rg. -- Das Recht, seine Meinung zu wechseln, ist eines der wichtigsten menschlichen Previlegien. (Robert Peel) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From sct at redhat.com Wed Feb 9 13:39:24 2005 From: sct at redhat.com (Stephen C. Tweedie) Date: Wed, 09 Feb 2005 13:39:24 +0000 Subject: Ext3 Journal corruption on hitachi deskstars In-Reply-To: References: Message-ID: <1107956364.1949.35.camel@sisko.sctweedie.blueyonder.co.uk> Hi, On Tue, 2005-02-08 at 20:38, greg.martin at ddos.com wrote: > I recently came across an enormous cluster of x86 clone machines running > fedora core 1 (2.4.24) which have typically all intel or amd have VIA IDE > chipsets. > > They frequently experience corrupted journals rendering the ext3 partition > in read-only mode. When ext3 first takes a filesystem offline/readonly, it always logs an initial message explaining why it's doing so. You'll need to capture that log message. Obviously, if it's the filesystem containing /var/ that went readonly, then /var/log/messages won't contain the information; but "dmesg" may, and you can set up serial or network console to capture it. --Stephen From jeff at jettis.com Wed Feb 9 14:07:41 2005 From: jeff at jettis.com (Jeff Dinisco) Date: Wed, 9 Feb 2005 06:07:41 -0800 Subject: mke2fs options for very large filesystems Message-ID: Ted, Could you explain why filesystem performance will degrade when the last 5% is used? I have several 1TB filesystems w/ only 1% reserved. The man page describes this space as "reserved for the super-user". I always assumed this was a buffer for root to be able to perform operations on a full filesystem. It seems as though 10GB is more than enough. Is this not the case? Thanks. - Jeff -----Original Message----- From: ext3-users-bounces at redhat.com [mailto:ext3-users-bounces at redhat.com] On Behalf Of Theodore Ts'o Sent: Tuesday, February 08, 2005 9:03 PM To: kwijibo at zianet.com; ext3 users list Subject: Re: mke2fs options for very large filesystems On Tue, Feb 08, 2005 at 12:59:53PM -0700, Andreas Dilger wrote: > On Feb 08, 2005 08:00 -0700, kwijibo at zianet.com wrote: > > Just as a side note but do you really want to reserve 5% > > of your 2TB partition for root? The filesystem performance will degrade if you eat into the last 5%, but as long as you're willing to live with this tradeoff, and assuming you don't need to reserve space for things like log files, then sure, you can always drop the percentage. > > Another side question on this. Is it possible to make mkfs > > accept non-integer values for the reserved percentage? > > It is getting to the point these days where partitions > > are getting so big that even 1% is becoming quite a waste. > > However it would still be good to have some space reserved. > > The value is stored in the kernel as a blocks count, so it is possible > to store any value there. It should be possible to change the mke2fs > and tune2fs option parsing to accept a float there. I think you can > set this manually via debugfs and "ssv" (in the latest e2fsprogs there > is also a command to set a single superblock value). The officially supported to set a specific number of reserved blocks is tune2fs -r Since mke2fs sometimes is used in install floppies, etc., I decided there was no point to drag in floating point routines, especially when you can always set the number of reserved blocks directly using tune2fs -r after you create the filesystem. - Ted _______________________________________________ Ext3-users mailing list Ext3-users at redhat.com https://www.redhat.com/mailman/listinfo/ext3-users From adilger at clusterfs.com Wed Feb 9 18:26:57 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Wed, 9 Feb 2005 11:26:57 -0700 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050208234449.GA8405@alea.gnuu.de> References: <20050204164527.GA24817@alea.gnuu.de> <20050208180547.GH2635@schnapps.adilger.int> <20050208234449.GA8405@alea.gnuu.de> Message-ID: <20050209182657.GO2635@schnapps.adilger.int> On Feb 09, 2005 00:44 +0100, J?rg Sommer wrote: > Andreas Dilger schrieb am Tue 08. Feb, 11:05 (-0700) : > > Also, if you run "\ls {dir}" does it display the files? If yes, then > > in debugfs run "stat {file}" and show the output here. > > Where? At the command line this fails, but in debugfs ls prints all files > and stat prints: This is from the command-line. Note specifically the '\' before ls, so that you don't get anything like color-ls trying to stat all of the files. This would tell us if the files are being found with readdir (they almost certainly are) and that the problem is during lookup. I assume if you do "ls -l {filename}" it fails. If you run "tune2fs -O ^dir_index" does the ls work again? Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From joerg at alea.gnuu.de Thu Feb 10 13:40:03 2005 From: joerg at alea.gnuu.de (=?iso-8859-1?Q?J=F6rg?= Sommer) Date: Thu, 10 Feb 2005 14:40:03 +0100 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050209182657.GO2635@schnapps.adilger.int> References: <20050204164527.GA24817@alea.gnuu.de> <20050208180547.GH2635@schnapps.adilger.int> <20050208234449.GA8405@alea.gnuu.de> <20050209182657.GO2635@schnapps.adilger.int> Message-ID: <20050210134003.GA14472@alea.gnuu.de> Andreas Dilger schrieb am Wed 09. Feb, 11:26 (-0700) : > On Feb 09, 2005 00:44 +0100, J?rg Sommer wrote: > > Andreas Dilger schrieb am Tue 08. Feb, 11:05 (-0700) : > > > Also, if you run "\ls {dir}" does it display the files? If yes, then > > > in debugfs run "stat {file}" and show the output here. > > > > Where? At the command line this fails, but in debugfs ls prints all files > > and stat prints: > > This is from the command-line. Note specifically the '\' before ls, so > that you don't get anything like color-ls trying to stat all of the files. There are no errors with \ls. > This would tell us if the files are being found with readdir (they almost > certainly are) and that the problem is during lookup. I assume if you do > "ls -l {filename}" it fails. $ stat archiv/text/finpr?f.pdf stat: Aufruf von stat f?r ,,archiv/text/finpr?f.pdf" nicht m?glich: Datei oder Verzeichnis nicht gefunden $ ls -l archiv/text/finpr?f.pdf ls: archiv/text/finpr?f.pdf: Datei oder Verzeichnis nicht gefunden > If you run "tune2fs -O ^dir_index" does the ls work again? yes. Can I add the index again? Thanks, Joerg. -- Viele Leute glauben, da? sie denken, wenn sie lediglich ihre Vorurteile neu ordnen. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From adilger at clusterfs.com Thu Feb 10 17:23:41 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Thu, 10 Feb 2005 10:23:41 -0700 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050210134003.GA14472@alea.gnuu.de> References: <20050204164527.GA24817@alea.gnuu.de> <20050208180547.GH2635@schnapps.adilger.int> <20050208234449.GA8405@alea.gnuu.de> <20050209182657.GO2635@schnapps.adilger.int> <20050210134003.GA14472@alea.gnuu.de> Message-ID: <20050210172341.GS2635@schnapps.adilger.int> On Feb 10, 2005 14:40 +0100, J?rg Sommer wrote: > > This is from the command-line. Note specifically the '\' before ls, so > > that you don't get anything like color-ls trying to stat all of the files. > > There are no errors with \ls. Ok, this means there is a bug with the htree indexing, possibly due to the use of 8-bit characters (though I can't see anything obvious in that regard). It might be that e2fsck is indexing them incorrectly, or the kernel, or both. > > If you run "tune2fs -O ^dir_index" does the ls work again? > > yes. > > Can I add the index again? Yes, "tune2fs -O dir_index". If you haven't created any large directories or added files to large directories since it was turned off then there isn't anything else to do (the htree indexing maintains internal consistency even when disabled, at worst reverting to linear searching until "e2fsck -fD" is run). After you do that can you run the following: mkdir {new_dir} cd {old_dir} \ls | (cd {new_dir}; xargs touch) and then debugfs {dev} debugfs> htree {old_dir} debugfs> htree {new_dir} What is of interest is whether the hash values of any of the entries are different (which would imply that e2fsck and the kernel disagree in the hash value of some entry). In particular, the kernel is looking for hash value H in a bucket (dir leaf block) that spans from (H-m) to (H+n) and isn't finding it, but a linear directory search IS finding it. Unfortunately, the debugfs "hash" function only calculates the dx_hash, and your filesystem almost certainly isn't using that. We probably need a new "tea_hash" and "md4_2_hash" commands, and then "hash" by default picks the current fs hash value (the seed optionally being specified for each command). Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tytso at mit.edu Thu Feb 10 17:55:28 2005 From: tytso at mit.edu (Theodore Ts'o) Date: Thu, 10 Feb 2005 12:55:28 -0500 Subject: mke2fs options for very large filesystems In-Reply-To: References: Message-ID: <20050210175528.GA10041@thunk.org> On Wed, Feb 09, 2005 at 06:07:41AM -0800, Jeff Dinisco wrote: > > Could you explain why filesystem performance will degrade when the last > 5% is used? I have several 1TB filesystems w/ only 1% reserved. The > man page describes this space as "reserved for the super-user". I > always assumed this was a buffer for root to be able to perform > operations on a full filesystem. It seems as though 10GB is more than > enough. Is this not the case? There are two reasons for the reserve. One is to reserve space on the partition containing /var and /etc for log files, etc. The other is to avoid the performance degredation when the last 5-10% of the disk space is used. (BSD actually reserves 10% by default.) Given that the cost of a 200 GB disks is under $100 USD the cost of reserving 10GB is approximately $5.00 USD, or approximately the cost of a large cup of coffee at Starbucks. Why is does the filesystem performance degrade? Because as the last blocks of a filesystem are consumed the probability of filesystem fragmentation goes up dramatically? Exactly when this happens depends on the write patterns of the workload and the distribution of the file sizes on the filesystem, but on average this tends to happen when the utilization rises to 90-95%. - Ted From greg.martin at ddos.com Fri Feb 11 00:55:37 2005 From: greg.martin at ddos.com (greg.martin at ddos.com) Date: Fri, 11 Feb 2005 01:55:37 +0100 Subject: Ext3 Journal corruption on hitachi deskstars In-Reply-To: <1107956364.1949.35.camel@sisko.sctweedie.blueyonder.co.uk> References: <1107956364.1949.35.camel@sisko.sctweedie.blueyonder.co.uk> Message-ID: Had another one fail today so I was able to tarball the /var directory this is the warning that comes from dmesg: EXT2-fs warning (device hda3): ext2_fill_super: mounting ext3 filesystem as ext2 EXT2-fs warning: mounting fs with errors, running e2fsck is recommended nothing much there... the console during the crash was flooding with this error: EXT-fs error (device ide(3,3)) in start)transation: Journal has aborted not much there... I noticed more google references to problems with Hitachi drives, the only thing thats common on these systems that are having the failures are the drive types and that they are all running some Redhat derivative fedora core 1, RHEL or Centos. Kernel is 2.4.24, some tech's claim you can wget flood apache for a few hours at 100mbps and cause this filesystem crash... Maybe a bug in the IDE driver? I just figured if I mentioned ext3 crash and hitachi in the same sentance someone would come back with a kernel patch for me but im never that lucky. Regards, Greg Stephen C. Tweedie writes: > Hi, > > On Tue, 2005-02-08 at 20:38, greg.martin at ddos.com wrote: >> I recently came across an enormous cluster of x86 clone machines running >> fedora core 1 (2.4.24) which have typically all intel or amd have VIA IDE >> chipsets. >> >> They frequently experience corrupted journals rendering the ext3 partition >> in read-only mode. > > When ext3 first takes a filesystem offline/readonly, it always logs an > initial message explaining why it's doing so. You'll need to capture > that log message. > > Obviously, if it's the filesystem containing /var/ that went readonly, > then /var/log/messages won't contain the information; but "dmesg" may, > and you can set up serial or network console to capture it. > > --Stephen > > From adail at estg.ipleiria.pt Sat Feb 12 16:59:26 2005 From: adail at estg.ipleiria.pt (=?iso-8859-1?Q?Ada=EDl_Oliveira?=) Date: Sat, 12 Feb 2005 16:59:26 -0000 Subject: Recover from mkfs -t ext3 Message-ID: <200502121659.j1CGxQve004256@mail.estg.ipleiria.pt> Hi, I ran mkfs -t ext3 in the wrong disk (/dev/sdc) and change de label of this partition (/dev/sdc1). It's possible to recover all data? Thanks A.O From joerg at alea.gnuu.de Sun Feb 13 21:26:39 2005 From: joerg at alea.gnuu.de (=?iso-8859-1?Q?J=F6rg?= Sommer) Date: Sun, 13 Feb 2005 22:26:39 +0100 Subject: Failures they e2fsck doesn't find In-Reply-To: <20050210172341.GS2635@schnapps.adilger.int> References: <20050204164527.GA24817@alea.gnuu.de> <20050208180547.GH2635@schnapps.adilger.int> <20050208234449.GA8405@alea.gnuu.de> <20050209182657.GO2635@schnapps.adilger.int> <20050210134003.GA14472@alea.gnuu.de> <20050210172341.GS2635@schnapps.adilger.int> Message-ID: <20050213212639.GA24996@alea.gnuu.de> Andreas Dilger schrieb am Thu 10. Feb, 10:23 (-0700) : > On Feb 10, 2005 14:40 +0100, J?rg Sommer wrote: > > > If you run "tune2fs -O ^dir_index" does the ls work again? > > > > yes. > > > > Can I add the index again? > > Yes, "tune2fs -O dir_index". If you haven't created any large directories Done. > After you do that can you run the following: > > mkdir {new_dir} > cd {old_dir} > \ls | (cd {new_dir}; xargs touch) > > and then > > debugfs {dev} > debugfs> htree {old_dir} Root node dump: Reserved zero: 0 Hash Version: 2 Info length: 8 Indirect levels: 0 Flags: 0 Number of entries (count): 2 Number of entries (limit): 124 Entry #0: Hash 0x00000000, block 1 Entry #1: Hash 0xb9114fc0, block 2 Entry #0: Hash 0x00000000, block 1 54501 0x088293ea (28) r?cksendung-wlan.dvi 734285 0x17aeabc8 (24) wasserwerk.latex 734252 0x1f55629c (24) Bewerbung.vor 1312 0x246cd2d4 (28) Bowlingpreise.sxw 734289 0x2e21b5e8 (28) hochzeitszeitung.txt 735073 0x30fd27b0 (16) steffi 519716 0x311a8596 (20) finpr?f.pdf 734270 0x31c25a06 (32) baf?g_r?ckmeldung.latex 734274 0x3238c734 (24) bkk-storno.dvi 734248 0x33c89cd4 (28) ASB-K?ndigung.sdw 734730 0x376f7cbe (16) Schule 734281 0x387f43be (24) wlan-plan.dvi 734269 0x3a82cb6a (16) lars.dvi 519707 0x4062be3c (36) darlehen-04-vertrag.latex 734253 0x48fa6bfa (28) Computerpreis.sdw 734287 0x491b3c3a (28) baf?g-?nderung.latex 1295 0x4a7ff138 (28) Getr?nkekarte.sxw 734263 0x4e435d4c (28) prog_alphabet.sdw 734277 0x534cbeb4 (28) darlehen-03.latex 734251 0x55ca18dc (16) wlan.dvi 734265 0x5d45e6f2 (28) thur k?ndigung.sdw 734268 0x5f508430 (20) lars.latex 734676 0x627516e6 (12) Inet 734255 0x693832e2 (28) ELV TempSensor.sdw 734256 0x6d176a3a (20) Selig2.sdw 734261 0x786912d6 (20) dorndorf.doc 734280 0x85e5d794 (24) wlan-plan.latex 734272 0x8819be28 (32) baf?g_ummeldung.latex 734657 0x89cb9c46 (16) spr?che 734735 0x8eb1a558 (16) Songs 734250 0x8f7e13cc (32) Allianz-K?ndigung.sdw 734254 0x91fef748 (28) ELV Interface.sdw 734273 0x92535b88 (24) bkk-storno.latex 734262 0x92b6b532 (24) idiotentest.sdw 734278 0x95f376da (24) darlehen-03.dvi 735103 0x9b1fdcc6 (16) lyrik 734279 0x9f547278 (20) kfz.latex 520001 0xa363e182 (12) 3m+1 734266 0xa8f58efe (32) Der saure Herrmann.sdw 734258 0xaa65d8f2 (24) Zivi Antrag.sdw 519714 0xab04fb7c (32) r?cksendung-etrade.dvi 734246 0xab082164 (40) Firma.sdw Entry #1: Hash 0xb9114fc0, block 2 734288 0xb9114fc0 (28) baf?g-?nderung.dvi 734257 0xba3bc724 (36) Versicherung-K?ndigung.sdw 734271 0xba9387ba (28) baf?g_ummeldung.dvi 734267 0xbf88efee (36) allianz r?ckforderung.sdw 54499 0xbff9f39e (32) r?cksendung-wlan.latex 734249 0xc02ce9de (32) Allianz-Diebstahl.sdw 734242 0xc0568f7c (16) Brigitte 734275 0xc0f16368 (16) kfz.dvi 735093 0xc4986dc0 (16) wurzel 734282 0xd613c70a (24) fax_ikj.latex 734286 0xd8a19672 (24) vollmacht.latex 734259 0xd90be2ae (28) Zivi Begr?ndung.sdw 519712 0xd9ef85e8 (32) r?cksendung-etrade.latex 734284 0xde7fa88e (24) wasserwerk.dvi 520417 0xe153df2e (28) geschichten-steffi 734276 0xe2b6c014 (32) baf?g_r?ckmeldung.dvi 519703 0xe872bd64 (28) brief-mutti.latex 734283 0xec40ae42 (20) fax_ikj.dvi 519713 0xee0f70b4 (24) finpr?f.latex 734247 0xef7a85b4 (20) wlan.latex 519705 0xf33d9d70 (24) brief-mutti.dvi 1300 0xf848ef3a (24) Essenkarte.sxw 519710 0xf9d41054 (32) darlehen-04-vertrag.dvi 734260 0xfd142cc4 (28) Zivi Lebenslauf.sdw 734264 0xffadc8ae (392) studium lebenslauf.sdw --------------------- > debugfs> htree {new_dir} Root node dump: Reserved zero: 0 Hash Version: 2 Info length: 8 Indirect levels: 0 Flags: 0 Number of entries (count): 2 Number of entries (limit): 124 Entry #0: Hash 0x00000000, block 1 Entry #1: Hash 0x92535b88, block 2 Entry #0: Hash 0x00000000, block 1 46180 0x8f7e13cc (32) Allianz-K?ndigung.sdw 46182 0x33c89cd4 (28) ASB-K?ndigung.sdw 46183 0x491b3c3a (28) baf?g-?nderung.latex 46186 0x8819be28 (32) baf?g_ummeldung.latex 46187 0x1f55629c (24) Bewerbung.vor 46188 0x3238c734 (24) bkk-storno.dvi 46190 0x246cd2d4 (28) Bowlingpreise.sxw 46194 0x48fa6bfa (28) Computerpreis.sdw 46196 0x534cbeb4 (28) darlehen-03.latex 46198 0x4062be3c (36) darlehen-04-vertrag.latex 46200 0x786912d6 (20) dorndorf.doc 46201 0x91fef748 (28) ELV Interface.sdw 46202 0x693832e2 (28) ELV TempSensor.sdw 46209 0x4a7ff138 (28) Getr?nkekarte.sxw 46210 0x2e21b5e8 (28) hochzeitszeitung.txt 46212 0x627516e6 (12) Inet 46215 0x3a82cb6a (16) lars.dvi 46216 0x5f508430 (20) lars.latex 46218 0x4e435d4c (28) prog_alphabet.sdw 46222 0x376f7cbe (16) Schule 46223 0x6d176a3a (20) Selig2.sdw 46224 0x8eb1a558 (16) Songs 46225 0x89cb9c46 (16) spr?che 46226 0x30fd27b0 (16) steffi 46228 0x5d45e6f2 (28) thur k?ndigung.sdw 46231 0x17aeabc8 (24) wasserwerk.latex 46232 0x55ca18dc (16) wlan.dvi 46234 0x387f43be (24) wlan-plan.dvi 46235 0x85e5d794 (352) wlan-plan.latex Entry #1: Hash 0x92535b88, block 2 46178 0xa363e182 (12) 3m+1 46179 0xc02ce9de (32) Allianz-Diebstahl.sdw 46181 0xbf88efee (36) allianz r?ckforderung.sdw 46184 0xe2b6c014 (32) baf?g_r?ckmeldung.dvi 46185 0x31c25a06 (32) baf?g_r?ckmeldung.latex 46189 0x92535b88 (24) bkk-storno.latex 46191 0xf33d9d70 (24) brief-mutti.dvi 46192 0xe872bd64 (28) brief-mutti.latex 46193 0xc0568f7c (16) Brigitte 46195 0x95f376da (24) darlehen-03.dvi 46197 0xf9d41054 (32) darlehen-04-vertrag.dvi 46199 0xa8f58efe (32) Der saure Herrmann.sdw 46203 0xf848ef3a (24) Essenkarte.sxw 46204 0xec40ae42 (20) fax_ikj.dvi 46205 0xd613c70a (24) fax_ikj.latex 46206 0x311a8596 (20) finpr?f.pdf 46207 0xab082164 (20) Firma.sdw 46208 0xe153df2e (28) geschichten-steffi 46211 0x92b6b532 (24) idiotentest.sdw 46213 0xc0f16368 (16) kfz.dvi 46214 0x9f547278 (20) kfz.latex 46217 0x9b1fdcc6 (16) lyrik 46219 0xab04fb7c (32) r?cksendung-etrade.dvi 46220 0x088293ea (28) r?cksendung-wlan.dvi 46221 0xbff9f39e (32) r?cksendung-wlan.latex 46227 0xffadc8ae (32) studium lebenslauf.sdw 46229 0xd8a19672 (24) vollmacht.latex 46230 0xde7fa88e (24) wasserwerk.dvi 46233 0xef7a85b4 (20) wlan.latex 46236 0xc4986dc0 (16) wurzel 46237 0xaa65d8f2 (24) Zivi Antrag.sdw 46238 0xfd142cc4 (256) Zivi Lebenslauf.sdw --------------------- Here are the differences: --- htree.old3 2005-02-13 22:16:56.570095080 +0100 +++ htree.new3 2005-02-13 22:16:46.256662960 +0100 -0xb9114fc0 (28) baf?g-?nderung.dvi -0xba3bc724 (36) Versicherung-K?ndigung.sdw -0xba9387ba (28) baf?g_ummeldung.dvi -0xd90be2ae (28) Zivi Begr?ndung.sdw -0xd9ef85e8 (32) r?cksendung-etrade.latex -0xee0f70b4 (24) finpr?f.latex These are all files I don't see with ls in the old directory. $ \ls archiv/text/fi* ls: archiv/text/finpr?f.pdf: Datei oder Verzeichnis nicht gefunden debugfs: stat archiv/text/finpr?f.latex Inode: 519713 Type: regular Mode: 0644 Flags: 0x0 Generation: 2062801501 User: 1000 Group: 100 Size: 1693 File ACL: 0 Directory ACL: 0 Links: 1 Blockcount: 4 Fragment: Address: 0 Number: 0 Size: 0 ctime: 0x41f8c231 -- Thu Jan 27 11:28:01 2005 atime: 0x420b63d1 -- Thu Feb 10 14:38:25 2005 mtime: 0x41f82a9e -- Thu Jan 27 00:41:18 2005 BLOCKS: (0-1):10240526-10240527 TOTAL: 2 But the hash of finpr?f.pdf doesn't change. 519716 0x311a8596 (20) finpr?f.pdf 46206 0x311a8596 (20) finpr?f.pdf Hope I could help you finding the bug. Do you need anymore informations? Joerg. -- "Computer games don't affect kids. If Pacman would have affected us as children, we would now run around in darkened rooms, munching pills and listening to repetetive music." -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: From vijayan at cs.wisc.edu Mon Feb 14 04:45:42 2005 From: vijayan at cs.wisc.edu (Vijayan Prabhakaran) Date: Sun, 13 Feb 2005 22:45:42 -0600 (CST) Subject: Disk errors Message-ID: Hi, Well, this might not be the right place to pose this question. But, I'm sure that there are people who can answer this. My question is regarding how disk errors are reported to ext3. From my basic understanding of the block driver routines, it seems that all the different types of disk errors (like sector failures, scsi bus timeout, etc) are converted into one single error code (EIO) and passed on to the file system (by the ll_rw_blk.c) So, irrespective of whether the underlying device is a SCSI or IDE, the disk errors are going to be reported as EIO to ext3, right ? Does anyone have any input on how this occurs in other open source operating systems ? I appreciate any help regarding this. Thanks much. Vijayan From adobriyan at mail.ru Tue Feb 15 10:46:06 2005 From: adobriyan at mail.ru (Alexey Dobriyan) Date: Tue, 15 Feb 2005 12:46:06 +0200 Subject: [PATCH] ext3: Fix sparse -Wbitwise warnings. Message-ID: <200502151246.06598.adobriyan@mail.ru> Signed-off-by: Alexey Dobriyan --- fs/ext3/resize.c | 20 ++++++++++---------- fs/ext3/super.c | 8 ++++---- include/linux/ext3_fs.h | 2 +- 3 files changed, 15 insertions(+), 15 deletions(-) Index: linux-warnings/include/linux/ext3_fs.h =================================================================== --- linux-warnings/include/linux/ext3_fs.h (revision 14) +++ linux-warnings/include/linux/ext3_fs.h (revision 20) @@ -450,7 +450,7 @@ */ __u8 s_prealloc_blocks; /* Nr of blocks to try to preallocate*/ __u8 s_prealloc_dir_blocks; /* Nr to preallocate for dirs */ - __u16 s_reserved_gdt_blocks; /* Per group desc for online growth */ + __le16 s_reserved_gdt_blocks; /* Per group desc for online growth */ /* * Journaling support valid if EXT3_FEATURE_COMPAT_HAS_JOURNAL set. */ Index: linux-warnings/fs/ext3/super.c =================================================================== --- linux-warnings/fs/ext3/super.c (revision 14) +++ linux-warnings/fs/ext3/super.c (revision 20) @@ -2105,13 +2105,13 @@ ext3_mark_recovery_complete(sb, es); } else { - __le32 ret; - if ((ret = EXT3_HAS_RO_COMPAT_FEATURE(sb, - ~EXT3_FEATURE_RO_COMPAT_SUPP))) { + int ret; + if ((ret = le32_to_cpu(EXT3_HAS_RO_COMPAT_FEATURE(sb, + ~EXT3_FEATURE_RO_COMPAT_SUPP)))) { printk(KERN_WARNING "EXT3-fs: %s: couldn't " "remount RDWR because of unsupported " "optional features (%x).\n", - sb->s_id, le32_to_cpu(ret)); + sb->s_id, ret); return -EROFS; } /* Index: linux-warnings/fs/ext3/resize.c =================================================================== --- linux-warnings/fs/ext3/resize.c (revision 14) +++ linux-warnings/fs/ext3/resize.c (revision 20) @@ -328,7 +328,7 @@ unsigned five = 5; unsigned seven = 7; unsigned grp; - __u32 *p = (__u32 *)primary->b_data; + __le32 *p = (__le32 *)primary->b_data; int gdbackups = 0; while ((grp = ext3_list_backups(sb, &three, &five, &seven)) < end) { @@ -371,7 +371,7 @@ struct buffer_head *dind; int gdbackups; struct ext3_iloc iloc; - __u32 *data; + __le32 *data; int err; if (test_opt(sb, DEBUG)) @@ -408,7 +408,7 @@ goto exit_bh; } - data = (__u32 *)dind->b_data; + data = (__le32 *)dind->b_data; if (le32_to_cpu(data[gdb_num % EXT3_ADDR_PER_BLOCK(sb)]) != gdblock) { ext3_warning(sb, __FUNCTION__, "new group %u GDT block %lu not reserved\n", @@ -510,7 +510,7 @@ struct buffer_head *dind; struct ext3_iloc iloc; unsigned long blk; - __u32 *data, *end; + __le32 *data, *end; int gdbackups = 0; int res, i; int err; @@ -527,15 +527,15 @@ } blk = EXT3_SB(sb)->s_sbh->b_blocknr + 1 + EXT3_SB(sb)->s_gdb_count; - data = (__u32 *)dind->b_data + EXT3_SB(sb)->s_gdb_count; - end = (__u32 *)dind->b_data + EXT3_ADDR_PER_BLOCK(sb); + data = (__le32 *)dind->b_data + EXT3_SB(sb)->s_gdb_count; + end = (__le32 *)dind->b_data + EXT3_ADDR_PER_BLOCK(sb); /* Get each reserved primary GDT block and verify it holds backups */ for (res = 0; res < reserved_gdb; res++, blk++) { if (le32_to_cpu(*data) != blk) { ext3_warning(sb, __FUNCTION__, "reserved block %lu not at offset %ld\n", - blk, (long)(data - (__u32 *)dind->b_data)); + blk, (long)(data - (__le32 *)dind->b_data)); err = -EINVAL; goto exit_bh; } @@ -550,7 +550,7 @@ goto exit_bh; } if (++data >= end) - data = (__u32 *)dind->b_data; + data = (__le32 *)dind->b_data; } for (i = 0; i < reserved_gdb; i++) { @@ -574,7 +574,7 @@ blk = input->group * EXT3_BLOCKS_PER_GROUP(sb); for (i = 0; i < reserved_gdb; i++) { int err2; - data = (__u32 *)primary[i]->b_data; + data = (__le32 *)primary[i]->b_data; /* printk("reserving backup %lu[%u] = %lu\n", primary[i]->b_blocknr, gdbackups, blk + primary[i]->b_blocknr); */ @@ -675,7 +675,7 @@ "can't update backup for group %d (err %d), " "forcing fsck on next reboot\n", group, err); sbi->s_mount_state &= ~EXT3_VALID_FS; - sbi->s_es->s_state &= ~cpu_to_le16(EXT3_VALID_FS); + sbi->s_es->s_state &= cpu_to_le16(~EXT3_VALID_FS); mark_buffer_dirty(sbi->s_sbh); } } From sct at redhat.com Tue Feb 15 14:12:09 2005 From: sct at redhat.com (Stephen C. Tweedie) Date: Tue, 15 Feb 2005 14:12:09 +0000 Subject: [PATCH] ext3: Fix sparse -Wbitwise warnings. In-Reply-To: <200502151246.06598.adobriyan@mail.ru> References: <200502151246.06598.adobriyan@mail.ru> Message-ID: <1108476729.3363.9.camel@sisko.sctweedie.blueyonder.co.uk> Hi, On Tue, 2005-02-15 at 10:46, Alexey Dobriyan wrote: > - if ((ret = EXT3_HAS_RO_COMPAT_FEATURE(sb, > - ~EXT3_FEATURE_RO_COMPAT_SUPP))) { > + if ((ret = le32_to_cpu(EXT3_HAS_RO_COMPAT_FEATURE(sb, > + ~EXT3_FEATURE_RO_COMPAT_SUPP)))) { NAK. EXT3_HAS_RO_COMPAT_FEATURE returns a boolean value. It happens to be implemented internally as #define EXT3_HAS_COMPAT_FEATURE(sb,mask) \ ( EXT3_SB(sb)->s_es->s_feature_compat & cpu_to_le32(mask) ) so the compiler, looking at the preprocessed code, will reasonably assume it's a genuine little-endian value. But it's only used as a boolean, so we shouldn't be requiring the callers to provide an le32_to_cpu() conversion. If we want to fix this, let's fix the macros: for example, convert EXT3_HAS_COMPAT_FEATURE to be ( le32_to_cpu(EXT3_SB(sb)->s_es->s_feature_compat) & (mask) ) so that we're doing the tests in native CPU endian-ness. --Stephen From Ralf.Hildebrandt at charite.de Tue Feb 15 15:20:43 2005 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 15 Feb 2005 16:20:43 +0100 Subject: Oops in 2.6.10-ac12 in kjournald (journal_commit_transaction) Message-ID: <20050215152043.GR24211@charite.de> Today our mailserver froze after just one day of uptime. I was able to capture the Oops on the screen using my digital camera: http://www.stahl.bau.tu-bs.de/~hildeb/bugreport/ Keywords: EIP is at journal_commit_transaction, process kjournald # mount /dev/cciss/c0d0p6 on / type ext3 (rw,errors=remount-ro) proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) devpts on /dev/pts type devpts (rw,gid=5,mode=620) tmpfs on /dev/shm type tmpfs (rw) /dev/cciss/c0d0p5 on /boot type ext3 (rw) /dev/shm on /var/amavis type tmpfs (rw,noatime,size=200m,mode=770,uid=104,gid=108) -- Ralf Hildebrandt (i.A. des IT-Zentrum) Ralf.Hildebrandt at charite.de Charite - Universit?tsmedizin Berlin Tel. +49 (0)30-450 570-155 Gemeinsame Einrichtung von FU- und HU-Berlin Fax. +49 (0)30-450 570-962 IT-Zentrum Standort CBF send no mail to spamtrap at charite.de From adobriyan at mail.ru Tue Feb 15 18:13:21 2005 From: adobriyan at mail.ru (Alexey Dobriyan) Date: Tue, 15 Feb 2005 20:13:21 +0200 Subject: [PATCH] ext3: Fix sparse -Wbitwise warnings. In-Reply-To: <1108476729.3363.9.camel@sisko.sctweedie.blueyonder.co.uk> References: <200502151246.06598.adobriyan@mail.ru> <1108476729.3363.9.camel@sisko.sctweedie.blueyonder.co.uk> Message-ID: <200502152013.21556.adobriyan@mail.ru> On Tuesday 15 February 2005 16:12, Stephen C. Tweedie wrote: > On Tue, 2005-02-15 at 10:46, Alexey Dobriyan wrote: > > > - if ((ret = EXT3_HAS_RO_COMPAT_FEATURE(sb, > > - ~EXT3_FEATURE_RO_COMPAT_SUPP))) { > > + if ((ret = le32_to_cpu(EXT3_HAS_RO_COMPAT_FEATURE(sb, > > + ~EXT3_FEATURE_RO_COMPAT_SUPP)))) { > > NAK. Argh... stupid me. super.c part should be just: --- linux-2.6.11-rc4/fs/ext3/super.c.orig 2005-02-15 20:01:52.000000000 +0200 +++ linux-2.6.11-rc4/fs/ext3/super.c 2005-02-15 20:02:47.000000000 +0200 @@ -2106,6 +2106,7 @@ static int ext3_remount (struct super_bl ext3_mark_recovery_complete(sb, es); } else { __le32 ret; + int ret1; if ((ret = EXT3_HAS_RO_COMPAT_FEATURE(sb, ~EXT3_FEATURE_RO_COMPAT_SUPP))) { printk(KERN_WARNING "EXT3-fs: %s: couldn't " @@ -2122,8 +2123,8 @@ static int ext3_remount (struct super_bl */ ext3_clear_journal_err(sb, es); sbi->s_mount_state = le16_to_cpu(es->s_state); - if ((ret = ext3_group_extend(sb, es, n_blocks_count))) - return ret; + if ((ret1 = ext3_group_extend(sb, es, n_blocks_count))) + return ret1; if (!ext3_setup_super (sb, es, 0)) sb->s_flags &= ~MS_RDONLY; } From mitch at sfgoth.com Tue Feb 15 23:29:39 2005 From: mitch at sfgoth.com (Mitchell Blank Jr) Date: Tue, 15 Feb 2005 15:29:39 -0800 Subject: [PATCH] ext3: Fix sparse -Wbitwise warnings. In-Reply-To: <1108476729.3363.9.camel@sisko.sctweedie.blueyonder.co.uk> References: <200502151246.06598.adobriyan@mail.ru> <1108476729.3363.9.camel@sisko.sctweedie.blueyonder.co.uk> Message-ID: <20050215232939.GD16892@gaz.sfgoth.com> Stephen C. Tweedie wrote: > If we want to fix this, let's fix the macros: for example, convert > EXT3_HAS_COMPAT_FEATURE to be > > ( le32_to_cpu(EXT3_SB(sb)->s_es->s_feature_compat) & (mask) ) Of course that's less efficient though since "mask" is probably constant.. so now the endian conversion changed from compile-time to run-time. Would something like ( ( EXT3_SB(sb)->s_es->s_feature_compat & cpu_to_le32(mask) ) != 0) be enough to satisfy sparse? -Mitch From sct at redhat.com Tue Feb 15 23:25:53 2005 From: sct at redhat.com (Stephen C. Tweedie) Date: Tue, 15 Feb 2005 23:25:53 +0000 Subject: [PATCH] ext3: Fix sparse -Wbitwise warnings. In-Reply-To: <20050215232939.GD16892@gaz.sfgoth.com> References: <200502151246.06598.adobriyan@mail.ru> <1108476729.3363.9.camel@sisko.sctweedie.blueyonder.co.uk> <20050215232939.GD16892@gaz.sfgoth.com> Message-ID: <1108509952.9776.101.camel@sisko.sctweedie.blueyonder.co.uk> Hi, On Tue, 2005-02-15 at 23:29, Mitchell Blank Jr wrote: > Of course that's less efficient though since "mask" is probably constant.. > so now the endian conversion changed from compile-time to run-time. > > Would something like > > ( ( EXT3_SB(sb)->s_es->s_feature_compat & cpu_to_le32(mask) ) != 0) > > be enough to satisfy sparse? Yes, but it breaks other places: there are a few places where callers actually want the real return value from the "&", so that (for example) they can tell the user which feature failed the compatibility test. --Stephen From adilger at clusterfs.com Tue Feb 15 23:46:37 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Tue, 15 Feb 2005 16:46:37 -0700 Subject: [PATCH] ext3: Fix sparse -Wbitwise warnings. In-Reply-To: <20050215232939.GD16892@gaz.sfgoth.com> References: <200502151246.06598.adobriyan@mail.ru> <1108476729.3363.9.camel@sisko.sctweedie.blueyonder.co.uk> <20050215232939.GD16892@gaz.sfgoth.com> Message-ID: <20050215234637.GI27352@schnapps.adilger.int> On Feb 15, 2005 15:29 -0800, Mitchell Blank Jr wrote: > Stephen C. Tweedie wrote: > > If we want to fix this, let's fix the macros: for example, convert > > EXT3_HAS_COMPAT_FEATURE to be > > > > ( le32_to_cpu(EXT3_SB(sb)->s_es->s_feature_compat) & (mask) ) > > Of course that's less efficient though since "mask" is probably constant.. > so now the endian conversion changed from compile-time to run-time. > > Would something like > > ( ( EXT3_SB(sb)->s_es->s_feature_compat & cpu_to_le32(mask) ) != 0) > > be enough to satisfy sparse? Or we could cast "mask" to the appropriate type (I'm not sure what sparse uses to determine this). Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From pegasus at nerv.eu.org Wed Feb 16 01:36:58 2005 From: pegasus at nerv.eu.org (Jure =?ISO-8859-2?Q?Pe=5Far?=) Date: Wed, 16 Feb 2005 02:36:58 +0100 Subject: Oops in 2.6.8.1 at __journal_drop_transaction Message-ID: <20050216023658.5aecf4dc.pegasus@nerv.eu.org> Hi all, One of our mx servers started misbehaving today (postfix would timeout internally, load rising) and after I tried to reboot it, I got this: Assertion failure in __journal_drop_transaction() at fs/jbd/checkpoint.c:613: "transaction->t_forget == NULL" ------------[ cut here ]------------ kernel BUG at fs/jbd/checkpoint.c:613! invalid operand: 0000 [#1] PREEMPT SMP Modules linked in: ipv6 aic79xx serverworks eepro100 sworks_agp agpgart floppy evdev pcspkr ohci_hcd usbcore e100 mii capability commoncap ide_cd ide_core cdrom rtc ext2 ext3 jbd mbcache sd_mod aic7xxx scsi_mod raid1 md unix font vesafb cfbcopyarea cfbimgblt cfbfillrect CPU: 0 EIP: 0060:[] Not tainted EFLAGS: 00010286 (2.6.8-1-686-smp) EIP is at __journal_drop_transaction+0x350/0x3f7 [jbd] eax: 00000071 ebx: c86f5620 ecx: c02d9fbc edx: c02d9fbc esi: f7af5800 edi: e9d5583c ebp: c86f5620 esp: f70e3d58 ds: 007b es: 007b ss: 0068 Process kjournald (pid: 523, threadinfo=f70e2000 task=f70a37f0) Stack: f88ac120 f88ab380 f88acefc 00000265 f88acf4c c86f5620 f7af5800 f88a731a f7af5800 c86f5620 dbc24db0 00000000 f88a66e6 e9d5583c e9d5583c f70e2000 f88a72c8 e9d5583c e9d5583c 000000e1 c86f55c0 d865c500 f70e2000 00000000 Call Trace: [] __journal_remove_checkpoint+0x4a/0xa0 [jbd] [] __try_to_free_cp_buf+0x76/0xc0 [jbd] [] __journal_clean_checkpoint_list+0xa8/0xb0 [jbd] [] journal_commit_transaction+0x2b8/0x1690 [jbd] [] autoremove_wake_function+0x0/0x60 [] netif_receive_skb+0x1ba/0x230 [] autoremove_wake_function+0x0/0x60 [] net_tx_action+0x5a/0x160 [] recalc_task_prio+0xa8/0x1a0 [] schedule+0x4b7/0x8a0 [] del_timer_sync+0x9a/0xe0 [] kjournald+0xf2/0x2e0 [jbd] [] autoremove_wake_function+0x0/0x60 [] autoremove_wake_function+0x0/0x60 [] ret_from_fork+0x6/0x14 [] commit_timeout+0x0/0x10 [jbd] [] kjournald+0x0/0x2e0 [jbd] [] kernel_thread_helper+0x5/0x10 Code: 0f 0b 65 02 fc ce 8a f8 e9 6e fd ff ff 8d 76 00 c7 04 24 20 <6>note: kjournald[523] exited with preempt_count 3 The machine is still alive and kicking (and would not reboot even with reboot -f), most probably because the system is on one disk and postfix spool is on another one, that seems to be the cause of the problem. Relevant /etc/fstab line: /dev/sdc1 /var/spool/postfix ext3 rw,noatime,nodiratime,data=journal,commit=60 0 0 It also has a large journal, 400MB if I remember correctly. Debian Sarge. -- Jure Pe?ar http://jure.pecar.org/ From menscher at uiuc.edu Wed Feb 16 21:02:50 2005 From: menscher at uiuc.edu (Damian Menscher) Date: Wed, 16 Feb 2005 15:02:50 -0600 (CST) Subject: mke2fs options for very large filesystems (and corruption!) Message-ID: [sorry if this isn't threaded right... I just subscribed] Theodore Ts'o wrote: > > There are two reasons for the reserve. One is to reserve space on the > partition containing /var and /etc for log files, etc. The other is > to avoid the performance degredation when the last 5-10% of the disk > space is used. (BSD actually reserves 10% by default.) Given that > the cost of a 200 GB disks is under $100 USD the cost of reserving > 10GB is approximately $5.00 USD, or approximately the cost of a large > cup of coffee at Starbucks. > > Why is does the filesystem performance degrade? Because as the last > blocks of a filesystem are consumed the probability of filesystem > fragmentation goes up dramatically? Exactly when this happens depends > on the write patterns of the workload and the distribution of the file > sizes on the filesystem, but on average this tends to happen when the > utilization rises to 90-95%. I'm working on something similar -- we're hoping to create a single 3.5 TB ext3 filesystem. For us, 5% is 175 gig. Since we're using hot-swap scsi disks instead of ide, that comes out to well over $100. So we'll be going with 1-2%. Even then, we're leaving several gig available, which will probably be fine. When we fill the disk, I expect we'll be more concerned with having no free space, and less concerned with its performance. More importantly, I have some questions about whether ext3 can realistically handle a filesystem this large. Since fdisk created a DOS partition table, and that couldn't handle such a large partition, I used parted to create a gpt partition of 3.5TB. We were then able to mkfs and mount the filesystem. As a test, I filled it with: cp 1_gig_file 1_gig_file.### (where ### ranged from 000 to 999) and dd if=/dev/zero of=bigfile bs=10M I expected bigfile to crash out at 2TB, but actually it filled the filesystem (created a 2.7TB file). The next bit is slightly muddled: 09:50 deleted the bigfile (this took a LONG time) 10:04 pulled a drive on the raid array (hardware raid5) 10:12 errors started appearing in the logs, and the filesystem remounted itself read-only Here are some sample errors: Feb 16 10:12:05 hera kernel: EXT3-fs error (device sdb1): ext3_free_blocks: Freeing blocks not in datazone - block = 1027000215, count = 1 Feb 16 10:12:05 hera kernel: Aborting journal on device sdb1. Feb 16 10:12:05 hera kernel: EXT3-fs error (device sdb1): ext3_free_blocks: Freeing blocks not in datazone - block = 3696983906, count = 1 Feb 16 10:12:05 hera kernel: EXT3-fs error (device sdb1): ext3_free_blocks: Freeing blocks not in datazone - block = 2229194010, count = 1 Feb 16 10:12:05 hera kernel: EXT3-fs error (device sdb1): ext3_free_blocks: Freeing blocks not in datazone - block = 1172112249, count = 1 Feb 16 10:12:05 hera kernel: EXT3-fs error (device sdb1): ext3_free_blocks: Freeing blocks not in datazone - block = 3315908307, count = 1 Feb 16 10:12:05 hera kernel: ext3_free_blocks_sb: aborting transaction: Journal has aborted in __ext3_journal_get_undo_access<2>EXT3-fs error (device sdb1) in ext3_free_blocks_sb: Journal has aborted ... Feb 16 10:12:10 hera kernel: EXT3-fs error (device sdb1) in ext3_delete_inode: Journal has aborted Feb 16 10:12:10 hera kernel: __journal_remove_journal_head: freeing b_committed_data Feb 16 10:12:10 hera last message repeated 5 times Feb 16 10:12:10 hera kernel: ext3_abort called. Feb 16 10:12:10 hera kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal Feb 16 10:12:10 hera kernel: Remounting filesystem read-only My best guess of what happened here is that "bigfile" became larger than the maximum filesize (it was a 2.7TB file, but the max filesize for ext3 is 2.0TB). So when we deleted it, it probably deleted data blocks for lots of other files. Or at least tried to... maybe these logs indicate failure. In any case, when we ran fsck, it had plenty of complaints. Here's a sample session: [root at hera /]# fsck /dev/sdb1 fsck 1.35 (28-Feb-2004) e2fsck 1.35 (28-Feb-2004) /dev/sdb1 contains a file system with errors, check forced. Pass 1: Checking inodes, blocks, and sizes Inode 17072 has illegal block(s). Clear? yes Illegal block #69645 (3666359908) in inode 17072. CLEARED. Illegal block #69646 (3602993593) in inode 17072. CLEARED. Illegal block #69647 (3034171662) in inode 17072. CLEARED. Illegal block #69649 (1280104119) in inode 17072. CLEARED. Illegal block #69650 (1548422666) in inode 17072. CLEARED. Illegal block #69651 (2791173364) in inode 17072. CLEARED. Illegal block #69652 (2279495497) in inode 17072. CLEARED. Illegal block #69653 (3744042344) in inode 17072. CLEARED. Illegal block #69655 (2639235121) in inode 17072. CLEARED. Illegal block #69656 (3190320101) in inode 17072. CLEARED. Illegal block #69657 (1758759856) in inode 17072. CLEARED. Too many illegal blocks in inode 17072. Clear inode? yes Inode 17124 has illegal block(s). Clear? yes Note that for each inode we cleared, we lost one of the 1_gig_files. For example, check out this directory listing: [root at hera newprojects]# ls -lart | more total 835484832 ?--------- ? ? ? ? ? rand_gig.797 ?--------- ? ? ? ? ? rand_gig.788 ?--------- ? ? ? ? ? rand_gig.786 ?--------- ? ? ? ? ? rand_gig.785 ?--------- ? ? ? ? ? rand_gig.733 ?--------- ? ? ? ? ? rand_gig.653 ?--------- ? ? ? ? ? rand_gig.628 ?--------- ? ? ? ? ? rand_gig.627 ?--------- ? ? ? ? ? rand_gig.621 ?--------- ? ? ? ? ? rand_gig.583 ?--------- ? ? ? ? ? rand_gig.577 ?--------- ? ? ? ? ? rand_gig.559 ?--------- ? ? ? ? ? rand_gig.405 ?--------- ? ? ? ? ? rand_gig.393 drwxr-xr-x 3 root root 4096 Feb 14 15:29 .. drwx------ 2 root root 16384 Feb 15 15:25 lost+found -rw-r--r-- 1 root root 1073741824 Feb 15 16:37 rand_gig.3 -rw-r--r-- 1 root root 1073741824 Feb 15 16:41 rand_gig.4 -rw-r--r-- 1 root root 1073741824 Feb 15 16:44 rand_gig.5 ... We never let fsck finish completely, though perhaps it would have just moved all of these to lost+found. Anyway, there are obviously some serious problems with this setup, and I suspect it's on the software side. I'd appreciate hearing any thoughts people might have on this. Are we safe if we just don't create files larger than 2TB in the future? Or did something else go wrong? If creating a >2TB file really is possible at the user level, and can really trash a filesystem, then this would appear to be a serious bug. Damian Menscher -- -=#| Physics Grad Student & SysAdmin @ U Illinois Urbana-Champaign |#=- -=#| 488 LLP, 1110 W. Green St, Urbana, IL 61801 Ofc:(217)333-0038 |#=- -=#| 4602 Beckman, VMIL/MS, Imaging Technology Group:(217)244-3074 |#=- -=#| www.uiuc.edu/~menscher/ Fax:(217)333-9819 |#=- -=#| The above opinions are not necessarily those of my employers. |#=- From subsume at gmail.com Mon Feb 21 17:23:00 2005 From: subsume at gmail.com (Yaego Stephen) Date: Mon, 21 Feb 2005 17:23:00 +0000 (UTC) Subject: e2fsck Looping? Message-ID: Helping a friend fix a computer that was having severe, weird troubles. Reformatted (from XP) and installed Fedora. Install went ok, on first boot, however, filesystem was READONLY for some reason. Knew hard drive was suspect, so I e2fsck'd it over and over all night long, fall asleep next to the computer. This morning, Fedora boots fine, login not a problem, so I shut down and fsck it again, for kicks. What happens: ---- e2fsck keeps complaining about too many illegal blocks in inode 7. Clear inode? Yeah Error reading block #### (Attempt...) while reading indirect blocks of inode ##### ignore error? Yeah Force rewrite? Yeah Restarting e2fsck from the beginning.... /dev/Volgroup00/LogVol00 contains a file system with errors, check forced. Inode 7 has illegal blocks. Clear? Yeah *GOTO top* ---- Took a nap today and its still doing it. At least the thing is booting now! Anyway, I read somewhere that e2fsck should not loop. Could someone please tell me what is going on? Prepare to elaborate/simplify. -Steve ps: Hunter Thompson died today. From adilger at clusterfs.com Mon Feb 21 17:46:46 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Mon, 21 Feb 2005 10:46:46 -0700 Subject: e2fsck Looping? In-Reply-To: References: Message-ID: <20050221174646.GL27352@schnapps.adilger.int> On Feb 21, 2005 17:23 +0000, Yaego Stephen wrote: > Helping a friend fix a computer that was having severe, weird troubles. > Reformatted (from XP) and installed Fedora. Install went ok, on first boot, > however, filesystem was READONLY for some reason. > > Knew hard drive was suspect, so I e2fsck'd it over and over all night long, fall > asleep next to the computer. This morning, Fedora boots fine, login not a > problem, so I shut down and fsck it again, for kicks. > > What happens: > > ---- > > e2fsck keeps complaining about too many illegal blocks in inode 7. > > Clear inode? Yeah > > Error reading block #### (Attempt...) while reading indirect blocks of inode > ##### ignore error? Yeah > > Force rewrite? Yeah > > Restarting e2fsck from the beginning.... > /dev/Volgroup00/LogVol00 contains a file system with errors, check forced. > > Inode 7 has illegal blocks. Clear? Yeah > > *GOTO top* This is a known bug, please update to e2fsprogs-1.36. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From howdy at cardinalpeak.com Tue Feb 22 00:01:50 2005 From: howdy at cardinalpeak.com (Howdy Pierce) Date: Mon, 21 Feb 2005 17:01:50 -0700 Subject: ext3 compatibility between 2.4 and 2.6 kernels Message-ID: Hello-- We have a system where a central server formats removable hard disks, which are then booted in an embedded system running a highly modified RH9. The removable disks themselves contain boot, root, and data filesystems. The problem we've encountered after upgrading to FC3 / kernel 2.6 on the central server is that the 2.4 kernel in the embedded system cannot read the root filesystem, and panics with a message about not being able to find init. The scenario is -- running FC3: 1. "mkfs -t ext3 -O none " on removable disk 2. "mount -Onoacl,oldalloc,nouser_xattr " 3. "tar xf" onto mountpoint 4. "umount " (repeat for each of /boot and / filesystems) I've tried various combinations of mkfs options and mount options, with no luck. Furthermore, I've tried performing the mkfs on a RH9/kernel 2.4.20 system, and then performing the mount/tar/unmount on a FC3/kernel 2.6.9 system. Even this combination fails -- it seems the untarring on FC3 somehow creates an ext3 filesystem that the embedded 2.4 kernel cannot read. Finally -- this may be a clue -- I've noticed that regardless of where the ext3 filesystem was created, symbolic links created on the FC3/2.6 kernel cannot be read on my 2.4 kernel. Are these known issues? Is this expected behavior? Can you recommend anything to fix the problem (aside from upgrading the embedded kernel to 2.6?) Thanks for your help, --Howdy ==================== Howdy Pierce Managing Partner Cardinal Peak, LLC www.cardinalpeak.com ==================== From adilger at clusterfs.com Tue Feb 22 03:01:25 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Mon, 21 Feb 2005 20:01:25 -0700 Subject: ext3 compatibility between 2.4 and 2.6 kernels In-Reply-To: References: Message-ID: <20050222030125.GN27352@schnapps.adilger.int> On Feb 21, 2005 17:01 -0700, Howdy Pierce wrote: > The problem we've encountered after upgrading to FC3 / kernel 2.6 on > the central server is that the 2.4 kernel in the embedded system > cannot read the root filesystem, and panics with a message about not > being able to find init. Having the specific messages from ext3 would help here. > The scenario is -- running FC3: > > 1. "mkfs -t ext3 -O none " on removable disk > 2. "mount -Onoacl,oldalloc,nouser_xattr " > 3. "tar xf" onto mountpoint > 4. "umount " What does "dumpe2fs -h" say for the filesystem after umount? The "oldalloc" mount option shouldn't leave any permanent change to the filesystem (it is just the inode allocation policy). Also, using "-O none" also disables sparse superblocks, which can consume a lot of space for large filesystems. > Furthermore, I've tried performing the mkfs on a RH9/kernel 2.4.20 > system, and then performing the mount/tar/unmount on a FC3/kernel > 2.6.9 system. Even this combination fails -- it seems the untarring > on FC3 somehow creates an ext3 filesystem that the embedded 2.4 kernel > cannot read. Does your FC3 kernel have selinux enabled? > Finally -- this may be a clue -- I've noticed that regardless of where > the ext3 filesystem was created, symbolic links created on the FC3/2.6 > kernel cannot be read on my 2.4 kernel. This sounds like there are EAs being placed on the symlinks, which can cause problems for older kernels (it was an oversight during ext3 development that this was allowed). > Can you recommend anything to fix the problem (aside from upgrading > the embedded kernel to 2.6?) I think there is a one-line fix to "ext3_inode_is_fast_symlink", adding a check for "i_file_acl" to determine if it is a fast (internal to inode) or slow (external block) symlink. This should already be done in newer 2.4 kernels also. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From adilger at clusterfs.com Tue Feb 22 17:26:46 2005 From: adilger at clusterfs.com ('Andreas Dilger') Date: Tue, 22 Feb 2005 10:26:46 -0700 Subject: ext3 compatibility between 2.4 and 2.6 kernels In-Reply-To: <20050222164734.DA7283100D6@moraine.clusterfs.com> References: <20050222030125.GN27352@schnapps.adilger.int> <20050222164734.DA7283100D6@moraine.clusterfs.com> Message-ID: <20050222172646.GP27352@schnapps.adilger.int> On Feb 22, 2005 09:47 -0700, Howdy Pierce wrote: > Thanks for your prompt help. The problem (as you hinted) turned out > to be SELinux. > > So -- the extended attributes get turned on using SE Linux, and this > happens regardless of whether you use the -Onouser_xattr flag to > mount. The user_xattr flag only allows/prevents regular users from storing EAs to the filesystem. > When I disabled SE Linux and repeated the test, the dumpe2fs from > before and after the mount/tar/umount was more similar. Also, > symbolic links were now readable from a 2.4 kernel, and my embedded > system boots correctly (and finds init). I would guess that /sbin/init is a symlink then? > I can't say I really understand enough about SE Linux to have an > opinion as to whether this is a bug or not. It seems like there > should be some option, even on an SE Linux system, to write ext2/3 > filesystems in a backward-compatible manner. Note that EAs (generally, this EA-on-symlink problem being an unfortunate exception) are compatible with older kernels. This functionality was added after EAs were developed and added to the kernel already. I agree it might be useful to have an option to prevent any EAs from being stored on a filesystem, since this also consumes space in the filesystem that has no value to your embedded device. I don't think that mounting the filesystem as ext2 will help as it also has EA support. As I mentioned previously, it is possible with a tiny patch to the 2.4 kernel to fix ext3_inode_is_fast_symlink() to understand symlinks with EAs. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From howdy at cardinalpeak.com Tue Feb 22 16:47:35 2005 From: howdy at cardinalpeak.com (Howdy Pierce) Date: Tue, 22 Feb 2005 09:47:35 -0700 Subject: ext3 compatibility between 2.4 and 2.6 kernels In-Reply-To: <20050222030125.GN27352@schnapps.adilger.int> Message-ID: <200502221747.j1MHlYcR001839@mx1.redhat.com> Andreas-- Thanks for your prompt help. The problem (as you hinted) turned out to be SELinux. On my FC3/kernel 2.6.9 system, I did the following: 1. mkfs -t ext3 -L "/boot" /dev/sda1 2. dumpe2fs -h /dev/sda1 Output: dumpe2fs 1.35 (28-Feb-2004) Filesystem volume name: /boot Last mounted on: Filesystem UUID: a6fb6a06-be6d-4716-8c39-861f97e77954 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal resize_inode filetype sparse_super Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 26208 Block count: 104420 Reserved block count: 5221 Free blocks: 95441 Free inodes: 26197 First block: 1 Block size: 1024 Fragment size: 1024 Blocks per group: 8192 Fragments per group: 8192 Inodes per group: 2016 Inode blocks per group: 252 Filesystem created: Tue Feb 22 09:00:18 2005 Last mount time: n/a Last write time: Tue Feb 22 09:00:20 2005 Mount count: 0 Maximum mount count: 30 Last checked: Tue Feb 22 09:00:18 2005 Check interval: 15552000 (6 months) Next check after: Sun Aug 21 10:00:18 2005 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 Default directory hash: tea Directory Hash Seed: 612d72f8-bf3c-4056-a0de-d4162a3a6c5e Journal backup: inode blocks 3. mount /dev/sda1 /mnt 4. tar xf (source file) -C /mnt 5. umount /dev/sda1 6. dumpe2fs -h /dev/sda1 Output: dumpe2fs 1.35 (28-Feb-2004) Filesystem volume name: /boot Last mounted on: Filesystem UUID: a6fb6a06-be6d-4716-8c39-861f97e77954 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode filetype sparse_super Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 26208 Block count: 104420 Reserved block count: 5221 Free blocks: 88714 Free inodes: 26161 First block: 1 Block size: 1024 Fragment size: 1024 Blocks per group: 8192 Fragments per group: 8192 Inodes per group: 2016 Inode blocks per group: 252 Filesystem created: Tue Feb 22 09:00:18 2005 Last mount time: Tue Feb 22 09:00:50 2005 Last write time: Tue Feb 22 09:01:09 2005 Mount count: 1 Maximum mount count: 30 Last checked: Tue Feb 22 09:00:18 2005 Check interval: 15552000 (6 months) Next check after: Sun Aug 21 10:00:18 2005 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 Default directory hash: tea Directory Hash Seed: 612d72f8-bf3c-4056-a0de-d4162a3a6c5e Journal backup: inode blocks So -- the extended attributes get turned on using SE Linux, and this happens regardless of whether you use the -Onouser_xattr flag to mount. When I disabled SE Linux and repeated the test, the dumpe2fs from before and after the mount/tar/umount was more similar. Also, symbolic links were now readable from a 2.4 kernel, and my embedded system boots correctly (and finds init). I can't say I really understand enough about SE Linux to have an opinion as to whether this is a bug or not. It seems like there should be some option, even on an SE Linux system, to write ext2/3 filesystems in a backward-compatible manner. In any event -- thanks once again for all the helpful responses... --Howdy -----Original Message----- From: Andreas Dilger [mailto:adilger at clusterfs.com] Sent: Monday, February 21, 2005 8:01 PM To: Howdy Pierce Cc: ext3-users at redhat.com Subject: Re: ext3 compatibility between 2.4 and 2.6 kernels On Feb 21, 2005 17:01 -0700, Howdy Pierce wrote: > The problem we've encountered after upgrading to FC3 / kernel 2.6 on > the central server is that the 2.4 kernel in the embedded system > cannot read the root filesystem, and panics with a message about not > being able to find init. Having the specific messages from ext3 would help here. > The scenario is -- running FC3: > > 1. "mkfs -t ext3 -O none " on removable disk > 2. "mount -Onoacl,oldalloc,nouser_xattr " > 3. "tar xf" onto mountpoint > 4. "umount " What does "dumpe2fs -h" say for the filesystem after umount? The "oldalloc" mount option shouldn't leave any permanent change to the filesystem (it is just the inode allocation policy). Also, using "-O none" also disables sparse superblocks, which can consume a lot of space for large filesystems. > Furthermore, I've tried performing the mkfs on a RH9/kernel 2.4.20 > system, and then performing the mount/tar/unmount on a FC3/kernel > 2.6.9 system. Even this combination fails -- it seems the untarring > on FC3 somehow creates an ext3 filesystem that the embedded 2.4 kernel > cannot read. Does your FC3 kernel have selinux enabled? > Finally -- this may be a clue -- I've noticed that regardless of where > the ext3 filesystem was created, symbolic links created on the FC3/2.6 > kernel cannot be read on my 2.4 kernel. This sounds like there are EAs being placed on the symlinks, which can cause problems for older kernels (it was an oversight during ext3 development that this was allowed). > Can you recommend anything to fix the problem (aside from upgrading > the embedded kernel to 2.6?) I think there is a one-line fix to "ext3_inode_is_fast_symlink", adding a check for "i_file_acl" to determine if it is a fast (internal to inode) or slow (external block) symlink. This should already be done in newer 2.4 kernels also. Cheers, Andreas From bhamal at wlink.com.np Tue Feb 22 14:26:41 2005 From: bhamal at wlink.com.np (bj) Date: Tue, 22 Feb 2005 20:11:41 +0545 Subject: moving some directories to the New partition but still preserving the links Message-ID: <000401c5194e$e6f88f80$0db3fea9@kath.state.gov> Hi ! I have Red Hat 8 on a intel P 4 machine with 512 MB memory . Thanks to all , I was able to partition my 60 GB hard drive creating extended drives . Now I want to move some of directories under my root to my extended partition that has already been formatted with mkfs.ext3. I have got the procedure to move /tmp and /var to their own shared partition . I want to move /usr , /applications , /backup , /home , misc , /proc , /tmp . I want to mount /dev/hda5 as /usr & /dev/hda6 as /home or what ever most suitable . But my greatest worry is how to make sure that the previously installed applications work after the move . How to preserve the symbolic & hard links as files are moved across different partitions. Please advice if it is possible to do a successful move of some of the directories under / and still preserve the links and make the applications work. Thanks in advance , cheers, bj My root directory structure is as ffs :- applications backup bin boot dev etc home initrd lib lost+found misc mnt opt proc root sbin tftpboot tmp usr var From gene at czarc.net Wed Feb 23 13:09:59 2005 From: gene at czarc.net (Gene Czarcinski) Date: Wed, 23 Feb 2005 08:09:59 -0500 Subject: 1.36 rpm Message-ID: <200502230809.59316.gene@czarc.net> Give some of the messages and bugzilla reports about file system problems which are fixed by using e2fsprogs 1.36, I am curious why an update/errata has not been issued yet. Anyone like to comment on thissituation? Gene From gene at czarc.net Fri Feb 25 13:24:45 2005 From: gene at czarc.net (Gene Czarcinski) Date: Fri, 25 Feb 2005 08:24:45 -0500 Subject: 1.36 again Message-ID: <200502250824.45902.gene@czarc.net> Given that no 1.36 has appeared from Fedora Core, I thought I would create the rpm myself. I took the 1.35-11.2 src.rpm from FC3 and updated the spec file for 1.36. However, the build fails. Has anyone successfully created a 1.36 rpm for FC3? If so, what did you need to do? Gene From brugolsky at telemetry-investments.com Fri Feb 25 13:42:38 2005 From: brugolsky at telemetry-investments.com (Bill Rugolsky Jr.) Date: Fri, 25 Feb 2005 08:42:38 -0500 Subject: 1.36 again In-Reply-To: <200502250824.45902.gene@czarc.net> References: <200502250824.45902.gene@czarc.net> Message-ID: <20050225134238.GC18310@ti64.telemetry-investments.com> On Fri, Feb 25, 2005 at 08:24:45AM -0500, Gene Czarcinski wrote: > Given that no 1.36 has appeared from Fedora Core, I thought I would create the > rpm myself. I took the 1.35-11.2 src.rpm from FC3 and updated the spec file > for 1.36. However, the build fails. When patching, probably. I expect that you need to drop a bunch of the patches that were applied to 1.35, and are now integrated into 1.36, or done differently. The only interesting patch to keep (if it still applies) is "nostrip", which is needed if you want the rpmbuild system to create a correct debuginfo rpm. Regards, Bill Rugolsky From gene at czarc.net Fri Feb 25 14:09:46 2005 From: gene at czarc.net (Gene Czarcinski) Date: Fri, 25 Feb 2005 09:09:46 -0500 Subject: 1.36 again In-Reply-To: <20050225134238.GC18310@ti64.telemetry-investments.com> References: <200502250824.45902.gene@czarc.net> <20050225134238.GC18310@ti64.telemetry-investments.com> Message-ID: <200502250909.46607.gene@czarc.net> On Friday 25 February 2005 08:42, Bill Rugolsky Jr. wrote: > On Fri, Feb 25, 2005 at 08:24:45AM -0500, Gene Czarcinski wrote: > > Given that no 1.36 has appeared from Fedora Core, I thought I would > > create the rpm myself. I took the 1.35-11.2 src.rpm from FC3 and updated > > the spec file for 1.36. However, the build fails. > > When patching, probably. I expect that you need to drop a bunch of the > patches that were applied to 1.35, and are now integrated into 1.36, or > done differently. > > The only interesting patch to keep (if it still applies) is "nostrip", > which is needed if you want the rpmbuild system to create a correct > debuginfo rpm. After checking all of the patches, it appeared to me that all of them had been picked up by 1.36 so I commented out their application ... the only exception was the "double_free" patch which still applied although, given the other comments by Ted Ts'o, I would question whether it is needed. Anyway, the "problem" is twofold -- First, during the compile phase, I notice that the compilations seem to have different parameters. Second, during install, the build fails because it finds no intl translations. If I comment out the "find_lang" to see how much further I can get, I get a missing "evms" library. Specifically, the whole "evms" subdirectory is missing from 1.36 although there is no mention in the Changelog. Gene From brugolsky at telemetry-investments.com Fri Feb 25 15:51:50 2005 From: brugolsky at telemetry-investments.com (Bill Rugolsky Jr.) Date: Fri, 25 Feb 2005 10:51:50 -0500 Subject: 1.36 again In-Reply-To: <200502250909.46607.gene@czarc.net> References: <200502250824.45902.gene@czarc.net> <20050225134238.GC18310@ti64.telemetry-investments.com> <200502250909.46607.gene@czarc.net> Message-ID: <20050225155150.GE18310@ti64.telemetry-investments.com> On Fri, Feb 25, 2005 at 09:09:46AM -0500, Gene Czarcinski wrote: > Second, during install, the build fails because it finds no intl translations. > If I comment out the "find_lang" to see how much further I can get, I get a > missing "evms" library. Specifically, the whole "evms" subdirectory is > missing from 1.36 although there is no mention in the Changelog. OK, *experimental* e2fsprogs-1.36 source and binary RPM packages based on Rawhide e2fsprogs-1.35-12.src.rpm can be found here: ftp://67.81.182.250/pub/experimental/rpms/ Changelog: * Fri Feb 25 2005 Bill Rugolsky 1.36-0.ti.1 - 1.36 - reenable resize2fs - disable evms - libuuid.3 now uuid.3 [Sorry, other trivial specfile tweaks are missing from the changelog.] No changes were made to ext2online. Standard disclaimers apply. Bill Rugolsky From tytso at mit.edu Fri Feb 25 16:28:17 2005 From: tytso at mit.edu (Theodore Ts'o) Date: Fri, 25 Feb 2005 11:28:17 -0500 Subject: 1.36 again In-Reply-To: <200502250824.45902.gene@czarc.net> References: <200502250824.45902.gene@czarc.net> Message-ID: <20050225162815.GA6082@thunk.org> On Fri, Feb 25, 2005 at 08:24:45AM -0500, Gene Czarcinski wrote: > Given that no 1.36 has appeared from Fedora Core, I thought I would create the > rpm myself. I took the 1.35-11.2 src.rpm from FC3 and updated the spec file > for 1.36. However, the build fails. > > Has anyone successfully created a 1.36 rpm for FC3? If so, what did you need > to do? Try using the e2fsprogs.spec file that is included in the e2fsprogs 1.36 distribution. It should be mostly suitable for FC3 --- if it doesn't work, please let me know, and send me any diffs that might be needed. - Ted From adilger at clusterfs.com Fri Feb 25 18:27:14 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Fri, 25 Feb 2005 11:27:14 -0700 Subject: 1.36 again In-Reply-To: <20050225162815.GA6082@thunk.org> References: <200502250824.45902.gene@czarc.net> <20050225162815.GA6082@thunk.org> Message-ID: <20050225182714.GT27352@schnapps.adilger.int> On Feb 25, 2005 11:28 -0500, Theodore Ts'o wrote: > On Fri, Feb 25, 2005 at 08:24:45AM -0500, Gene Czarcinski wrote: > > Given that no 1.36 has appeared from Fedora Core, I thought I would create > > the rpm myself. I took the 1.35-11.2 src.rpm from FC3 and updated the > > spec file for 1.36. However, the build fails. > > > > Has anyone successfully created a 1.36 rpm for FC3? If so, what did you > > need to do? > > Try using the e2fsprogs.spec file that is included in the e2fsprogs > 1.36 distribution. It should be mostly suitable for FC3 --- if it > doesn't work, please let me know, and send me any diffs that might be > needed. At one time I created a "make rpm" target for e2fsprogs: e2fsprogs/Makefile.in: +rpm: + sh contrib/build-rpm Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From brugolsky at telemetry-investments.com Fri Feb 25 19:18:07 2005 From: brugolsky at telemetry-investments.com (Bill Rugolsky Jr.) Date: Fri, 25 Feb 2005 14:18:07 -0500 Subject: 1.36 again In-Reply-To: <20050225182714.GT27352@schnapps.adilger.int> References: <200502250824.45902.gene@czarc.net> <20050225162815.GA6082@thunk.org> <20050225182714.GT27352@schnapps.adilger.int> Message-ID: <20050225191807.GA21103@ti64.telemetry-investments.com> On Fri, Feb 25, 2005 at 11:27:14AM -0700, Andreas Dilger wrote: > At one time I created a "make rpm" target for e2fsprogs: > > e2fsprogs/Makefile.in: > > +rpm: > + sh contrib/build-rpm Makefile targets like these are useful for developers who are mucking around in a development tree, and just want a package of their current tree; the kernel targets are great for that. End-users should prefer that the tarball contain a working distro-neutral spec file, so that they can just invoke rpmbuild: rpmbuild -ta e2fsprogs-1.36.tar.gz Unfortunately, naming conventions and macros differ just enought between the various RPM-based distros that plenty of packages have instead a foo.spec.in. In e2fsprogs-1.36, the only configure macro is the version number, @E2FSPROGS_VERSION at . :-( IMHO, one should ship a .spec file with the correct version number, or at least a macro conditional that allows the user to do the following: rpmbuild -ta --define 'version 1.36' e2fsprogs-1.36.tar.gz Regards, Bill Rugolsky From adilger at clusterfs.com Fri Feb 25 21:37:18 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Fri, 25 Feb 2005 14:37:18 -0700 Subject: 1.36 again In-Reply-To: <20050225191807.GA21103@ti64.telemetry-investments.com> References: <200502250824.45902.gene@czarc.net> <20050225162815.GA6082@thunk.org> <20050225182714.GT27352@schnapps.adilger.int> <20050225191807.GA21103@ti64.telemetry-investments.com> Message-ID: <20050225213718.GV27352@schnapps.adilger.int> On Feb 25, 2005 14:18 -0500, Bill Rugolsky Jr. wrote: > Makefile targets like these are useful for developers who are mucking > around in a development tree, and just want a package of their current > tree; the kernel targets are great for that. End-users should prefer > that the tarball contain a working distro-neutral spec file, so that > they can just invoke rpmbuild: > > rpmbuild -ta e2fsprogs-1.36.tar.gz > > Unfortunately, naming conventions and macros differ just enought between > the various RPM-based distros that plenty of packages have instead a > foo.spec.in. In e2fsprogs-1.36, the only configure macro is the version > number, @E2FSPROGS_VERSION at . :-( IMHO, one should ship a .spec file with > the correct version number, or at least a macro conditional that allows > the user to do the following: > > rpmbuild -ta --define 'version 1.36' e2fsprogs-1.36.tar.gz Patches greatfully accepted, as you know a lot more about RPM than I. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From alazarev at itg.uiuc.edu Fri Feb 25 22:34:42 2005 From: alazarev at itg.uiuc.edu (Alexander Lazarevich) Date: Fri, 25 Feb 2005 16:34:42 -0600 (CST) Subject: ext3 +2TB fs Message-ID: I've got a 3.3TB ext3 on a FC3 64-bit system, running kernel 2.6.10-1.766FC3smp. I create the partition with parted 1.6.21, and I make the fs via: mkfs.ext3 -m1 -b 4096 -T largefile4 /dev/sda1 Works fine. bonnie++ running on it multiple times for days on end, no problems. However, I do the exact same setup on a RHEL4-AS i686 system, 32-bit, and the fs is totally hosed, get all kinds of errors with the filesystem, just trying to mount it doesn't work. Crud, I blew away the system, so I don't have the error logs, but it was an error about the group descriptors being corrupted. I ran fsck but it couldn't fix it. That kernel is 2.6.9-5.0.3smp. I also compile with the CONFIG_LDB flag which enables large block device support. I'm told that should be all I have to do. But it just doesn't work. Not at all. Any ideas? I'm stuck running FC3 now unless I can get some quick fix from you guys. Thanks! Alex From alazarev at itg.uiuc.edu Fri Feb 25 22:36:33 2005 From: alazarev at itg.uiuc.edu (Alexander Lazarevich) Date: Fri, 25 Feb 2005 16:36:33 -0600 (CST) Subject: ext3 +2TB fs In-Reply-To: References: Message-ID: ah, found the error logs in a previous email i never sent: Feb 25 11:23:28 xxx kernel: sda: sda1 Feb 25 11:23:30 xxx kernel: SCSI device sda: 2927173632 512-byte hdwr sectors (1498713 MB) Feb 25 11:23:30 xxx kernel: SCSI device sda: drive cache: write back Feb 25 11:23:30 xxx kernel: sda: sda1 Feb 25 11:28:25 xxx kernel: EXT3-fs error (device sda1): ext3_check_descriptors: Inode table for group 5642 not in group (block 184811522)! Feb 25 11:28:25 xxx kernel: EXT3-fs: group descriptors corrupted ! Feb 25 11:28:34 xxx kernel: EXT3-fs error (device sdb1): ext3_check_descriptors: Inode table for group 826 not in group (block 27000834)! Feb 25 11:28:34 xxx kernel: EXT3-fs: group descriptors corrupted ! At that point I can fsck and fix the fs, but as soon as I try to mount the problem happens again. that is the error i get on a RHEL4-AS 2.6.9-5.0.3smp kernel using parted 1.6.21 and mkfs command from below. any ideas? thanks! On Fri, 25 Feb 2005, Alexander Lazarevich wrote: > I've got a 3.3TB ext3 on a FC3 64-bit system, running kernel > 2.6.10-1.766FC3smp. I create the partition with parted 1.6.21, and I make the > fs via: > > mkfs.ext3 -m1 -b 4096 -T largefile4 /dev/sda1 > > Works fine. bonnie++ running on it multiple times for days on end, no > problems. > > However, I do the exact same setup on a RHEL4-AS i686 system, 32-bit, and the > fs is totally hosed, get all kinds of errors with the filesystem, just trying > to mount it doesn't work. Crud, I blew away the system, so I don't have the > error logs, but it was an error about the group descriptors being corrupted. > I ran fsck but it couldn't fix it. That kernel is 2.6.9-5.0.3smp. I also > compile with the CONFIG_LDB flag which enables large block device support. > I'm told that should be all I have to do. But it just doesn't work. Not at > all. > > Any ideas? I'm stuck running FC3 now unless I can get some quick fix from you > guys. > > Thanks! > > Alex > > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > From alazarev at itg.uiuc.edu Fri Feb 25 22:38:08 2005 From: alazarev at itg.uiuc.edu (Alexander Lazarevich) Date: Fri, 25 Feb 2005 16:38:08 -0600 (CST) Subject: ext3 +2TB fs In-Reply-To: References: Message-ID: ah, yes, and the biggest difference between the two systems is the fact that one is a 64-bit kernel, and the other is 32-bit. should have made that more clear. alex On Fri, 25 Feb 2005, Alexander Lazarevich wrote: > ah, found the error logs in a previous email i never sent: > > Feb 25 11:23:28 xxx kernel: sda: sda1 > Feb 25 11:23:30 xxx kernel: SCSI device sda: 2927173632 512-byte hdwr > sectors (1498713 MB) > Feb 25 11:23:30 xxx kernel: SCSI device sda: drive cache: write back > Feb 25 11:23:30 xxx kernel: sda: sda1 > Feb 25 11:28:25 xxx kernel: EXT3-fs error (device sda1): > ext3_check_descriptors: Inode table for group 5642 not in group (block > 184811522)! > Feb 25 11:28:25 xxx kernel: EXT3-fs: group descriptors corrupted ! > Feb 25 11:28:34 xxx kernel: EXT3-fs error (device sdb1): > ext3_check_descriptors: Inode table for group 826 not in group (block > 27000834)! > Feb 25 11:28:34 xxx kernel: EXT3-fs: group descriptors corrupted ! > > At that point I can fsck and fix the fs, but as soon as I try to mount the > problem happens again. > > that is the error i get on a RHEL4-AS 2.6.9-5.0.3smp kernel using parted > 1.6.21 and mkfs command from below. > > any ideas? thanks! > > On Fri, 25 Feb 2005, Alexander Lazarevich wrote: > >> I've got a 3.3TB ext3 on a FC3 64-bit system, running kernel >> 2.6.10-1.766FC3smp. I create the partition with parted 1.6.21, and I make >> the fs via: >> >> mkfs.ext3 -m1 -b 4096 -T largefile4 /dev/sda1 >> >> Works fine. bonnie++ running on it multiple times for days on end, no >> problems. >> >> However, I do the exact same setup on a RHEL4-AS i686 system, 32-bit, and >> the fs is totally hosed, get all kinds of errors with the filesystem, just >> trying to mount it doesn't work. Crud, I blew away the system, so I don't >> have the error logs, but it was an error about the group descriptors being >> corrupted. I ran fsck but it couldn't fix it. That kernel is >> 2.6.9-5.0.3smp. I also compile with the CONFIG_LDB flag which enables >> large block device support. I'm told that should be all I have to do. But >> it just doesn't work. Not at all. >> >> Any ideas? I'm stuck running FC3 now unless I can get some quick fix from >> you guys. >> >> Thanks! >> >> Alex >> >> >> _______________________________________________ >> Ext3-users mailing list >> Ext3-users at redhat.com >> https://www.redhat.com/mailman/listinfo/ext3-users >> > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > From adilger at clusterfs.com Fri Feb 25 23:02:41 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Fri, 25 Feb 2005 16:02:41 -0700 Subject: ext3 +2TB fs In-Reply-To: References: Message-ID: <20050225230241.GW27352@schnapps.adilger.int> On Feb 25, 2005 16:36 -0600, Alexander Lazarevich wrote: > Feb 25 11:23:28 xxx kernel: sda: sda1 > Feb 25 11:23:30 xxx kernel: SCSI device sda: 2927173632 512-byte hdwr > sectors (1498713 MB) > Feb 25 11:23:30 xxx kernel: SCSI device sda: drive cache: write back > Feb 25 11:23:30 xxx kernel: sda: sda1 > Feb 25 11:28:25 xxx kernel: EXT3-fs error (device sda1): > ext3_check_descriptors: Inode table for group 5642 not in group (block > 184811522)! > Feb 25 11:28:25 xxx kernel: EXT3-fs: group descriptors corrupted ! > Feb 25 11:28:34 xxx kernel: EXT3-fs error (device sdb1): > ext3_check_descriptors: Inode table for group 826 not in group (block > 27000834)! > Feb 25 11:28:34 xxx kernel: EXT3-fs: group descriptors corrupted ! The group descriptors are stored right at the start of the filesystem, so if there is a 32-bit overflow it would corrupt them right away. I would start by testing whether the large device works properly by writing some pattern (e.g. 64-bit byte offset) to the start of each 4k block on disk, and then read them back to verify nothing has been overwritten. Next, create directories (you may need as many as 16k) to get one that is in the >2TB part of the disk. You can tell by the inode number and the output from dumpe2fs. If you write a file in that directory it should allocate space at > 2TB on the disk, and debugfs "stat file" will tell you the block layout of the file. > On Fri, 25 Feb 2005, Alexander Lazarevich wrote: > >I've got a 3.3TB ext3 on a FC3 64-bit system, running kernel > >2.6.10-1.766FC3smp. I create the partition with parted 1.6.21, and I make > >the fs via: > > > >mkfs.ext3 -m1 -b 4096 -T largefile4 /dev/sda1 > > > >Works fine. bonnie++ running on it multiple times for days on end, no > >problems. > > > >However, I do the exact same setup on a RHEL4-AS i686 system, 32-bit, and > >the fs is totally hosed, get all kinds of errors with the filesystem, just > >trying to mount it doesn't work. Crud, I blew away the system, so I don't > >have the error logs, but it was an error about the group descriptors being > >corrupted. I ran fsck but it couldn't fix it. That kernel is > >2.6.9-5.0.3smp. I also compile with the CONFIG_LDB flag which enables > >large block device support. I'm told that should be all I have to do. But > >it just doesn't work. Not at all. > > > >Any ideas? I'm stuck running FC3 now unless I can get some quick fix from > >you guys. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From menscher at uiuc.edu Sat Feb 26 00:58:03 2005 From: menscher at uiuc.edu (Damian Menscher) Date: Fri, 25 Feb 2005 18:58:03 -0600 (CST) Subject: ext3 +2TB fs In-Reply-To: <20050225230241.GW27352@schnapps.adilger.int> References: <20050225230241.GW27352@schnapps.adilger.int> Message-ID: On Fri, 25 Feb 2005, Andreas Dilger wrote: > On Feb 25, 2005 16:36 -0600, Alexander Lazarevich wrote: > > Feb 25 11:23:28 xxx kernel: sda: sda1 > > Feb 25 11:23:30 xxx kernel: SCSI device sda: 2927173632 512-byte hdwr > > sectors (1498713 MB) > > Feb 25 11:23:30 xxx kernel: SCSI device sda: drive cache: write back > > Feb 25 11:23:30 xxx kernel: sda: sda1 > > Feb 25 11:28:25 xxx kernel: EXT3-fs error (device sda1): > > ext3_check_descriptors: Inode table for group 5642 not in group (block > > 184811522)! > > Feb 25 11:28:25 xxx kernel: EXT3-fs: group descriptors corrupted ! > > Feb 25 11:28:34 xxx kernel: EXT3-fs error (device sdb1): > > ext3_check_descriptors: Inode table for group 826 not in group (block > > 27000834)! > > Feb 25 11:28:34 xxx kernel: EXT3-fs: group descriptors corrupted ! > > The group descriptors are stored right at the start of the filesystem, > so if there is a 32-bit overflow it would corrupt them right away. > > I would start by testing whether the large device works properly by > writing some pattern (e.g. 64-bit byte offset) to the start of each > 4k block on disk, and then read them back to verify nothing has been > overwritten. Out of curiosity, how would one do this? All I can think of is to script something to call dd with the seek/skip argument. But making 3.5TB/4k = a billion calls to dd out a shell seems kinda silly. What do you suggest? > Next, create directories (you may need as many as 16k) to get one that > is in the >2TB part of the disk. You can tell by the inode number and > the output from dumpe2fs. If you write a file in that directory it > should allocate space at > 2TB on the disk, and debugfs "stat file" will > tell you the block layout of the file. As I understand it, the first test is to identify if the flaw exists in the kernel block-device code, and the second test whether the bug is in the ext2 code? Anyone out there actually using a >2TB filesystem on a 32-bit machine? Damian [works with Alex] -- -=#| Physics Grad Student & SysAdmin @ U Illinois Urbana-Champaign |#=- -=#| 488 LLP, 1110 W. Green St, Urbana, IL 61801 Ofc:(217)333-0038 |#=- -=#| www.uiuc.edu/~menscher/ Fax:(217)333-9819 |#=- From adilger at clusterfs.com Sat Feb 26 02:40:59 2005 From: adilger at clusterfs.com (Andreas Dilger) Date: Fri, 25 Feb 2005 19:40:59 -0700 Subject: ext3 +2TB fs In-Reply-To: References: <20050225230241.GW27352@schnapps.adilger.int> Message-ID: <20050226024059.GX27352@schnapps.adilger.int> On Feb 25, 2005 18:58 -0600, Damian Menscher wrote: > On Fri, 25 Feb 2005, Andreas Dilger wrote: > >I would start by testing whether the large device works properly by > >writing some pattern (e.g. 64-bit byte offset) to the start of each > >4k block on disk, and then read them back to verify nothing has been > >overwritten. > > Out of curiosity, how would one do this? All I can think of is to > script something to call dd with the seek/skip argument. But making > 3.5TB/4k = a billion calls to dd out a shell seems kinda silly. What do > you suggest? I'd say a simple C program like the following (not much error checking, untested, be careful ;-): #define _GNU_SOURCE #define _LARGEFILE64_SOURCE #include #include #include #include #define BUFSZ 4096 int main() { char buf[BUFSZ] = { 0 }; long long offset, *bufp = (long long *)buf; int fd = open(argv[1], O_RDWR | O_LARGEFILE); int rc; while (write(fd, buf, sizeof(buf)) == sizeof(buf)) *bufp += sizeof(buf); printf("end at %llu: %s\n", lseek64(fd, 0, SEEK_CUR), strerror(errno)); offset = lseek64(fd, 0, SEEK_SET); while (read(fd, buf, sizeof(buf)) == sizeof(buf)) { if (*bufp != offset) fprintf(stderr, "offset %llu data is %llu\n", offset, *bufp); offset += sizeof(buf); } printf("end at %llu: %s\n", lseek64(fd, 0, SEEK_CUR), strerror(errno)); return 0; } > >Next, create directories (you may need as many as 16k) to get one that > >is in the >2TB part of the disk. You can tell by the inode number and > >the output from dumpe2fs. If you write a file in that directory it > >should allocate space at > 2TB on the disk, and debugfs "stat file" will > >tell you the block layout of the file. > > As I understand it, the first test is to identify if the flaw exists in > the kernel block-device code, and the second test whether the bug is in > the ext2 code? Correct. > Anyone out there actually using a >2TB filesystem on a 32-bit machine? I've heard sporadic reports about it, but there is definitely a problem somewhere after 2TB. For Lustre we don't care so much about gigantic individual filesystems because we aggregate them at a higher level (100's of TB) and having "smaller" (i.e. 2TB) filesystems allows more parallelism for IO, e2fsck, etc. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://members.shaw.ca/adilger/ http://members.shaw.ca/golinux/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From tytso at mit.edu Fri Feb 25 19:00:49 2005 From: tytso at mit.edu (Theodore Ts'o) Date: Fri, 25 Feb 2005 14:00:49 -0500 Subject: 1.36 again In-Reply-To: <200502250909.46607.gene@czarc.net> References: <200502250824.45902.gene@czarc.net> <20050225134238.GC18310@ti64.telemetry-investments.com> <200502250909.46607.gene@czarc.net> Message-ID: <20050225190049.GA8315@thunk.org> On Fri, Feb 25, 2005 at 09:09:46AM -0500, Gene Czarcinski wrote: > After checking all of the patches, it appeared to me that all of > them had been picked up by 1.36 so I commented out their application > ... the only exception was the "double_free" patch which still > applied although, given the other comments by Ted Ts'o, I would > question whether it is needed. It is definitely not needed. I haven't looked at the patch in a while so I can't tell you off the top of my head whether applying it will lead to something actively harmful happening, or just a harmless memory leak. (I think the latter, but I am not 100% sure). > If I comment out the "find_lang" to see how much further I can get, I get a > missing "evms" library. Specifically, the whole "evms" subdirectory is > missing from 1.36 although there is no mention in the Changelog. >From the release notes: Remove obsolete EVMS 1.x and a.out DLL support. - Ted From tytso at mit.edu Sat Feb 26 16:38:12 2005 From: tytso at mit.edu (Theodore Ts'o) Date: Sat, 26 Feb 2005 11:38:12 -0500 Subject: 1.36 again In-Reply-To: <20050225191807.GA21103@ti64.telemetry-investments.com> References: <200502250824.45902.gene@czarc.net> <20050225162815.GA6082@thunk.org> <20050225182714.GT27352@schnapps.adilger.int> <20050225191807.GA21103@ti64.telemetry-investments.com> Message-ID: <20050226163812.GA15346@thunk.org> On Fri, Feb 25, 2005 at 02:18:07PM -0500, Bill Rugolsky Jr. wrote: > Makefile targets like these are useful for developers who are mucking > around in a development tree, and just want a package of their current > tree; the kernel targets are great for that. End-users should prefer > that the tarball contain a working distro-neutral spec file, so that > they can just invoke rpmbuild: > > rpmbuild -ta e2fsprogs-1.36.tar.gz > > Unfortunately, naming conventions and macros differ just enought between > the various RPM-based distros that plenty of packages have instead a > foo.spec.in. In e2fsprogs-1.36, the only configure macro is the version > number, @E2FSPROGS_VERSION at . :-( IMHO, one should ship a .spec file with > the correct version number, or at least a macro conditional that allows > the user to do the following: > > rpmbuild -ta --define 'version 1.36' e2fsprogs-1.36.tar.gz The e2fsprogs.tar.gz tarball *does* come shipped with a e2fsprogs.spec file that has the version number defined, so that "rpmbuild -ta e2fsprogs-1.36.tar.gz" should work correctly, and has for quite some time now. The e2fsprogs.spec.in file that has @E2FSPROGS_VERSION@ is there so that the e2fsprogs.spec file can be built and included automatically in the .tar.gz file when the util/gen-tarball script is executed. E2fsprogs.spec *is* supposed to be a distro-neutral spec file, but I don't regularly use an rpm-based distribution these days, so I am depending on others to report bugs and suggest patches. It would be helpful though if people actually *tried* to use it as opposed making incorrect assertions on the mailing list. :-) - Ted From brugolsky at telemetry-investments.com Mon Feb 28 17:39:35 2005 From: brugolsky at telemetry-investments.com (Bill Rugolsky Jr.) Date: Mon, 28 Feb 2005 12:39:35 -0500 Subject: 1.36 again In-Reply-To: <20050226163812.GA15346@thunk.org> References: <200502250824.45902.gene@czarc.net> <20050225162815.GA6082@thunk.org> <20050225182714.GT27352@schnapps.adilger.int> <20050225191807.GA21103@ti64.telemetry-investments.com> <20050226163812.GA15346@thunk.org> Message-ID: <20050228173935.GE30585@ti64.telemetry-investments.com> On Sat, Feb 26, 2005 at 11:38:12AM -0500, Theodore Ts'o wrote: > E2fsprogs.spec *is* supposed to be a distro-neutral spec file, but I > don't regularly use an rpm-based distribution these days, so I am > depending on others to report bugs and suggest patches. It would be > helpful though if people actually *tried* to use it as opposed making > incorrect assertions on the mailing list. :-) Sorry Ted, ENOCAFFEINE. It turns out that rpmbuild -ta is incompatible with my ~/.rpmmacro file (which include %{name} in the paths). Using the standard layout, the %find_lang fails on FC3. As penance, I'll figure out what's wrong and offer up a patch. :-) Bill