From m4rtntns at gmail.com Sat May 10 18:42:50 2014 From: m4rtntns at gmail.com (Martin T) Date: Sat, 10 May 2014 18:42:50 +0000 Subject: location of file-system information on ext4 Message-ID: Hi, I zero-filled first 10MiB of my SSD(dd if=/dev/zero of=/dev/sda bs=10M count=1). As expected, this wiped my primary GPD header and first partition. Before the wipe, GPT was following: Disk /dev/sda: 250069680 sectors, 119.2 GiB Logical sector size: 512 bytes Disk identifier (GUID): 2EFD285D-F8E6-4262-B380-232E866AF15C Partition table holds up to 128 entries First usable sector is 34, last usable sector is 250069646 Partitions will be aligned on 1-sector boundaries Total free space is 16 sectors (8.0 KiB) Number Start (sector) End (sector) Size Code Name 1 34 58593784 27.9 GiB EF00 root part 2 58593785 234375035 83.8 GiB 0700 home part 3 234375036 250069630 7.5 GiB 8200 swap part However, to my surprise, the file(1) and dumpe2fs(8) utilities were still able to show information regarding file system: root at T60:~# file -s /dev/sda1 /dev/sda1: sticky Linux rev 1.0 ext4 filesystem data, UUID=88e70e1b-4c45-4e7e-931f-9e97d7257ee8 (needs journal recovery) (extents) (large files) (huge files) root at T60:~# dumpe2fs /dev/sda1 dumpe2fs 1.42.5 (29-Jul-2012) Filesystem volume name: Last mounted on: / Filesystem UUID: 88e70e1b-4c45-4e7e-931f-9e97d7257ee8 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 1831424 Block count: 7324218 Reserved block count: 366210 Free blocks: 4511822 Free inodes: 1671701 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1022 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8176 Inode blocks per group: 511 Flex block group size: 16 Filesystem created: Wed Jul 10 23:01:50 2013 Last mount time: Mon May 5 19:28:01 2014 Last write time: Mon May 5 19:28:01 2014 Mount count: 14 Maximum mount count: 34 Last checked: Sun Jan 26 15:32:00 2014 Check interval: 15552000 (6 months) Next check after: Fri Jul 25 15:32:00 2014 Lifetime writes: 61 GB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Journal inode: 8 First orphan inode: 1439005 Default directory hash: half_md4 Directory Hash Seed: f7476ad6-3067-4697-93b3-f4a368d76eab Journal backup: inode blocks Journal superblock magic number invalid! root at T60:~# Am I correct that there are multiple super-blocks scattered around the HDD and if first one is missing, then next one is read by file or dumpe2fs utilities? In the first place, what (important) file-system data is written at the beginning of the partition? regards, Martin From tytso at mit.edu Sun May 11 18:58:52 2014 From: tytso at mit.edu (Theodore Ts'o) Date: Sun, 11 May 2014 14:58:52 -0400 Subject: location of file-system information on ext4 In-Reply-To: References: Message-ID: <20140511185852.GA5480@thunk.org> On Sat, May 10, 2014 at 06:42:50PM +0000, Martin T wrote: > > Am I correct that there are multiple super-blocks scattered around the > HDD and if first one is missing, then next one is read by file or > dumpe2fs utilities? In the first place, what (important) file-system > data is written at the beginning of the partition? There are multiple super blocks scattered around the HDD, but they are used primarily by e2fsck, for the purposes of trying to repair a corrupted file system. As far as I know dumpe2fs (definitely) and file (almost certainly) doesn't fall back and try the alternate superblocks. Was the file system mounted at the time when you tried running the dd command to zero the first 10MB of your SSD? The only thing I can think of is that the file system was mounted, and so the superblock was written back when you zero'ed the SSD, via some artifact of buffer cache (non-)aliasing. You could check by running the command od /dev/sda1 | head and seeing if the first 8k or so of the file system really is all zero's or not. In answer to your question, portions of the inode table are located at the begining of the file system, so the root inode would have been lost if the first 10MB or so of the file system really was smashed. - Ted From bensberg at justemail.net Mon May 26 19:20:11 2014 From: bensberg at justemail.net (Benno Schulenberg) Date: Mon, 26 May 2014 21:20:11 +0200 Subject: more PO files available at the TP Message-ID: <1401132011.17846.121708233.2D911640@webmail.messagingengine.com> Hi, At the translationproject there are four more languages available than are included in the 1.42.10 tarball: Danish, Esperanto, Malay, and Ukrainian. Please include these in your next release. Attached patch adds the missing language codes to the po/LINGUAS file. The easiest way to fetch the missing files (and the latest updates) is to run: rsync -Lrtvz translationproject.org::tp/latest/e2fsprogs/ po Regards, Benno -- http://www.fastmail.fm - Or how I learned to stop worrying and love email again -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-po-Add-the-codes-of-the-other-four-languages-availab.patch Type: text/x-diff Size: 602 bytes Desc: not available URL: From tytso at mit.edu Thu May 29 06:33:36 2014 From: tytso at mit.edu (Theodore Ts'o) Date: Thu, 29 May 2014 02:33:36 -0400 Subject: more PO files available at the TP In-Reply-To: <1401132011.17846.121708233.2D911640@webmail.messagingengine.com> References: <1401132011.17846.121708233.2D911640@webmail.messagingengine.com> Message-ID: <20140529063336.GE25041@thunk.org> On Mon, May 26, 2014 at 09:20:11PM +0200, Benno Schulenberg wrote: > > At the translationproject there are four more languages available > than are included in the 1.42.10 tarball: Danish, Esperanto, Malay, > and Ukrainian. Please include these in your next release. I deliberately didn't include some of these because the number of translated messages were pathetic. Danish has 139 messages translated out of 1305. Malay for a long time had 72 messages translated; it's now up 235 messages out 1305. Eperanto has gotten better; it's now up to 499 / 1305. Ukranian is pretty new, but it's actually in pretty good shape; 1055 / 1305. It seems to me that if a message catalog has only 5% of the messages translated, including them is very likely to result in a poorer user experience than if everything was left in English. > Attached patch adds the missing language codes to the po/LINGUAS > file. The easiest way to fetch the missing files (and the latest > updates) is to run: > > rsync -Lrtvz translationproject.org::tp/latest/e2fsprogs/ po I actually use a script to pull down the changes: #!/bin/bash # # git-tp-sync - downloads the latest PO files from translationproject.org # and commits changes to your GIT repository. # # Features: # - commit per PO file (no more huge commits for all .po files) # - the commit "Author:" field is set from the Last-Translator # # # Copyright (C) 2007 Karel Zak # what I generally do is take a look at the new translations, and if they look OK, I'll add them to the repository. I admit that I didn't look at the Ukarian translation when it first showed up because my experience has been that nearly all of the translation files had a vanishly small percentage of the messages translated, and I had assumed the Ukranian translation would follow the same sad pattern. Cheers, - Ted From bensberg at justemail.net Thu May 29 08:15:27 2014 From: bensberg at justemail.net (Benno Schulenberg) Date: Thu, 29 May 2014 10:15:27 +0200 Subject: more PO files available at the TP In-Reply-To: <20140529063336.GE25041@thunk.org> References: <1401132011.17846.121708233.2D911640@webmail.messagingengine.com> <20140529063336.GE25041@thunk.org> Message-ID: <1401351327.16611.122784917.7710A6AA@webmail.messagingengine.com> On Thu, May 29, 2014, at 8:33, Theodore Ts'o wrote: > It seems to me that if a message catalog has only 5% of the messages > translated, including them is very likely to result in a poorer user > experience than if everything was left in English. When it's just five percent, yes, that may be more of a nuisance than a help. But when it's ten to twenty percent, and when it's the entire part of the program that the average user gets to see, then it can be quite useful. For Esperanto I have tried to do that: the average user of e2fsck would see all of its messages translated. However, even if I had fully translated the POT file, some things would still be untranslated. The entire output of 'tune2fs -l ' is always in English -- lib/e2p/ls.c is lacking gettextization. And here and there in other files several messages haven't been gettextized either. Over the past years I have sent you several patches for these, but none of them seem to have reached you. Regards, Benno -- http://www.fastmail.fm - Or how I learned to stop worrying and love email again From mvolaski at aecom.yu.edu Thu May 29 15:37:42 2014 From: mvolaski at aecom.yu.edu (Maurice Volaski) Date: Thu, 29 May 2014 15:37:42 -0000 Subject: [DRBD-user] [Q] What would cause fsck running on a drbd device to just stop? Message-ID: <20140529153503.0F76A14F51C@mx2.netorek.fi> drbd-0.7.19 under kernel 2.6.17-rc4 is running on a primary node standalone. There are 8 resources in the same group. fsck.ext3 -fv is being run simultaneously on all of them. Each of the drbd devices are running on an lv, which all belong to a single pv. The actual "disk" is a hardware RAID connected via SCSI (i.e., the mpt driver). Five of the fsck finished their tasks successfully and reported no problems. The remaining three got "stuck". There was no activity either on the physical RAID itself or listed in top. They were just listed as "D," uninterruptible sleep. Two of the fscks were at the end, giving the final summary information--no problems---for their respective filesystems and were stuck at that point. The last one was stuck in the first pass. Attempts to kill them failed; even kill -9 and attempting to shutdown were ignored. I rebooted manually and ran the three stuck ones again without a hitch. -- Maurice Volaski, mvolaski at aecom.yu.edu Computing Support, Rose F. Kennedy Center Albert Einstein College of Medicine of Yeshiva University _______________________________________________ drbd-user mailing list drbd-user at lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user From mvolaski at aecom.yu.edu Thu May 29 15:39:44 2014 From: mvolaski at aecom.yu.edu (Maurice Volaski) Date: Thu, 29 May 2014 15:39:44 -0000 Subject: [DRBD-user] [Q] What would cause fsck running on a drbd device to just stop? Message-ID: <20140529153557.7FE12205E29@mx2.netorek.fi> drbd-0.7.19 under kernel 2.6.17-rc4 is running on a primary node standalone. There are 8 resources in the same group. fsck.ext3 -fv is being run simultaneously on all of them. Each of the drbd devices are running on an lv, which all belong to a single pv. The actual "disk" is a hardware RAID connected via SCSI (i.e., the mpt driver). Five of the fsck finished their tasks successfully and reported no problems. The remaining three got "stuck". There was no activity either on the physical RAID itself or listed in top. They were just listed as "D," uninterruptible sleep. Two of the fscks were at the end, giving the final summary information--no problems---for their respective filesystems and were stuck at that point. The last one was stuck in the first pass. Attempts to kill them failed; even kill -9 and attempting to shutdown were ignored. I rebooted manually and ran the three stuck ones again without a hitch. -- Maurice Volaski, mvolaski at aecom.yu.edu Computing Support, Rose F. Kennedy Center Albert Einstein College of Medicine of Yeshiva University _______________________________________________ drbd-user mailing list drbd-user at lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user From tytso at mit.edu Thu May 29 19:52:59 2014 From: tytso at mit.edu (Theodore Ts'o) Date: Thu, 29 May 2014 15:52:59 -0400 Subject: more PO files available at the TP In-Reply-To: <1401351327.16611.122784917.7710A6AA@webmail.messagingengine.com> References: <1401132011.17846.121708233.2D911640@webmail.messagingengine.com> <20140529063336.GE25041@thunk.org> <1401351327.16611.122784917.7710A6AA@webmail.messagingengine.com> Message-ID: <20140529195259.GK25041@thunk.org> On Thu, May 29, 2014 at 10:15:27AM +0200, Benno Schulenberg wrote: > When it's just five percent, yes, that may be more of a nuisance > than a help. But when it's ten to twenty percent, and when it's > the entire part of the program that the average user gets to see, > then it can be quite useful. If it's the right 10 to 20%, that's probably true, I agree. I'll take a closer look for the next release. > However, even if I had fully translated the POT file, some things > would still be untranslated. The entire output of 'tune2fs -l ' > is always in English -- lib/e2p/ls.c is lacking gettextization. And > here and there in other files several messages haven't been gettextized > either. Over the past years I have sent you several patches for these, > but none of them seem to have reached you. Sorry, I must have missed that/those patch(es); my apologies. If you send such patches to linux-ext4 at vger.kernel.org, they will get tracked using patchwork[1], which makes it much less likely that patches get dropped. http://patchwork.ozlabs.org/project/linux-ext4/list/ Cheers, - Ted From kkeller at wombat.san-francisco.ca.us Sat May 31 18:56:07 2014 From: kkeller at wombat.san-francisco.ca.us (Keith Keller) Date: Sat, 31 May 2014 11:56:07 -0700 Subject: [long] major problems on fs; e2fsck running out of memory Message-ID: <20140531185607.GA12748@wombat.san-francisco.ca.us> Hello ext3 list, I am having an odd issue with one of my filesystems, and I am hoping someone here can help out. Yes, I do have backups. :) But as is often the case, it's nice to avoid restoring from backup if possible. If there is a more appropriate place for this question please let me know. After quite a while between reboots, I saw a report on the console that the filesystem was inconsistent and could not be automatically repaired. After some aborted tests (which I did not log, unfortunately, I was able to get this far: # head fsck.out fsck from util-linux-ng 2.17.2 /dev/mapper/vg1--sdb-lv_vz contains a file system with errors, check forced. # time passes, progress bar gets to 51.8% with no problems, then Pass 1: Checking inodes, blocks, and sizes Inode 266338321 has imagic flag set. Clear? yes Inode 266338321 has a extra size (34120) which is invalid Fix? yes Inode 266338321 has compression flag set on filesystem without compression support. Clear? yes # some 150k messages later Inode 266349409, i_blocks is 94855766560840, should be 0. Fix? yes Inode 266349363 has a bad extended attribute block 1262962006. Clear? yes Inode 266349363 has illegal block(s). Clear? yes Illegal block #6 (1447645766) in inode 266349363. CLEARED. Illegal indirect block (1447642454) in inode 266349363. CLEARED. Illegal block #270533644 (1702521203) in inode 266349363. CLEARED. Warning... fsck.ext4 for device /dev/mapper/vg1--sdb-lv_vz exited with signal 11. I wasn't sure what that meant, and somewhat without thinking, I made more attempts to repair the fs (including, IIRC, some attempts to mount the filesystem ro). Here's the next fsck attempt: fsck from util-linux-ng 2.17.2 fsck.ext4: Group descriptors look bad... trying backup blocks... One or more block group descriptor checksums are invalid. Fix? yes Group descriptor 0 checksum is invalid. FIXED. Group descriptor 1 checksum is invalid. FIXED. # many group descriptors fixed Group descriptor 40834 checksum is invalid. FIXED. Group descriptor 40836 checksum is invalid. FIXED. /dev/mapper/vg1--sdb-lv_vz contains a file system with errors, check forced. This doesn't bode well, but we'll try to go on... Pass 1: Checking inodes, blocks, and sizes # again gets to 51.8% with no problems # again over 100k lines of errors Inode 266356018 is too big. Truncate? yes Block #537922572 (62467) causes directory to be too big. CLEARED. Warning... fsck.ext4 for device /dev/mapper/vg1--sdb-lv_vz exited with signal 11. I tried once more with e2fsck 1.41.12 (stock from CentOS 6), then on a search, tried e2fsck 1.42.10 from source. ext2fs_check_desc: Corrupt group descriptor: bad block for block bitmap ./e2fsprogs-1.42.10/e2fsck/e2fsck: Group descriptors look bad... trying backup blocks... /dev/mapper/vg1--sdb-lv_vz contains a file system with errors, check forced. ./e2fsprogs-1.42.10/e2fsck/e2fsck: A block group is missing an inode table while reading bad blocks inode This doesn't bode well, but we'll try to go on... Pass 1: Checking inodes, blocks, and sizes # again gets to 51.8% with no problems # again over 100k lines of errors Illegal block #6 (1498565709) in inode 266374005. CLEARED. Inode 266374005 is too big. Truncate? yes Block #73401356 (909652270) causes directory to be too big. CLEARED. Error storing directory block information (inode=266374005, block=0, num=22224176): Memory allocation failed /dev/mapper/vg1--sdb-lv_vz: ***** FILE SYSTEM WAS MODIFIED ***** /dev/mapper/vg1--sdb-lv_vz: ***** FILE SYSTEM WAS MODIFIED ***** Repeated attempts seem to get farther into repairs, but there's still a large number of repairs reported, which seems scary, and it still runs out of memory on a 128GB-memory server. I don't currently have a filesystem with more than 128GB of free space if I wanted to use the scratch_files option (though if that's really the solution, I'll make a way). The 51.8% seems very suspicious to me. A few weeks ago, I did an online resize2fs, and the original filesystem was about 52% the size of the new one (from 2.7TB to 5.3TB). The resize2fs didn't report any errors, and I haven't seen any physical errors in the logs, so this is the first indication I've had of a problem. My tune2fs output and other possible information is below. Is there any hope for this filesytem? The "doesn't bode well" message doesn't give me hope, but perhaps there's some last-ditch efforts I can make to try to recover. If you need any other information about the filesystem please let me know. --keith # uname -a Linux XXX.XXX 2.6.32-042stab090.2 #1 SMP Wed May 21 19:25:03 MSK 2014 x86_64 x86_64 x86_64 GNU/Linux # free -g total used free shared buffers cached Mem: 125 0 125 0 0 0 -/+ buffers/cache: 0 125 Swap: 0 0 0 # lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert lv_local vg0-sda -wi-ao 19.53g lv_swap vg0-sda -wi-a- 2.00g lv_tmp vg0-sda -wi-ao 19.53g lv_usr vg0-sda -wi-ao 19.53g lv_var vg0-sda -wi-ao 19.53g lv_vz vg1-sdb -wi-a- 5.36t # vgs VG #PV #LV #SN Attr VSize VFree vg0-sda 1 5 0 wz--n- 96.09g 15.96g vg1-sdb 1 1 0 wz--n- 5.36t 0 # pvs PV VG Fmt Attr PSize PFree /dev/sda3 vg0-sda lvm2 a-- 96.09g 15.96g /dev/sdb1 vg1-sdb lvm2 a-- 5.36t 0 tune2fs 1.41.12 (17-May-2010) Filesystem volume name: Last mounted on: /vz Filesystem UUID: 74a4ea8b-03ed-4e9c-ab01-8574517cd5af Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: (none) Filesystem state: not clean with errors Errors behavior: Continue Filesystem OS type: Linux Inode count: 359661568 Block count: 1438622720 Reserved block count: 60203550 Free blocks: 1030108897 Free inodes: 357427346 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 681 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 8192 Inode blocks per group: 512 Flex block group size: 16 Filesystem created: Thu May 31 14:47:29 2012 Last mount time: Sun Oct 27 21:48:21 2013 Last write time: Fri May 30 23:22:31 2014 Mount count: 1 Maximum mount count: 21 Last checked: Sun Oct 27 21:38:53 2013 Check interval: 15552000 (6 months) Next check after: Fri Apr 25 21:38:53 2014 Lifetime writes: 14 TB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Default directory hash: half_md4 Directory Hash Seed: 37c55228-9d7b-4a34-bd88-8322f435b9cb Journal backup: inode blocks -- kkeller at wombat.san-francisco.ca.us