From Ralf.Hildebrandt at charite.de Mon Feb 1 16:15:49 2010 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 1 Feb 2010 17:15:49 +0100 Subject: name_count maxed, losing inode data: dev=00:05, inode=5221 Message-ID: <20100201161549.GA3864@charite.de> What does name_count maxed, losing inode data: dev=00:05, inode=5221 mean? Today we booted from 2.6.31 -> 2.6.31.12 and got: ... [ 66.829882] EXT3 FS on sda1, internal journal [ 67.892792] name_count maxed, losing inode data: dev=00:05, inode=5221 [ 67.892875] name_count maxed, losing inode data: dev=00:05, inode=5214 [ 67.892883] name_count maxed, losing inode data: dev=00:05, inode=5225 [ 67.893033] name_count maxed, losing inode data: dev=00:05, inode=5225 [ 67.893045] name_count maxed, losing inode data: dev=00:05, inode=5225 [ 67.893110] name_count maxed, losing inode data: dev=00:05, inode=5214 [ 67.893118] name_count maxed, losing inode data: dev=00:05, inode=5229 [ 67.893270] name_count maxed, losing inode data: dev=00:05, inode=5229 [ 67.893278] name_count maxed, losing inode data: dev=00:05, inode=5229 [ 67.893392] name_count maxed, losing inode data: dev=00:05, inode=5214 [ 67.893401] name_count maxed, losing inode data: dev=00:05, inode=5233 [ 67.893555] name_count maxed, losing inode data: dev=00:05, inode=5233 [ 67.893563] name_count maxed, losing inode data: dev=00:05, inode=5233 [ 67.893634] name_count maxed, losing inode data: dev=00:05, inode=5214 [ 67.893643] name_count maxed, losing inode data: dev=00:05, inode=5237 [ 67.893792] name_count maxed, losing inode data: dev=00:05, inode=5237 [ 67.893801] name_count maxed, losing inode data: dev=00:05, inode=5237 [ 67.893868] name_count maxed, losing inode data: dev=00:05, inode=5214 [ 67.893877] name_count maxed, losing inode data: dev=00:05, inode=5241 [ 67.894010] name_count maxed, losing inode data: dev=00:05, inode=5241 [ 67.894019] name_count maxed, losing inode data: dev=00:05, inode=5241 [ 67.993863] name_count maxed, losing inode data: dev=00:05, inode=5258 [ 67.993934] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.993943] name_count maxed, losing inode data: dev=00:05, inode=5262 [ 67.994130] name_count maxed, losing inode data: dev=00:05, inode=5262 [ 67.994138] name_count maxed, losing inode data: dev=00:05, inode=5262 [ 67.994268] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.994277] name_count maxed, losing inode data: dev=00:05, inode=5266 [ 67.994593] name_count maxed, losing inode data: dev=00:05, inode=5266 [ 67.994602] name_count maxed, losing inode data: dev=00:05, inode=5266 [ 67.994670] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.994684] name_count maxed, losing inode data: dev=00:05, inode=5270 [ 67.995138] name_count maxed, losing inode data: dev=00:05, inode=5270 [ 67.995148] name_count maxed, losing inode data: dev=00:05, inode=5270 [ 67.995222] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.995231] name_count maxed, losing inode data: dev=00:05, inode=5274 [ 67.995369] name_count maxed, losing inode data: dev=00:05, inode=5274 [ 67.995377] name_count maxed, losing inode data: dev=00:05, inode=5274 [ 67.995440] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.995450] name_count maxed, losing inode data: dev=00:05, inode=5278 [ 67.995590] name_count maxed, losing inode data: dev=00:05, inode=5278 [ 67.995599] name_count maxed, losing inode data: dev=00:05, inode=5278 [ 67.995663] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.995672] name_count maxed, losing inode data: dev=00:05, inode=5282 [ 67.995833] name_count maxed, losing inode data: dev=00:05, inode=5282 [ 67.995841] name_count maxed, losing inode data: dev=00:05, inode=5282 [ 67.995906] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.995914] name_count maxed, losing inode data: dev=00:05, inode=5286 [ 67.996126] name_count maxed, losing inode data: dev=00:05, inode=5286 [ 67.996135] name_count maxed, losing inode data: dev=00:05, inode=5286 [ 67.996202] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.996211] name_count maxed, losing inode data: dev=00:05, inode=5290 [ 67.996391] name_count maxed, losing inode data: dev=00:05, inode=5290 [ 67.996400] name_count maxed, losing inode data: dev=00:05, inode=5290 [ 67.996483] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.996491] name_count maxed, losing inode data: dev=00:05, inode=5294 [ 67.996948] name_count maxed, losing inode data: dev=00:05, inode=5294 [ 67.996958] name_count maxed, losing inode data: dev=00:05, inode=5294 [ 67.997028] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.997036] name_count maxed, losing inode data: dev=00:05, inode=5298 [ 67.997190] name_count maxed, losing inode data: dev=00:05, inode=5298 [ 67.997198] name_count maxed, losing inode data: dev=00:05, inode=5298 [ 67.997272] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.997280] name_count maxed, losing inode data: dev=00:05, inode=5302 [ 67.997726] name_count maxed, losing inode data: dev=00:05, inode=5302 [ 67.997737] name_count maxed, losing inode data: dev=00:05, inode=5302 [ 67.997808] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.997817] name_count maxed, losing inode data: dev=00:05, inode=5306 [ 67.998233] name_count maxed, losing inode data: dev=00:05, inode=5306 [ 67.998247] name_count maxed, losing inode data: dev=00:05, inode=5306 [ 67.998322] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.998330] name_count maxed, losing inode data: dev=00:05, inode=5310 [ 67.998722] name_count maxed, losing inode data: dev=00:05, inode=5310 [ 67.998732] name_count maxed, losing inode data: dev=00:05, inode=5310 [ 67.998928] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.998937] name_count maxed, losing inode data: dev=00:05, inode=5314 [ 67.999206] name_count maxed, losing inode data: dev=00:05, inode=5314 [ 67.999216] name_count maxed, losing inode data: dev=00:05, inode=5314 [ 67.999295] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.999304] name_count maxed, losing inode data: dev=00:05, inode=5318 [ 67.999600] name_count maxed, losing inode data: dev=00:05, inode=5318 [ 67.999608] name_count maxed, losing inode data: dev=00:05, inode=5318 [ 67.999680] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.999688] name_count maxed, losing inode data: dev=00:05, inode=5322 [ 67.999844] name_count maxed, losing inode data: dev=00:05, inode=5322 [ 67.999853] name_count maxed, losing inode data: dev=00:05, inode=5322 [ 67.999915] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 67.999924] name_count maxed, losing inode data: dev=00:05, inode=5326 [ 68.000152] name_count maxed, losing inode data: dev=00:05, inode=5326 [ 68.000161] name_count maxed, losing inode data: dev=00:05, inode=5326 [ 68.000225] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 68.000233] name_count maxed, losing inode data: dev=00:05, inode=5330 [ 68.000415] name_count maxed, losing inode data: dev=00:05, inode=5330 [ 68.000423] name_count maxed, losing inode data: dev=00:05, inode=5330 [ 68.000487] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 68.000495] name_count maxed, losing inode data: dev=00:05, inode=5334 [ 68.000741] name_count maxed, losing inode data: dev=00:05, inode=5334 [ 68.000750] name_count maxed, losing inode data: dev=00:05, inode=5334 [ 68.000842] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 68.000851] name_count maxed, losing inode data: dev=00:05, inode=5338 [ 68.001034] name_count maxed, losing inode data: dev=00:05, inode=5338 [ 68.001043] name_count maxed, losing inode data: dev=00:05, inode=5338 [ 68.001104] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 68.001112] name_count maxed, losing inode data: dev=00:05, inode=5342 [ 68.001278] name_count maxed, losing inode data: dev=00:05, inode=5342 [ 68.001286] name_count maxed, losing inode data: dev=00:05, inode=5342 [ 68.001350] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 68.001359] name_count maxed, losing inode data: dev=00:05, inode=5346 [ 68.001518] name_count maxed, losing inode data: dev=00:05, inode=5346 [ 68.001527] name_count maxed, losing inode data: dev=00:05, inode=5346 [ 68.001596] name_count maxed, losing inode data: dev=00:05, inode=5251 [ 68.001604] name_count maxed, losing inode data: dev=00:05, inode=5350 [ 68.001810] name_count maxed, losing inode data: dev=00:05, inode=5350 [ 68.001819] name_count maxed, losing inode data: dev=00:05, inode=5350 [ 68.119718] EXT4-fs (dm-0): barriers enabled ... -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From lists at nerdbynature.de Tue Feb 2 19:40:09 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Tue, 2 Feb 2010 11:40:09 -0800 (PST) Subject: name_count maxed, losing inode data: dev=00:05, inode=5221 In-Reply-To: <20100201161549.GA3864@charite.de> References: <20100201161549.GA3864@charite.de> Message-ID: On Mon, 1 Feb 2010 at 17:15, Ralf Hildebrandt wrote: > What does > name_count maxed, losing inode data: dev=00:05, inode=5221 > mean? I don't know *what* it means exactly, but this has been reported earlier and it's still an open issue: https://bugzilla.redhat.com/show_bug.cgi?id=445757 https://bugzilla.redhat.com/show_bug.cgi?id=495207 Christian. -- BOFH excuse #196: Me no internet, only janitor, me just wax floors. From sandeen at redhat.com Tue Feb 2 19:50:25 2010 From: sandeen at redhat.com (Eric Sandeen) Date: Tue, 02 Feb 2010 13:50:25 -0600 Subject: name_count maxed, losing inode data: dev=00:05, inode=5221 In-Reply-To: References: <20100201161549.GA3864@charite.de> Message-ID: <4B688201.5080306@redhat.com> Christian Kujau wrote: > On Mon, 1 Feb 2010 at 17:15, Ralf Hildebrandt wrote: > >> What does >> name_count maxed, losing inode data: dev=00:05, inode=5221 >> mean? > > I don't know *what* it means exactly, but this has been reported earlier > and it's still an open issue: > > https://bugzilla.redhat.com/show_bug.cgi?id=445757 > https://bugzilla.redhat.com/show_bug.cgi?id=495207 > > Christian. This is all from the audit subsystem; it's not an ext3 issue AFAICT. I'd push on those bugs if you're concerned. :) -Eric /* AUDIT_NAMES is the number of slots we reserve in the audit_context * for saving names from getname(). */ #define AUDIT_NAMES 20 static int audit_inc_name_count(struct audit_context *context, const struct inode *inode) { if (context->name_count >= AUDIT_NAMES) { if (inode) printk(KERN_DEBUG "name_count maxed, losing inode data: " "dev=%02x:%02x, inode=%lu\n", MAJOR(inode->i_sb->s_dev), MINOR(inode->i_sb->s_dev), inode->i_ino); else printk(KERN_DEBUG "name_count maxed, losing inode data\n"); return 1; } context->name_count++; #if AUDIT_DEBUG context->ino_count++; #endif return 0; } From lists at nerdbynature.de Tue Feb 2 20:19:24 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Tue, 2 Feb 2010 12:19:24 -0800 (PST) Subject: name_count maxed, losing inode data: dev=00:05, inode=5221 In-Reply-To: <4B688201.5080306@redhat.com> References: <20100201161549.GA3864@charite.de> <4B688201.5080306@redhat.com> Message-ID: On Tue, 2 Feb 2010 at 13:50, Eric Sandeen wrote: > This is all from the audit subsystem; it's not an ext3 issue AFAICT. Yeah, after grepping for it I've seen this now. However, I still don't understand why it's printed at all and if it's something to worry about. Well, probably not, since it's printed with KERN_DEBUG, but then these messages should be supressed, no? Also, can we alter the prinkt somewhat so it says where the message is coming from? Something like: diff --git a/linux-2.6-git/kernel/auditsc.c.orig b/linux-2.6-git/kernel/auditsc.c index fc0f928..17d8708 100644 --- a/linux-2.6-git/kernel/auditsc.c.orig +++ b/linux-2.6-git/kernel/auditsc.c @@ -1893,14 +1893,14 @@ static int audit_inc_name_count(struct audit_context *context, { if (context->name_count >= AUDIT_NAMES) { if (inode) - printk(KERN_DEBUG "name_count maxed, losing inode data: " + printk(KERN_DEBUG "audit: name_count maxed, losing inode data: " "dev=%02x:%02x, inode=%lu\n", MAJOR(inode->i_sb->s_dev), MINOR(inode->i_sb->s_dev), inode->i_ino); else - printk(KERN_DEBUG "name_count maxed, losing inode data\n"); + printk(KERN_DEBUG "audit: name_count maxed, losing inode data\n"); return 1; } context->name_count++; Thanks, Christian. -- BOFH excuse #298: Not enough interrupts From sandeen at redhat.com Wed Feb 3 17:18:54 2010 From: sandeen at redhat.com (Eric Sandeen) Date: Wed, 03 Feb 2010 11:18:54 -0600 Subject: name_count maxed, losing inode data: dev=00:05, inode=5221 In-Reply-To: References: <20100201161549.GA3864@charite.de> <4B688201.5080306@redhat.com> Message-ID: <4B69AFFE.9070306@redhat.com> Christian Kujau wrote: > On Tue, 2 Feb 2010 at 13:50, Eric Sandeen wrote: >> This is all from the audit subsystem; it's not an ext3 issue AFAICT. > > Yeah, after grepping for it I've seen this now. However, I still don't > understand why it's printed at all and if it's something to worry about. > Well, probably not, since it's printed with KERN_DEBUG, but then these > messages should be supressed, no? > > Also, can we alter the prinkt somewhat so it says where the message is > coming from? Something like: Sure, good idea - can you send that patch upstream, cc: eparis at redhat.com ? -Eric > diff --git a/linux-2.6-git/kernel/auditsc.c.orig b/linux-2.6-git/kernel/auditsc.c > index fc0f928..17d8708 100644 > --- a/linux-2.6-git/kernel/auditsc.c.orig > +++ b/linux-2.6-git/kernel/auditsc.c > @@ -1893,14 +1893,14 @@ static int audit_inc_name_count(struct audit_context *context, > { > if (context->name_count >= AUDIT_NAMES) { > if (inode) > - printk(KERN_DEBUG "name_count maxed, losing inode data: " > + printk(KERN_DEBUG "audit: name_count maxed, losing inode data: " > "dev=%02x:%02x, inode=%lu\n", > MAJOR(inode->i_sb->s_dev), > MINOR(inode->i_sb->s_dev), > inode->i_ino); > > else > - printk(KERN_DEBUG "name_count maxed, losing inode data\n"); > + printk(KERN_DEBUG "audit: name_count maxed, losing inode data\n"); > return 1; > } > context->name_count++; > > > Thanks, > Christian. From lists at nerdbynature.de Wed Feb 3 17:39:18 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Wed, 3 Feb 2010 09:39:18 -0800 (PST) Subject: [PATCH] Re: name_count maxed, losing inode data: dev=00:05, inode=5221 Message-ID: Ralf Hildebrandt reported[0] the following messages on ext3-users: name_count maxed, losing inode data: dev=00:05, inode=5221 because the filesystem in question is indeed ext3. However, this warning is not generated by ext3 code but by the audit framework. While the origins of these messages are discussed elsewhere[1] the following patch modifies the printks in question so that users know where these messages are coming from. There may be other places within the audit framework needing the same treatment. [0] http://www.redhat.com/archives/ext3-users/2010-February/msg00000.html [1] https://bugzilla.redhat.com/show_bug.cgi?id=495207 Signed-off-by: Christian Kujau Reported-by: Ralf Hildebrandt Cc: Eric Paris auditsc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/linux-2.6-git/kernel/auditsc.c.orig b/linux-2.6-git/kernel/auditsc.c index fc0f928..17d8708 100644 --- a/linux-2.6-git/kernel/auditsc.c.orig +++ b/linux-2.6-git/kernel/auditsc.c @@ -1893,14 +1893,14 @@ static int audit_inc_name_count(struct audit_context *context, { if (context->name_count >= AUDIT_NAMES) { if (inode) - printk(KERN_DEBUG "name_count maxed, losing inode data: " + printk(KERN_DEBUG "audit: name_count maxed, losing inode data: " "dev=%02x:%02x, inode=%lu\n", MAJOR(inode->i_sb->s_dev), MINOR(inode->i_sb->s_dev), inode->i_ino); else - printk(KERN_DEBUG "name_count maxed, losing inode data\n"); + printk(KERN_DEBUG "audit: name_count maxed, losing inode data\n"); return 1; } context->name_count++; -- BOFH excuse #298: Not enough interrupts From samix_119 at yahoo.com Fri Feb 5 21:22:51 2010 From: samix_119 at yahoo.com (Muhammed Sameer) Date: Fri, 5 Feb 2010 13:22:51 -0800 (PST) Subject: Going readonly too frequently In-Reply-To: <326877.26414.qm@web112608.mail.gq1.yahoo.com> Message-ID: <218286.90905.qm@web112614.mail.gq1.yahoo.com> Hey, * We upgraded the firmware on our RAID device and the FS seems to be stable for now. * Thank you guys for all your inputs, I really appreciate it! Regards, Muhammed Sameer --- On Tue, 1/26/10, Muhammed Sameer wrote: > From: Muhammed Sameer > Subject: Going readonly too frequently > To: ext3-users at redhat.com > Date: Tuesday, January 26, 2010, 8:15 AM > Hey, > > * Our server is going readonly atleast 10 - 15 times a day > with the below error > > > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device > sdb1): ext3_free_blocks_sb: bit already cleared for block > 53771686 > Jan 26 12:57:36 mailbox kernel: Aborting journal on device > sdb1. > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) > in ext3_reserve_inode_write: Journal has aborted > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) > in ext3_truncate: Journal has aborted > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) > in ext3_reserve_inode_write: Journal has aborted > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) > in ext3_orphan_del: Journal has aborted > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) > in ext3_reserve_inode_write: Journal has aborted > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device sdb1) > in ext3_delete_inode: Journal has aborted > Jan 26 12:57:36 mailbox kernel: > __journal_remove_journal_head: freeing b_committed_data > Jan 26 12:57:36 mailbox last message repeated 3 times > Jan 26 12:57:36 mailbox kernel: ext3_abort called. > Jan 26 12:57:36 mailbox kernel: EXT3-fs error (device > sdb1): ext3_journal_start_sb: Detected aborted journal > Jan 26 12:57:36 mailbox kernel: Remounting filesystem > read-only > Jan 26 12:58:04 mailbox kernel: > __journal_remove_journal_head: freeing b_committed_data > Jan 26 12:58:05 mailbox kernel: > __journal_remove_journal_head: freeing b_committed_data > > > * Our kernel is > 2.6.18-182.el5 > > * Our OS is > Red Hat Enterprise Linux Server release 5.2 (Tikanga) > > * We even tried fsck? but the problem persists > > > Regards, > Muhammed Sameer > > > ? ? ? > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > From lists at nerdbynature.de Mon Feb 8 08:38:33 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Mon, 8 Feb 2010 00:38:33 -0800 (PST) Subject: [PATCH] Re: name_count maxed, losing inode data: dev=00:05, inode=5221 In-Reply-To: References: Message-ID: On Wed, 3 Feb 2010 at 09:39, Christian Kujau wrote: > Ralf Hildebrandt reported[0] the following messages on ext3-users: > > name_count maxed, losing inode data: dev=00:05, inode=5221 > > because the filesystem in question is indeed ext3. However, this warning > is not generated by ext3 code but by the audit framework. While the > origins of these messages are discussed elsewhere[1] the following > patch modifies the printks in question so that users know where these > messages are coming from. There may be other places within the audit > framework needing the same treatment. ...ping? > > [0] http://www.redhat.com/archives/ext3-users/2010-February/msg00000.html > [1] https://bugzilla.redhat.com/show_bug.cgi?id=495207 > > Signed-off-by: Christian Kujau > Reported-by: Ralf Hildebrandt > Cc: Eric Paris > > auditsc.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/linux-2.6-git/kernel/auditsc.c.orig b/linux-2.6-git/kernel/auditsc.c > index fc0f928..17d8708 100644 > --- a/linux-2.6-git/kernel/auditsc.c.orig > +++ b/linux-2.6-git/kernel/auditsc.c > @@ -1893,14 +1893,14 @@ static int audit_inc_name_count(struct audit_context *context, > { > if (context->name_count >= AUDIT_NAMES) { > if (inode) > - printk(KERN_DEBUG "name_count maxed, losing inode data: " > + printk(KERN_DEBUG "audit: name_count maxed, losing inode data: " > "dev=%02x:%02x, inode=%lu\n", > MAJOR(inode->i_sb->s_dev), > MINOR(inode->i_sb->s_dev), > inode->i_ino); > > else > - printk(KERN_DEBUG "name_count maxed, losing inode data\n"); > + printk(KERN_DEBUG "audit: name_count maxed, losing inode data\n"); > return 1; > } > context->name_count++; > > -- > BOFH excuse #298: > > Not enough interrupts > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo at vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > > -- BOFH excuse #258: That's easy to fix, but I can't be bothered. From gpodevij at gmail.com Mon Feb 8 20:59:57 2010 From: gpodevij at gmail.com (=?ISO-8859-1?Q?Ga=EBtan_Podevijn?=) Date: Mon, 8 Feb 2010 21:59:57 +0100 Subject: index directories in ext2 / ext3 Message-ID: Hello Everyone, I am a Belgian student in computer sciences at the Free University of Brussels ( ULB ). I am currently working on a little article. I would like to explain the differences between ext2, ext3 and ext4. During my researches, I read that ext2's directories are implemented with a linked-list structure. I also read Daniel Phillips's article "A Directory index for ext2" (2002) and I'm a bit troubled. Indeed, I see that the HTree data structure was developed for ext2 but I cannot see anywhere that the current ext2 implementation uses that data structure. I opened the dir.c file from ext2 on the last stable kernel sources and I do not see anything about HTree, but the dir.c file from ext3 uses it, if I do not go wrong. Could you explain me why people talk about "index directories for ext2" but there's no ext2 implementation that uses the HTree structure ? Is it only for ext3 ? Since when ? Could you give me some source that explain when was implemented HTree in ext{2,3} ? Thanks a lot. -------------- next part -------------- An HTML attachment was scrubbed... URL: From adilger at sun.com Mon Feb 8 21:47:54 2010 From: adilger at sun.com (Andreas Dilger) Date: Mon, 08 Feb 2010 14:47:54 -0700 Subject: index directories in ext2 / ext3 In-Reply-To: References: Message-ID: On 2010-02-08, at 13:59, Ga?tan Podevijn wrote: > I am currently working on a little article. I would like to explain > the differences between ext2, ext3 and ext4. > > During my researches, I read that ext2's directories are implemented > with a linked-list structure. I also read Daniel Phillips's article > "A Directory index for ext2" (2002) and I'm a bit troubled. > > Indeed, I see that the HTree data structure was developed for ext2 > but I cannot see anywhere that the current ext2 implementation uses > that data structure. That is correct - it was developed for ext2, but since the new filesystem development was happening in ext3 the feature was landed only in ext3. > Could you explain me why people talk about "index directories for > ext2" but there's no ext2 implementation that uses the HTree > structure ? Is it only for ext3 ? Since when ? Could you give me > some source that explain when was implemented HTree in ext{2,3} ? Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. From lakshmipathi.g at gmail.com Thu Feb 11 14:26:47 2010 From: lakshmipathi.g at gmail.com (lakshmi pathi) Date: Thu, 11 Feb 2010 19:56:47 +0530 Subject: giis-ext4 undelete Message-ID: Hi all, Here it's http://www.giis.co.in/giis/, ext4 undelete tool.giis-ext4 uses ext2fs lib and sqlite,thus provides better/best performance than giis for ext3. giis-ext4 has less than 1000 LOC which is almost 5 times less original than giis and giis-ext4 took just around 6 weekends (approx 15 days , 2hrs / day) thanks to ext2fs library :) Feedback/comments on giis-ext4 are most welcome. ps: don't forget to check out the giis-ext4 screencasting. :) -- ---- Cheers, Lakshmipathi.G www.giis.co.in -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at nerdbynature.de Sat Feb 13 07:47:59 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Fri, 12 Feb 2010 23:47:59 -0800 (PST) Subject: giis-ext4 undelete In-Reply-To: References: Message-ID: On Thu, 11 Feb 2010 at 19:56, lakshmi pathi wrote: > Feedback/comments on giis-ext4 are most welcome. I'm always curious about these undeletion attempts and so I've tried it. But without luck again: # giis-ext4 -g Enter your option: 1 giis-ext4: giis-ext4.c:863: open_db: Assertion `error == 0' failed. Aborted # echo $? 134 This has been compiled with - e2fslibs-dev 1.41.10 - libsqlite3-dev 3.6.22 - gcc 4.4.3, with -g -ggdb I've attached the gdb output and tune2fs output to this email. HTH, Christian. -- BOFH excuse #24: network packets travelling uphill (use a carrier pigeon) -------------- next part -------------- # # Makefile # CFLAGS = -g -ggdb LDFLAGS = -lext2fs -lsqlite3 giis-ext4: gcc $(CFLAGS) $(LDFLAGS) src/giis-ext4.c -o giis-ext4 clean: rm -f giis-ext4 all: giis-ext4 .PHONY: clean .PHONY: giis-ext4 all -------------- next part -------------- tune2fs 1.41.10 (10-Feb-2009) Filesystem volume name: Last mounted on: /mnt/d1 Filesystem UUID: 65de9933-1be8-49cc-909c-e324a49f62d6 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 128016 Block count: 512000 Reserved block count: 25600 Free blocks: 485334 Free inodes: 128005 First block: 1 Block size: 1024 Fragment size: 1024 Reserved GDT blocks: 256 Blocks per group: 8192 Fragments per group: 8192 Inodes per group: 2032 Inode blocks per group: 254 Flex block group size: 16 Filesystem created: Sat Feb 13 07:35:00 2010 Last mount time: Sat Feb 13 07:35:09 2010 Last write time: Sat Feb 13 07:35:09 2010 Mount count: 1 Maximum mount count: 21 Last checked: Sat Feb 13 07:35:00 2010 Check interval: 15552000 (6 months) Next check after: Thu Aug 12 08:35:00 2010 Lifetime writes: 24 MB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 Default directory hash: half_md4 Directory Hash Seed: 3990aa48-db93-4905-82dc-6e09c49ef971 Journal backup: inode blocks -------------- next part -------------- # grep /mnt/d1 /proc/mounts /dev/mapper/vg0-lv0 /mnt/d1 ext4 rw,relatime,barrier=1,data=ordered 0 0 # gdb /usr/local/src/giis-ext4/bin/giis-ext4 GNU gdb (GDB) 7.0.1-debian This GDB was configured as "x86_64-linux-gnu". Reading symbols from /usr/local/src/giis-ext4/bin/giis-ext4...(no debugging symbols found)...done. (gdb) run -g Starting program: /usr/local/src/giis-ext4/bin/giis-ext4 -g [Thread debugging using libthread_db enabled] Device Found : /dev/xvda1 press 1: get all user files press 2: get specific user files press 3: get specific file type press 4: get specific file press 5: get it by deleted date Enter your option:1 giis-ext4: giis-ext4.c:863: open_db: Assertion `error == 0' failed. Program received signal SIGABRT, Aborted. 0x00007ffff7603f45 in *__GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. in ../nptl/sysdeps/unix/sysv/linux/raise.c (gdb) bt #0 0x00007ffff7603f45 in *__GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007ffff7606d80 in *__GI_abort () at abort.c:88 #2 0x00007ffff75fd08a in *__GI___assert_fail (assertion=0x404304 "error == 0", file=, line=863, function=0x404dbf "open_db") at assert.c:78 #3 0x0000000000403bdc in open_db () #4 0x00000000004024ce in giis_ext4_recover_all () #5 0x0000000000401831 in main () (gdb) bt full #0 0x00007ffff7603f45 in *__GI_raise (sig=) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 pid = selftid = #1 0x00007ffff7606d80 in *__GI_abort () at abort.c:88 act = {__sigaction_handler = {sa_handler = 0x404304 <__dso_handle+1276>, sa_sigaction = 0x404304 <__dso_handle+1276>}, sa_mask = {__val = {140737344616945, 140737488348880, 863, 140737488349120, 140737343861862, 206158430232, 140737488349136, 140737488348912, 140737343776536, 206158430256, 140737488349160, 6357648, 6357632, 1, 4211460, 140737488350776}}, sa_flags = -143746404, sa_restorer = 0x4042f8 <__dso_handle+1264>} sigs = {__val = {32, 0 }} #2 0x00007ffff75fd08a in *__GI___assert_fail (assertion=0x404304 "error == 0", file=, line=863, function=0x404dbf "open_db") at assert.c:78 buf = 0x610290 "" #3 0x0000000000403bdc in open_db () No symbol table info available. #4 0x00000000004024ce in giis_ext4_recover_all () No symbol table info available. #5 0x0000000000401831 in main () No symbol table info available. (gdb) quit A debugging session is active. Inferior 1 [process 5065] will be killed. Quit anyway? (y or n) y # file /usr/local/src/giis-ext4/bin/giis-ext4 /usr/local/src/giis-ext4/bin/giis-ext4: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked (uses shared libs), for GNU/Linux 2.6.18, not stripped # ldd /usr/local/src/giis-ext4/bin/giis-ext4 linux-vdso.so.1 => (0x00007fffbbdff000) libsqlite3.so.0 => /usr/lib/libsqlite3.so.0 (0x00007fdb7e324000) libext2fs.so.2 => /lib/libext2fs.so.2 (0x00007fdb7e0f6000) libc.so.6 => /lib/libc.so.6 (0x00007fdb7dda1000) libpthread.so.0 => /lib/libpthread.so.0 (0x00007fdb7db85000) libdl.so.2 => /lib/libdl.so.2 (0x00007fdb7d981000) libcom_err.so.2 => /lib/libcom_err.so.2 (0x00007fdb7d77d000) /lib64/ld-linux-x86-64.so.2 (0x00007fdb7e5b8000) # ls -lgo /usr/local/src/giis-ext4/bin/giis-ext4 -rwx------ 1 30800 Feb 10 20:24 /usr/local/src/giis-ext4/bin/giis-ext4 From lakshmipathi.g at gmail.com Sat Feb 13 14:46:26 2010 From: lakshmipathi.g at gmail.com (lakshmi pathi) Date: Sat, 13 Feb 2010 20:16:26 +0530 Subject: giis-ext4 undelete In-Reply-To: References: Message-ID: Thanks for your feedback/comments.I have following queries, 1.Is your root file system (/dev/xvda1) is ext4? 2.Have you added the directory /mnt/d1 using "giis-ext4 --install" option? Because you need to configure it to protect important directories, in this case add /mnt/d1 before running "giis-ext4 -g". I get the same error message,when giis-ext4 -g is invoked without configuring it using "giis-ext4 --install" ----------- giis-ext4: giis-ext4.c:863: open_db: Assertion `error == 0' failed. Aborted (core dumped) ------------- See my comments (/* */) on right-side in following output: ------------------------- # giis-ext4 --install Device Found : /dev/mapper/vg_space-lv_root /* root device taken from /etc/mtab */ giis : Taking snapshot of current File system giis-ext4:Installation begins.. giis-ext4: header table created giis-ext4: file table created What's the maximum directory depth?5 /*Max. directory depth involved */ Enter the dirname name,that you would like to protect(Max. 7 directories) Enter dirname:/mnt/d1 /* Enter important directories to protect */ Press 1 to add/protect another directory else Press 0 to complete: 1 Enter dirname:/opt/testing /* Enter important directories */ Press 1 to add/protect another directory else Press 0 to complete: 0 Check for newly files every 'auto update time' minutes. Enter auto update time: 10 /* Every 10 minutes giis-ext4 , searches for any modified/new files under protected dirs*/ ------------------- Now if you delete files from protected dirs (say /mnt/d1),then you can recover them using "giis-ext4 -g " option. Thanks for the Makefile too,I'll use it :) If you are stilling getting same error , please let me know. -- ---- Cheers, Lakshmipathi.G www.giis.co.in On Sat, Feb 13, 2010 at 1:17 PM, Christian Kujau wrote: > On Thu, 11 Feb 2010 at 19:56, lakshmi pathi wrote: > > Feedback/comments on giis-ext4 are most welcome. > > I'm always curious about these undeletion attempts and so I've tried it. > But without luck again: > > # giis-ext4 -g > Enter your option: 1 > giis-ext4: giis-ext4.c:863: open_db: Assertion `error == 0' failed. > Aborted > # echo $? > 134 > > This has been compiled with > - e2fslibs-dev 1.41.10 > - libsqlite3-dev 3.6.22 > - gcc 4.4.3, with -g -ggdb > > I've attached the gdb output and tune2fs output to this email. > > HTH, > Christian. > -- > BOFH excuse #24: > > network packets travelling uphill (use a carrier pigeon) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at nerdbynature.de Sat Feb 13 21:20:36 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Sat, 13 Feb 2010 13:20:36 -0800 (PST) Subject: giis-ext4 undelete In-Reply-To: References: Message-ID: On Sat, 13 Feb 2010 at 20:16, lakshmi pathi wrote: > 1.Is your root file system (/dev/xvda1) is ext4? My root filesystem is ext4 too, yet, but I tried to somehow convince giis-ext4 to undeleted files from another device, mounted to /mnt/d1. > 2.Have you added the directory /mnt/d1 using "giis-ext4 --install" option? No, I did not do that. I have the the INSTALL file mentioning the "-i" switch but I did not know what it meant with "which directories to protect". I take it that giis-ext4 will be running as a cronjob, scanning the "protected directories" and keeping a log of changes in there, to be able to recover them later on? > Because you need to configure it to protect important directories, in this > case add /mnt/d1 before running "giis-ext4 -g". I've tried that now. However, I cannot add /mnt/d1 (which is a ext4 mountpoint) to the list, as giis-ext4 will segfault then. When I add some directory from the root filesystem, --install will run successfully. Christian. Script started on Sat Feb 13 21:59:34 2010 # df -h . Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg0-lv0 485M 35M 425M 8% /mnt/d1 # gdb /usr/local/src/giis-ext4/bin/giis-ext4 GNU gdb (GDB) 7.0.1-debian Reading symbols from /usr/local/src/giis-ext4/bin/giis-ext4...(no debugging symbols found)...done. (gdb) run -i Starting program: /usr/local/src/giis-ext4/bin/giis-ext4 -i [Thread debugging using libthread_db enabled] Device Found : /dev/xvda1 giis : Taking snapshot of current File system [New Thread 0x7ffff6fae910 (LWP 6592)] [Thread 0x7ffff6fae910 (LWP 6592) exited] giis-ext4:Installation begins.. giis-ext4: header table created giis-ext4: file table created What's the maximum directory depth?5 Enter the dirname name,that you would like to protect(Max. 7 directories) Enter dirname:/mnt/d1 Press 1 to add/protect another directory else Press 0 to complete: 0 Check for newly files every 'auto update time' minutes. Enter auto update time: 10 Parsing directory : /mnt/d1 Program received signal SIGSEGV, Segmentation fault. strlen () at ../sysdeps/x86_64/strlen.S:31 31 ../sysdeps/x86_64/strlen.S: No such file or directory. in ../sysdeps/x86_64/strlen.S Current language: auto The current source language is "auto; currently asm". (gdb) bt #0 strlen () at ../sysdeps/x86_64/strlen.S:31 #1 0x0000000000402134 in giis_ext4_sqlite_insert_record () #2 0x0000000000401c05 in giis_ext4_parse_dir () #3 0x0000000000401ad2 in giis_ext4_parse_dir () #4 0x0000000000403ae6 in giis_ext4_update_dirs () #5 0x0000000000401707 in main () (gdb) bt full #0 strlen () at ../sysdeps/x86_64/strlen.S:31 No locals. #1 0x0000000000402134 in giis_ext4_sqlite_insert_record () No symbol table info available. #2 0x0000000000401c05 in giis_ext4_parse_dir () No symbol table info available. #3 0x0000000000401ad2 in giis_ext4_parse_dir () No symbol table info available. #4 0x0000000000403ae6 in giis_ext4_update_dirs () No symbol table info available. #5 0x0000000000401707 in main () No symbol table info available. (gdb) quit A debugging session is active. Inferior 1 [process 6589] will be killed. Quit anyway? (y or n) y # exit Script done on Sat Feb 13 22:01:05 2010 -- BOFH excuse #283: Lawn mower blade in your fan need sharpening From lakshmipathi.g at gmail.com Sun Feb 14 04:10:39 2010 From: lakshmipathi.g at gmail.com (lakshmi pathi) Date: Sun, 14 Feb 2010 09:40:39 +0530 Subject: giis-ext4 undelete In-Reply-To: References: Message-ID: My root filesystem is ext4 too, yet, but I tried to somehow convince giis-ext4 to undeleted files from another device, mounted to /mnt/d1. okay,mounting another ext4 device should be fine with giis-ext4. No, I did not do that. So that's reason for previous error message on "open_db". giis-ext4 will be running as a cronjob, scanning the "protected directories" and keeping a log of changes in there, to be able to recover them later on? giis-ext4 will run from cronjob from every "auto update time" and scanning protected directories and keeps track of files meta-data (inode) but not actual file contents.(data blocks) If something deleted and assuming the deleted files contents are not overwritten,yet. It recovers them using files meta-data. So the disadvantage is , giis-ext4 won't recover files deleted before it's installation. Program received signal SIGSEGV, Segmentation fault. strlen () at ../sysdeps/x86_64/strlen.S:31 I haven't seen this error with Fedora/Ubuntu , I'll debug more about this and hopefully find a solution and post it here. Thanks. -- ---- Cheers, Lakshmipathi.G www.giis.co.in On Sun, Feb 14, 2010 at 2:50 AM, Christian Kujau wrote: > On Sat, 13 Feb 2010 at 20:16, lakshmi pathi wrote: > > 1.Is your root file system (/dev/xvda1) is ext4? > > My root filesystem is ext4 too, yet, but I tried to somehow convince > giis-ext4 to undeleted files from another device, mounted to /mnt/d1. > > > 2.Have you added the directory /mnt/d1 using "giis-ext4 --install" > option? > > No, I did not do that. I have the the INSTALL file mentioning the "-i" > switch but I did not know what it meant with "which directories to > protect". I take it that giis-ext4 will be running as a cronjob, scanning > the "protected directories" and keeping a log of changes in there, to be > able to recover them later on? > > > Because you need to configure it to protect important directories, in > this > > case add /mnt/d1 before running "giis-ext4 -g". > > I've tried that now. However, I cannot add /mnt/d1 (which is a ext4 > mountpoint) to the list, as giis-ext4 will segfault then. When I add some > directory from the root filesystem, --install will run successfully. > > Christian. > > > Script started on Sat Feb 13 21:59:34 2010 > # df -h . > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/vg0-lv0 485M 35M 425M 8% /mnt/d1 > > # gdb /usr/local/src/giis-ext4/bin/giis-ext4 > GNU gdb (GDB) 7.0.1-debian > Reading symbols from /usr/local/src/giis-ext4/bin/giis-ext4...(no debugging > symbols found)...done. > (gdb) run -i > Starting program: /usr/local/src/giis-ext4/bin/giis-ext4 -i > [Thread debugging using libthread_db enabled] > > Device Found : /dev/xvda1 > giis : Taking snapshot of current File system > [New Thread 0x7ffff6fae910 (LWP 6592)] > > [Thread 0x7ffff6fae910 (LWP 6592) exited] > giis-ext4:Installation begins.. > giis-ext4: header table created > giis-ext4: file table created > What's the maximum directory depth?5 > > Enter the dirname name,that you would like to protect(Max. 7 directories) > Enter dirname:/mnt/d1 > > Press 1 to add/protect another directory else Press 0 to complete: 0 > > Check for newly files every 'auto update time' minutes. > Enter auto update time: 10 > > Parsing directory : /mnt/d1 > > Program received signal SIGSEGV, Segmentation fault. > strlen () at ../sysdeps/x86_64/strlen.S:31 > 31 ../sysdeps/x86_64/strlen.S: No such file or directory. > in ../sysdeps/x86_64/strlen.S > Current language: auto > The current source language is "auto; currently asm". > (gdb) bt > #0 strlen () at ../sysdeps/x86_64/strlen.S:31 > #1 0x0000000000402134 in giis_ext4_sqlite_insert_record () > #2 0x0000000000401c05 in giis_ext4_parse_dir () > #3 0x0000000000401ad2 in giis_ext4_parse_dir () > #4 0x0000000000403ae6 in giis_ext4_update_dirs () > #5 0x0000000000401707 in main () > (gdb) bt full > #0 strlen () at ../sysdeps/x86_64/strlen.S:31 > No locals. > #1 0x0000000000402134 in giis_ext4_sqlite_insert_record () > No symbol table info available. > #2 0x0000000000401c05 in giis_ext4_parse_dir () > No symbol table info available. > #3 0x0000000000401ad2 in giis_ext4_parse_dir () > No symbol table info available. > #4 0x0000000000403ae6 in giis_ext4_update_dirs () > No symbol table info available. > #5 0x0000000000401707 in main () > No symbol table info available. > (gdb) quit > A debugging session is active. > > Inferior 1 [process 6589] will be killed. > > Quit anyway? (y or n) y > # exit > > Script done on Sat Feb 13 22:01:05 2010 > -- > BOFH excuse #283: > > Lawn mower blade in your fan need sharpening > -------------- next part -------------- An HTML attachment was scrubbed... URL: From reza at parvan.net Sat Feb 20 04:15:30 2010 From: reza at parvan.net (Reza Roboubi) Date: Fri, 19 Feb 2010 20:15:30 -0800 Subject: Is my data checksummed? Message-ID: <4B7F61E2.9090404@parvan.net> What checksumming is done for the actual data? I know that storage devices often do their own checksumming too, but how can I be sure my data is integrity checked every time I read it? PS: ext2/ext4? Thank you very much. Reza. From lists at nerdbynature.de Sun Feb 21 00:42:25 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Sat, 20 Feb 2010 16:42:25 -0800 (PST) Subject: Is my data checksummed? In-Reply-To: <4B7F61E2.9090404@parvan.net> References: <4B7F61E2.9090404@parvan.net> Message-ID: On Fri, 19 Feb 2010 at 20:15, Reza Roboubi wrote: > What checksumming is done for the actual data? Ext4 introduced journal checksumming[0], but not for the actual data. Ext2/3 don't have checksums at all, IIRC. > often do their own checksumming too, but how can I be sure my data is > integrity checked every time I read it? Btrfs has data checksumming :-) Christian. [0] http://ext4.wiki.kernel.org/index.php/Ext4_Howto#Journal_checksumming -- BOFH excuse #144: Too few computrons available. From whats at wekk.net Sun Feb 21 11:51:17 2010 From: whats at wekk.net (Albert =?ISO-8859-1?Q?Sellar=E8s?=) Date: Sun, 21 Feb 2010 12:51:17 +0100 Subject: Invalid superblock after e2fsck Message-ID: <1266753077.6554.24.camel@x61s> Hi all, I'm trying to fix a 7.5Tb ext3 filesystem using e2fsck on a x86_64 machine with plenty of memory ram. The filesystem is corrupted, but I can mount it. Before starting the filesystem check, I did a LVM snapshot to be able to start it again from the same point in case of error. After 12 hours checking the filesystem, I got this error message: Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: 65673.66/1550.92/1371.06 Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s Restarting e2fsck from the beginning... e2fsck: Superblock invalid, trying backup blocks... e2fsck: Bad magic number in super-block while trying to open /dev/storage/cabina_snapshot Once I saw this message, my first thought was that e2fsck didn't manage to fix the filesystem and it corrupted the superblock. To be sure I dumped the entire block and compared it against the original superblock. Doing that I realized that the entire superblock only contained zeros. Any ideas of what can I do? Thanks you very much. -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Aix? ?s una part d'un missatge signada digitalment URL: From darkonc at gmail.com Sun Feb 21 12:51:06 2010 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Sun, 21 Feb 2010 04:51:06 -0800 Subject: Invalid superblock after e2fsck In-Reply-To: <1266753077.6554.24.camel@x61s> References: <1266753077.6554.24.camel@x61s> Message-ID: <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> The system is using the backup superblock -- which sounds reasonable, under the circumstances, and should result in a half-decent recovery. A couple of questions: Was the superblock zero to begin with, or did it become zero during the FSCK? In either case, I'm worried about the zero data having been written. This is obviously worth further investigation. Did the system abort the FSCK after this error? or did you stop it? did you try explicitly using one of the backup superblocks? a list of backup superblocks can be found by using -n on mkfs check the -b option on fsck.ext2 for more details on the backup superblock defaults. 2010/2/21 Albert Sellar?s > Hi all, > > I'm trying to fix a 7.5Tb ext3 filesystem using e2fsck on a x86_64 > machine with plenty of memory ram. The filesystem is corrupted, but I > can mount it.is > > Before starting the filesystem check, I did a LVM snapshot to be able to > start it again from the same point in case of error. > > After 12 hours checking the filesystem, I got this error message: > > Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: > 65673.66/1550.92/1371.06 > Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s > Restarting e2fsck from the beginning... > e2fsck: Superblock invalid, trying backup blocks... > e2fsck: Bad magic number in super-block while trying to > open /dev/storage/cabina_snapshot > > Once I saw this message, my first thought was that e2fsck didn't manage > to fix the filesystem and it corrupted the superblock. To be sure I > dumped the entire block and compared it against the original superblock. > Doing that I realized that the entire superblock only contained zeros. > > Any ideas of what can I do? > > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From whats at wekk.net Sun Feb 21 14:00:15 2010 From: whats at wekk.net (Albert =?ISO-8859-1?Q?Sellar=E8s?=) Date: Sun, 21 Feb 2010 15:00:15 +0100 Subject: Invalid superblock after e2fsck In-Reply-To: <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> References: <1266753077.6554.24.camel@x61s> <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> Message-ID: <1266760815.6554.59.camel@x61s> Hi, e2fsck quits after the bad magic number message. I didn't pasted the entire e2fsck output (sorry!). The full output was that: root at pacs:~# e2fsck -C 0 -y -t /dev/storage/cabina_snapshot2 e2fsck 1.41.9 (22-Aug-2009) /dev/storage/cabina_snapshot2 contains a file system with errors, check forced. Pass 1: Checking inodes, blocks, and sizes /dev/storage/cabina_snapshot2: |================== / 60.0% [...] Until 60% of the progress bar, there was no error messages. After reach the 60%, e2fsck started fixing a lot of things. In the output I recognized the inode that it was analyzing. Once e2fsck analyzed the last inode of the filesystem, it printed this message and exited: Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: 65673.66/1550.92/1371.06 Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s Restarting e2fsck from the beginning... e2fsck: Superblock invalid, trying backup blocks... e2fsck: Bad magic number in super-block while trying to open /dev/storage/cabina_snapshot The superblock could not be read or does not describe a correct ext2 filesystem. If the device is valid and it really contains an ext2 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate superblock: e2fsck -b 8193 About your questions: The content of the default superblock (I mean the superblock located at block 0) was correct before to launch e2fsck because I was able to mount it and use the filesystem. (I also dumped their content and checked it). After the filesystem check, the superblock was filled only with zeros (no magic number, no inode counts, etc...), all the 4096 bytes at zero. I've not tried to launch the e2fsck over a backup superblock instead of over the default one because I think that this would produce the same result. I'm sure that the original superblock was not corrupted then I guess that with a backup superblock, e2fsck would have the same behavior. What do you think? Should I try to launch it over a backup superblock at the first time? Any other ideas? Thanks you very much! El dg 21 de 02 de 2010 a les 04:51 -0800, en/na Stephen Samuel (gmail) va escriure: > The system is using the backup superblock -- which sounds reasonable, > under the circumstances, and should result in a half-decent recovery. > > A couple of questions: > Was the superblock zero to begin with, or did it become zero during > the FSCK? > In either case, I'm worried about the zero data having been written. > This is obviously worth further investigation. > Did the system abort the FSCK after this error? or did you stop it? > did you try explicitly using one of the backup superblocks? > a list of backup superblocks can be found by using -n on mkfs > check the -b option on fsck.ext2 for more details on the backup > superblock defaults. > > 2010/2/21 Albert Sellar?s > Hi all, > > I'm trying to fix a 7.5Tb ext3 filesystem using e2fsck on a > x86_64 > machine with plenty of memory ram. The filesystem is > corrupted, but I > can mount it.is > > Before starting the filesystem check, I did a LVM snapshot to > be able to > start it again from the same point in case of error. > > After 12 hours checking the filesystem, I got this error > message: > > Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: > 65673.66/1550.92/1371.06 > Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s > Restarting e2fsck from the beginning... > e2fsck: Superblock invalid, trying backup blocks... > e2fsck: Bad magic number in super-block while trying to > open /dev/storage/cabina_snapshot > > Once I saw this message, my first thought was that e2fsck > didn't manage > to fix the filesystem and it corrupted the superblock. To be > sure I > dumped the entire block and compared it against the original > superblock. > Doing that I realized that the entire superblock only > contained zeros. > > Any ideas of what can I do? > > > -- > Stephen Samuel http://www.bcgreen.com Software, like love, > 778-861-7641 grows when you give it away -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Aix? ?s una part d'un missatge signada digitalment URL: From darkonc at gmail.com Sun Feb 21 17:38:43 2010 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Sun, 21 Feb 2010 09:38:43 -0800 Subject: Invalid superblock after e2fsck In-Reply-To: <1266760815.6554.59.camel@x61s> References: <1266753077.6554.24.camel@x61s> <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> <1266760815.6554.59.camel@x61s> Message-ID: <6cd50f9f1002210938o568dd1e6ga3b38c75aa3cdcc1@mail.gmail.com> I'm worried toat fsck is corrupting your filesystem.... That implies that something else is seriously wrong -- either the controller, the kernel or the copy of e2fsck that you're using. At this point, I'd suggest doing the FSCK from a CD. If you have the time to work on this, you can try an 'fsck -n' to see if you have good resuts without writing to the drive. In the meantime, I would also verify the FSCK instance. If you're running Red Hat, that would be an rpm --verify {package-name} {package-name} is probably e2fstools, but I'm not currently on a redhat system to check that right now. I'm not sure what the debian equivalent is. manually starting fsck also has the advantage that the system will be more resliant to errors. It asks questions before it does things, but it will ask to fix problems that would cause an abort during a pre-mount preen. In any case, boot from a CD, do a system memory check (to make sure you have no bad RAM), and then manually start the FSCK. If all of that works, then I would say that your root filesystem is corrupt. Try using chkrootkit to see if there's a root kit installed, or just reload the OS. 2010/2/21 Albert Sellar?s > Hi, > > e2fsck quits after the bad magic number message. I didn't pasted the > entire e2fsck output (sorry!). The full output was that: > > root at pacs:~# e2fsck -C 0 -y -t /dev/storage/cabina_snapshot2 > e2fsck 1.41.9 (22-Aug-2009) > /dev/storage/cabina_snapshot2 contains a file system with errors, check > forced. > Pass 1: Checking inodes, blocks, and sizes > /dev/storage/cabina_snapshot2: |================== / 60.0% > > [...] > > Until 60% of the progress bar, there was no error messages. After reach > the 60%, e2fsck started fixing a lot of things. In the output I > recognized the inode that it was analyzing. Once e2fsck analyzed the > last inode of the filesystem, it printed this message and exited: > > Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: > 65673.66/1550.92/1371.06 > Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s > Restarting e2fsck from the beginning... > e2fsck: Superblock invalid, trying backup blocks... > e2fsck: Bad magic number in super-block while trying to > open /dev/storage/cabina_snapshot > > The superblock could not be read or does not describe a correct ext2 > filesystem. If the device is valid and it really contains an ext2 > filesystem (and not swap or ufs or something else), then the superblock > is corrupt, and you might try running e2fsck with an alternate > superblock: > e2fsck -b 8193 > > About your questions: > > The content of the default superblock (I mean the superblock located at > block 0) was correct before to launch e2fsck because I was able to mount > it and use the filesystem. (I also dumped their content and checked it). > > After the filesystem check, the superblock was filled only with zeros > (no magic number, no inode counts, etc...), all the 4096 bytes at zero. > > > I've not tried to launch the e2fsck over a backup superblock instead of > over the default one because I think that this would produce the same > result. I'm sure that the original superblock was not corrupted then I > guess that with a backup superblock, e2fsck would have the same > behavior. > > What do you think? Should I try to launch it over a backup superblock at > the first time? > > Any other ideas? > > Thanks you very much! > > > El dg 21 de 02 de 2010 a les 04:51 -0800, en/na Stephen Samuel (gmail) > va escriure: > > The system is using the backup superblock -- which sounds reasonable, > > under the circumstances, and should result in a half-decent recovery. > > > > A couple of questions: > > Was the superblock zero to begin with, or did it become zero during > > the FSCK? > > In either case, I'm worried about the zero data having been written. > > This is obviously worth further investigation. > > Did the system abort the FSCK after this error? or did you stop it? > > did you try explicitly using one of the backup superblocks? > > a list of backup superblocks can be found by using -n on mkfs > > check the -b option on fsck.ext2 for more details on the backup > > superblock defaults. > > > > 2010/2/21 Albert Sellar?s > > Hi all, > > > > I'm trying to fix a 7.5Tb ext3 filesystem using e2fsck on a > > x86_64 > > machine with plenty of memory ram. The filesystem is > > corrupted, but I > > can mount it.is > > > > Before starting the filesystem check, I did a LVM snapshot to > > be able to > > start it again from the same point in case of error. > > > > After 12 hours checking the filesystem, I got this error > > message: > > > > Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: > > 65673.66/1550.92/1371.06 > > Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s > > Restarting e2fsck from the beginning... > > e2fsck: Superblock invalid, trying backup blocks... > > e2fsck: Bad magic number in super-block while trying to > > open /dev/storage/cabina_snapshot > > > > Once I saw this message, my first thought was that e2fsck > > didn't manage > > to fix the filesystem and it corrupted the superblock. To be > > sure I > > dumped the entire block and compared it against the original > > superblock. > > Doing that I realized that the entire superblock only > > contained zeros. > > > > Any ideas of what can I do? > > > > > > -- > > Stephen Samuel http://www.bcgreen.com Software, like love, > > 778-861-7641 grows when you give it away > -- > Albert Sellar?s GPG id: 0x13053FFE > http://www.wekk.net whats at jabber.org > Linux User: 324456 > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From jlforrest at berkeley.edu Sun Feb 21 17:56:05 2010 From: jlforrest at berkeley.edu (Jon Forrest) Date: Sun, 21 Feb 2010 09:56:05 -0800 Subject: Invalid superblock after e2fsck In-Reply-To: <1266753077.6554.24.camel@x61s> References: <1266753077.6554.24.camel@x61s> Message-ID: <4B8173B5.9050907@berkeley.edu> On 2/21/2010 3:51 AM, Albert Sellar?s wrote: > Hi all, > > I'm trying to fix a 7.5Tb ext3 filesystem using e2fsck on a x86_64 > machine with plenty of memory ram. The filesystem is corrupted, but I > can mount it. What partition table format are you using? I sure hope it's not msdos. If it is your problem might be caused by this. I once had similar problems. I was able to replace the msdos table with a GPT table, and then use the gparted 'rescue' command to recover without loosing a single bit of data. Cordially, -- Jon Forrest Research Computing Support College of Chemistry 173 Tan Hall University of California Berkeley Berkeley, CA 94720-1460 510-643-1032 jlforrest at berkeley.edu From whats at wekk.net Sun Feb 21 18:57:14 2010 From: whats at wekk.net (Albert =?ISO-8859-1?Q?Sellar=E8s?=) Date: Sun, 21 Feb 2010 19:57:14 +0100 Subject: Invalid superblock after e2fsck In-Reply-To: <6cd50f9f1002210938o568dd1e6ga3b38c75aa3cdcc1@mail.gmail.com> References: <1266753077.6554.24.camel@x61s> <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> <1266760815.6554.59.camel@x61s> <6cd50f9f1002210938o568dd1e6ga3b38c75aa3cdcc1@mail.gmail.com> Message-ID: <1266778634.6554.102.camel@x61s> Hi, the filesystem is on a SAN device and was connected to a 32bits RHEL4 system. To be able to check it with enough ram and with a new kernel, I installed an ubuntu karmic on a new x86_64 machine and I plugged the SAN to it. The new machine have 4Gb of RAM and 90Gb of SWAP. Thinking in what you say, I think that the only thing that can be "broken" in the new system is the hard drive because is the only part not completely new, however, I will verify memory RAM. By the way, today before to send the first email, I've downloaded the last version of e2fstools (only one version upper that the ubuntu package), I've compiled it and I've launched the filesystem check using the new version of e2fsck. Now, the process e2fsck is at 60% and it is using 28Gb of memory. I don't understand why the first filesystem check (the one that I explained in the first email) didn't almost use SWAP space. It is weird to see that there is a big difference in memory usage between both versions. Is it "normal"? What can be happening? Sorry for bother you with so many questions. :( Thanks you very much. El dg 21 de 02 de 2010 a les 09:38 -0800, en/na Stephen Samuel (gmail) va escriure: > I'm worried toat fsck is corrupting your filesystem.... That implies > that something else is seriously wrong -- either the controller, the > kernel or the copy of e2fsck that you're using. > > At this point, I'd suggest doing the FSCK from a CD. If you have the > time to work on this, you can try an 'fsck -n' to see if you have good > resuts without writing to the drive. > In the meantime, I would also verify the FSCK instance. If you're > running Red Hat, that would be an > rpm --verify {package-name} > {package-name} is probably e2fstools, but I'm not currently on a > redhat system to check that right now. I'm not sure what the debian > equivalent is. > > manually starting fsck also has the advantage that the system will be > more resliant to errors. It asks questions before it does things, but > it will ask to fix problems that would cause an abort during a > pre-mount preen. > > In any case, boot from a CD, do a system memory check (to make sure > you have no bad RAM), and then manually start the FSCK. If all of > that works, then I would say that your root filesystem is corrupt. > Try using chkrootkit to see if there's a root kit installed, or just > reload the OS. > > > 2010/2/21 Albert Sellar?s > Hi, > > e2fsck quits after the bad magic number message. I didn't > pasted the > entire e2fsck output (sorry!). The full output was that: > > root at pacs:~# e2fsck -C 0 -y -t /dev/storage/cabina_snapshot2 > e2fsck 1.41.9 (22-Aug-2009) > /dev/storage/cabina_snapshot2 contains a file system with > errors, check > forced. > Pass 1: Checking inodes, blocks, and sizes > /dev/storage/cabina_snapshot2: |================== > / 60.0% > > [...] > > Until 60% of the progress bar, there was no error messages. > After reach > the 60%, e2fsck started fixing a lot of things. In the output > I > recognized the inode that it was analyzing. Once e2fsck > analyzed the > last inode of the filesystem, it printed this message and > exited: > > Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: > 65673.66/1550.92/1371.06 > Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s > Restarting e2fsck from the beginning... > e2fsck: Superblock invalid, trying backup blocks... > e2fsck: Bad magic number in super-block while trying to > open /dev/storage/cabina_snapshot > > > The superblock could not be read or does not describe a > correct ext2 > filesystem. If the device is valid and it really contains an > ext2 > filesystem (and not swap or ufs or something else), then the > superblock > is corrupt, and you might try running e2fsck with an alternate > superblock: > e2fsck -b 8193 > > About your questions: > > The content of the default superblock (I mean the superblock > located at > block 0) was correct before to launch e2fsck because I was > able to mount > it and use the filesystem. (I also dumped their content and > checked it). > > After the filesystem check, the superblock was filled only > with zeros > (no magic number, no inode counts, etc...), all the 4096 bytes > at zero. > > > I've not tried to launch the e2fsck over a backup superblock > instead of > over the default one because I think that this would produce > the same > result. I'm sure that the original superblock was not > corrupted then I > guess that with a backup superblock, e2fsck would have the > same > behavior. > > What do you think? Should I try to launch it over a backup > superblock at > the first time? > > Any other ideas? > > Thanks you very much! > > > El dg 21 de 02 de 2010 a les 04:51 -0800, en/na Stephen Samuel > (gmail) > va escriure: > > > The system is using the backup superblock -- which sounds > reasonable, > > under the circumstances, and should result in a half-decent > recovery. > > > > A couple of questions: > > Was the superblock zero to begin with, or did it become zero > during > > the FSCK? > > In either case, I'm worried about the zero data having been > written. > > This is obviously worth further investigation. > > Did the system abort the FSCK after this error? or did you > stop it? > > did you try explicitly using one of the backup superblocks? > > a list of backup superblocks can be found by using -n on > mkfs > > check the -b option on fsck.ext2 for more details on the > backup > > superblock defaults. > > > > 2010/2/21 Albert Sellar?s > > Hi all, > > > > I'm trying to fix a 7.5Tb ext3 filesystem using > e2fsck on a > > x86_64 > > machine with plenty of memory ram. The filesystem is > > corrupted, but I > > can mount it.is > > > > Before starting the filesystem check, I did a LVM > snapshot to > > be able to > > start it again from the same point in case of error. > > > > After 12 hours checking the filesystem, I got this > error > > message: > > > > Pass 1: Memory used: 268k/18014398508105072k > (59k/210k), time: > > 65673.66/1550.92/1371.06 > > Pass 1: I/O read: 253456MB, write: 23885MB, rate: > 4.22MB/s > > Restarting e2fsck from the beginning... > > e2fsck: Superblock invalid, trying backup blocks... > > e2fsck: Bad magic number in super-block while trying > to > > open /dev/storage/cabina_snapshot > > > > Once I saw this message, my first thought was that > e2fsck > > didn't manage > > to fix the filesystem and it corrupted the > superblock. To be > > sure I > > dumped the entire block and compared it against the > original > > superblock. > > Doing that I realized that the entire superblock > only > > contained zeros. > > > > Any ideas of what can I do? > > > > > > -- > > Stephen Samuel http://www.bcgreen.com Software, like love, > > 778-861-7641 grows when you > give it away > > > -- > Albert Sellar?s GPG id: 0x13053FFE > http://www.wekk.net whats at jabber.org > Linux User: 324456 > > > > > -- > Stephen Samuel http://www.bcgreen.com Software, like love, > 778-861-7641 grows when you give it away -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Aix? ?s una part d'un missatge signada digitalment URL: From whats at wekk.net Sun Feb 21 19:00:18 2010 From: whats at wekk.net (Albert =?ISO-8859-1?Q?Sellar=E8s?=) Date: Sun, 21 Feb 2010 20:00:18 +0100 Subject: Invalid superblock after e2fsck In-Reply-To: <4B8173B5.9050907@berkeley.edu> References: <1266753077.6554.24.camel@x61s> <4B8173B5.9050907@berkeley.edu> Message-ID: <1266778818.6554.105.camel@x61s> Hi, I'm not using any partition table, I'm directly using a Logical Volume. :) Greetings. > What partition table format are you using? I sure hope > it's not msdos. If it is your problem might be caused by > this. I once had similar problems. I was able to replace > the msdos table with a GPT table, and then use the > gparted 'rescue' command to recover without loosing > a single bit of data. > > Cordially, > -- > Jon Forrest > Research Computing Support > College of Chemistry > 173 Tan Hall > University of California Berkeley > Berkeley, CA > 94720-1460 > 510-643-1032 > jlforrest at berkeley.edu > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Aix? ?s una part d'un missatge signada digitalment URL: From tytso at mit.edu Mon Feb 22 02:41:39 2010 From: tytso at mit.edu (tytso at mit.edu) Date: Sun, 21 Feb 2010 21:41:39 -0500 Subject: Is my data checksummed? In-Reply-To: <4B7F61E2.9090404@parvan.net> References: <4B7F61E2.9090404@parvan.net> Message-ID: <20100222024139.GD23832@thunk.org> On Fri, Feb 19, 2010 at 08:15:30PM -0800, Reza Roboubi wrote: > What checksumming is done for the actual data? I know that storage > devices often do their own checksumming too, but how can I be sure > my data is integrity checked every time I read it? If you use disks that support the Data Integrity Field (DIF) extension, Linux will use it to provide end-to-end data checksum support. Otherwise, there are checksums on the disk and between disk controller and the CPU, but those are obviously not end-to-end checksums. Adding data-level checksums is not something that we are planning on adding to the ext2/3/4 file systems. BTRFS is the only file system that has data-level checksums, but it's not yet production ready. Best regards, - Ted From rwheeler at redhat.com Mon Feb 22 12:49:25 2010 From: rwheeler at redhat.com (Ric Wheeler) Date: Mon, 22 Feb 2010 07:49:25 -0500 Subject: Is my data checksummed? In-Reply-To: <20100222024139.GD23832@thunk.org> References: <4B7F61E2.9090404@parvan.net> <20100222024139.GD23832@thunk.org> Message-ID: <4B827D55.4050801@redhat.com> On 02/21/2010 09:41 PM, tytso at mit.edu wrote: > On Fri, Feb 19, 2010 at 08:15:30PM -0800, Reza Roboubi wrote: > >> What checksumming is done for the actual data? I know that storage >> devices often do their own checksumming too, but how can I be sure >> my data is integrity checked every time I read it? >> > If you use disks that support the Data Integrity Field (DIF) > extension, Linux will use it to provide end-to-end data checksum > support. Otherwise, there are checksums on the disk and between disk > controller and the CPU, but those are obviously not end-to-end > checksums. > Just to be clear, even with a storage path that supports DIF/DIX, we don't currently do anything for applications on top of file systems. The primary application to target storage path is covered mainly for raw devices. ric > Adding data-level checksums is not something that we are planning on > adding to the ext2/3/4 file systems. BTRFS is the only file system > that has data-level checksums, but it's not yet production ready. > > Best regards, > > - Ted > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > From sandeen at redhat.com Mon Feb 22 20:52:57 2010 From: sandeen at redhat.com (Eric Sandeen) Date: Mon, 22 Feb 2010 14:52:57 -0600 Subject: Invalid superblock after e2fsck In-Reply-To: <1266753077.6554.24.camel@x61s> References: <1266753077.6554.24.camel@x61s> Message-ID: <4B82EEA9.8050402@redhat.com> Albert Sellar?s wrote: > Hi all, > > I'm trying to fix a 7.5Tb ext3 filesystem using e2fsck on a x86_64 > machine with plenty of memory ram. The filesystem is corrupted, but I > can mount it. > > Before starting the filesystem check, I did a LVM snapshot to be able to > start it again from the same point in case of error. > > After 12 hours checking the filesystem, I got this error message: > > Pass 1: Memory used: 268k/18014398508105072k (59k/210k), time: > 65673.66/1550.92/1371.06 > Pass 1: I/O read: 253456MB, write: 23885MB, rate: 4.22MB/s > Restarting e2fsck from the beginning... > e2fsck: Superblock invalid, trying backup blocks... > e2fsck: Bad magic number in super-block while trying to > open /dev/storage/cabina_snapshot > > Once I saw this message, my first thought was that e2fsck didn't manage > to fix the filesystem and it corrupted the superblock. To be sure I > dumped the entire block and compared it against the original superblock. > Doing that I realized that the entire superblock only contained zeros. > > Any ideas of what can I do? What is the blocksize of your filesystem? RHEL4 should be safe for 2^31 blocks, which -should- get you to 8T at 4k blocks. dumpe2fs -h info would show us (long as you have an intact superblock somewehre...) But if you were beyond 2^31 blocks I imagine you'd have hit troubles early on. -Eric From darkonc at gmail.com Mon Feb 22 21:36:39 2010 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Mon, 22 Feb 2010 13:36:39 -0800 Subject: Invalid superblock after e2fsck In-Reply-To: <1266778634.6554.102.camel@x61s> References: <1266753077.6554.24.camel@x61s> <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> <1266760815.6554.59.camel@x61s> <6cd50f9f1002210938o568dd1e6ga3b38c75aa3cdcc1@mail.gmail.com> <1266778634.6554.102.camel@x61s> Message-ID: <6cd50f9f1002221336u67d33a4chde6f58f3141bb379@mail.gmail.com> Unh, hold on... Somebody else can hopefully answer this question: Haven't there been problems moving an ext2 filesystem between 32bit and 64bit kernels? If so, are all of those problems known solved, or may we have run into one here? 2010/2/21 Albert Sellar?s > Hi, > > the filesystem is on a SAN device and was connected to a *32bits RHEL4 > system.* > > To be able to check it with enough ram and with a new kernel, I > installed an ubuntu karmic on* a new x86_64 machine* and I plugged the SAN > to it. The new machine have 4Gb of RAM and 90Gb of SWAP. > > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From tytso at mit.edu Mon Feb 22 22:10:47 2010 From: tytso at mit.edu (tytso at mit.edu) Date: Mon, 22 Feb 2010 17:10:47 -0500 Subject: Invalid superblock after e2fsck In-Reply-To: <1266760815.6554.59.camel@x61s> References: <1266753077.6554.24.camel@x61s> <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> <1266760815.6554.59.camel@x61s> Message-ID: <20100222221047.GF23832@thunk.org> On Sun, Feb 21, 2010 at 03:00:15PM +0100, Albert Sellar?s wrote: > > e2fsck quits after the bad magic number message. I didn't pasted the > entire e2fsck output (sorry!). The full output was that: > > root at pacs:~# e2fsck -C 0 -y -t /dev/storage/cabina_snapshot2 > e2fsck 1.41.9 (22-Aug-2009) > /dev/storage/cabina_snapshot2 contains a file system with errors, check > forced. > Pass 1: Checking inodes, blocks, and sizes > /dev/storage/cabina_snapshot2: |================== / 60.0% > > [...] > > Until 60% of the progress bar, there was no error messages. After reach > the 60%, e2fsck started fixing a lot of things. In the output I > recognized the inode that it was analyzing. Once e2fsck analyzed the > last inode of the filesystem, it printed this message and exited: So *do* you have the full e2fsck output? I'd like to look at it to see what it did. It's _possible_ that a really badly corrupted file system block might have caused it to get confused enough such that it managed to overwrite the superblock while trying to fix the filesystem. But unless I see the whole e2fsck output, it's hard to say for sure what happened. - Ted From whats at wekk.net Tue Feb 23 07:54:34 2010 From: whats at wekk.net (=?UTF-8?Q?Albert_Sellar=C3=A8s?=) Date: Tue, 23 Feb 2010 08:54:34 +0100 Subject: Invalid superblock after e2fsck In-Reply-To: <4B82EEA9.8050402@redhat.com> References: <1266753077.6554.24.camel@x61s> <4B82EEA9.8050402@redhat.com> Message-ID: <76957c4508f13d87ccfd919d88399180@localhost> Hi, I have the original filesystem (on a logical volume) "intact" because I've always worked on snapshots. The filesystem has less than 2^31 blocks. This system has been correctly running for four years and corruption become because of a hardware problem with a SAN device. The output of dumpe2fs -h /dev/storage/cabina is: dumpe2fs 1.41.9 (22-Aug-2009) Filesystem volume name: Last mounted on: Filesystem UUID: 607db553-3dc2-4304-9059-37eb99f1047b Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal resize_inode filetype sparse_super large_file Filesystem flags: signed_directory_hash Default mount options: (none) Filesystem state: clean with errors Errors behavior: Continue Filesystem OS type: Linux Inode count: 1038548992 Block count: 2077085696 Reserved block count: 0 Free blocks: 220295654 Free inodes: 1004004295 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 877 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 16384 Inode blocks per group: 512 Filesystem created: Fri Jul 7 16:45:37 2006 Last mount time: Wed Jan 20 11:53:00 2010 Last write time: Thu Feb 11 10:16:13 2010 Mount count: 65 Maximum mount count: 35 Last checked: Sat Jul 15 05:53:09 2006 Check interval: 15552000 (6 months) Next check after: Thu Jan 11 04:53:09 2007 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 Default directory hash: tea Directory Hash Seed: 94a8b51f-6ba4-484d-bb92-167cb75c3eb6 Journal backup: inode blocks Journal size: 32M Thanks you! > What is the blocksize of your filesystem? RHEL4 should be safe for > 2^31 blocks, which -should- get you to 8T at 4k blocks. > > dumpe2fs -h info would show us (long as you have an intact superblock > somewehre...) > > But if you were beyond 2^31 blocks I imagine you'd have hit troubles early > on. > > -Eric -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 From whats at wekk.net Tue Feb 23 08:06:13 2010 From: whats at wekk.net (=?UTF-8?Q?Albert_Sellar=C3=A8s?=) Date: Tue, 23 Feb 2010 09:06:13 +0100 Subject: Invalid superblock after e2fsck In-Reply-To: <20100222221047.GF23832@thunk.org> References: <1266753077.6554.24.camel@x61s> <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> <1266760815.6554.59.camel@x61s> <20100222221047.GF23832@thunk.org> Message-ID: <2259c32aeb892f1e6edfac6f844cb776@localhost> Hi, I don't have the full e2fsck output now, but I can relaunch it to produce the same behaviour and save the output. I will do it now, then I hope to have the full output in +12h more or less. Thanks all for your help and interest. > So *do* you have the full e2fsck output? I'd like to look at it to > see what it did. It's _possible_ that a really badly corrupted file > system block might have caused it to get confused enough such that it > managed to overwrite the superblock while trying to fix the > filesystem. But unless I see the whole e2fsck output, it's hard to > say for sure what happened. > > - Ted -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 From reza at parvan.net Tue Feb 23 09:56:54 2010 From: reza at parvan.net (Reza Roboubi) Date: Tue, 23 Feb 2010 01:56:54 -0800 Subject: Is my data checksummed? In-Reply-To: <4B827D55.4050801@redhat.com> References: <4B7F61E2.9090404@parvan.net> <20100222024139.GD23832@thunk.org> <4B827D55.4050801@redhat.com> Message-ID: <4B83A666.4010000@parvan.net> Thanks a lot for all the responses. Ric Wheeler wrote: > On 02/21/2010 09:41 PM, tytso at mit.edu wrote: >> If you use disks that support the Data Integrity Field (DIF) >> extension, Linux will use it to provide end-to-end data checksum >> support. Otherwise, there are checksums on the disk and between disk >> controller and the CPU, but those are obviously not end-to-end >> checksums. >> > > Just to be clear, even with a storage path that supports DIF/DIX, we > don't currently do anything for applications on top of file systems. The > primary application to target storage path is covered mainly for raw > devices. Sorry Ric, The last sentence loses me. I mean, I know what raw devices are! :) Would you please elaborate a little. Reza. From eparis at redhat.com Wed Feb 17 16:37:59 2010 From: eparis at redhat.com (Eric Paris) Date: Wed, 17 Feb 2010 11:37:59 -0500 Subject: [PATCH] Re: name_count maxed, losing inode data: dev=00:05, inode=5221 In-Reply-To: References: Message-ID: <1266424679.9304.3.camel@localhost> On Mon, 2010-02-08 at 00:38 -0800, Christian Kujau wrote: > On Wed, 3 Feb 2010 at 09:39, Christian Kujau wrote: > > Ralf Hildebrandt reported[0] the following messages on ext3-users: > > > > name_count maxed, losing inode data: dev=00:05, inode=5221 > > > > because the filesystem in question is indeed ext3. However, this warning > > is not generated by ext3 code but by the audit framework. While the > > origins of these messages are discussed elsewhere[1] the following > > patch modifies the printks in question so that users know where these > > messages are coming from. There may be other places within the audit > > framework needing the same treatment. > > ...ping? I'm tracking these issues as Red Hat bz 445757. I've got a patch which removes the restriction of 20 inodes being accessed in one syscall which I've floated to Al Viro to see what he thinks. Since the restriction is gone we won't lose names and thus it removes these messages altogether. You are not forgotten! -Eric From sandeen at redhat.com Tue Feb 23 15:16:07 2010 From: sandeen at redhat.com (Eric Sandeen) Date: Tue, 23 Feb 2010 09:16:07 -0600 Subject: Invalid superblock after e2fsck In-Reply-To: <76957c4508f13d87ccfd919d88399180@localhost> References: <1266753077.6554.24.camel@x61s> <4B82EEA9.8050402@redhat.com> <76957c4508f13d87ccfd919d88399180@localhost> Message-ID: <4B83F137.8030600@redhat.com> Albert Sellar?s wrote: > Hi, > > I have the original filesystem (on a logical volume) "intact" because I've > always worked on snapshots. > > The filesystem has less than 2^31 blocks. This system has been correctly > running for four years and corruption become because of a hardware problem > with a SAN device. > > The output of dumpe2fs -h /dev/storage/cabina is: > > dumpe2fs 1.41.9 (22-Aug-2009) ... > Block count: 2077085696 ... > Block size: 4096 Yep, right you are, just figured it was worth a check. Since you can safely operate on a snapshot, -maybe- it would be worth testing with an e2fsprogs which is newer than that which ships with rhel4... this wouldn't be a RHEL-supported option, but if there is something which has been fixed upstream it would help identify the problem if we knew which release fixed it. -Eric From whats at wekk.net Tue Feb 23 17:11:08 2010 From: whats at wekk.net (Albert =?ISO-8859-1?Q?Sellar=E8s?=) Date: Tue, 23 Feb 2010 18:11:08 +0100 Subject: Invalid superblock after e2fsck In-Reply-To: <4B83F137.8030600@redhat.com> References: <1266753077.6554.24.camel@x61s> <4B82EEA9.8050402@redhat.com> <76957c4508f13d87ccfd919d88399180@localhost> <4B83F137.8030600@redhat.com> Message-ID: <1266945068.5772.19.camel@x61s> Hi, I already tried to check the filesystem with last e2fsprogs version (1.41.10) but with this version I did get a segfault. I opened a bug with all the information [1] Thank you! [1] http://sourceforge.net/tracker/?func=detail&aid=2956459&group_id=2406&atid=102406 El dt 23 de 02 de 2010 a les 09:16 -0600, en/na Eric Sandeen va escriure: > Albert Sellar?s wrote: > > Hi, > > > > I have the original filesystem (on a logical volume) "intact" because I've > > always worked on snapshots. > > > > The filesystem has less than 2^31 blocks. This system has been correctly > > running for four years and corruption become because of a hardware problem > > with a SAN device. > > > > The output of dumpe2fs -h /dev/storage/cabina is: > > > > dumpe2fs 1.41.9 (22-Aug-2009) > ... > > > Block count: 2077085696 > ... > > > Block size: 4096 > > > Yep, right you are, just figured it was worth a check. > > Since you can safely operate on a snapshot, -maybe- it would be worth > testing with an e2fsprogs which is newer than that which ships with rhel4... > this wouldn't be a RHEL-supported option, but if there is something which > has been fixed upstream it would help identify the problem if > we knew which release fixed it. > > -Eric -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Aix? ?s una part d'un missatge signada digitalment URL: From mirjafarali at gmail.com Tue Feb 23 21:56:08 2010 From: mirjafarali at gmail.com (MirJafar Ali) Date: Tue, 23 Feb 2010 15:56:08 -0600 Subject: File layput Visualization Message-ID: Hello, Can someone inform me whether there are some File Layout ( not at the directory and file level) but at the superblock, inode and group level visualization software in ext3 ? Thanks. Mir -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirjafarali at gmail.com Tue Feb 23 21:59:36 2010 From: mirjafarali at gmail.com (MirJafar Ali) Date: Tue, 23 Feb 2010 15:59:36 -0600 Subject: Ext2/Ext3 File layout policies Message-ID: Hello, I studied some of the papers on file layout in ext2/ext3, but all the papers are at least 10-12 years old. Can someone give hints/reference of survey of changes in file layout policies in the last 4-5 years ? I am interesting in knowing some weakness in ext2/ext3 that that possibly our group can work on. Thanks. Mir -------------- next part -------------- An HTML attachment was scrubbed... URL: From rwheeler at redhat.com Tue Feb 23 23:46:21 2010 From: rwheeler at redhat.com (Ric Wheeler) Date: Tue, 23 Feb 2010 18:46:21 -0500 Subject: Is my data checksummed? In-Reply-To: <4B83A666.4010000@parvan.net> References: <4B7F61E2.9090404@parvan.net> <20100222024139.GD23832@thunk.org> <4B827D55.4050801@redhat.com> <4B83A666.4010000@parvan.net> Message-ID: <4B8468CD.10800@redhat.com> On 02/23/2010 04:56 AM, Reza Roboubi wrote: > Thanks a lot for all the responses. > > Ric Wheeler wrote: >> On 02/21/2010 09:41 PM, tytso at mit.edu wrote: >>> If you use disks that support the Data Integrity Field (DIF) >>> extension, Linux will use it to provide end-to-end data checksum >>> support. Otherwise, there are checksums on the disk and between disk >>> controller and the CPU, but those are obviously not end-to-end >>> checksums. >> >> Just to be clear, even with a storage path that supports DIF/DIX, we >> don't currently do anything for applications on top of file systems. >> The primary application to target storage path is covered mainly for >> raw devices. > > Sorry Ric, > > The last sentence loses me. I mean, I know what raw devices are! :) > Would you please elaborate a little. > > Reza. Sorry for the confusion. It confuses me now that I try to reread it as well :-( What I was trying to say is that in this first implementation, the only applications that can take advantage of DIF/DIX extra data protection will need to do it all themselves and will need to use raw devices (no file systems). Oracle's DB will be a likely candidate as will a very few other sophisticated applications. Note that the other mode of operation (HBA -> target storage device) is invisible and will work for everyone. It just does not protect against data corruption higher in the stack (above the HBA/device driver). ric From lists at nerdbynature.de Wed Feb 24 00:07:57 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Tue, 23 Feb 2010 16:07:57 -0800 (PST) Subject: File layput Visualization In-Reply-To: References: Message-ID: On Tue, 23 Feb 2010 at 15:56, MirJafar Ali wrote: > Can someone inform me whether there are some File Layout ( not at the > directory and file level) > but at the superblock, inode and group level visualization software in ext3 Do you know of any examples/implementations for other filesystems? What are you trying to accomplish with such a visualization? Christian. -- BOFH excuse #289: Interference between the keyboard and the chair. From lists at nerdbynature.de Wed Feb 24 00:24:12 2010 From: lists at nerdbynature.de (Christian Kujau) Date: Tue, 23 Feb 2010 16:24:12 -0800 (PST) Subject: Ext2/Ext3 File layout policies In-Reply-To: References: Message-ID: On Tue, 23 Feb 2010 at 15:59, MirJafar Ali wrote: > I studied some of the papers on file layout in ext2/ext3, but all the papers > are at least 10-12 years old. Do you mean "file layout" or "filsystem layout", as in "FHS"[0]? > Can someone give hints/reference of survey of changes in file > layout policies in the last 4-5 years ? What do you mean by "survey of changes"? The changes to ext2/3 can be seen in the kernel's version history (git log), starting at 2.6.12, IIRC. > I am interesting in knowing some weakness in ext2/ext3 that that possibly > our group can work on. A "weakness", you mean like a security flaw? Or perhaps I mis-parsed the sentence completely.... Christian. [0] http://www.pathname.com/fhs/ -- BOFH excuse #289: Interference between the keyboard and the chair. From tytso at mit.edu Wed Feb 24 07:17:46 2010 From: tytso at mit.edu (Theodore Tso) Date: Wed, 24 Feb 2010 02:17:46 -0500 Subject: Ext2/Ext3 File layout policies In-Reply-To: References: Message-ID: On Feb 23, 2010, at 4:59 PM, MirJafar Ali wrote: > I studied some of the papers on file layout in ext2/ext3, but all the papers are at least 10-12 > years old. Can someone give hints/reference of survey of changes in file layout policies in the > last 4-5 years ? There are a number of newer papers in the ext4 Wiki: http://ext4.wiki.kernel.org. Click on the Publications / Articles link. I'm not sure what you mean by "file layout policies". Do you mean how the filesystem lays out its metadata? Do you mean the block allocation algorithms? And for what purpose are you interested in knowing about layout policies in particular. > I am interesting in knowing some weakness in ext2/ext3 that that possibly our group can work > on. Can you say a bit more about "our group", and what sort of work you are interested in doing. There is development going on in the ext4 file system, and the linux-ext4 at vger.kernel.org mailing list is where you will find the ext2/3/4 developers. Regards, - Ted From whats at wekk.net Wed Feb 24 08:44:58 2010 From: whats at wekk.net (Albert =?ISO-8859-1?Q?Sellar=E8s?=) Date: Wed, 24 Feb 2010 09:44:58 +0100 Subject: Invalid superblock after e2fsck In-Reply-To: <2259c32aeb892f1e6edfac6f844cb776@localhost> References: <1266753077.6554.24.camel@x61s> <6cd50f9f1002210451m19b62cd0gbaef50a32f444767@mail.gmail.com> <1266760815.6554.59.camel@x61s> <20100222221047.GF23832@thunk.org> <2259c32aeb892f1e6edfac6f844cb776@localhost> Message-ID: <1267001098.5806.33.camel@x61s> Hi, I already have the e2fsck output. If you would manage to find what happened and/or a possible fix, please let me know. This is the file: http://88.87.193.250/e2fsck.output.txt.bz2 Thank you very much! El dt 23 de 02 de 2010 a les 09:06 +0100, en/na Albert Sellar?s va escriure: > Hi, > > I don't have the full e2fsck output now, but I can relaunch it to produce > the same behaviour and save the output. I will do it now, then I hope to > have the full output in +12h more or less. > > Thanks all for your help and interest. > > > So *do* you have the full e2fsck output? I'd like to look at it to > > see what it did. It's _possible_ that a really badly corrupted file > > system block might have caused it to get confused enough such that it > > managed to overwrite the superblock while trying to fix the > > filesystem. But unless I see the whole e2fsck output, it's hard to > > say for sure what happened. > > > > - Ted > > -- > Albert Sellar?s GPG id: 0x13053FFE > http://www.wekk.net whats at jabber.org > Linux User: 324456 > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users -- Albert Sellar?s GPG id: 0x13053FFE http://www.wekk.net whats at jabber.org Linux User: 324456 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Aix? ?s una part d'un missatge signada digitalment URL: From F.Schrittesser at hofburg.com Wed Feb 24 08:49:02 2010 From: F.Schrittesser at hofburg.com (Schrittesser, Florian) Date: Wed, 24 Feb 2010 09:49:02 +0100 Subject: Overwrite Mechanism Message-ID: <7199FA8C213F514695F080BF85BBB06801248822@sbs1.hofburg-vie.local> Hi all, i wanted to ask you all about the exact overwrite mechanism used in ext3; is the information in the datablocks overwritten or written to a new location and the information in the referring inode is changed? Is it depending on the application used (and the functions it calls) or independent from that? Please don't hesitate to go into further details and thanks for your help! cheers, flo This message has been scanned for malware by Websense. www.websense.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pg_ext3 at ext3.for.sabi.co.UK Wed Feb 24 16:38:14 2010 From: pg_ext3 at ext3.for.sabi.co.UK (Peter Grandi) Date: Wed, 24 Feb 2010 16:38:14 +0000 Subject: Is my data checksummed? In-Reply-To: <4B827D55.4050801@redhat.com> References: <4B7F61E2.9090404@parvan.net> <20100222024139.GD23832@thunk.org> <4B827D55.4050801@redhat.com> Message-ID: <19333.22006.509954.895274@tree.ty.sabi.co.uk> >>> What checksumming is done for the actual data? I know that >>> storage devices often do their own checksumming too, but how >>> can I be sure my data is integrity checked every time I read >>> it? These things ("storage devices often do their own checksumming" and "my data is integrity checked") are rather unrelated. Various parts of storage subsystems do things like checksumming not to protect your data, but to detect potential faults. That is mainly as a diagnostic not for integrity. Part of the reason is that it is very difficult and needlessly expensive to do comprehensivce integrity checking within the storage subsystem, automagically. >> If you use disks that support the Data Integrity Field (DIF) >> extension, Linux will use it to provide end-to-end data >> checksum support. Otherwise, there are checksums on the disk >> and between disk controller and the CPU, but those are >> obviously not end-to-end checksums. Yes. But I'll add that the only way to ensure that "data is integrity checked" is to do it truly end-to-end, with data and application specific checks. For example as a weak but useful measure I 'zip' or gzip' (sometimes with zero compression if already compressed) data that I want to be able to move around across years and many storage devices. Consider for example bugs in the IO subsystem itself, where the wrong data ends up being written and checksummed, and gets validated every time even if it is not the right data. > Just to be clear, even with a storage path that supports > DIF/DIX, we don't currently do anything for applications on > top of file systems. The primary application to target storage > path is covered mainly for raw devices. Which makes it not that generally useful. In effect DIF is a hw accelerator of a weak form of per-block checksumming. I think that most current CPUs are fast enough to do it without it beoing that noticeable. >> Adding data-level checksums is not something that we are >> planning on adding to the ext2/3/4 file systems. BTRFS is >> the only file system that has data-level checksums, but it's >> not yet production ready. But again that's not end-to-end. It is just as far as the current storage system goes, and the biggest value, like for ZFS, is to detect issues with the storage system itself (e.g. bugs as well as hw issues). From adilger at sun.com Wed Feb 24 21:37:26 2010 From: adilger at sun.com (Andreas Dilger) Date: Wed, 24 Feb 2010 14:37:26 -0700 Subject: Overwrite Mechanism In-Reply-To: <7199FA8C213F514695F080BF85BBB06801248822@sbs1.hofburg-vie.local> References: <7199FA8C213F514695F080BF85BBB06801248822@sbs1.hofburg-vie.local> Message-ID: <35797B9E-E09E-4ED5-818D-99CCE78ACDD7@sun.com> On 2010-02-24, at 01:49, Schrittesser, Florian wrote: > i wanted to ask you all about the exact overwrite mechanism used in > ext3; is the information in the datablocks overwritten or written to > a new location and the information in the referring inode is > changed? Is it depending on the application used (and the functions > it calls) or independent from that? > > Please don't hesitate to go into further details and thanks for your > help! It depends on what your application is doing. If it is opening the file with open(O_TRUNC) then writing it, the new data is written to wherever it is allocated. If O_TRUNC is not used, then the data is written into the same blocks as were previously allocated to the file. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. From adilger at sun.com Thu Feb 25 18:05:48 2010 From: adilger at sun.com (Andreas Dilger) Date: Thu, 25 Feb 2010 11:05:48 -0700 Subject: [PATCH] fix dblist size accounting (was Re: Invalid superblock after e2fsck) In-Reply-To: <1266945068.5772.19.camel@x61s> References: <1266753077.6554.24.camel@x61s> <4B82EEA9.8050402@redhat.com> <76957c4508f13d87ccfd919d88399180@localhost> <4B83F137.8030600@redhat.com> <1266945068.5772.19.camel@x61s> Message-ID: <392E3076-B50A-48D7-A4D3-8E03F7AE4574@sun.com> On 2010-02-23, at 10:11, Albert Sellar?s wrote: > I already tried to check the filesystem with last e2fsprogs version > (1.41.10) but with this version I did get a segfault. > > I opened a bug with all the information [1] > > Thank you! > > [1] > http://sourceforge.net/tracker/?func=detail&aid=2956459&group_id=2406&atid=102406 I'm not sure if it is related, but I noticed that the above bug is segfaulting in ext2fs_add_dir_block(), and I have a patch in my tree that fixes the block dblist accounting in the case that realloc() fails its allocation. I'm just in the process of sending out all of the patches I've accumulated, so I may as well send this one now. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. -------------- next part -------------- A non-text attachment was scrubbed... Name: e2fsprogs-dblist.patch Type: application/octet-stream Size: 574 bytes Desc: not available URL: From c-koenig at freenet.de Fri Feb 26 10:31:10 2010 From: c-koenig at freenet.de (c-koenig at freenet.de) Date: Fri, 26 Feb 2010 11:31:10 +0100 Subject: Probs with the Superblock Message-ID: <4b9ec464a9c9d4d114e286ed13425ad2@email.freenet.de> Hello, ? when the free Inode-List in the superblock is empty, there? will be a function how updates the inode-list with free inodes. On whoch location does the kernel prove thes requirement? The requirement is: Free Inode-List=Empty!I checked the linux source but i can?t find the function which checks if the superblock free inode list is empty! ? Thanks for your help and sorry for my bad english best regards Chris -- -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirjafarali at gmail.com Sat Feb 27 17:51:23 2010 From: mirjafarali at gmail.com (MirJafar Ali) Date: Sat, 27 Feb 2010 11:51:23 -0600 Subject: File Affinities Message-ID: Hello, Can someone give hints/reference about file affinities ? I am doing one project where files which are accessed together are closely placed in the file system. What is the best way to find the affinities ? Thanks. Mir -------------- next part -------------- An HTML attachment was scrubbed... URL: From mirjafarali at gmail.com Sat Feb 27 17:52:38 2010 From: mirjafarali at gmail.com (MirJafar Ali) Date: Sat, 27 Feb 2010 11:52:38 -0600 Subject: e2fsprogs Help. Message-ID: Hello, Hope you will forgive me for asking some very simple question about e2fsprogs. I am very new to the kernel as well as file system programming. My task is to collect superblock, inode, bitmap ( or free list) information from ext2/ext3 filesysteam. After searching the google, I came to know about e2fsprogs, which I was able to install and use at least "dumpe2fs" utility. This utility do not provide timestamp of each allocated inode and I am thinking of modifying source code of e2fsprogs. My two questions are: 1. Is it correct to use "dumpe2fs" program and modify for timestamp ? 2. I could not understand the main purpose of e2fsprogs. Am I correct to assume that this program extract information from the existing ext2/ext3 filesystem. Why there are ext2_inode datastructure defined in these programs ? Why couldn't these programs read existing datastructures of inode ? 3. Are these user level or kernel level programs ? 4. I want to have my own policies of allocatiing datablocks in some cylinder group instead of using ext2 default policy ? What should I do for that ? I know, these must be silly questions, but I will learn from your response. Mir. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tytso at mit.edu Sun Feb 28 05:56:48 2010 From: tytso at mit.edu (tytso at mit.edu) Date: Sun, 28 Feb 2010 00:56:48 -0500 Subject: e2fsprogs Help. In-Reply-To: References: Message-ID: <20100228055648.GF14646@thunk.org> On Sat, Feb 27, 2010 at 11:52:38AM -0600, MirJafar Ali wrote: > > Hope you will forgive me for asking some very simple question about > e2fsprogs. I am very new to the kernel as > well as file system programming. > > My task is to collect superblock, inode, bitmap ( or free list) information > from ext2/ext3 filesysteam. After searching > the google, I came to know about e2fsprogs, which I was able to install and > use at least "dumpe2fs" utility. This > utility do not provide timestamp of each allocated inode and I am thinking > of modifying source code of e2fsprogs. Um, this wouldn't be for a school assignment, would it? Why are you being asked to do this? What is the high level problem are you trying to solve, or the high-level goal you are trying to accomplish? > My two questions are: > > 1. Is it correct to use "dumpe2fs" program and modify for timestamp ? Probably not. You might want to look at the "debugfs" program, which is also included in e2fsprogs. Reading the man pages would alos be a good idea. Do you realize that these programs were probably already installed on your Linux system? > 2. I could not understand the main purpose of e2fsprogs. Am I correct to > assume that this program extract information > from the existing ext2/ext3 filesystem. E2fsprogs is a suite of programs that provide the userspace support for the ext2, ext3, and ext4 file systems. So mke2fs wil create a new file system; debugfs is the program that allows you to look at and manipulate low-level file system structures conveniently from userspace (for debugging or developer convenience; users of file systems shouldn't be using this for their everyday activities); e2fsck is the filesystem consistency checker, and so on. > Why there are ext2_inode > datastructure defined in these programs ? Why couldn't > these programs read existing datastructures of inode ? > 3. Are these user level or kernel level programs ? OK, now I'm *sure* this is for a school assignment. > 4. I want to have my own policies of allocatiing datablocks in some > cylinder group instead of using ext2 default policy ? What > should I do for that ? If you can't tell the difference between user and kernel level programs, you're probably way out of your depth to be trying to do this. I suggest you talk to your professor or recitation instructor.... - Ted > I know, these must be silly questions, but I will learn from your > response. Mir. From lakshmipathi.g at gmail.com Sun Feb 28 06:25:07 2010 From: lakshmipathi.g at gmail.com (lakshmi pathi) Date: Sun, 28 Feb 2010 11:55:07 +0530 Subject: ext3 file hide Message-ID: Hi all, I was thinking about file hiding in ext3 and thought about this method. I want know whether this will work or not. 1. creat an immutable file (/usr/share/exthide/file1.txt) with content "Tool to hide files" 2. Now creat an symbolic link (exthide.tmp) to (/usr/share/exthide/file1.txt) 3. Now get file that needs to be hidden from user. (/home/laks/userfile.txt) 4. Get user files (/home/laks/userfile.txt) inode,assume it is small-sized file( that if i_block[13] !=0 , then exit) copy block number at i_block[0] to i_block[13] and i_size to i_block[14] 5. Now Get exthide.tmp's (symbolic link) i_block[0] value and assign it to i_block[0] of /home/laks/userfile.txt and assign exthide.tmp i_size and i_mode to /home/laks/userfile.txt inode's size and mode. ( or should i leave /home/laks/userfile.txt inode's i_size to its original value ?) 6. Now move to /home/laks/userfile.txt 's directory (ie /home/laks) and change this userfile.txt file type as 7 and write this parent inode. Thus the file /home/laks/userfile.txt content is hidden now,Since /home/laks/userfile.txt file type is set as symbolic link and points to (/usr/share/exthide/file1.txt). If user wants to view hidden file , then again copy back blocks number from i_block[13] to i_block[0] and set file type as regular in inode and pareant inode. Do you think , this method will work ? or It has serious issues ? -- ---- Cheers, Lakshmipathi.G FOSS Programmer. www.giis.co.in -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris.j.taylor at optusnet.com.au Sat Feb 27 04:13:41 2010 From: chris.j.taylor at optusnet.com.au (Chris Taylor) Date: Sat, 27 Feb 2010 04:13:41 -0000 Subject: ext2 IF windows Xp Pro with Ubuntu 9.10 64amd Message-ID: Hi To all I have just built a new System 3.4Gb AMD Athlon 64bit 1GB RAM 500Gb SATA HDD Disk /dev/sda: 500.1 GB, 500107862016 bytes 255 heads, 63 sectors/track, 60801 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x00e600e6 Device Boot Start End Blocks Id System /dev/sda1 * 1 1912 15358108+ 7 HPFS/NTFS /dev/sda2 1913 3824 15358140 83 Linux /dev/sda3 3825 60801 457667752+ 5 Extended /dev/sda5 60194 60801 4883760 82 Linux swap / Solaris /dev/sda6 3825 60193 452783929+ 83 Linux Partition table entries are not in disk order. On sda1 I have windows Xp Pro sp2 on sda2 I have Ubuntu 9.10 64bit just upgraded via web On sda6 I have my home partition (according to gparted) I have installed EXT2IFS so I can have XP and Ubuntu use the same place for files. Every time I try to access F: drive from Windows I get "do you want to format the drive " I'm thinking that I have a Inodes problem, Thinking they are 256 not 128. I have tried to format the drive with Gparted to EXT3 a few times and get the same problem still " Large inodes The current version of Ext2 IFS only mounts volumes with an inode size of 128 like old Linux kernels have. Some very new Linux distributions create an Ext3 file systems with inodes of 256 bytes. Ext2 IFS 1.11 is not able to access them. Currently there is only one workaround: Please back up the files and create the Ext3 file system again. Give the mkfs.ext3 tool the -I 128 switch. Finally, restore all files with the backup. " If I'm write I need to unmount the /Home partition but I don't know how to do this :-( Please if you would be so kind as to help me with any info Chris