From tytso at mit.edu Mon Jun 1 12:57:04 2009 From: tytso at mit.edu (Theodore Tso) Date: Mon, 1 Jun 2009 08:57:04 -0400 Subject: -o extents (ext4 capabilities for newly-created files) In-Reply-To: <200905312107.06449.d_baron@012.net.il> References: <200905312107.06449.d_baron@012.net.il> Message-ID: <20090601125704.GG31943@mit.edu> On Sun, May 31, 2009 at 09:07:05PM +0300, David Baron wrote: > Is it desirable to use this to make current ext3 filesystems hybrid ext4 > systems? You will get some of the benefits of ext4, without having to do a dump/restore. > Is there any advantage to extents for numerous, non-huge, more normal-sized > files. Define "non-huge". If the file is less than 48k on a 4k block filesystem, it doesn't make difference. For files larger than 48k, it is likely, if the filesystem isn't horrifically fragmented, that you will be able to avoid needing to use one or more indirect blocks, which will slightly speed up your filesystem and save a bit of space. Simply moving to the ext4 filesystem will use a more advanced block allocation system which can give some wins, although not as much as a freshly formatted ext4 filesystem. > Stability? Note that to get full stability, you really want to be using either the RHEL tech preview or Feodora 11 kernel and e2fsprogs (where Eric has been very good about backporting the necessary stability fixes), or 2.6.30 (once it is released) and e2fsprogs 1.41.6 (which was just released). We're fairly confident that ext4 has most of the scary bugs worked out at this point, but keep in mind that for really critical, production-critical enterprise systems, a paranoid sysadmin might want to stick on ext3 for a while longer. That being said, I've used it on my laptop since last July, and haven't lost any data or needed to resort to backups. (Other people have turned up bugs, which in a few cases have caused data loss, but that's why we all keep regular backups, *RIGHT*?) - Ted From lakshmipathi.g at gmail.com Mon Jun 1 16:10:36 2009 From: lakshmipathi.g at gmail.com (lakshmi pathi) Date: Mon, 1 Jun 2009 12:10:36 -0400 Subject: a question on mount count and maximum mount count In-Reply-To: <4A22404D.9080600@powercraft.nl> References: <20090530170409.GF1898@wolff.to> <4A22404D.9080600@powercraft.nl> Message-ID: thank you,I think using these three solution smartctl ,last reboot and mount count ,will help in arriving a (possible) solution. Cheers, Lakshmipathi.G On Sun, May 31, 2009 at 4:31 AM, Jelle de Jong wrote: > lakshmi pathi wrote: >> This is a case of where knowing what you are really trying to do would >> be useful >> >> sorry :) I came across this interesting question in Linux forum , I >> thought about digging "/var/log/messages" and grep >> "Freeing initrd memory" ?or some unique message might give the required answer. >> I posted the question here, because I wanted the professional answer :) >> >> Thanks guyz, today I learned by last command.But how reliable ? "last >> reboot" is ? ?I quickly checked the man page it said /var/log/wtmp >> used by last command.- what will happen if ?the log file >> modified/erased or archived. > > You can also use de S.M.A.R.T. statistics of the hard drive, it will not > show you the reboots but it will show the spin ups of the hard drive so > power off power on, you can use this with the last reboot command and > get a good indication. > > Best regards, > > Jelle de Jong > > # /usr/sbin/smartctl -A /dev/sda | grep -E "Start_Stop_Count|Power_On_Hours|Power_Cycle_Count" > ?4 Start_Stop_Count ? ? ? ?0x0032 ? 100 ? 100 ? 000 ? ?Old_age ? Always ? ? ? - ? ? ? 9 > ?9 Power_On_Hours ? ? ? ? ?0x0032 ? 098 ? 098 ? 000 ? ?Old_age ? Always ? ? ? - ? ? ? 8843 > ?12 Power_Cycle_Count ? ? ? 0x0032 ? 100 ? 100 ? 000 ? ?Old_age ? Always ? ? ? - ? ? ? 9 > > /usr/sbin/smartctl -A /dev/sdb | grep -E "Start_Stop_Count|Power_On_Hours|Power_Cycle_Count" > ?4 Start_Stop_Count ? ? ? ?0x0032 ? 100 ? 100 ? 000 ? ?Old_age ? Always ? ? ? - ? ? ? 53 > ?9 Power_On_Hours ? ? ? ? ?0x0032 ? 098 ? 098 ? 000 ? ?Old_age ? Always ? ? ? - ? ? ? 8844 > ?12 Power_Cycle_Count ? ? ? 0x0032 ? 100 ? 100 ? 000 ? ?Old_age ? Always ? ? ? - ? ? ? 53 > > -- Cheers, Lakshmipathi.G From neoscreenager at gmail.com Tue Jun 2 08:04:07 2009 From: neoscreenager at gmail.com (NeYo) Date: Tue, 2 Jun 2009 01:04:07 -0700 (PDT) Subject: kernel panic due to file system corruption Message-ID: <23827734.post@talk.nabble.com> Hi I have RHEL 4 AS running on HP ProLiant DL580 G5. The machine was running fine last week but was shutted down abruptly due to power and ups faliure. When i booted it again i got following error : Redhat Nash Version 4.2.1.8 can't access (null) exec of init ((null)) failed!!! : 14 umount /initrd/dev failed :2 Kernel panic -not syncing! Attempted to kill init!! When i booted from the RHEL 4 CD#1 into the rescue mode, it lodes the cciss drivers and ask for whether to mount /mnt/sysimage or directly dropped to the shell. I choose continue so that /mnt/sysimage could be mounted but it shows an error that no linux partitions found , dropping to the shell. At the shell when i did fdisk -l , it gives following output : Disk /dev/cciss/c0d0: 146.7 GB, 146745262080 bytes 255 heads, 63 sectors/track, 17840 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x000ba26d Device Boot Start End Blocks Id System /dev/cciss/c0d0p1 * 1 127 1020096 83 Linux /dev/cciss/c0d0p2 128 7775 61432560 83 Linux /dev/cciss/c0d0p3 7776 13894 49150867+ 82 Linux swap / Solaris /dev/cciss/c0d0p4 13895 17840 31696245 f W95 Ext'd (LBA) /dev/cciss/c0d0p5 13895 17718 30716248+ 83 Linux Here c0d0p1 is /boot, c0d0p2 is / and c0d0p5 is /opt. I tried to mount all the partitions and they are mounting fine but the / partition (c0d0p2) does not have /bin and /etc folders, insted of which there are files of same name. when i tried mkinitrd, it says command not found. I am not very litrate about RAID . Please help me in correcting this problem, its very urgent coz we have a demo next monday. Here is my grub.conf file:: # grub.conf generated by anaconda # # Note that you do not have to rerun grub after making changes to this file # NOTICE: You have a /boot partition. This means that # all kernel and initrd paths are relative to /boot/, eg. # root (hd0,0) # kernel /vmlinuz-version ro root=/dev/cciss/c0d0p2 # initrd /initrd-version.img boot=/dev/cciss/c0d0 default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux AS (2.6.9-42.ELsmp) root (hd0,0) kernel /vmlinuz-2.6.9-42.ELsmp ro root=LABEL=/ rhgb quiet initrd /initrd-2.6.9-42.ELsmp.img title Red Hat Enterprise Linux AS-up (2.6.9-42.EL) root (hd0,0) kernel /vmlinuz-2.6.9-42.EL ro root=LABEL=/ rhgb quiet initrd /initrd-2.6.9-42.EL.img To make things worse, I booted from fedora 9 cd in rescue mode and tried following commands: device (hd0) /dev/cciss/c0d0 root (hd0,0) setup (hd0) Now even the grub is not loading, it just displays Grub Error without any error number I've ran fsck on /dev/cciss/c0d0p1, p2 and p5, but nothing.. my /bin and /etc folders are not deleted but are corrupted i.e when i ls into my root directory, i can see bin and etc but as files (not as folder ). Is there any way to repair the etc and bin to convert them back to normal /etc and /bin folders with all the configuration files intact.. here is the ls -l of my root directory mounted in mnt after booting with knoppix live disk::: total 728 -rwxr-xr-x 1 root root 88468 Feb 10 2006 bin <--- file insted of directory drwxr-xr-x 4 root root 4096 Mar 11 2008 boot drwxr-xr-x 2 root root 4096 Mar 11 2008 dev -rwxr-xr-x 1 root root 464080 Jul 4 2006 etc <--- file insted of directory -rw-r--r-- 1 root root 0 May 27 02:21 halt drwxr-xr-x 19 root root 4096 May 8 06:38 home drwxr-xr-x 2 root root 4096 Aug 12 2004 initrd drwx------ 86 root root 16384 Mar 12 2008 lost+found drwxr-xr-x 3 root root 4096 May 27 00:34 media drwxr-xr-x 2 root root 4096 Jul 11 2006 misc drwxr-xr-x 2 root root 4096 Aug 12 2004 mnt drwxr-xr-x 10 root root 4096 May 4 01:11 opt drwxr-xr-x 2 root root 4096 Mar 12 2008 proc -rwxr-xr-x 1 root root 1484 Dec 5 2005 root <--- file insted of directory lrwxrwxrwx 1 root root 19 Mar 11 2008 sbin -> ../init.d/haldaemon drwxr-xr-x 2 root root 4096 Mar 11 2008 selinux drwxr-xr-x 2 root root 4096 Aug 12 2004 srv drwxr-xr-x 2 root root 4096 Mar 12 2008 sys drwxr-xr-x 3 root root 4096 Mar 11 2008 tftpboot drwxrwxrwt 44 root root 16384 May 27 02:21 tmp drwxr-xr-x 2 root root 4096 Jul 21 2008 user drwxr-xr-x 17 root root 4096 Sep 25 2008 usr drwxr-xr-x 24 root root 4096 Mar 13 2008 vars Can the original folders be recovered... -- View this message in context: http://www.nabble.com/kernel-panic-due-to-file-system-corruption-tp23827734p23827734.html Sent from the Ext3 - User mailing list archive at Nabble.com. From bitzsk at gmail.com Tue Jun 2 08:56:33 2009 From: bitzsk at gmail.com (Zhang Shukun) Date: Tue, 2 Jun 2009 16:56:33 +0800 Subject: what's relationship between VFS inode and ext3_inode? Message-ID: hi, in kernel space ,if i get the i_ino(inode mumber ) of a file, and next how could i get the block number of this file? in VFS layer the inode struct haven't i_block[] array to point to the blocks. but in ext3_inode struct, it has. what's the relationship between the VFS inode and ext3_inode? how do i get the block number of a file ? the path is as follow: filp_open("/bin/ls",O_RDONLY,0)---- -->struct file *------->struct dentry *f_dentry----->struct inode *d_inode-------->struct inode{}-------------> ???------------>struct ext3_inode{}---------->__le32 i_block[EXT2_N_BLOCKS]; from VFS inode to ext3_inode , i don't know how to transfer?? Thanks! -- Best regards, ??? ???????????????? Sucan Computer Department,Beijing Institute of Technology,China -------------- next part -------------- An HTML attachment was scrubbed... URL: From tytso at mit.edu Tue Jun 2 11:39:52 2009 From: tytso at mit.edu (Theodore Tso) Date: Tue, 2 Jun 2009 07:39:52 -0400 Subject: what's relationship between VFS inode and ext3_inode? In-Reply-To: References: Message-ID: <20090602113952.GQ31943@mit.edu> On Tue, Jun 02, 2009 at 04:56:33PM +0800, Zhang Shukun wrote: > hi, > > in kernel space ,if i get the i_ino(inode mumber ) of a file, and next how > could i get the block number of this file? in VFS layer the inode struct > haven't i_block[] array to point to the blocks. but in ext3_inode struct, it > has. what's the relationship between the VFS inode and ext3_inode? > > > how do i get the block number of a file ? the path is as follow: > > filp_open("/bin/ls",O_RDONLY,0)---- -->struct file *------->struct dentry > *f_dentry----->struct inode *d_inode-------->struct inode{}-------------> > ???------------>struct ext3_inode{}---------->__le32 > i_block[EXT2_N_BLOCKS]; > > from VFS inode to ext3_inode , i don't know how to transfer?? Thanks! What are you trying to do? For userspace programs, just use the FIBMAP (or FIEMAP) ioctl's. - Ted From bitzsk at gmail.com Tue Jun 2 12:05:48 2009 From: bitzsk at gmail.com (Zhang Shukun) Date: Tue, 2 Jun 2009 20:05:48 +0800 Subject: what's relationship between VFS inode and ext3_inode? In-Reply-To: <20090602113952.GQ31943@mit.edu> References: <20090602113952.GQ31943@mit.edu> Message-ID: thanks for your answer! my goal is : in kernel space , get the specific disk block numbers(or sector number) of a file(such as /bin/ls). because i want to use the block number in DomU, which is a VM in xen. when i read or write a file in DomU, front end driver will send the block number to back-end driver Dom0 , and then in dom0 i can check if the block could be write or not. 2009/6/2 Theodore Tso > On Tue, Jun 02, 2009 at 04:56:33PM +0800, Zhang Shukun wrote: > > hi, > > > > in kernel space ,if i get the i_ino(inode mumber ) of a file, and next > how > > could i get the block number of this file? in VFS layer the inode struct > > haven't i_block[] array to point to the blocks. but in ext3_inode struct, > it > > has. what's the relationship between the VFS inode and ext3_inode? > > > > > > how do i get the block number of a file ? the path is as follow: > > > > filp_open("/bin/ls",O_RDONLY,0)---- -->struct file *------->struct dentry > > *f_dentry----->struct inode *d_inode-------->struct inode{}-------------> > > ???------------>struct ext3_inode{}---------->__le32 > > i_block[EXT2_N_BLOCKS]; > > > > from VFS inode to ext3_inode , i don't know how to transfer?? Thanks! > > What are you trying to do? For userspace programs, just use the > FIBMAP (or FIEMAP) ioctl's. > > - Ted > -- Best regards, ??? ???????????????? Sucan Computer Department,Beijing Institute of Technology,China -------------- next part -------------- An HTML attachment was scrubbed... URL: From tytso at mit.edu Tue Jun 2 12:29:43 2009 From: tytso at mit.edu (Theodore Tso) Date: Tue, 2 Jun 2009 08:29:43 -0400 Subject: what's relationship between VFS inode and ext3_inode? In-Reply-To: References: <20090602113952.GQ31943@mit.edu> Message-ID: <20090602122942.GR31943@mit.edu> On Tue, Jun 02, 2009 at 08:05:48PM +0800, Zhang Shukun wrote: > thanks for your answer! > > my goal is : in kernel space , get the specific disk block numbers(or sector > number) of a file(such as /bin/ls). > > because i want to use the block number in DomU, which is a VM in xen. when i > read or write a file in DomU, front end driver will send the block number to > back-end driver Dom0 , and then in dom0 i can check if the block could be > write or not. The filesystem does that already (send the block number to the back-end driver) when it wants to read or write the block form the filesystem. If the goal is to do this for security purposes, you should be calculating the inode->block mapping in dom0, *not* in domU. After all, if you don't trust the domU to enforce the normal security restrictions which the kernel enforces, why would you trust domU to give you the correct inode->block mapping so that dom0 can do some kind of security access check? In any case, to answer your specific question, to convert from a struct inode to a struct ext3_inode_info, *if* you know that a particular struct inode from an ext3 filesystem (as opposed to an ext2 filesystem, ext4 filesystem, FAT filesystem, a networking socket, etc.) you can use the EXT3_I function to get from an struct inode to struct ext3_inode_info: static inline struct ext3_inode_info *EXT3_I(struct inode *inode) { return container_of(inode, struct ext3_inode_info, vfs_inode); } What happens here is that the struct inode is actually embedded inside the ext3_inode_info structure, and what is cached is the ext3_inode_info structure. So container_of() basically takes a pointer to the struct inode, and substracts the offset from the beginning of ext3_inode_info and the vfs_inode element in ext3_inode_info, to get a pointer to the struct ext3_inode_info. If this isn't an ext3 inode, though, your kernel code will likely crash from the nonsense it will get when it tries to interpret data that isn't a struct ext3_inode_info. In general, the *only* place where well-written, maintainable, kernel code should try to use EXT3_I is inside fs/ext3/*.c. - Ted From bitzsk at gmail.com Tue Jun 2 13:29:43 2009 From: bitzsk at gmail.com (Zhang Shukun) Date: Tue, 2 Jun 2009 21:29:43 +0800 Subject: what's relationship between VFS inode and ext3_inode? In-Reply-To: <20090602122942.GR31943@mit.edu> References: <20090602113952.GQ31943@mit.edu> <20090602122942.GR31943@mit.edu> Message-ID: thanks! your reply it's very useful to me!! yes, it' security purposes! i just trust dom0 , but not trust domU. i want to save the blocks of improtant files of domU into dom0, when write operation happend in domU, domU will send the request struct , which contain the sectors need to write, to dom0, next, in dom0 compare the sector number. if the sector numbers is which i want to protect, the write operation will be stoped by dom0. even if intruder get the root privilege in domU, he can't change the file (for example :files in /bin dir) which i protected. now, another question confused me: in the request struct , there are two group of sector info. and i don't know what's the diff between them. and how to get the all sector numbers of a request for a file? sector_t sector; /* next sector to submit */ unsigned long nr_sectors; /* no. of sectors left to submit */ /* no. of sectors left to submit in the current segment */ unsigned int current_nr_sectors; /***************************************************************************************************/ sector_t hard_sector; /* next sector to complete */ unsigned long hard_nr_sectors; /* no. of sectors left to complete */ /* no. of sectors left to complete in the current segment */ unsigned int hard_cur_sectors; Thanks very much! 2009/6/2 Theodore Tso > On Tue, Jun 02, 2009 at 08:05:48PM +0800, Zhang Shukun wrote: > > thanks for your answer! > > > > my goal is : in kernel space , get the specific disk block numbers(or > sector > > number) of a file(such as /bin/ls). > > > > because i want to use the block number in DomU, which is a VM in xen. > when i > > read or write a file in DomU, front end driver will send the block number > to > > back-end driver Dom0 , and then in dom0 i can check if the block could be > > write or not. > > The filesystem does that already (send the block number to the > back-end driver) when it wants to read or write the block form the > filesystem. If the goal is to do this for security purposes, you > should be calculating the inode->block mapping in dom0, *not* in domU. > After all, if you don't trust the domU to enforce the normal security > restrictions which the kernel enforces, why would you trust domU to > give you the correct inode->block mapping so that dom0 can do some > kind of security access check? > > In any case, to answer your specific question, to convert from a > struct inode to a struct ext3_inode_info, *if* you know that a > particular struct inode from an ext3 filesystem (as opposed to an ext2 > filesystem, ext4 filesystem, FAT filesystem, a networking socket, > etc.) you can use the EXT3_I function to get from an struct inode to > struct ext3_inode_info: > > static inline struct ext3_inode_info *EXT3_I(struct inode *inode) > { > return container_of(inode, struct ext3_inode_info, vfs_inode); > } > > What happens here is that the struct inode is actually embedded inside > the ext3_inode_info structure, and what is cached is the > ext3_inode_info structure. So container_of() basically takes a > pointer to the struct inode, and substracts the offset from the > beginning of ext3_inode_info and the vfs_inode element in > ext3_inode_info, to get a pointer to the struct ext3_inode_info. > > If this isn't an ext3 inode, though, your kernel code will likely > crash from the nonsense it will get when it tries to interpret data > that isn't a struct ext3_inode_info. In general, the *only* place > where well-written, maintainable, kernel code should try to use EXT3_I > is inside fs/ext3/*.c. > > - Ted > -- Best regards, ??? ???????????????? Sucan Computer Department,Beijing Institute of Technology,China -------------- next part -------------- An HTML attachment was scrubbed... URL: From bitzsk at gmail.com Wed Jun 3 05:13:30 2009 From: bitzsk at gmail.com (Zhang Shukun) Date: Wed, 3 Jun 2009 13:13:30 +0800 Subject: How to get file info of a directory in linux kernel space? Message-ID: hi, all you know in the linux userspace , i can compile and run the following program to get the name or inode number of the files in the /bin directory. i want to know how to do this in linux kernelspace ? thank you very much! /*********************************************************************/ #include #include #include #include #include #include void printdir(char *dir, int depth){ DIR *dp; struct dirent *entry; struct stat statbuf; if((dp = opendir(dir)) == NULL){ fprintf(stderr,"cannot open directory: %s\n",dir); return; } chdir(dir); while((entry = readdir(dp)) != NULL){ lstat(entry->d_name,&statbuf); if(S_ISDIR(statbuf.st_mode)){ if(strcmp("..",entry->d_name) == 0|| strcmp(".",entry->d_name) == 0) continue; printf("%*s%s/\n",depth,"",entry->d_name); printdir(entry->d_name,depth+4); } else printf("%*s%s\n",depth,"",entry->d_name); } chdir(".."); closedir(dp); } int main(){ printdir("/bin",0); } /*************************************************************************************************/ -- Best regards, ??? ???????????????? Sucan Computer Department,Beijing Institute of Technology,China -------------- next part -------------- An HTML attachment was scrubbed... URL: From morty+redhat at frakir.org Wed Jun 3 09:09:47 2009 From: morty+redhat at frakir.org (Morty) Date: Wed, 3 Jun 2009 05:09:47 -0400 Subject: command to recreate an FS? Message-ID: <20090603090947.GE19727@red-sonja.frakir.org> Is there a lightweight command to recreate an ext2/ext3/ext4 FS with original options, e.g. based on tune2fs -l output? That is, for backup/restore purposes, it would be nice to run a command that saves the options to recreate each FS. Then, when restoring, the FSs would be recreated exactly -- original label, UUID, reserved block count, check interval, block size, etc. e2image can recreate an FS, but it takes a while to run, and the output file is large, even taking into account that the output file is sparse. It would be nice if e2image had an option to save just the bare minimum metadata required to recreate the FS -- that would run faster and take up less space. If nothing like this already exists, I'll whip up something in Perl. Just hoping not to reinvent the wheel. :) - Morty From tytso at mit.edu Thu Jun 4 12:42:16 2009 From: tytso at mit.edu (Theodore Tso) Date: Thu, 4 Jun 2009 08:42:16 -0400 Subject: what's relationship between VFS inode and ext3_inode? In-Reply-To: References: <20090602113952.GQ31943@mit.edu> <20090602122942.GR31943@mit.edu> Message-ID: <20090604124216.GF21314@mit.edu> On Tue, Jun 02, 2009 at 09:29:43PM +0800, Zhang Shukun wrote: > thanks! your reply it's very useful to me!! > > yes, it' security purposes! i just trust dom0 , but not trust domU. i want > to save the blocks of improtant files of domU into dom0, when write > operation happend in domU, domU will send the request struct , which contain > the sectors need to write, to dom0, next, in dom0 compare the sector number. > if the sector numbers is which i want to protect, the write operation will > be stoped by dom0. even if intruder get the root privilege in domU, he > can't change the file (for example :files in /bin dir) which i protected. So what I would strongly suggest that you do is to create a userspace program, using the libext2fs library, which reads an unmounted filesystem image, and creates a list of block ranges which should be protected. This program would be run in dom0, and it would then pass the block ranges down a device driver which would restrict write access to the domU guest program. It will be much simpler than trying to figure out this information inside the kernel. Also, note that you need to restrict more information than just actual files; you will also need to disallow changes to certain directories. After all, what use is it to protect the blocks associated with /lib/libc.so.6, if the attacker can simply modify /lib by replacing /lib/libc.so.6 with a new file that has modified contents. And protecting /lib isn't enough, since the attacker could just modify the root directory and replace the /lib directory.... You also have to consider the fact that the attacker could simply modify the inode table and simply change the blocks which a particular file is pointing at. But there are multiple inodes stored in a single block, so if you protect a block in the inode table, that will be effectively preventing multiple inodes in the filesystem from being change, and that might not be desireable depending on what other inodes are sharing a particular block in the inode table with the file you are trying to protect. The bottom line is what you are trying to do is pretty hard; you are trying to write protect part of a filesystem from an untrustworthy root user by restricting write operations on a per-block basis. This is an abstraction violation, and when you violate abstractions, you can expect that (a) things won't be easy, and (b) unless you are very careful, you might not achieve the goal you were hoping for anyway. It might be better, if you really don't trust the domU OS, that you use a fully write protected block device, plus some kind of union filesystem. - Ted From bitzsk at gmail.com Fri Jun 5 03:50:55 2009 From: bitzsk at gmail.com (Zhang Shukun) Date: Fri, 5 Jun 2009 11:50:55 +0800 Subject: why block number changed after change of file? Message-ID: hi, i touch a new file and input the content "a" to it, next i use filefrag to see the block number of the file , and then open the file to change the content of the file to "aa", next use filefrag to check the block number again, but why block number changed after change of file? is the block number the hard-disk block number or memory block number? if it is memory block number ,how to get the unchangable hard-disk block number? Thanks! the commands displays as follow: zsk at zsk-laptop:~$ sudo filefrag -v a Checking a Filesystem type is: ef53 Filesystem cylinder groups is approximately 104 Blocksize of file a is 4096 File size of a is 3 (1 blocks) First block: 2501741 Last block: 2501741 a: 1 extent found zsk at zsk-laptop:~$ vim a zsk at zsk-laptop:~$ sudo filefrag -v a Checking a Filesystem type is: ef53 Filesystem cylinder groups is approximately 104 Blocksize of file a is 4096 File size of a is 4 (1 blocks) First block: 3013843 Last block: 3013843 a: 1 extent found -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From sandeen at redhat.com Fri Jun 5 04:00:27 2009 From: sandeen at redhat.com (Eric Sandeen) Date: Thu, 04 Jun 2009 23:00:27 -0500 Subject: why block number changed after change of file? In-Reply-To: References: Message-ID: <4A28985B.3050408@redhat.com> Zhang Shukun wrote: > > hi, > > i touch a new file and input the content "a" to it, next i use filefrag > to see the block number of the file , and then open the file to change > the content of the file to "aa", next use filefrag to check the block > number again, but why block number changed after change of file? is the > block number the hard-disk block number or memory block number? filefrag reports the location on-disk. > if it is > memory block number ,how to get the unchangable hard-disk block number? filefrag (or the underlying ioctl it calls, FIBMAP (or better, FIEMAP)) is the right tool / interface. vim truncates & rewrites the file; this goes back to the allocator in ext3, and for whatever reason, it picks a new block this time. $ echo foo > testfile $ strace -otrace vim testfile ... look at the trace ... open("testfile", O_WRONLY|O_CREAT|O_TRUNC, 0664) = 3 This sequence does the same: # echo bar > testfile # filefrag -v testfile | grep ^First First block: 1284630 # echo baz > testfile # filefrag -v testfile | grep ^First First block: 1284530 Slightly strange, but nothing really wrong with it .... -Eric > Thanks! > > the commands displays as follow: > > zsk at zsk-laptop:~$ sudo filefrag -v a > Checking a > Filesystem type is: ef53 > Filesystem cylinder groups is approximately 104 > Blocksize of file a is 4096 > File size of a is 3 (1 blocks) > First block: 2501741 > Last block: 2501741 > a: 1 extent found > zsk at zsk-laptop:~$ vim a > zsk at zsk-laptop:~$ sudo filefrag -v a > Checking a > Filesystem type is: ef53 > Filesystem cylinder groups is approximately 104 > Blocksize of file a is 4096 > File size of a is 4 (1 blocks) > First block: 3013843 > Last block: 3013843 > a: 1 extent found > -- > Regards, > Sucan From davids at webmaster.com Fri Jun 5 04:15:40 2009 From: davids at webmaster.com (David Schwartz) Date: Thu, 4 Jun 2009 21:15:40 -0700 Subject: why block number changed after change of file? In-Reply-To: Message-ID: Zhang Shukun wrote: > i touch a new file and input the content "a" to it, > next i use filefrag to see the block number of the file , > and then open the file to change the content of the file to > "aa", next use filefrag to check the block number again, > but why block number changed after change of file? The implementation is free to change a file's block number at any time. > is the block number the hard-disk block number or memory > block number? It is the current hard-disk block number. But there is no law that says a file's data must always be on the same place on the disk. In fact, it cannot be, because the new version of the data and the old version cannot be in the same place at the same time, and there's no point in putting the new data over the old data. > if it is memory block number, > how to get the unchangable hard-disk block number? There is no "unchangable block number" for the first data byte of a particular file. That will change if the data at the beginning of the file changes, because the old and new data cannot be at the same place at the same time. Imagine if you have a bunch of buckets on your desk, these are like hard disk blocks. And you have a pointer to a bucket, this is like the first data block. Someone says "replace the data with this new data". How can you do this safely? If you just try to replace the data in a given bucket, and you half-finish, you leave the data corrupt. So you do this: 1) You find an empty bucket. 2) You put the new data in it. 3) You move the pointer to the new bucket. 4) You mark the old bucket free. This is safe, but the net result is that changing the data the pointer points to changes the pointer. Defragmentation also changes the block numbers, and the filesystem has no way to promise you it won't defragment either. What is your outer problem? Odds are there's a right way to solve it. DS From bitzsk at gmail.com Sat Jun 6 06:25:27 2009 From: bitzsk at gmail.com (Zhang Shukun) Date: Sat, 6 Jun 2009 14:25:27 +0800 Subject: Relationship between request->sector and request->bio->bi_sector? In-Reply-To: References: Message-ID: HI, do you know what is the relationship between request->sector and request->bio->bi_sector? Thank you! -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From d.capitella at tin.it Sat Jun 6 12:23:43 2009 From: d.capitella at tin.it (Donato Capitella) Date: Sat, 06 Jun 2009 14:23:43 +0200 Subject: kernel beginner - ext3 hacking help Message-ID: <1244291023.6287.11.camel@kiba> Hi there, I just started hacking the linux kernel. I need to know how to do 2 things: 1) Given a virtual inode, how do I get the block list associated to it? I tried to convert the inode to an ext3_inode_info using EXT3_I(), but I can't find the block list there. 2) Given a block number, how do I write data to that block? Is there any function like write_to_block(int block_n, char *buf, int bytes) or something? Thank you in advance, I'm young and this is my first approach to kernel programming and don't know where to find documentation. Donato Capitella From bitzsk at gmail.com Sun Jun 7 03:33:46 2009 From: bitzsk at gmail.com (Zhang Shukun) Date: Sun, 7 Jun 2009 11:33:46 +0800 Subject: kernel beginner - ext3 hacking help In-Reply-To: <1244291023.6287.11.camel@kiba> References: <1244291023.6287.11.camel@kiba> Message-ID: 1) Given a virtual inode, how do I get the block list associated to it? I tried to convert the inode to an ext3_inode_info using EXT3_I(), but I can't find the block list there. block list is in the ext3_inode_info struct: struct ext3_inode_info { __le32 i_data[15]; /* this is block list */ __u32 i_flags; ....... }; and i do'nt know the second question. 2009/6/6 Donato Capitella > Hi there, > I just started hacking the linux kernel. I need to know how to do 2 > things: > > 2) Given a block number, how do I write data to that block? Is there any > function like write_to_block(int block_n, char *buf, int bytes) or > something? > > Thank you in advance, I'm young and this is my first approach to kernel > programming and don't know where to find documentation. > > Donato Capitella > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at nerdbynature.de Sun Jun 7 07:29:32 2009 From: lists at nerdbynature.de (Christian Kujau) Date: Sun, 7 Jun 2009 00:29:32 -0700 (PDT) Subject: kernel panic due to file system corruption In-Reply-To: <23827734.post@talk.nabble.com> References: <23827734.post@talk.nabble.com> Message-ID: On Tue, 2 Jun 2009, NeYo wrote: > I have RHEL 4 AS running on HP ProLiant DL580 G5. RHEL 4, so you're running kernel 2.6.9 or something? Pretty old...hm. > The machine was running > fine last week but was shutted down abruptly due to power and ups faliure. > When i booted it again i got following error : [...] > I tried to mount all the partitions and they are mounting fine but the / > partition (c0d0p2) does not have /bin and /etc folders, insted of which > there are files of same name. So, after the power failure "something" happened to your root filesystem. I'd suggest to boot off a rescue CD, backup your root partition (dd) and then try to e2fsck. The filesystem should then be in a consistent state, not sure if it will be usable though. Oh, and try to log all the e2fsck output in case you want to analyze it later on. Christian. -- BOFH excuse #75: There isn't any problem From lists at nerdbynature.de Sun Jun 7 07:43:04 2009 From: lists at nerdbynature.de (Christian Kujau) Date: Sun, 7 Jun 2009 00:43:04 -0700 (PDT) Subject: command to recreate an FS? In-Reply-To: <20090603090947.GE19727@red-sonja.frakir.org> References: <20090603090947.GE19727@red-sonja.frakir.org> Message-ID: On Wed, 3 Jun 2009, Morty wrote: > Is there a lightweight command to recreate an ext2/ext3/ext4 FS with > original options, If no options have been specified to mke2fs, newer versions of e2fsprogs are shipping with /etc/mke2fs.conf which holds a few defaults. > e.g. based on tune2fs -l output? No, I don't know of such a tool. Parsing tune2fs -l output and passing this to mke2fs seems the way to go here... > If nothing like this already exists, I'll whip up something in Perl. Be sure to publish the download URL once it's ready ;-) Christian. -- BOFH excuse #286: Telecommunications is downgrading. From bitzsk at gmail.com Sun Jun 7 08:00:58 2009 From: bitzsk at gmail.com (Sucan Zhang) Date: Sun, 7 Jun 2009 16:00:58 +0800 Subject: How to get the sector numbers of a file ? Message-ID: Hi, you know command filefrag can get the block numbers of a file, but how to get the sector numbers of a file ? thanks ! -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From neoscreenager at gmail.com Sun Jun 7 08:18:07 2009 From: neoscreenager at gmail.com (NeYo) Date: Sun, 7 Jun 2009 01:18:07 -0700 (PDT) Subject: kernel panic due to file system corruption In-Reply-To: References: <23827734.post@talk.nabble.com> Message-ID: <23908661.post@talk.nabble.com> I have already done that, the fsck.ext3 gives the message that partition is clean. Christian Kujau-3 wrote: > > On Tue, 2 Jun 2009, NeYo wrote: >> I have RHEL 4 AS running on HP ProLiant DL580 G5. > > RHEL 4, so you're running kernel 2.6.9 or something? Pretty old...hm. > >> The machine was running >> fine last week but was shutted down abruptly due to power and ups >> faliure. >> When i booted it again i got following error : > [...] >> I tried to mount all the partitions and they are mounting fine but the / >> partition (c0d0p2) does not have /bin and /etc folders, insted of which >> there are files of same name. > > So, after the power failure "something" happened to your root filesystem. > I'd suggest to boot off a rescue CD, backup your root partition (dd) and > then try to e2fsck. The filesystem should then be in a consistent state, > not sure if it will be usable though. Oh, and try to log all the e2fsck > output in case you want to analyze it later on. > > Christian. > -- > BOFH excuse #75: > > There isn't any problem > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > > -- View this message in context: http://www.nabble.com/kernel-panic-due-to-file-system-corruption-tp23827734p23908661.html Sent from the Ext3 - User mailing list archive at Nabble.com. From bothie at gmx.de Sun Jun 7 09:05:12 2009 From: bothie at gmx.de (Bodo Thiesen) Date: Sun, 7 Jun 2009 11:05:12 +0200 Subject: How to get the sector numbers of a file ? In-Reply-To: References: Message-ID: <20090607110512.2945a652@30_bodo> * Sucan Zhang hat geschrieben: > you know command filefrag can get the block numbers of a file, but how to > get the sector numbers of a file ? multiply the block number with the block_size divide by sector_size. Regards, Bodo From bothie at gmx.de Sun Jun 7 09:40:10 2009 From: bothie at gmx.de (Bodo Thiesen) Date: Sun, 7 Jun 2009 11:40:10 +0200 Subject: kernel panic due to file system corruption In-Reply-To: <23908661.post@talk.nabble.com> References: <23827734.post@talk.nabble.com> <23908661.post@talk.nabble.com> Message-ID: <20090607114010.31a4c408@30_bodo> * NeYo hat geschrieben: > I have already done that, the fsck.ext3 gives the message that partition is > clean. ext2 file systems with journal are clean literally EVERY TIME ;) e2fsck -f Regards, Bodo From bitzsk at gmail.com Tue Jun 9 08:38:50 2009 From: bitzsk at gmail.com (Sucan Zhang) Date: Tue, 9 Jun 2009 16:38:50 +0800 Subject: why sector number not match block number? In-Reply-To: References: Message-ID: Hi, i want to get the exact match between block number and sector number of a file in domU. so i do follows . root at feisty:~# ls -l total 1 -rw-r--r-- 1 root root 4 Jun 9 08:16 a root at feisty:~# filefrag -v a Checking a Filesystem type is: ef53 Filesystem cylinder groups is approximately 122 Blocksize of file a is 1024 File size of a is 2 (1 blocks) First block: 247208 Last block: 247208 a: 1 extent found the file has one block and block size is 247208, so the secoter size is 247208 *2 +1 == 78B51 and 247208*2 +2 == 78B52 (block number count begin from 0 , sector number count begin from 1). root at feisty:~# echo "a" >> a command above is a write to a file root at feisty:~# tail /var/log/messages Jun 9 08:15:54 feisty kernel: do_blk_req cf9d1de8: cmd cf9d1e64, sec ae85e, (2/2) buffer:cf7a1800 [write] Jun 9 08:16:14 feisty kernel: do_blk_req cf9d152c: cmd cf9d15a8, sec ae7c6, (2/2) buffer:ce81ec00 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d152c: cmd cf9d15a8, sec 2, (2/2) buffer:d06ee400 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d1f40: cmd cf9d1fbc, sec 6, (2/2) buffer:d06eec00 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d1e94: cmd cf9d1f10, sec 880d0, (2/2) buffer:d0392000 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d1de8: cmd cf9d1e64, sec 880d4, (2/2) buffer:d0392800 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d11d0: cmd cf9d124c, sec 880d8, (2/2) buffer:cecdf000 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d127c: cmd cf9d12f8, sec ac002, (2/2) buffer:cf1e4400 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d1124: cmd cf9d11a0, sec ae25a, (2/2) buffer:cf652400 [write] Jun 9 08:16:19 feisty kernel: do_blk_req cf9d1b38: cmd cf9d1bb4, sec ae67c, (2/2) buffer:ce6d1800 [write] and then to show sector number i had written, i expect to see 78B51 or 78B52 sector number. but none of then display, can you tell me why? -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bitzsk at gmail.com Wed Jun 10 03:44:20 2009 From: bitzsk at gmail.com (Sucan Zhang) Date: Wed, 10 Jun 2009 11:44:20 +0800 Subject: what's relationship between VFS inode and ext3_inode? In-Reply-To: <20090604124216.GF21314@mit.edu> References: <20090602113952.GQ31943@mit.edu> <20090602122942.GR31943@mit.edu> <20090604124216.GF21314@mit.edu> Message-ID: Dear Ted, Thank you very much for your patient answer. you always Shine a *light on me *for some new idea. i have tested in domU for protect file writen, but failed. the reason is that each time when i open a file for writting, and save back to the disk. the block number of the file will change. (i use filefrag to see the blocks of a file). but success from a file for reading, by first get the block number of the file and then save the block number to dom0, each time when the file is read in domU, it will send request to dom0. if i detect the block number is the same to which i saved before, the read operation will be stopped. is that the filesystem mechanism to change bock number of a file when written even if the operation is to make the file to smaller than before? and how could i make the file written protection in the block level when the block number always change? Thanks a lot ;) 2009/6/4 Theodore Tso > On Tue, Jun 02, 2009 at 09:29:43PM +0800, Zhang Shukun wrote: > > thanks! your reply it's very useful to me!! > > > > yes, it' security purposes! i just trust dom0 , but not trust domU. i > want > > to save the blocks of improtant files of domU into dom0, when write > > operation happend in domU, domU will send the request struct , which > contain > > the sectors need to write, to dom0, next, in dom0 compare the sector > number. > > if the sector numbers is which i want to protect, the write operation > will > > be stoped by dom0. even if intruder get the root privilege in domU, he > > can't change the file (for example :files in /bin dir) which i protected. > > So what I would strongly suggest that you do is to create a userspace > program, using the libext2fs library, which reads an unmounted > filesystem image, and creates a list of block ranges which should be > protected. This program would be run in dom0, and it would then pass > the block ranges down a device driver which would restrict write > access to the domU guest program. It will be much simpler than trying > to figure out this information inside the kernel. > > Also, note that you need to restrict more information than just actual > files; you will also need to disallow changes to certain directories. > After all, what use is it to protect the blocks associated with > /lib/libc.so.6, if the attacker can simply modify /lib by replacing > /lib/libc.so.6 with a new file that has modified contents. And > protecting /lib isn't enough, since the attacker could just modify the > root directory and replace the /lib directory.... > > You also have to consider the fact that the attacker could simply > modify the inode table and simply change the blocks which a particular > file is pointing at. But there are multiple inodes stored in a single > block, so if you protect a block in the inode table, that will be > effectively preventing multiple inodes in the filesystem from being > change, and that might not be desireable depending on what other > inodes are sharing a particular block in the inode table with the file > you are trying to protect. > > The bottom line is what you are trying to do is pretty hard; you are > trying to write protect part of a filesystem from an untrustworthy > root user by restricting write operations on a per-block basis. This > is an abstraction violation, and when you violate abstractions, you > can expect that (a) things won't be easy, and (b) unless you are very > careful, you might not achieve the goal you were hoping for anyway. > > It might be better, if you really don't trust the domU OS, that you > use a fully write protected block device, plus some kind of union > filesystem. > > - Ted > -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From bitzsk at gmail.com Wed Jun 10 05:40:20 2009 From: bitzsk at gmail.com (Sucan Zhang) Date: Wed, 10 Jun 2009 13:40:20 +0800 Subject: What is the difference between cat and vim command to file system? Message-ID: Hi, why use cat command to edit a file , the blocks of the file will not change. but use vim command to edit a file, the blocks of the file will change? Thanks in advance! -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From darkonc at gmail.com Wed Jun 10 09:30:39 2009 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Wed, 10 Jun 2009 02:30:39 -0700 Subject: What is the difference between cat and vim command to file system? In-Reply-To: References: Message-ID: <6cd50f9f0906100230l1193c2daj8e0f2eaa900d53f0@mail.gmail.com> an example test case might be useful. I'm not clear what you're trying to do, or what the problem is. On Tue, Jun 9, 2009 at 10:40 PM, Sucan Zhang wrote: > Hi, > > why use cat command to edit a file , the blocks of the file will not > change. > > but use vim command to edit a file, the blocks of the file will change? > > Thanks in advance! > > -- > Regards, > Sucan > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From bitzsk at gmail.com Wed Jun 10 11:57:33 2009 From: bitzsk at gmail.com (Sucan Zhang) Date: Wed, 10 Jun 2009 19:57:33 +0800 Subject: What is the difference between cat and vim command to file system? In-Reply-To: <6cd50f9f0906100230l1193c2daj8e0f2eaa900d53f0@mail.gmail.com> References: <6cd50f9f0906100230l1193c2daj8e0f2eaa900d53f0@mail.gmail.com> Message-ID: Thank you ! when use vim command the block number will change. but when use echo "something" > a file. the block number will not change. zsk at zsk-laptop:~$ sudo filefrag -v a Checking a Filesystem type is: ef53 Filesystem cylinder groups is approximately 104 Blocksize of file a is 4096 File size of a is 13 (1 blocks) First block: 3013938 Last block: 3013938 a: 1 extent found zsk at zsk-laptop:~$ cat a hello,world! zsk at zsk-laptop:~$ vim a zsk at zsk-laptop:~$ cat a hi,hello,world! zsk at zsk-laptop:~$ sudo filefrag -v a Checking a Filesystem type is: ef53 Filesystem cylinder groups is approximately 104 Blocksize of file a is 4096 File size of a is 16 (1 blocks) First block: 3013924 Last block: 3013924 a: 1 extent found 2009/6/10 Stephen Samuel (gmail) > an example test case might be useful. > I'm not clear what you're trying to do, or what the problem is. > > On Tue, Jun 9, 2009 at 10:40 PM, Sucan Zhang wrote: > >> Hi, >> >> why use cat command to edit a file , the blocks of the file will not >> change. >> >> but use vim command to edit a file, the blocks of the file will change? >> >> Thanks in advance! >> >> -- >> Regards, >> Sucan >> >> _______________________________________________ >> Ext3-users mailing list >> Ext3-users at redhat.com >> https://www.redhat.com/mailman/listinfo/ext3-users >> > > > > -- > Stephen Samuel http://www.bcgreen.com Software, like love, > 778-861-7641 grows when you give it away > -- Regards, Sucan -------------- next part -------------- An HTML attachment was scrubbed... URL: From tytso at mit.edu Wed Jun 10 13:39:03 2009 From: tytso at mit.edu (Theodore Tso) Date: Wed, 10 Jun 2009 09:39:03 -0400 Subject: What is the difference between cat and vim command to file system? In-Reply-To: References: Message-ID: <20090610133903.GC6647@mit.edu> On Wed, Jun 10, 2009 at 01:40:20PM +0800, Sucan Zhang wrote: > > why use cat command to edit a file , the blocks of the file will not change. If you do "cat > file" or "echo foo > file", the file is first getting truncated, and so _normally_ the block of the file which will get reallocated will be the same. It doesn't have to be, however. It also means that if you do: cat > file foo foo foo ^D and you crash before the ^D, the file will be lost. > but use vim command to edit a file, the blocks of the file will change? An carefully written editor such as vim will tend to do something like this: 1) write the new contest to file.new 2) fsync file.new to make sure the blocks are safely on disk 3) rename file.new to file, which will delete file and replace it with file.new That way if you crash in the middle of writing out the file, you don't lose the contents of the file. The bottom line is if you are really trying to do file-level security controls using block numbers, you need to reestablish the block numbers each time the filesystem has changed. As I mentioned, this also means you have to include the blocks for the inode table and the blocks containing the directory. It works only if the filesystem is static --- and you want to prevent anyone from changing directories and filenames. In general, it's really not a great way of doing partial security on filesystems. You may be better off using SELinux or Apparmor. - Ted From lakshmipathi.g at gmail.com Thu Jun 11 11:39:20 2009 From: lakshmipathi.g at gmail.com (lakshmi pathi) Date: Thu, 11 Jun 2009 17:09:20 +0530 Subject: How to optimize ext3/ext4 file system performance? Message-ID: Hi, I want to create ext3FS or ext4FS with it's maximum performace. I thought about below points (If i'm wrong please let me know ) : 1)Using writeback mode . 2)Use larger block size (8K) 3)Enable noaccess time, in fstab. I'm not sure about below option,but i read,its about directory optimization. 4)e2fsck -D -f /dev/sda Anyother tips to optimize the file system? I'm planning to spend sometime with "Linux From scratch" in comings days. So I want "my" system to have better performace :) -- Cheers, Lakshmipathi.G From sandeen at redhat.com Thu Jun 11 14:34:34 2009 From: sandeen at redhat.com (Eric Sandeen) Date: Thu, 11 Jun 2009 09:34:34 -0500 Subject: How to optimize ext3/ext4 file system performance? In-Reply-To: References: Message-ID: <4A3115FA.9070002@redhat.com> lakshmi pathi wrote: > Hi, > I want to create ext3FS or ext4FS with it's maximum performace. You left out the part about what workload you want to maximize... > I thought about below points (If i'm wrong please let me know ) : > > 1)Using writeback mode . Acceptable and helpful in some situations, but be aware of data integrity issues here if you crash - files may end up with wrong data in them, either your own wrong data, or someone else's. > 2)Use larger block size (8K) Only possible if you have 8K or greater pages, and if you're on x86/x86_64 etc you're limited to 4K. > 3)Enable noaccess time, in fstab. noatime can make a difference. > I'm not sure about below option,but i read,its about directory optimization. > 4)e2fsck -D -f /dev/sda -D is only useful/applicable on aged filesystems, it won't make any difference on a new fs. > Anyother tips to optimize the file system? I'm planning to spend > sometime with "Linux From scratch" in comings days. > So I want "my" system to have better performace :) By and large, the defaults provide the best performance for "normal" situations. That's why they're defaults. :) -Eric From lm at bitmover.com Thu Jun 11 20:43:56 2009 From: lm at bitmover.com (Larry McVoy) Date: Thu, 11 Jun 2009 13:43:56 -0700 Subject: ext4-users mailing list? In-Reply-To: <4A158FA6.5050702@nasa.gov> References: <4A158FA6.5050702@nasa.gov> Message-ID: <20090611204356.GB16756@bitmover.com> On Thu, May 21, 2009 at 10:30:14AM -0700, Sean McCauliff wrote: > I'm going to be making the transition to ext4 in the next few months. > Is there an ext4-users mailing list? The google knows all: http://vger.kernel.org/vger-lists.html#linux-ext4 -- --- Larry McVoy lm at bitmover.com http://www.bitkeeper.com From lakshmipathi.g at gmail.com Fri Jun 12 04:10:23 2009 From: lakshmipathi.g at gmail.com (lakshmi pathi) Date: Fri, 12 Jun 2009 09:40:23 +0530 Subject: How to optimize ext3/ext4 file system performance? In-Reply-To: <4A3115FA.9070002@redhat.com> References: <4A3115FA.9070002@redhat.com> Message-ID: Thanks for all your valuable comments :) That's why they're defaults. :) :) -- Cheers, Lakshmipathi.G On Thu, Jun 11, 2009 at 8:04 PM, Eric Sandeen wrote: > lakshmi pathi wrote: >> Hi, >> I want to create ext3FS or ext4FS with it's maximum performace. > > You left out the part about what workload you want to maximize... > >> I thought about below points (If i'm wrong please let me know ) : >> >> 1)Using writeback mode . > > Acceptable and helpful in some situations, but be aware of data > integrity issues here if you crash - files may end up with wrong data in > them, either your own wrong data, or someone else's. > >> 2)Use larger block size (8K) > > Only possible if you have 8K or greater pages, and if you're on > x86/x86_64 etc you're limited to 4K. > >> 3)Enable noaccess time, in fstab. > > noatime can make a difference. > >> I'm not sure about below option,but i read,its about directory optimization. >> 4)e2fsck -D -f /dev/sda > > -D is only useful/applicable on aged filesystems, it won't make any > difference on a new fs. > >> Anyother tips to optimize the file system? ?I'm planning to spend >> sometime with "Linux From scratch" in comings days. >> So I want "my" system to have better performace ?:) > > By and large, the defaults provide the best performance for "normal" > situations. ?That's why they're defaults. ?:) > > -Eric > From d.capitella at tin.it Fri Jun 12 19:20:15 2009 From: d.capitella at tin.it (Donato Capitella) Date: Fri, 12 Jun 2009 21:20:15 +0200 Subject: Prevent umounting Message-ID: <1244834415.15659.3.camel@kiba> I'm writing a patch to the ext3 fs. I thought that if I incremented the s_active field in the super_block struct, the user would be unable to umount the filesystem until I decremented the s_active count. But things seem to be different... If the user issues an umount command, he can still umount the filesystem and just gets a warning. My question is: how do I prevent the user from umounting the filesystem? Thank you, Donato Capitella From lists at nerdbynature.de Sun Jun 14 17:09:11 2009 From: lists at nerdbynature.de (Christian Kujau) Date: Sun, 14 Jun 2009 10:09:11 -0700 (PDT) Subject: Prevent umounting In-Reply-To: <1244834415.15659.3.camel@kiba> References: <1244834415.15659.3.camel@kiba> Message-ID: On Fri, 12 Jun 2009, Donato Capitella wrote: > If the user issues an umount command, he can still umount the filesystem > and just gets a warning. Why is the user (!root) allowed to unmount anyway? I guess the "user" or "owner" option has been used when the fileystem was mounted? > My question is: how do I prevent the user from umounting the filesystem? What are you really trying to do? If a mere user was allowed to /mount/ why do you want to disallow /unmounting/? Maybe sudo(8) can help: allow the user to /mount/ but remove the "user" and "owner" option from fstab - that way a user can still mount but it's not "her" mount any more, thus umount will fail. Christian. -- BOFH excuse #377: Someone hooked the twisted pair wires into the answering machine. From lists at nerdbynature.de Sun Jun 14 18:46:21 2009 From: lists at nerdbynature.de (Christian Kujau) Date: Sun, 14 Jun 2009 11:46:21 -0700 (PDT) Subject: Prevent umounting In-Reply-To: <1245001465.10498.2.camel@kiba> References: <1244834415.15659.3.camel@kiba> <1245001465.10498.2.camel@kiba> Message-ID: [Cc'ing ext3-users again] On Sun, 14 Jun 2009, Donato Capitella wrote: > Well, that's not it. > > I modified the ext3 fs and added a kernel thread that performs deferred > operations. As long as this thread is working, I would like the file > system not to be unmounted. I still don't understand the reasoning here: "not to be unmounted" as in a permission-bound issue ("is not allowed to unmount") or a technical issue? For the latter: simply keep an open file on the filesystem, unmount will fail with -EBUSY. Or it's a matter of "because we can", as in "because I want to be able to disallow mounting on the filesystem level". But then again, I did not have my coffee yet and my brain keeps asking why, why, why? :) Christian. -- BOFH excuse #202: kernel panic: write-only-memory (/dev/wom0) capacity exceeded. From darkonc at gmail.com Mon Jun 15 19:58:26 2009 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Mon, 15 Jun 2009 12:58:26 -0700 Subject: Prevent umounting In-Reply-To: References: <1244834415.15659.3.camel@kiba> <1245001465.10498.2.camel@kiba> Message-ID: <6cd50f9f0906151258n1eb2aa47x17cb172dff8e06c8@mail.gmail.com> If you want to keep people from unmounting, create a file: /mnt/my_file_system/tmp/TMPxxxxxx then unlink that file, but *do not close your open file descriptor.* The name will disappear but the inode will continue to exist. As long as your program keeps the file descriptor for the unlinked file open, users will be unable to unmount the filesystem A simple example: touch /media/disk-1/nothinge tail -f /media/disk-1/nothing& rm /media/disk-1/nothing umount /media/disk-1 eject /media/disk-1 # (both the last two commands should fail).. kill %% # kill the background 'tail' process eject /media/disk-1 # that last 'eject' should work On Sun, Jun 14, 2009 at 11:46 AM, Christian Kujau wrote: > [Cc'ing ext3-users again] > > On Sun, 14 Jun 2009, Donato Capitella wrote: > > Well, that's not it. > > > > I modified the ext3 fs and added a kernel thread that performs deferred > > operations. As long as this thread is working, I would like the file > > system not to be unmounted. > > I still don't understand the reasoning here: "not to be unmounted" as in > a permission-bound issue ("is not allowed to unmount") or a technical > issue? For the latter: simply keep an open file on the filesystem, unmount > will fail with -EBUSY. Or it's a matter of "because we can", as in > "because I want to be able to disallow mounting on the filesystem level". > > But then again, I did not have my coffee yet and my brain keeps asking > why, why, why? :) > > Christian. > -- > BOFH excuse #202: > > kernel panic: write-only-memory (/dev/wom0) capacity exceeded. > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From morty+redhat at frakir.org Wed Jun 17 18:07:44 2009 From: morty+redhat at frakir.org (Morty) Date: Wed, 17 Jun 2009 14:07:44 -0400 Subject: command to recreate an FS? In-Reply-To: References: <20090603090947.GE19727@red-sonja.frakir.org> Message-ID: <20090617180743.GA2825@red-sonja.frakir.org> On Sun, Jun 07, 2009 at 12:43:04AM -0700, Christian Kujau wrote: > On Wed, 3 Jun 2009, Morty wrote: > > Is there a lightweight command to recreate an ext2/ext3/ext4 FS with > > original options, > > If no options have been specified to mke2fs, newer versions of e2fsprogs > are shipping with /etc/mke2fs.conf which holds a few defaults. Even if no options were specified, some distros use UUID as the basis for FS mounts. That's great if disks move around, but not so good if I upgrade from one disk to another. And if I did specify options to mke2fs when creating the FS, or if I ran tune2fs to change options later, it would be nice to have an easy way to recreate those options. > > If nothing like this already exists, I'll whip up something in Perl. > > Be sure to publish the download URL once it's ready ;-) Will do. Speaking of which, is there a way to set the directory hash seed? tune2fs -l will show me the current hash seed, but I don't see an option in mke2fs or tune2fs to set it. - Morty From adilger at sun.com Wed Jun 17 23:16:05 2009 From: adilger at sun.com (Andreas Dilger) Date: Wed, 17 Jun 2009 17:16:05 -0600 Subject: command to recreate an FS? In-Reply-To: <20090617180743.GA2825@red-sonja.frakir.org> References: <20090603090947.GE19727@red-sonja.frakir.org> <20090617180743.GA2825@red-sonja.frakir.org> Message-ID: <20090617231605.GI13073@webber.adilger.int> On Jun 17, 2009 14:07 -0400, Morty wrote: > Even if no options were specified, some distros use UUID as the basis > for FS mounts. That's great if disks move around, but not so good if I > upgrade from one disk to another. And if I did specify options to > mke2fs when creating the FS, or if I ran tune2fs to change options > later, it would be nice to have an easy way to recreate those options. The UUID can be specified or set during/after mke2fs I believe. > Speaking of which, is there a way to set the directory hash seed? > tune2fs -l will show me the current hash seed, but I don't see an > option in mke2fs or tune2fs to set it. Probably debugfs, since this isn't expected to be something that users do normally. It will cause all existing hash indexes to become invalid, and "e2fsck -fD" would be needed to recreate them. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. From bothie at gmx.de Thu Jun 18 11:22:19 2009 From: bothie at gmx.de (Bodo Thiesen) Date: Thu, 18 Jun 2009 13:22:19 +0200 Subject: command to recreate an FS? In-Reply-To: <20090617180743.GA2825@red-sonja.frakir.org> References: <20090603090947.GE19727@red-sonja.frakir.org> <20090617180743.GA2825@red-sonja.frakir.org> Message-ID: <20090618132219.4c133706@30_bodo> * Morty hat geschrieben: >* On Sun, Jun 07, 2009 at 12:43:04AM -0700, Christian Kujau wrote: >>* On Wed, 3 Jun 2009, Morty wrote: >>> Is there a lightweight command to recreate an ext2/ext3/ext4 FS with >>> original options, > Even if no options were specified, some distros use UUID as the basis > for FS mounts. [...] That's great if disks move around, but not so good if I > upgrade from one disk to another. [...] > Speaking of which, is there a way to set the directory hash seed? > tune2fs -l will show me the current hash seed, but I don't see an > option in mke2fs or tune2fs to set it. If you only intend on moving a file system from one disk to another, I'd suggest the following procedure: 1.) Install the new disk leaving the old installed. 2.) Make any partitioning necessary. 3.) Remount the file system to be moved r/o if it's mounted r/w currently (?init S? or ?init 1? and ?lsof? is your friend ;) 4.) dd if=old_partition of=new_partition 5.) resize2fs new_partition 6.) install lilo/grub on the new disk 7.) shutdown, remove the old disk, reboot This approach should leave the uuid and directory hash seed unchanged. If you however intend to recreate the file system after severe data loss without resizing of the file system itself, then you could do this: Save the old super block (the best would be to do this BEFORE you loose the data ;) You can use the following command for that: dd if=partition of=filename4superblock bs=1024 skip=1 count=1 After you have recreated the file system with the same arguments to mke2fs, you should be able to restore the old super block using this command: dd if=filename4superblock of=partition bs=1024 seek=1 count=1 After that, you will have to run e2fsck -fD, to update the copies of the super block and to recreate the directory hashes for root and lost+found. Now you can fill up the file system with the most recent backup. Regards, Bodo PS: If you're gonna write your Perl script to store the bare minimum data, I'd suggest you store a) the (first) super block, b) the group descriptor table c) the first three blocks (one block is 512 byte as we all know) of the inode table. To save space, just compress the data using lzma or some other compressor. Writing this information to the right places of an empty file system should be enough to recreate the filesystem, however, e2fsck would be neccessarry then too, because you will loose the contents of the root and lost+found directory by this approach which have to be recreated. Zeroing out the device before writing back this data would be a good thing, too. From sean.d.mccauliff at nasa.gov Fri Jun 12 20:11:11 2009 From: sean.d.mccauliff at nasa.gov (Mccauliff, Sean D. (ARC-PX)[LOCKHEED MARTIN CORP]) Date: Fri, 12 Jun 2009 15:11:11 -0500 Subject: ext4-users mailing list? In-Reply-To: <20090611204356.GB16756@bitmover.com> References: <4A158FA6.5050702@nasa.gov>,<20090611204356.GB16756@bitmover.com> Message-ID: <341DAA96EE3A8444B6E4657BE8A846EA75DB2C8F@NDJSSCC06.ndc.nasa.gov> Thanks! Sean ________________________________________ From: Larry McVoy [lm at bitmover.com] Sent: Thursday, June 11, 2009 1:43 PM To: Mccauliff, Sean D. (ARC-PX)[LOCKHEED MARTIN CORP] Cc: ext3-users at redhat.com Subject: Re: ext4-users mailing list? On Thu, May 21, 2009 at 10:30:14AM -0700, Sean McCauliff wrote: > I'm going to be making the transition to ext4 in the next few months. > Is there an ext4-users mailing list? The google knows all: http://vger.kernel.org/vger-lists.html#linux-ext4 -- --- Larry McVoy lm at bitmover.com http://www.bitkeeper.com From william.twomey at gmail.com Sun Jun 14 23:56:08 2009 From: william.twomey at gmail.com (Will Twomey) Date: Sun, 14 Jun 2009 18:56:08 -0500 Subject: ext3 Saved my life... Message-ID: <9a6de10906141656s409d9d9cw14a8da2707459721@mail.gmail.com> Using ext3 recently saved my life, and just wanted to send out a big thank-you to the ext3 developers. I have a 1.5TB array (RAID1) that, during an install, I accidentally ran mkfs on. I shut off power to the machine shortly after (30sec-1min) figuring that I couldn't do any more damage to it. I booted up a live CD, assembled the array and ran fsck... Out of the 1.2TB I had been using, I was able to recover over 900GB! Thanks for the great filesystem! :) -Will From gregoire at gentil.com Fri Jun 19 23:42:54 2009 From: gregoire at gentil.com (Gregoire Gentil) Date: Fri, 19 Jun 2009 16:42:54 -0700 Subject: How to optimize the cloning of a disk with a big empty ext3 partition? In-Reply-To: References: Message-ID: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> Hello, I need to duplicate a huge number of identical 8GB SD cards on which I have 1GB of data at the beginning of the disk on various partitions and then a 7GB ext3 partition which is rather empty (just a few MB of data). The duplicator device enables me to select which sectors I can binary-duplicate. I would love to divide by 8 my duplication time, by duplicating only the first GB and then the beginning of the 7GB ext3 partition. When I do a mkfs.ext3, I see that the super-block is written to various locations of the partition which is not good for the optimization I would like to do. Anyone can advise how I could optimize my duplication? What are my options? Would it be the same situation for ext2? Meaning where is the journal written on the ext3 partition? Is there a single version or multiple versions? Thanks in advance for any answer or pointer, Gregoire From tytso at mit.edu Sat Jun 20 01:33:03 2009 From: tytso at mit.edu (Theodore Tso) Date: Fri, 19 Jun 2009 21:33:03 -0400 Subject: How to optimize the cloning of a disk with a big empty ext3 partition? In-Reply-To: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> References: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> Message-ID: <20090620013303.GB7644@mit.edu> On Fri, Jun 19, 2009 at 04:42:54PM -0700, Gregoire Gentil wrote: > Hello, > > I need to duplicate a huge number of identical 8GB SD cards on which I > have 1GB of data at the beginning of the disk on various partitions and > then a 7GB ext3 partition which is rather empty (just a few MB of data). > > The duplicator device enables me to select which sectors I can > binary-duplicate. I would love to divide by 8 my duplication time, by > duplicating only the first GB and then the beginning of the 7GB ext3 > partition. > > When I do a mkfs.ext3, I see that the super-block is written to various > locations of the partition which is not good for the optimization I > would like to do. The metadata for ext3 is scattered across the disk; it's not just the superblock, but it's also parts of the inode table, and bitmap allocation blocks, which must be initialized. It's possible to determine the list of blocks that need to be initialized, but it's going to be a rather long list. If you need to type the list of sectors into your duplicator, it's probably not practical. If on the other hand you can feed it a file with a set of sector numbers, you could extract that list of blocks in use from the filesystem (which listed by dumpe2fs, although not in the most convenient format), and convert it to sector numbers counting from the beginning of the disk (as opposed to block numbers counting from the beginning of the partitions), and then feed the whole long list of sector block ranges to your duplicator device. - Ted From gregoire at gentil.com Sat Jun 20 02:11:00 2009 From: gregoire at gentil.com (Gregoire Gentil) Date: Fri, 19 Jun 2009 19:11:00 -0700 Subject: How to optimize the cloning of a disk with a big empty ext3 partition? In-Reply-To: <20090620013303.GB7644@mit.edu> References: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> <20090620013303.GB7644@mit.edu> Message-ID: <1245463860.13939.80.camel@alwaysinnovating.earthlink.net> On Fri, 2009-06-19 at 21:33 -0400, Theodore Tso wrote: > On Fri, Jun 19, 2009 at 04:42:54PM -0700, Gregoire Gentil wrote: > > Hello, > > > > I need to duplicate a huge number of identical 8GB SD cards on which I > > have 1GB of data at the beginning of the disk on various partitions and > > then a 7GB ext3 partition which is rather empty (just a few MB of data). > > > > The duplicator device enables me to select which sectors I can > > binary-duplicate. I would love to divide by 8 my duplication time, by > > duplicating only the first GB and then the beginning of the 7GB ext3 > > partition. > > > > When I do a mkfs.ext3, I see that the super-block is written to various > > locations of the partition which is not good for the optimization I > > would like to do. > > The metadata for ext3 is scattered across the disk; it's not just the > superblock, but it's also parts of the inode table, and bitmap > allocation blocks, which must be initialized. > > It's possible to determine the list of blocks that need to be > initialized, but it's going to be a rather long list. If you need to > type the list of sectors into your duplicator, it's probably not > practical. If on the other hand you can feed it a file with a set of > sector numbers, you could extract that list of blocks in use from the > filesystem (which listed by dumpe2fs, although not in the most > convenient format), and convert it to sector numbers counting from the > beginning of the disk (as opposed to block numbers counting from the > beginning of the partitions), and then feed the whole long list of > sector block ranges to your duplicator device. > > - Ted Ted, Many thanks for the explanation. It's what I was afraid of! Gr?goire From darkonc at gmail.com Sat Jun 20 04:39:32 2009 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Fri, 19 Jun 2009 21:39:32 -0700 Subject: How to optimize the cloning of a disk with a big empty ext3 partition? In-Reply-To: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> References: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> Message-ID: <6cd50f9f0906192139o39a03275s4589c0bde34429b8@mail.gmail.com> A program like partimage will copy only the sectors that are already in use, but it requires that the destination partition be at least as big as the original (otherwise it would be required to relocate used sectors that are near the end of the parttition). The only real way around your problem would be to resize2fs to resize the filesystem (not the partition) to a bit over 1GB At that point you could safely copy only the first Gig of the partition (whatever size you changed the filesystem to). You could then (if you wanted to) resize the filesystem back to fill the partition. If you just need the data from the partition archived, and don't need it to be live, then you can just use partimage to dump the used sectors, into a file... When you need to reconstitute the filesystem, however, partimage will want to create a filesystem the same size as the original. On Fri, Jun 19, 2009 at 4:42 PM, Gregoire Gentil wrote: > Hello, > > I need to duplicate a huge number of identical 8GB SD cards on which I > have 1GB of data at the beginning of the disk on various partitions and > then a 7GB ext3 partition which is rather empty (just a few MB of data). > > The duplicator device enables me to select which sectors I can > binary-duplicate. I would love to divide by 8 my duplication time, by > duplicating only the first GB and then the beginning of the 7GB ext3 > partition. > > When I do a mkfs.ext3, I see that the super-block is written to various > locations of the partition which is not good for the optimization I > would like to do. > > Anyone can advise how I could optimize my duplication? What are my > options? Would it be the same situation for ext2? Meaning where is the > journal written on the ext3 partition? Is there a single version or > multiple versions? > > Thanks in advance for any answer or pointer, > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex at alex.org.uk Sat Jun 20 08:00:10 2009 From: alex at alex.org.uk (Alex Bligh) Date: Sat, 20 Jun 2009 09:00:10 +0100 Subject: How to optimize the cloning of a disk with a big empty ext3 partition? In-Reply-To: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> References: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> Message-ID: <7C3F486409A1EB92420E6D28@nimrod.local> --On 19 June 2009 16:42:54 -0700 Gregoire Gentil wrote: > I need to duplicate a huge number of identical 8GB SD cards on which I > have 1GB of data at the beginning of the disk on various partitions and > then a 7GB ext3 partition which is rather empty (just a few MB of data). > > The duplicator device enables me to select which sectors I can > binary-duplicate. I would love to divide by 8 my duplication time, by > duplicating only the first GB and then the beginning of the 7GB ext3 > partition. I am guessing here about your application, but how about: 1. Leave the 7GB partition unformatted in the SD card image, set the partition type in the partition table to a magic value 2. Make a gzip'd version of a disk image of that partition. Do this by using a loopback file system on a file the size of the partition which is initially full of a single magic byte/word. The resultant file should be tiny as the image will be mostly blocks full of the magic byte/word. Put this on another partition. 3. In the boot code for the device (I am assuming you can change that or better still it is on the CF device), put a little bit of code that a) checks the partition table for the magic number, and if set, do this: b) mounts the partition with the gzip image, extracts it, writes it (only writing blocks that are not full of the magic byte/word, so the process will be quick) to the new partition, then unmounts the partition with the gzip image so the boot process can continue c) change the partition table entry to ext3 d) continue the boot process Of course this only happens on the first boot, and is only writing a few Mb of data. 4. Only duplicate the first 1GB of the disk The program described is a few tens of lines of C linked with zlib. It also gives you a relatively easy way to restore the partition if you should ever need to (change partition table magic number back, reboot). If this is the only r/w partition, that might be useful. -- Alex Bligh From gregoire at gentil.com Sun Jun 21 00:42:09 2009 From: gregoire at gentil.com (Gregoire Gentil) Date: Sat, 20 Jun 2009 17:42:09 -0700 Subject: How to optimize the cloning of a disk with a big empty ext3 partition? In-Reply-To: <6cd50f9f0906192139o39a03275s4589c0bde34429b8@mail.gmail.com> References: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> <6cd50f9f0906192139o39a03275s4589c0bde34429b8@mail.gmail.com> Message-ID: <1245544929.5192.55.camel@alwaysinnovating.earthlink.net> Thanks for the suggestion, Gr?goire On Fri, 2009-06-19 at 21:39 -0700, Stephen Samuel (gmail) wrote: > A program like partimage will copy only the sectors that are already > in use, but it > requires that the destination partition be at least as big as the > original (otherwise > it would be required to relocate used sectors that are near the end of > the parttition). > > The only real way around your problem would be to resize2fs to resize > the filesystem > (not the partition) to a bit over 1GB At that point you could safely > copy only the first > Gig of the partition (whatever size you changed the filesystem to). > You could then > (if you wanted to) resize the filesystem back to fill the partition. > > If you just need the data from the partition archived, and don't need > it to be live, > then you can just use partimage to dump the used sectors, into a > file... When you > need to reconstitute the filesystem, however, partimage will want to > create a filesystem > the same size as the original. > > On Fri, Jun 19, 2009 at 4:42 PM, Gregoire Gentil > wrote: > Hello, > > I need to duplicate a huge number of identical 8GB SD cards on > which I > have 1GB of data at the beginning of the disk on various > partitions and > then a 7GB ext3 partition which is rather empty (just a few MB > of data). > > The duplicator device enables me to select which sectors I can > binary-duplicate. I would love to divide by 8 my duplication > time, by > duplicating only the first GB and then the beginning of the > 7GB ext3 > partition. > > When I do a mkfs.ext3, I see that the super-block is written > to various > locations of the partition which is not good for the > optimization I > would like to do. > > Anyone can advise how I could optimize my duplication? What > are my > options? Would it be the same situation for ext2? Meaning > where is the > journal written on the ext3 partition? Is there a single > version or > multiple versions? > > Thanks in advance for any answer or pointer, > > -- > Stephen Samuel http://www.bcgreen.com Software, like love, > 778-861-7641 grows when you give it away From gregoire at gentil.com Sun Jun 21 00:43:36 2009 From: gregoire at gentil.com (Gregoire Gentil) Date: Sat, 20 Jun 2009 17:43:36 -0700 Subject: How to optimize the cloning of a disk with a big empty ext3 partition? In-Reply-To: <7C3F486409A1EB92420E6D28@nimrod.local> References: <1245454974.13939.60.camel@alwaysinnovating.earthlink.net> <7C3F486409A1EB92420E6D28@nimrod.local> Message-ID: <1245545016.5192.57.camel@alwaysinnovating.earthlink.net> I though about something like that. It's a trade-off between duplication hassle (bear by me) vs. hassle for the user during the first boot. I'm already some squashfs so it's a little bit the same point. Thanks for the suggestion, Gr?goire On Sat, 2009-06-20 at 09:00 +0100, Alex Bligh wrote: > > --On 19 June 2009 16:42:54 -0700 Gregoire Gentil > wrote: > > > I need to duplicate a huge number of identical 8GB SD cards on which I > > have 1GB of data at the beginning of the disk on various partitions and > > then a 7GB ext3 partition which is rather empty (just a few MB of data). > > > > The duplicator device enables me to select which sectors I can > > binary-duplicate. I would love to divide by 8 my duplication time, by > > duplicating only the first GB and then the beginning of the 7GB ext3 > > partition. > > I am guessing here about your application, but how about: > > 1. Leave the 7GB partition unformatted in the SD card image, set the > partition type in the partition table to a magic value > 2. Make a gzip'd version of a disk image of that partition. Do this by > using a loopback file system on a file the size of the partition > which is initially full of a single magic byte/word. The resultant > file should be tiny as the image will be mostly blocks full of the > magic byte/word. Put this on another partition. > 3. In the boot code for the device (I am assuming you can change that > or better still it is on the CF device), put a little bit of code > that > a) checks the partition table for the magic number, and if set, > do this: > b) mounts the partition with the gzip image, extracts it, > writes it (only writing blocks that are not full of the magic > byte/word, so the process will be quick) to the new partition, > then unmounts the partition with the gzip image so the boot > process can continue > c) change the partition table entry to ext3 > d) continue the boot process > Of course this only happens on the first boot, and is only writing > a few Mb of data. > 4. Only duplicate the first 1GB of the disk > > The program described is a few tens of lines of C linked with zlib. > It also gives you a relatively easy way to restore the partition > if you should ever need to (change partition table magic number back, > reboot). If this is the only r/w partition, that might be useful. > From jelledejong at powercraft.nl Tue Jun 23 09:12:54 2009 From: jelledejong at powercraft.nl (Jelle de Jong) Date: Tue, 23 Jun 2009 11:12:54 +0200 Subject: how to calculate correct values for resize2fs Message-ID: <4A409C96.7040703@powercraft.nl> Hello everybody, I want to make a new 16 MB partition on the end of my /dev/sda SSD. Currently the SSD is partitioned in one ext2 /dev/sda1. My question is how can i make this new partition with the CLI tools. I tried several things but they keep failing because I don't know how to calculate with blocks , sectors and MB. Would somebody willing to learn me some things and maybe show some examples. I also have some parted question on how to just make a partition from the free space on the end of the disk. Thanks in advance, Best regards, Jelle current situation ext2 /dev/sda1, wanted situation ext2 /dev/sda1 and 16MB 0xef /dev/sda2 on the end of /dev/sda 16MB 0xef PS. I know I got some fsck -n /dev/sda1 errors here. Working on that too. $ dumpe2fs -h /dev/sda1 dumpe2fs 1.41.6 (30-May-2009) Filesystem volume name: Last mounted on: Filesystem UUID: 3e6384e0-dcaf-4c93-972f-f54c00a7dd22 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: filetype sparse_super Default mount options: (none) Filesystem state: not clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 492032 Block count: 983973 Reserved block count: 49198 Free blocks: 119386 Free inodes: 355584 First block: 0 Block size: 4096 Fragment size: 4096 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 15872 Inode blocks per group: 496 Last mount time: Tue Jun 23 10:52:00 2009 Last write time: Tue Jun 23 10:52:11 2009 Mount count: 9 Maximum mount count: 30 Last checked: Tue Jun 23 00:23:53 2009 Check interval: 0 () Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 $ sfdisk -d /dev/sda # partition table of /dev/sda unit: sectors /dev/sda1 : start= 63, size= 7871787, Id=83, bootable /dev/sda2 : start= 0, size= 0, Id= 0 /dev/sda3 : start= 0, size= 0, Id= 0 /dev/sda4 : start= 0, size= 0, Id= 0 $ fdisk -l /dev/sda Disk /dev/sda: 4034 MB, 4034838528 bytes 255 heads, 63 sectors/track, 490 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xfb38fb38 Device Boot Start End Blocks Id System /dev/sda1 * 1 490 3935893+ 83 Linux bc <<< "16*1024" 16384 bc <<< "3935893 - 16384" 3919509 echo "3935893 - 16384" | bc 3919509 resize2fs /dev/sda1 3919509 bc <<< "4034-16" resize2fs /dev/sda1 4018M # result: partition is only 983973 (4K) blocks # requested: 1028608 blocks # will not fit exiting # how can I convert the free space to a partition with parted? sudo /sbin/parted --script /dev/sda \ mkpart primary 3919509 100% \ print From samix_119 at yahoo.com Tue Jun 23 09:38:48 2009 From: samix_119 at yahoo.com (samix_119 at yahoo.com) Date: Tue, 23 Jun 2009 02:38:48 -0700 (PDT) Subject: EXT3-fs error - Freeing blocks not in datazone Message-ID: <859087.16103.qm@web112615.mail.gq1.yahoo.com> Hello, * One of our servers experienced a file system corruption and gave this error in the /var/log/messages file. {code} Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1): ext3_free_blocks: Freeing blocks not in datazone - blo ck = 2684399216, count = 1 Jun 17 05:53:47 myhost kernel: Aborting journal on device sdb1. Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) in ext3_free_blocks_sb: Journal has aborted Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) in ext3_free_blocks_sb: Journal has aborted Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) in ext3_reserve_inode_write: Journal has aborted Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) in ext3_truncate: Journal has aborted Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) in ext3_reserve_inode_write: Journal has aborted Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) in ext3_orphan_del: Journal has aborted Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) in ext3_reserve_inode_write: Journal has aborted Jun 17 05:53:47 myhost kernel: ext3_abort called. Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1): ext3_journal_start_sb: Detected aborted journal Jun 17 05:53:47 myhost kernel: Remounting filesystem read-only Jun 17 05:53:47 myhost kernel: __journal_remove_journal_head: freeing b_committed_data {code} * The file system then went into read only mode * We tried to unmount and then remount the file system,but on trying to unmount the server simply hung and we had to restart the machine * Fsck found errors on the file system and fixed it * This filesystem is on a raid partition (RAID 1) and the physical and virtual disk caches are turned on. * Even the barrier option for this file system was on to prevent any corruption of the data in case of a sudden reset * Can some one please point to me what is that went wrong exactly and how we can avoid this in the future Regards, Muhammed Sameer From adilger at sun.com Tue Jun 23 10:58:26 2009 From: adilger at sun.com (Andreas Dilger) Date: Tue, 23 Jun 2009 12:58:26 +0200 Subject: EXT3-fs error - Freeing blocks not in datazone In-Reply-To: <859087.16103.qm@web112615.mail.gq1.yahoo.com> References: <859087.16103.qm@web112615.mail.gq1.yahoo.com> Message-ID: <20090623105707.GA31668@webber.adilger.int> On Jun 23, 2009 02:38 -0700, samix_119 at yahoo.com wrote: > * This filesystem is on a raid partition (RAID 1) and the physical and virtual disk caches are turned on. Running with cache on is dangerous for this reason. > * Even the barrier option for this file system was on to prevent any corruption of the data in case of a sudden reset For software RAID the barrier option does not work. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. From samix_119 at yahoo.com Tue Jun 23 12:29:50 2009 From: samix_119 at yahoo.com (Muhammed Sameer) Date: Tue, 23 Jun 2009 05:29:50 -0700 (PDT) Subject: EXT3-fs error - Freeing blocks not in datazone Message-ID: <340707.77646.qm@web112610.mail.gq1.yahoo.com> > > * This filesystem is on a raid partition (RAID 1) and > the physical and virtual disk caches are turned on. > > Running with cache on is dangerous for this reason. > > > * Even the barrier option for this file system was on > to prevent any corruption of the data in case of a sudden > reset > > For software RAID the barrier option does not work. Thank you for your input Andreas, we are actually using hardware raid > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. > > From tytso at mit.edu Tue Jun 23 16:46:31 2009 From: tytso at mit.edu (Theodore Tso) Date: Tue, 23 Jun 2009 12:46:31 -0400 Subject: EXT3-fs error - Freeing blocks not in datazone In-Reply-To: <859087.16103.qm@web112615.mail.gq1.yahoo.com> References: <859087.16103.qm@web112615.mail.gq1.yahoo.com> Message-ID: <20090623164631.GG26079@mit.edu> On Tue, Jun 23, 2009 at 02:38:48AM -0700, samix_119 at yahoo.com wrote: > > > * Can some one please point to me what is that went wrong exactly and how we can avoid this in the future > It looks like some kind of on-disk corruption, but whether that was caused by a memory hiccup, or a controller hiccup, or a disk hiccup is hard to say. Was this a one-time problem, or are you have you sufferred problems more than once? - - Ted From samix_119 at yahoo.com Wed Jun 24 10:48:37 2009 From: samix_119 at yahoo.com (Muhammed Sameer) Date: Wed, 24 Jun 2009 03:48:37 -0700 (PDT) Subject: EXT3-fs error - Freeing blocks not in datazone Message-ID: <260631.34167.qm@web112618.mail.gq1.yahoo.com> This was the first time that we faced this kind of a problem Regards, Muhammed Sameer > wrote: > > > > > > * Can some one please point to me what is that went > wrong exactly and how we can avoid this in the future > > > > It looks like some kind of on-disk corruption, but whether > that was > caused by a memory hiccup, or a controller hiccup, or a > disk hiccup is > hard to say.? Was this a one-time problem, or are you > have you > sufferred problems more than once? > > ??? ? -??? > ??? ??? ??? > ??? - Ted > From lists at nerdbynature.de Thu Jun 25 17:13:40 2009 From: lists at nerdbynature.de (Christian Kujau) Date: Thu, 25 Jun 2009 10:13:40 -0700 (PDT) Subject: how to calculate correct values for resize2fs In-Reply-To: <4A409C96.7040703@powercraft.nl> References: <4A409C96.7040703@powercraft.nl> Message-ID: On Tue, June 23, 2009 02:12, Jelle de Jong wrote: > I want to make a new 16 MB partition on the end of my /dev/sda SSD. > Currently the SSD is partitioned in one ext2 /dev/sda1. > My question is how can i make this new partition with the CLI tools. Creating partitions is done with {c,s}fdisk. I think cfdisk ask where to create this partition (at the end/at the beginning of the disk). > I also have some parted question on how to just make a partition from > the free space on the end of the disk. "man parted", but I guess if cfdisk can do it, parted should be able to do so as well. Hm, I just checked: (parted) help mkpart mkpart PART-TYPE [FS-TYPE] START END make a partition So, you can actually specify where you want the partition to start. After creating a new partition, you can mkfs this new one. I did not understand at which point you want to "resize" a partition. Resizing a filesystem is usually done on existing, resized partitions. C. -- BOFH excuse #51: Cosmic ray particles crashed through the hard disk platter From mvolaski at aecom.yu.edu Thu Jun 25 18:29:46 2009 From: mvolaski at aecom.yu.edu (Maurice Volaski) Date: Thu, 25 Jun 2009 14:29:46 -0400 Subject: [Q] What might cause modification dates to shift later by an hour? Message-ID: Recently, our backup software oddly decided to rebackup a good portion of our file server instead of just doing an incremental. When I examined various sets of presumably identical files, I discovered that the modification dates on these files were no longer the same. Many files were re-dated to exactly one hour later such that if a file had been modified on 3/24/04 at 2:24:53 PM, it's modification date had somehow been changed to 3/24/04 at 3:24:53 PM. A point to note that is that this server is actually two separate servers with independent storage mirrored by drbd. And it's possible that the first backup was done on one server and the second on the other. However, drbd mirrors the storage bit by bit, so it's hard to see how files on each of them could have different modification dates. Also, even if the time on the computers were different by an hour (they're current the same to the second), it's not clear how the modification dates could somehow get re-dated. It's also not clear what mechanism could have redated them. I've almost certainly run fsck on the affected filesystems. Would there ever be a circumstance when fsck might second guess a file's modification time and change it? e2fsck is currently 1.41.4, though I don't know if that's the version that I ran when I ran it. However, there seems to be a field in the backup software called status modification date and a lot of these affected files had their status modified around the same time, implying something did iterate over the files and do something to them. -- Maurice Volaski, mvolaski at aecom.yu.edu Computing Support, Rose F. Kennedy Center Albert Einstein College of Medicine of Yeshiva University From jelledejong at powercraft.nl Thu Jun 25 19:28:29 2009 From: jelledejong at powercraft.nl (Jelle de Jong) Date: Thu, 25 Jun 2009 21:28:29 +0200 Subject: how to calculate correct values for resize2fs In-Reply-To: References: <4A409C96.7040703@powercraft.nl> Message-ID: <4A43CFDD.1020200@powercraft.nl> Christian Kujau wrote: > On Tue, June 23, 2009 02:12, Jelle de Jong wrote: >> I want to make a new 16 MB partition on the end of my /dev/sda SSD. >> Currently the SSD is partitioned in one ext2 /dev/sda1. >> My question is how can i make this new partition with the CLI tools. > > Creating partitions is done with {c,s}fdisk. I think cfdisk ask where to > create this partition (at the end/at the beginning of the disk). > >> I also have some parted question on how to just make a partition from >> the free space on the end of the disk. > > "man parted", but I guess if cfdisk can do it, parted should be able to do > so as well. Hm, I just checked: > > (parted) help mkpart > mkpart PART-TYPE [FS-TYPE] START END make a partition > > So, you can actually specify where you want the partition to start. > > After creating a new partition, you can mkfs this new one. I did not > understand at which point you want to "resize" a partition. Resizing a > filesystem is usually done on existing, resized partitions. > > C. Thanks for the feedback, I seem to have made some mistakes in explaining the situation. I am not really a newbie I know most of the fdisk and parted tools almost out of my head. The question is more about resizing partitions and changing disk layouts. How to calculate with block sizes and get the wanted partition sizes. I show the details of my file system in my previous mail, what would be the exact commands to to resize my /dev/sda1 partition and what would be the exact command to create a additional partition that fits the new created free space: so: resize2fs /dev/sda1 ???????? sudo /sbin/parted --script /dev/sda \ mkpart primary ?????? 100% \ print fdisk /dev/sda # enter 'm, t, 2, ef ,w' Hope you can show me how to calculate the value to get a 16 MB free space on /dev/sda with the resize2fs tool. Best regards, Jelle de Jong From bothie at gmx.de Thu Jun 25 21:20:15 2009 From: bothie at gmx.de (Bodo Thiesen) Date: Thu, 25 Jun 2009 23:20:15 +0200 Subject: how to calculate correct values for resize2fs In-Reply-To: <4A43CFDD.1020200@powercraft.nl> References: <4A409C96.7040703@powercraft.nl> <4A43CFDD.1020200@powercraft.nl> Message-ID: <20090625232015.398e8603@30_bodo> * Jelle de Jong hat geschrieben: > Christian Kujau wrote: >> After creating a new partition, you can mkfs this new one. I did not >> understand at which point you want to "resize" a partition. Resizing a >> filesystem is usually done on existing, resized partitions. He wants to SHRINK the partition, so he will have to resize the file system, then shrink the partition. > resize2fs /dev/sda1 ???????? [...] > Hope you can show me how to calculate the value to get a 16 MB free > space on /dev/sda with the resize2fs tool. Just resize the file system on sda1 to 3GB (or another value which leaves enough space - just overcalculate here). Then make the partition smaller to the final size you want it to be, then re-erlarge the file system to it's final size. And another advice: Do NOT create the new partition(s), before you completed the TWO resize-steps. Regards, Bodo From getsmit at gmail.com Fri Jun 26 01:58:04 2009 From: getsmit at gmail.com (Smit Shah) Date: Thu, 25 Jun 2009 18:58:04 -0700 Subject: How to get device name with device id? Message-ID: <8770d98c0906251858j1312e881k33ada185f88bb5e4@mail.gmail.com> Hey, I just ran through this post, and since no solution was posted, here's a piece of code that i had written Infact when i wrote this i wanted both the device name and also the file system on that partition. But for this, one needs the root privileges. #include #include #include #include #include #include #include int main(int argc,char *argv[]) { struct blkid_struct_dev *dev; struct blkid_struct_cache *cache; char *type,*dev_name; struct stat buf; assert(argc == 2); assert(argv[1]); assert(lstat(argv[1],&buf) == 0); dev_name=blkid_devno_to_devname(buf.st_dev); assert(dev_name); printf("Device name :%s\n",dev_name); assert(blkid_get_cache(&cache,"/dev/null") == 0); dev = (struct blkid_struct_dev *)blkid_get_dev(cache,dev_name, BLKID_DEV_NORMAL); assert(dev); type = blkid_get_tag_value(cache,"TYPE",dev_name); printf("File System type is : %s\n",type); blkid_free_dev(dev); return (0); } You have to compile it with the libblkid. Do man for libblkid, that should be really helpful. -Smit Shah. --- On Thu, 20/3/08, liuyue wrote: > From: liuyue > Subject: Re: How to get device name with device id? > To: "ashitpro yahoo co in" , "ext3-users redhat com" > Date: Thursday, 20 March, 2008, 4:34 PM > ashish mahamuni, > > I guess maybe the following function does what you want. > But it is a kernel function, sorry :( > > int __file_to_disk (char * file_name, char *disk_name) { > int err = 0; > struct nameidata nd; > struct super_block * sb; > struct vfsmount *mnt; > err = path_lookup(file_name, LOOKUP_FOLLOW, > &nd); > if(err){ > DCFS3_ERROR("error to parse the file > name, %s\n", file_name); > goto exit; > } > mnt = nd.mnt; > sb = mnt->mnt_sb; > strcpy (disk_name, > sb->s_bdev->bd_disk->disk_name); > path_release(&nd); > exit: > return err; > } > > ======= 2008-03-20 19:21:04 > ????????======= > > >Hi all, > > > >I want to open a device(/dev/sda1, /dev/hda2 etc) in > which my file exists. > >I've used 'stat' system call to get the > device id. > > > >But now I want the device name from this id(st_dev). > >How to get that one? > >Or > >Do you have any other method to know the device name > where my file resides? > > > >Thanks > > > > > > Bollywood, fun, friendship, sports and more. You > name it, we have it on http://in.promos.yahoo.com/groups > > > > > >_______________________________________________ > >Ext3-users mailing list > >Ext3-users redhat com > >https://www.redhat.com/mailman/listinfo/ext3-users > > > > > > = = = = = = = = = = = = = = = = = = = = > > > ????????? > ?? > > > ????????liuyue > ????????liuyue ncic ac cn > ??????????2008-03-20 -------------- next part -------------- An HTML attachment was scrubbed... URL: From adilger at sun.com Fri Jun 26 04:49:50 2009 From: adilger at sun.com (Andreas Dilger) Date: Fri, 26 Jun 2009 06:49:50 +0200 Subject: How to get device name with device id? In-Reply-To: <8770d98c0906251858j1312e881k33ada185f88bb5e4@mail.gmail.com> References: <8770d98c0906251858j1312e881k33ada185f88bb5e4@mail.gmail.com> Message-ID: <20090626044950.GR3385@webber.adilger.int> On Jun 25, 2009 18:58 -0700, Smit Shah wrote: > I just ran through this post, and since no solution was posted, here's > a piece of code that i had written > Infact when i wrote this i wanted both the device name and also the > file system on that partition. What is wrong with just using the "blkid" program? It doesn't need root permission (if it is using the cache): [adilger ~]$ blkid /dev/sda1: LABEL="boot" UUID="1fe1d719-1a8c-45a0-969a-cba8b101cc57" TYPE="ext3" SEC_TYPE="ext2" /dev/sda2: LABEL="FC8" UUID="3bf693a3-a288-45ea-9f8b-1d140254b44c" TYPE="ext3" SEC_TYPE="ext2" /dev/sda3: TYPE="swap" LABEL="swap" UUID="d37ac19b-c8e1-42d5-8802-f1c98a6d7062" /dev/sda5: UUID="zm1xpW-mGQT-wlXO-dgMR-Gtx0-F5iI-9SSlxK" TYPE="lvm2pv" /dev/dm-0: LABEL="home" UUID="1bcd7718-3d7b-41ba-ab1a-f38e0fe40cf9" TYPE="ext3" SEC_TYPE="ext2" and you can get the filesystem type of all or a specific device easily: [adilger ~]$ blkid -s TYPE /dev/sda1: TYPE="ext3" /dev/sda2: TYPE="ext3" /dev/sda3: TYPE="swap" /dev/sda5: TYPE="lvm2pv" /dev/dm-0: TYPE="ext3" [adilger ~]$ blkid -s TYPE /dev/sda2 /dev/sda2: TYPE="ext3" or other information such as the filesystem label: [adilger ~]$ blkid -s LABEL /dev/sda2 /dev/sda1: LABEL="boot" Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. From darkonc at gmail.com Fri Jun 26 05:36:17 2009 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Thu, 25 Jun 2009 22:36:17 -0700 Subject: [Q] What might cause modification dates to shift later by an hour? In-Reply-To: References: Message-ID: <6cd50f9f0906252236i58cae1daj85600a276308ab4a@mail.gmail.com> Time zones are the culprit -- Specifically DST shifts. If you check the absolute time stamp (which is GMT), it hasn't changed... but the distance between you and GMT changed by one hour when the DST change kicked in. Nothing has gone wrong, unless your backup software works internally with TZ shifted date stamps, in which case it might get confused twice per yer. On Thu, Jun 25, 2009 at 11:29 AM, Maurice Volaski wrote: > Recently, our backup software oddly decided to rebackup a good portion of > our file server instead of just doing an incremental. When I examined > various sets of presumably identical files, I discovered that the > modification dates on these files were no longer the same. Many files were > re-dated to exactly one hour later such that if a file had been modified on > 3/24/04 at 2:24:53 PM, it's modification date had somehow been changed to > 3/24/04 at 3:24:53 PM. > > A point to note that is that this server is actually two separate servers > with independent storage mirrored by drbd. And it's possible that the first > backup was done on one server and the second on the other. However, drbd > mirrors the storage bit by bit, so it's hard to see how files on each of > them could have different modification dates. Also, even if the time on the > computers were different by an hour (they're current the same to the > second), it's not clear how the modification dates could somehow get > re-dated. > > It's also not clear what mechanism could have redated them. I've almost > certainly run fsck on the affected filesystems. Would there ever be a > circumstance when fsck might second guess a file's modification time and > change it? e2fsck is currently 1.41.4, though I don't know if that's the > version that I ran when I ran it. > > However, there seems to be a field in the backup software called status > modification date and a lot of these affected files had their status > modified around the same time, implying something did iterate over the files > and do something to them. > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From getsmit at gmail.com Fri Jun 26 06:25:49 2009 From: getsmit at gmail.com (Smit Shah) Date: Thu, 25 Jun 2009 23:25:49 -0700 Subject: How to get device name with device id? In-Reply-To: <20090626044950.GR3385@webber.adilger.int> References: <8770d98c0906251858j1312e881k33ada185f88bb5e4@mail.gmail.com> <20090626044950.GR3385@webber.adilger.int> Message-ID: <8770d98c0906252325pdb29c9en45591222a26f39c1@mail.gmail.com> On Thu, Jun 25, 2009 at 9:49 PM, Andreas Dilger wrote: > On Jun 25, 2009 18:58 -0700, Smit Shah wrote: > > I just ran through this post, and since no solution was posted, here's > > a piece of code that i had written > > Infact when i wrote this i wanted both the device name and also the > > file system on that partition. > > What is wrong with just using the "blkid" program? I guess blkid uses the same interfaces but the command doesn't provide option to specify the device number. But when i wrote this what i wanted to do is to get the file system of the device on which the file resides. > It doesn't need > root permission (if it is using the cache): > Yes. > > [adilger ~]$ blkid > /dev/sda1: LABEL="boot" UUID="1fe1d719-1a8c-45a0-969a-cba8b101cc57" > TYPE="ext3" SEC_TYPE="ext2" > /dev/sda2: LABEL="FC8" UUID="3bf693a3-a288-45ea-9f8b-1d140254b44c" > TYPE="ext3" SEC_TYPE="ext2" > /dev/sda3: TYPE="swap" LABEL="swap" > UUID="d37ac19b-c8e1-42d5-8802-f1c98a6d7062" > /dev/sda5: UUID="zm1xpW-mGQT-wlXO-dgMR-Gtx0-F5iI-9SSlxK" TYPE="lvm2pv" > /dev/dm-0: LABEL="home" UUID="1bcd7718-3d7b-41ba-ab1a-f38e0fe40cf9" > TYPE="ext3" SEC_TYPE="ext2" > > and you can get the filesystem type of all or a specific device easily: > > [adilger ~]$ blkid -s TYPE > /dev/sda1: TYPE="ext3" > /dev/sda2: TYPE="ext3" > /dev/sda3: TYPE="swap" > /dev/sda5: TYPE="lvm2pv" > /dev/dm-0: TYPE="ext3" > > [adilger ~]$ blkid -s TYPE /dev/sda2 > /dev/sda2: TYPE="ext3" > > or other information such as the filesystem label: > > [adilger ~]$ blkid -s LABEL /dev/sda2 > /dev/sda1: LABEL="boot" > > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darkonc at gmail.com Fri Jun 26 23:21:12 2009 From: darkonc at gmail.com (Stephen Samuel (gmail)) Date: Fri, 26 Jun 2009 16:21:12 -0700 Subject: [Q] What might cause modification dates to shift later by an hour? In-Reply-To: References: <6cd50f9f0906252236i58cae1daj85600a276308ab4a@mail.gmail.com> Message-ID: <6cd50f9f0906261621j74be340exf5cfae8d840dd69d@mail.gmail.com> One easy workaround would be to, when you start the backup program, to set it to a timezone that does not have daylight savings time shifts. (eg: " TZ=MST run_my_backup" ) This can be done on a per-process (or per-user) basis by setting the TZ environment variable before starting the process (or in a user's .*profile file) On Fri, Jun 26, 2009 at 3:00 PM, Maurice Volaski wrote: > Time zones are the culprit -- Specifically DST shifts. >> If you check the absolute time stamp (which is GMT), it hasn't changed... >> but the distance between you and GMT changed by one hour when the DST >> change kicked in. >> Nothing has gone wrong, unless your backup software works internally with >> TZ shifted date stamps, in which >> case it might get confused twice per yer. >> >> Thanks for you answer. It looks like a bug in the backup software. > -- Stephen Samuel http://www.bcgreen.com Software, like love, 778-861-7641 grows when you give it away -------------- next part -------------- An HTML attachment was scrubbed... URL: From jelledejong at powercraft.nl Sat Jun 27 08:50:54 2009 From: jelledejong at powercraft.nl (Jelle de Jong) Date: Sat, 27 Jun 2009 10:50:54 +0200 Subject: how to calculate correct values for resize2fs In-Reply-To: <20090625232015.398e8603@30_bodo> References: <4A409C96.7040703@powercraft.nl> <4A43CFDD.1020200@powercraft.nl> <20090625232015.398e8603@30_bodo> Message-ID: <4A45DD6E.3060000@powercraft.nl> Bodo Thiesen wrote: > * Jelle de Jong hat geschrieben: > >> Christian Kujau wrote: >>> After creating a new partition, you can mkfs this new one. I did not >>> understand at which point you want to "resize" a partition. Resizing a >>> filesystem is usually done on existing, resized partitions. > > He wants to SHRINK the partition, so he will have to resize the > file system, then shrink the partition. > >> resize2fs /dev/sda1 ???????? > [...] >> Hope you can show me how to calculate the value to get a 16 MB free >> space on /dev/sda with the resize2fs tool. > > Just resize the file system on sda1 to 3GB (or another value which leaves > enough space - just overcalculate here). Then make the partition smaller > to the final size you want it to be, then re-erlarge the file system to > it's final size. > > And another advice: Do NOT create the new partition(s), before you > completed the TWO resize-steps. > > Regards, Bodo Hi Bodo thanks for the feedback, The problem is I have to automate this process for a lot of systems. I just want a good scientific explanation on how to calculated the needed sizes for resize2fs if you look at my first post I attached a lot of data that should provide enough info for a good exact calculation. I just don't know how this calculation should be made. Just shrinking the partition to 3GB is not an option because there is not so much free space. There is around 200MB free space and I need a 16 MB new fresh partition. I can do this with gparted with 3 mouse clicks, but i need a way to do it with commandline tools and I want to know how to do the calculations. Thanks in advance, Jelle From bothie at gmx.de Sat Jun 27 10:46:47 2009 From: bothie at gmx.de (Bodo Thiesen) Date: Sat, 27 Jun 2009 12:46:47 +0200 Subject: how to calculate correct values for resize2fs In-Reply-To: <4A409C96.7040703@powercraft.nl> References: <4A409C96.7040703@powercraft.nl> Message-ID: <20090627124647.1099de35@30_bodo> Hello Jelle 1. Please don't CC to poster if you answer, we are all on this list and get the messages anyways. 2. And please don't full-quote if not *really* necessary. 3. From your other mail: > The problem is I have to automate this process That's an important information you forgot to mention in the initial post ;) * Jelle de Jong hat geschrieben: > $ fdisk -l /dev/sda > Disk /dev/sda: 4034 MB, 4034838528 bytes > bc <<< "4034-16" > resize2fs /dev/sda1 4018M > > # result: partition is only 983973 (4K) blocks > # requested: 1028608 blocks > # will not fit exiting 4034838528 is 3847 MB and not 4034 like fdisk told you. Obviously fdisk is using that bogus SI-numbering. 4018*1024^2 is 1028608 clusters and the underlaying block device of the file system only supports 983973 clusters (which it already provides). So: Take the bytes value, devide it by 1024^2 to gain the real number of MBs and subtract 18 (not 16) MB in the first call to resize2fs like this: resize2fs -p /dev/sda1 $(echo " scale=0; $( export LANG=C fdisk -l /dev/sda | grep sda: | awk '{ print $5 }' )/1024^2-18 " | bc) Then shrink the partition by 16 MB then resize2fs /dev/sda1 without argument again to fill up the unused space again. Remember my advice from the previous mail I'v written as well. Regards, Bodo From jelledejong at powercraft.nl Sat Jun 27 13:18:07 2009 From: jelledejong at powercraft.nl (Jelle de Jong) Date: Sat, 27 Jun 2009 15:18:07 +0200 Subject: how to calculate correct values for resize2fs In-Reply-To: <20090627124647.1099de35@30_bodo> References: <4A409C96.7040703@powercraft.nl> <20090627124647.1099de35@30_bodo> Message-ID: <4A461C0F.40104@powercraft.nl> Bodo Thiesen wrote: > 1. Please don't CC to poster if you answer, we are all on this list and > get the messages anyways. Sorry I just hit the reply to all button on my MUA > 2. And please don't full-quote if not *really* necessary. Ok > 3. From your other mail: > > The problem is I have to automate this process > That's an important information you forgot to mention in the initial > post ;) Hmm I tough the question was clear engough sorry for that Ok I seems to have missed a part and that is that after the resizing of the file system I have to resize the partition to make the free space. Then I have to make the file system fit the partition again, and after that I can make a new partition in the free space. So I set up an test enviorment and started calculating and testing, I think I now understand how to calculate values for resize2fs. However I am now stuck on resizing the partition after the file system has been resized. I hope I am now not two off-topic since the question is now not resize2fs related. Some checks on the calculation would be nice too. Please see: dd count=102400 bs=1024 if=/dev/zero of=temp /sbin/parted --script temp \ mklabel msdos \ mkpart primary 0 50% \ mkpart primary 50% 100% \ toggle 1 boot \ print sudo losetup /dev/loop0 temp fdisk -ul /dev/loop0 Disk /dev/loop0: 104 MB, 104857600 bytes 4 heads, 32 sectors/track, 1600 cylinders, total 204800 sectors Units = sectors of 1 * 512 = 512 bytes Disk identifier: 0x0000cef3 Device Boot Start End Blocks Id System /dev/loop0p1 * 32 102399 51184 83 Linux /dev/loop0p2 102400 204799 51200 83 Linux echo "32*512" | bc 16384 losetup -o 16384 /dev/loop1 /dev/loop0 echo "32*102400" | bc 3276800 losetup -o 3276800 /dev/loop2 /dev/loop0 sudo losetup -a fdisk -l /dev/loop0 mkfs.ext2 /dev/loop1 mkfs.ext4 /dev/loop2 mkdir disk1 mkdir disk2 sudo mount /dev/loop1 disk1/ sudo mount /dev/loop2 disk2/ mount | grep disk /dev/loop1 on /home/jelle/disk1 type ext2 (rw) /dev/loop2 on /home/jelle/disk2 type ext4 (rw) sudo touch disk1/test1 ls -hal disk1/ total 21K drwxr-xr-x 3 jelle jelle 1.0K 2009-06-27 14:15 . drwx--x--x 96 jelle jelle 8.0K 2009-06-27 14:14 .. drwx------ 2 root root 12K 2009-06-27 14:13 lost+found -rw-r--r-- 1 root root 0 2009-06-27 14:15 test1 sudo touch disk2/test2 ls -hal disk2/ total 21K drwxr-xr-x 3 jelle jelle 1.0K 2009-06-27 14:15 . drwx--x--x 96 jelle jelle 8.0K 2009-06-27 14:14 .. drwx------ 2 root root 12K 2009-06-27 14:13 lost+found -rw-r--r-- 1 root root 0 2009-06-27 14:15 test2 sudo umount /dev/loop1 sudo umount /dev/loop2 losetup -d /dev/loop1 losetup -d /dev/loop2 losetup -a /sbin/parted --script temp \ mklabel msdos \ mkpart primary 0 100% \ toggle 1 boot \ print fdisk -ul /dev/loop0 echo "32*512" | bc 16384 losetup -o 16384 /dev/loop1 /dev/loop0 mkfs.ext2 /dev/loop1 fdisk -l /dev/loop0 | head -n 2 | tail -n 1 | awk '{ print $5 }' 104857600 # bytes dumpe2fs -h /dev/loop1 | grep "Block size:" | awk '{ print $3 }' 1024 # blocksize echo "104857600/1024^2" | bc 100 # MiB echo "104857600/1024" | bc 102400 # filesystem blocksize echo "16*1024^2" | bc 16777216 # bytes of 16 MiB echo "16777216/1024" | bc 16384 # filesystem blocksize echo "102400-16384" | bc 86016 # to get a 16 MiB free space echo "2*1024^2/1024" | bc 2048 # 2 MiB extra for shrinking echo "102400-16384-2048" | bc 83968 # to get a 18 MiB free space # check the filesystem e2fsck -f /dev/loop1 # shrink the filesystem resize2fs /dev/loop1 83968 # check new block count dumpe2fs -h /dev/loop1 | grep "Block count:" | awk '{ print $3 }' 83968 # block count # calculate the resize of the partition fdisk -l /dev/loop0 | head -n 2 | tail -n 1 | awk '{ print $5 }' 104857600 # bytes echo "16*1024^2" | bc 16777216 # bytes of 16 MiB echo "104857600-16777216" | bc 88080384 # bytes /sbin/parted --script /dev/loop0 \ resize 1 0 88080384B \ print Error: File system has an incompatible feature enabled. Compatible features are has_journal, dir_index, filetype, sparse_super and large_file. Use tune2fs or debugfs to remove features. # help! dumpe2fs -h /dev/loop1 | grep "Filesystem features:" Filesystem features: ext_attr resize_inode dir_index filetype sparse_super tune2fs -O ^ext_attr /dev/loop1 Clearing filesystem feature 'ext_attr' not supported. # partition table of /dev/loop0 sfdisk -d /dev/loop0 /sbin/parted --script /dev/loop0 print Model: Unknown (unknown) Disk /dev/loop0: 105MB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 16.4kB 105MB 105MB primary ext2 boot # grow the filesystem to fit the partition resize2fs /dev/loop1 # then I need to create a new 16 MiB partition... Best regards, Jelle From bruno at wolff.to Sat Jun 27 14:13:36 2009 From: bruno at wolff.to (Bruno Wolff III) Date: Sat, 27 Jun 2009 09:13:36 -0500 Subject: how to calculate correct values for resize2fs In-Reply-To: <20090627124647.1099de35@30_bodo> References: <4A409C96.7040703@powercraft.nl> <20090627124647.1099de35@30_bodo> Message-ID: <20090627141336.GC25277@wolff.to> On Sat, Jun 27, 2009 at 12:46:47 +0200, Bodo Thiesen wrote: > Hello Jelle > > 1. Please don't CC to poster if you answer, we are all on this list and > get the messages anyways. That isn't necessary true. Posts from nonsubscribers are moderated, but can make it through. Posters who don't want to be replied to directly can set mail-followup-to to indicate that desire. From bothie at gmx.de Sat Jun 27 18:54:27 2009 From: bothie at gmx.de (Bodo Thiesen) Date: Sat, 27 Jun 2009 20:54:27 +0200 Subject: how to calculate correct values for resize2fs In-Reply-To: <4A461C0F.40104@powercraft.nl> References: <4A409C96.7040703@powercraft.nl> <20090627124647.1099de35@30_bodo> <4A461C0F.40104@powercraft.nl> Message-ID: <20090627205427.6779869f@30_bodo> * Jelle de Jong hat geschrieben: > Ok I seems to have missed a part and that is that after the resizing of > the file system I have to resize the partition to make the free space. > Then I have to make the file system fit the partition again, and after > that I can make a new partition in the free space. It's not 100%ly needed to do it this way, but doing it by exactly calculating the needed size is just error prone, so this is the way, *I* would do it (because it doesn't do any harm this way but helps preventing errors). > Disk /dev/loop0: 104 MB, 104857600 bytes > 4 heads, 32 sectors/track, 1600 cylinders, total 204800 sectors > Units = sectors of 1 * 512 = 512 bytes > Disk identifier: 0x0000cef3 > > Device Boot Start End Blocks Id System > /dev/loop0p1 * 32 102399 51184 83 Linux > /dev/loop0p2 102400 204799 51200 83 Linux > > echo "32*512" | bc > 16384 > > losetup -o 16384 /dev/loop1 /dev/loop0 > > echo "32*102400" | bc > 3276800 104857600/512=204800, so the disk image is 204800 blocks. The start values are in blocks. So, 32*512 is ok, but the other value has to be 102400*512 then ... > losetup -o 3276800 /dev/loop2 /dev/loop0 ... which leads to losetup -o 52428800 /dev/loop2 /dev/loop0 > sudo losetup -a > fdisk -l /dev/loop0 > mkfs.ext2 /dev/loop1 > mkfs.ext4 /dev/loop2 Because of the wrong second losetup, the latter mkfs damages the first file system obviously (which doesn't neccessarily cause troubles like in this test case - but it might have). > /sbin/parted --script /dev/loop0 \ > resize 1 0 88080384B \ > print > Error: File system has an incompatible feature enabled. Compatible > features are has_journal, dir_index, filetype, sparse_super and > large_file. Use tune2fs or debugfs to remove features. You should be able to 'rm' the partition and then 'mkpart' it with the new size. (I don't know parted, maybe you will have to use fdisk for that, if parted doesn't want you to create a partition without destroying the data in it, just test it.) Regards, Bodo From samix_119 at yahoo.com Tue Jun 30 13:45:56 2009 From: samix_119 at yahoo.com (Muhammed Sameer) Date: Tue, 30 Jun 2009 06:45:56 -0700 (PDT) Subject: EXT3-fs error - Freeing blocks not in datazone Message-ID: <867510.30862.qm@web112604.mail.gq1.yahoo.com> Hello, * For further analysis the server that we use is a quad core 64bit server with 2 MB of RAM we faced a file system error as mentioned below on a RAID 1. We have a hardware raid and the raid controller is ADAPTEC * Our kernel is 2.6.18-53.1.14.el5 * The error that our machine faced is as below * One of our servers experienced a file system corruption and gave this error in the /var/log/messages file. > > {code} > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1): > ext3_free_blocks: Freeing blocks not in datazone - blo > ck = 2684399216, count = 1 > Jun 17 05:53:47 myhost kernel: Aborting journal on device > sdb1. > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) > in ext3_free_blocks_sb: Journal has aborted > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) > in ext3_free_blocks_sb: Journal has aborted > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) > in ext3_reserve_inode_write: Journal has aborted > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) > in ext3_truncate: Journal has aborted > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) > in ext3_reserve_inode_write: Journal has aborted > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) > in ext3_orphan_del: Journal has aborted > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1) > in ext3_reserve_inode_write: Journal has aborted > Jun 17 05:53:47 myhost kernel: ext3_abort called. > Jun 17 05:53:47 myhost kernel: EXT3-fs error (device sdb1): > ext3_journal_start_sb: Detected aborted journal > Jun 17 05:53:47 myhost kernel: Remounting filesystem > read-only > Jun 17 05:53:47 myhost kernel: > __journal_remove_journal_head: freeing b_committed_data > {code} > > * The file system then went into read only mode > > * We tried to unmount and then remount the file system,but > on trying to unmount the server simply hung and we had to > restart the machine > > * Fsck found errors on the file system and fixed it > > * This filesystem is on a raid partition (RAID 1) and the > physical and virtual disk caches are turned on. > > * Even the barrier option for this file system was on to > prevent any corruption of the data in case of a sudden > reset > > * Can some one please point to me what is that went wrong > exactly and how we can avoid this in the future > > Regards, > Muhammed Sameer > > > > > > ? ? ? > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users > From V.Ly at library.usyd.edu.au Sat Jun 27 01:14:14 2009 From: V.Ly at library.usyd.edu.au (Van Ly) Date: Sat, 27 Jun 2009 01:14:14 -0000 Subject: free blocks count vs df() stat() Message-ID: <148ED1460B12F447BA90843866E244BF09F0C6@EXPRSV04.mcs.usyd.edu.au> Hi, Before going ahead with a fsck() and applying the patches on a RedHat 5.x system that has important data, I checked the numbers on the file system and depending on which way to look at it, the number for free/available space left could be 116Gb (see #1,3) or 24Gb (see #2). Does anyone know which to believe, 116Gb or 24Gb is free? I have diagnostic details appended below. Also, if the volume was created with logvol() (see #4) with the maxsize parameter used, does that affect later growth of the file system beyond maxsize (see #3)? My superficial understanding of inodes and blocks does not extend to lvm() and how that affects creation-time parameters shown by tune2fs. Thanks in advance, Van Ly -- # 1 # this comes close to 116Gb free <- `Available: 30246714` # stat -f /common File: "/common" ID: 0 Namelen: 255 Type: ext2/ext3 Block size: 4096 Fundamental block size: 4096 Blocks: Total: 60944400 Free: 33391866 Available: 30246714 Inodes: Total: 62914560 Free: 62037005 # 2 # free blocks suggests there is 24Gb free tune2fs 1.39 (29-May-2006) Filesystem volume name: Last mounted on: Filesystem UUID: b78a127a-80d9-43e2-8f9c-143a82137eb8 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_re\ covery sparse_super large_file Default mount options: user_xattr acl Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 62914560 Block count: 62914560 Reserved block count: 3145152 Free blocks: 6417762 Free inodes: 15443849 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1009 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 32768 Inode blocks per group: 1024 Filesystem created: Thu Dec 18 12:06:47 2008 Last mount time: Sun Feb 1 07:25:25 2009 Last write time: Sun Feb 1 07:25:25 2009 Mount count: 3 Maximum mount count: -1 Last checked: Thu Dec 18 12:06:47 2008 Check interval: 0 () Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 First orphan inode: 30212898 Default directory hash: tea Directory Hash Seed: 15dea504-b626-476b-8513-fdc11b43abaf Journal backup: inode blocks # 3 # volume has since been grown to 233Gb # df -h /common Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg0-commonlv 233G 105G 116G 48% /common # 4 # redhat kickstart created volume logvol /common --fstype ext3 --name=commonlv --vgname=vg0 --size=512 --grow --maxsize=61440 -------------- next part -------------- An HTML attachment was scrubbed... URL: From V.Ly at library.usyd.edu.au Mon Jun 29 01:06:07 2009 From: V.Ly at library.usyd.edu.au (Van Ly) Date: Mon, 29 Jun 2009 01:06:07 -0000 Subject: free blocks count vs df() stat() References: <148ED1460B12F447BA90843866E244BF09F0C6@EXPRSV04.mcs.usyd.edu.au> Message-ID: <148ED1460B12F447BA90843866E244BF09F0D0@EXPRSV04.mcs.usyd.edu.au> Hi everyone, In follow-up, I found an explanation of the superblock and free blocks at http://www.angelfire.com/myband/binusoman/Unix.html#blocks which suggests free blocks represents a segment of available storage and the last value of the segment points to another segment of free blocks, so it is reasonable for it to be smaller than the block values shown by df() and stat(). But the interaction of lvm, ext3 and maxsize set at creation time and the consequence of going beyond that is still uncertain. -- Van Ly -----Original Message----- From: Van Ly Sent: Sat 6/27/2009 11:13 AM To: ext3-users at redhat.com Subject: free blocks count vs df() stat() Hi, Before going ahead with a fsck() and applying the patches on a RedHat 5.x system that has important data, I checked the numbers on the file system and depending on which way to look at it, the number for free/available space left could be 116Gb (see #1,3) or 24Gb (see #2). Does anyone know which to believe, 116Gb or 24Gb is free? I have diagnostic details appended below. Also, if the volume was created with logvol() (see #4) with the maxsize parameter used, does that affect later growth of the file system beyond maxsize (see #3)? My superficial understanding of inodes and blocks does not extend to lvm() and how that affects creation-time parameters shown by tune2fs. Thanks in advance, Van Ly -- # 1 # this comes close to 116Gb free <- `Available: 30246714` # stat -f /common File: "/common" ID: 0 Namelen: 255 Type: ext2/ext3 Block size: 4096 Fundamental block size: 4096 Blocks: Total: 60944400 Free: 33391866 Available: 30246714 Inodes: Total: 62914560 Free: 62037005 # 2 # free blocks suggests there is 24Gb free tune2fs 1.39 (29-May-2006) Filesystem volume name: Last mounted on: Filesystem UUID: b78a127a-80d9-43e2-8f9c-143a82137eb8 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_re\ covery sparse_super large_file Default mount options: user_xattr acl Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 62914560 Block count: 62914560 Reserved block count: 3145152 Free blocks: 6417762 Free inodes: 15443849 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1009 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 32768 Inode blocks per group: 1024 Filesystem created: Thu Dec 18 12:06:47 2008 Last mount time: Sun Feb 1 07:25:25 2009 Last write time: Sun Feb 1 07:25:25 2009 Mount count: 3 Maximum mount count: -1 Last checked: Thu Dec 18 12:06:47 2008 Check interval: 0 () Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 First orphan inode: 30212898 Default directory hash: tea Directory Hash Seed: 15dea504-b626-476b-8513-fdc11b43abaf Journal backup: inode blocks # 3 # volume has since been grown to 233Gb # df -h /common Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg0-commonlv 233G 105G 116G 48% /common # 4 # redhat kickstart created volume logvol /common --fstype ext3 --name=commonlv --vgname=vg0 --size=512 --grow --maxsize=61440 -------------- next part -------------- An HTML attachment was scrubbed... URL: