<div class="gmail_quote">On Tue, Nov 29, 2011 at 9:09 PM, Douglas McClendon <span dir="ltr"><<a href="mailto:dmc.lists@filteredperception.org">dmc.lists@filteredperception.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">On 11/29/2011 12:22 AM, Frederick Grose wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
On Sun, Nov 27, 2011 at 8:08 PM, Douglas McClendon<br>
<<a href="mailto:dmc.lists@filteredperception.org" target="_blank">dmc.lists@filteredperception.<u></u>org</a><br></div><div class="im">
<mailto:<a href="mailto:dmc.lists@filteredperception.org" target="_blank">dmc.lists@<u></u>filteredperception.org</a>>> wrote:<br>
<br>
On 11/27/2011 04:31 PM, Frederick Grose wrote:<br>
<br>
On Sat, Oct 29, 2011 at 4:49 PM, Alasdair G Kergon<br>
<<a href="mailto:agk@redhat.com" target="_blank">agk@redhat.com</a> <mailto:<a href="mailto:agk@redhat.com" target="_blank">agk@redhat.com</a>><br></div><div class="im">
<mailto:<a href="mailto:agk@redhat.com" target="_blank">agk@redhat.com</a> <mailto:<a href="mailto:agk@redhat.com" target="_blank">agk@redhat.com</a>>>> wrote:<br>
<br>
markmc did some code that shows how to read the format a few<br>
years<br>
ago here:<br></div>
<a href="http://people.gnome.org/~__markmc/code/merge-dm-snapshot.__c" target="_blank">http://people.gnome.org/~__<u></u>markmc/code/merge-dm-snapshot.<u></u>__c</a><br>
<<a href="http://people.gnome.org/~markmc/code/merge-dm-snapshot.c" target="_blank">http://people.gnome.org/~<u></u>markmc/code/merge-dm-snapshot.<u></u>c</a>><br>
<<a href="http://people.gnome.org/%__7Emarkmc/code/merge-dm-__snapshot.c" target="_blank">http://people.gnome.org/%__<u></u>7Emarkmc/code/merge-dm-__<u></u>snapshot.c</a><div><div class="h5"><br>
<<a href="http://people.gnome.org/%7Emarkmc/code/merge-dm-snapshot.c" target="_blank">http://people.gnome.org/%<u></u>7Emarkmc/code/merge-dm-<u></u>snapshot.c</a>>><br>
<br>
<br>
Otherwise look at dm-snap-persistent.c in the kernel tree.<br>
<br>
Alasdair<br>
<br>
<br>
Users can easily exhaust a LiveUSB snapshot overlay by writing<br>
too many changes to the OS filesystem, such as by performing<br>
a yum update.<br>
<br>
Would such an 'exhausted' overlay be amenable to some sort of<br>
data recovery or forensics?<br>
<br>
If so, how so; if not, why not?<br>
<br>
<br>
Can you not mount the exhausted dm snapshot? If you mean data<br>
recovery or forensics beyond that, please elaborate.<br>
<br>
I know I've had issues 'read-only' mounting ext3(2?3?4?) filesystems<br>
before (on actually read-only block devices). Maybe some flags that<br>
I've forgotten get past that, but worst case you can always fake a<br>
writable device with a 2nd overlay/snapshot going to a writable<br>
device, i.e. if you wanted to let fsck repair things.<br>
<br>
-dmc<br>
<br>
<br>
A typical scenario goes like this:<br>
<br>
A LiveCD is loaded onto a USB device with a relatively small<br>
persistent overlay and no separate home.img filesystem. The user<br>
proceeds to use the system and save some information. They may<br>
execute a df command and see that there is lots of space<br>
available on the rootfs. Then they decide to execute a yum<br>
update or yum install when there is insufficient space available.<br>
(They do not know about dmsetup status.) Their session will now<br>
not shutdown normally. dmsetup status now shows<br>
<br>
live-rw: 0 8388608 snapshot Invalid.<br>
<br>
The device fails to complete a subsequent boot attempt. The<br>
startup log shows<br>
</div></div></blockquote>
<br>
You probably know this, but note there is a reset_overlay (or similar) boot flag which I think causes the overlay/snapshot to be reset, to get a booting system.</blockquote><div><br></div><div><font face="'courier new', monospace">Yes, the relevant code for Fedora is here,</font></div>
<div><a href="http://git.kernel.org/?p=boot/dracut/dracut.git;a=blob;f=modules.d/90dmsquash-live/dmsquash-live-root;hb=HEAD#l101"><font face="'courier new', monospace">http://git.kernel.org/?p=boot/dracut/dracut.git;a=blob;f=modules.d/90dmsquash-live/dmsquash-live-root;hb=HEAD#l101</font></a></div>
<div><font face="'courier new', monospace"><br></font></div><div><span style="white-space: pre; background-color: rgb(255, 255, 255); "><font face="'courier new', monospace">dd if=/dev/zero of=$OVERLAY_LOOPDEV bs=64k count=1 2>/dev/null</font></span></div>
<div><font face="'courier new', monospace"><br></font></div><div><font face="'courier new', monospace">although it seems to need an extra parameter, conv=notrunc</font></div><div><font face="'courier new', monospace">to prevent truncation of the overlay file.</font></div>
<div><font face="'courier new', monospace"><br></font></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"> So the rest of this is with the use-case in mind of trying to recover prior contents of the overlay/snapshot in a useful way.<div class="im">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
mount: /dev/mapper/live-rw: can't read superblock<br>
<br>
e2fsck /dev/dm-0 shows<br>
<br>
Buffer I/O error on device dm-0, logical block 0 (and 1, 2, 3, 0)<br>
e2fsck: Attempt to read block from the filesystem resulted in<br>
short read while trying to open /dev/dm-0<br>
Could this be a zero-length partition?<br>
<br>
I recreated the above scenario on Fedora-16-Live-Desktop<br>
installed with an overlay-size-mb of 100. After updating to ~70%<br>
of the overlay, I wrote with<br>
dd if=/dev/zero of=/boot/load bs=1M count=10<br>
3 times to exhaust the overlay.<br>
<br>
With the defective device mounted on a good system, and loop<br>
devices set up for ext3fs.img and the overlay file,<br>
<br>
dmsetup create item --table "0 8388608 snapshot 7:1 7:2 P 8"</blockquote></div></blockquote></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">dmsetup status reports<br>
item: 0 8388608 snapshot Invalid<br>
</blockquote>
<br></div>
I don't know what that last line of output specifically means as far as subsequent expectations should go, though presumably others on this list can elaborate, or worst grep for the message in the code.<br>
<br>
Maybe dmsetup is more cooperative with full overlays if they are created in read-only mode?(wild guess)<br></blockquote><div><br></div><div><font face="'courier new', monospace">I tested this with --readonly added to the</font></div>
<div><font face="'courier new', monospace">dmsetup create command, both with and without a</font></div><div><font face="'courier new', monospace">-r parameter on the overlay loop device set up command.</font></div>
<div><font face="'courier new', monospace">Both resulted in the Invalid snapshot report from</font></div><div><font face="'courier new', monospace">dmsetup status.</font></div><div><font face="'courier new', monospace"><br>
</font></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">I.e. there is certainly the expectation that you won't be able to write to the device (or at a minimum to any chunk not already in the overlay(?)). So as I mentioned, the way I'd go about it would be to recreate it (I'm taking for granted above cmdline is correct), but in read-only mode, then either just dup that to a new larger device, or add a second read-write snapshot layer on top (pointing to a looped tmpfile somewhere). Then try to mount and/or fsck that.<br>
</blockquote><div> </div><div><font face="'courier new', monospace">Attempting to mount the Invalid snapshot results in,</font></div><div><font face="'courier new', monospace">mount: /dev/mapper/item: can't read superblock</font></div>
<div><font face="'courier new', monospace"><br></font></div><div><font face="'courier new', monospace">A second snapshot set up over the first also fails to mount</font></div><div><font face="'courier new', monospace">or e2fsck with the same errors.</font></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">But that presumes that the 'snapshot Invalid' isn't evidence that something is preventing even read-only access to the full snapshot.</blockquote>
<div><br></div><div><font face="'courier new', monospace">snapshot Invalid does seem to prevent even read-only access.</font></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
For that I defer to the people more familiar with the relevent code.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
e2fsck /dev/dm-0 reports<br>
<br>
e2fsck 1.41.14 (22-Dec-2010)<br>
e2fsck: Attempt to read block from filesystem resulted in short<br>
read while trying to open /dev/dm-0<br>
Could this be a zero-length partition?<br>
<br>
A hex editor view of the overlay file shows lots of content with<br>
lots of zeros at the end of the file.<br>
<br>
My questions are these:<br>
<br>
Might there be a way to edit the overlay file to restore its<br>
utility for the information written before the damaging write?<br>
<br>
If so, how so; if not, why not?<br>
</blockquote>
<br></div>
Were it actually the case that full persistent snapshots become unusuable (short of 'editing'), that would seem like a bug needing to be fixed.<br>
<br>
-dmc<br>
<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
Suitable references would do as well.<br>
<br>
Thanks! --Fred</div></blockquote></blockquote></div>