[Fedora-livecd-list] Re: Review? LiveUSB persistence

[Jeremy Katz]

On Mon, 2008-02-04 at 17:34 -0600, Douglas McClendon wrote:
> Jeremy Katz wrote:
> > Moving to the more appropriate list, original is at
> > https://www.redhat.com/archives/fedora-devel-list/2008-January/msg03188.html for those not on fedora-devel
> > 
> > On Thu, 2008-01-31 at 16:11 -0600, Douglas McClendon wrote:
> >> I keep hearing these crazy rumors that RedHat actually wants to subject 
> >> actual users to my LiveUSB persistence feature next/this month.
> > 
> > Lots of people want lots of different things.  I've definitely left this
> > sitting hanging too long and I apologize.  I just didn't have time to
> > poke over the holidays and my January was a bitty nutty.  But I should
> > be back to just my normal mail lag now ;-)
> > 
> >> I for one think the feature is really cool, useful, and know of no 
> >> showstopping or even known bugs.  But I also would not subject actual 
> >> users to it, without having had more people review it and sign off on 
> >> it.  Anyway, to absolve myself of any responsibility, I once again 
> >> request code review and feedback-
> > 
> > Overall, I think that it looks good.  I think the big thing to work out
> > is how best to integrate the initscripts changes.  Especially as we're
> > in parallel switching things over to upstart for F9.  Just to make sure
> > that I'm clear about what's there so that when we try to bribe Bill, we
> > can do so effectively...
> > * The unmount loops in /etc/init.d/functions are modified so that we
> > don't unmount the underlying fs of the overlay
> 
> Correct.  And the method currently in use is adding any subcomponents of 
> the rootfs to /dev/.fstab.live.special.  I chose that location just 
> because it is available from both the initramfs and then later during 
> shutdown.

*nod*

> > * We don't want to sync the hwclock on shutdown of a live image
> > * Do an overlay teardown on halt to replace the rw snapshot with a ro
> > version
> 
> Yup, basically the old phase of 'remount rootfs readonly' just gets 
> quite a bit more complicated, but still essentially just that.

Right.  I really am leaning towards thinking that the teardown of the rw
mount to be ro probably needs to be from initramfs land as that allows
it to not require changes anywhere else when adding a new rootfs scheme.
I'll have to see what we can cobble together to make that work (it has
the added bonus of we'd be able to eject the cd on reboot from a livecd)

> > The latter feels like we're going to want a generic hook to be able to
> > run, perhaps from back in the initramfs.  The middle feels largely
> > unrelated to persistence in general, but just a "needs fixing" which
> > will fit in nicely with other hwclock changes that are underway in
> > rawhide.  And the first I'm less clear on I guess.
> > 
> > Outside of those changes, everything looks pretty straight-forward and
> > reasonable.  We'll want a little more error checking in
> > livecd-iso-to-disk and it might also be nice to have another tool that
> > lets you create your usb stick to use with the actual cd form (I did
> > this by hand and it was pretty nice that I could easily get it
> > working!).  As for the findoverlay bits, for right now, leaving them as
> > a separate script is fine.  In the longer term, we're hopefully going to
> > kill mayflower and be able to use more "normal" initrds[1], at which
> > point pulling it in more directly would be nice.
> 
> I have no preference where the code lives.  The main thing I would 
> highlight is how originally I wrote findoverlay, with the mindset that 
> the overlay file/layer could be located anywhere.  But then, to focus on 
> getting something really usable, I honed in on the assumption of the 
> overlay file living together with the LiveOS on a LiveUSB.  As a result 
> the findoverlay script definitely does not look like entirely grade A 
> code right now.  But I have no objection to it getting merged in any 
> form, and perhaps later having cleanup/refactoring passes, perhaps with 
> even later passes that add back the functionality I originally had in mind.

*nod*  I think it's the best way so that we get things in sooner rather
than later.  You've had it basically working for quite a while and we
should just bite the bullet and say "good enough" rather than waiting
for "perfect"

> > As far as reliability... shutting down cleanly, I can't cause a problem.
> > I pulled the usb stick from a box and got some garbage in a file I had
> > written.  I want to do some more playing here, but I also wanted to get
> > this mail out today.  Worst case, if we say "you need to shut down
> > cleanly", then so be it.  It might be nice if we flag clean vs not
> > shutdown so that we can force a fsck in unclean cases.  But no clue if
> > that's even feasible; I'll take a look, though.
> 
> My vague understanding right now would that it would be treated just 
> like a normal rootfs.  I.e. the same things happening if it is flagged 
> as dirty, as if a normal system.  But...  I guess then there is the 
> issue of fscking the host filesystem, e.g. vfat(ext3?) of the LiveUSB.  Ugh.

Right.  It's ... less than pretty.  But could easily be a second pass
once we get things merged in.

> One immediate enhancement I thought of, is that perhaps the interface 
> for initialization can be taken out of livecd-iso-to-disk, and put into 
> the initramfs.  I.e. a new kernel cmdline parameter of "initoverlay=256MB"
> would just invoke the dd if=/dev/zero... during that boot.  Maybe even a 
> default value that just uses half the free space on the LiveUSB.

This would definitely make things simpler for Luke who's working on a
Win32 version of livecd-iso-to-disk ;-)  I'm not really sure where I
stand on this one, I could probably be moved in either direction

> Thus perhaps livecd-iso-to-disk just by default adds another couple menu 
> entries to syslinux, one for 'boot with persistence' and one for 
> 'initialize persistence'.  Or perhaps the former automatically 
> initializes if none is found, and the latter, or some post-boot system 
> command could 'reinitialize persistence'...

The problem with adding entries is that conceivably, you could have
multiple kernels included and editing the syslinux.cfg at that point
gets tricky.  But I can think of a few ways to handle that, including
having the persistence option in CD form and letting people choose (with
a timeout) a location for their persistent bits.  

I'm not going to get hung up on figuring that out today, though.  I kind
of like the fact that to easily use it, you use livecd-iso-to-disk and
it sets things up for you.  Since if you're going to have persistence,
then a USB stick is a pretty good idea.  And by using USB instead, you
end up getting better performance

Jeremy