From tim.wood at datawranglers.com Sun Jul 1 20:45:12 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Sun, 1 Jul 2007 14:45:12 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor is looking for busybox In-Reply-To: <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> Message-ID: <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> I've (finally) been trying to build a live cd. I've been trying to use Revisor. After I sorted out some issues so that Fedora played nice with my repository, Revisor runs and gets to the point where it's trying to build the cd. And it complains that busybox is a required package. My repository has busybox-anaconda. I look at the os and all portions of one of the official Fedora 7 repositories and I see the same thing: busybox-anaconda and no busybox. Does anyone know how to resolve this? From kanarip at kanarip.com Sun Jul 1 21:24:37 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Sun, 01 Jul 2007 23:24:37 +0200 Subject: [Fedora-livecd-list] Revisor is looking for busybox In-Reply-To: <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> Message-ID: <46881B95.7030907@kanarip.com> Tim Wood wrote: > I've (finally) been trying to build a live cd. I've been trying to use > Revisor. After I sorted out some issues so that Fedora played nice with > my repository, Revisor runs and gets to the point where it's trying to > build the cd. And it complains that busybox is a required package. My > repository has busybox-anaconda. I look at the os and all portions of one > of the official Fedora 7 repositories and I see the same thing: > busybox-anaconda and no busybox. Does anyone know how to resolve this? > Hi Tim, busybox is in the Everything repository. The default configs of the version of Revisor you run may be pointing to the Fedora repository, holding just the packages that came with the released DVD. Later versions of Revisor have this fixed. Kind regards, Jeroen van Meeuwen -kanarip From tim.wood at datawranglers.com Sun Jul 1 22:17:27 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Sun, 1 Jul 2007 16:17:27 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor is looking for busybox In-Reply-To: <46881B95.7030907@kanarip.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> Message-ID: <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> When I ran yum update revisor, no luck, but I was able to download the latest off of download.fedora.redhat.com/.../development/... and it seems to be happily working. Once I can generate one via the defaults I'm going to (scary) try one off of the repository I'm running here. Should I add some or all of the development rpms to my repository? > Tim Wood wrote: >> I've (finally) been trying to build a live cd. I've been trying to use >> Revisor. After I sorted out some issues so that Fedora played nice with >> my repository, Revisor runs and gets to the point where it's trying to >> build the cd. And it complains that busybox is a required package. My >> repository has busybox-anaconda. I look at the os and all portions of >> one >> of the official Fedora 7 repositories and I see the same thing: >> busybox-anaconda and no busybox. Does anyone know how to resolve this? >> > > Hi Tim, > > busybox is in the Everything repository. The default configs of the > version of Revisor you run may be pointing to the Fedora repository, > holding just the packages that came with the released DVD. > > Later versions of Revisor have this fixed. > > Kind regards, > > Jeroen van Meeuwen > -kanarip > > From zeter90 at yahoo.com Sun Jul 1 23:20:48 2007 From: zeter90 at yahoo.com (Varoon) Date: Sun, 1 Jul 2007 16:20:48 -0700 (PDT) Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? Message-ID: <883270.46965.qm@web36506.mail.mud.yahoo.com> How to compile c programs in fedora 7 KDE? I need to command to do this thanks --------------------------------- Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jonathansteffan at gmail.com Mon Jul 2 00:57:47 2007 From: jonathansteffan at gmail.com (Jonathan Steffan) Date: Sun, 01 Jul 2007 18:57:47 -0600 Subject: [Fedora-livecd-list] Revisor is looking for busybox In-Reply-To: <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> Message-ID: <1183337867.20961.4.camel@damaestro> On Sun, 2007-07-01 at 16:17 -0600, Tim Wood wrote: > When I ran yum update revisor, no luck, This is due to the update sitting in the signing que. It should be signed and pushed out to the mirrors sometime here soon. For future reference, you can download Revisor builds directly from Koji (if you don't want to wait for us to approve the builds for public consumption): http://koji.fedoraproject.org/koji/packageinfo?packageID=4446 > but I was able to download the > latest off of download.fedora.redhat.com/.../development/... and it seems > to be happily working. > > Once I can generate one via the defaults I'm going to (scary) try one off > of the repository I'm running here. Should I add some or all of the > development rpms to my repository? No. The issue is that we were shipping default configs that only pointed at the "Fedora" repo, which only has the bits that were shipped on the DVD. We currently list "busybox" as a required package. Make sure you have busybox in the repo you point Revisor to. > > > > Tim Wood wrote: > >> I've (finally) been trying to build a live cd. I've been trying to use > >> Revisor. ;-) > After I sorted out some issues so that Fedora played nice with > >> my repository, Revisor runs and gets to the point where it's trying to > >> build the cd. And it complains that busybox is a required package. My > >> repository has busybox-anaconda. I look at the os and all portions of > >> one > >> of the official Fedora 7 repositories and I see the same thing: > >> busybox-anaconda and no busybox. Does anyone know how to resolve this? > >> > > > > Hi Tim, > > > > busybox is in the Everything repository. The default configs of the > > version of Revisor you run may be pointing to the Fedora repository, > > holding just the packages that came with the released DVD. > > > > Later versions of Revisor have this fixed. What version are you using? (rpm -q revisor) 2.0.4.0 is the latest. Jonathan Steffan daMaestro From kanarip at kanarip.com Mon Jul 2 07:58:02 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Mon, 02 Jul 2007 09:58:02 +0200 Subject: [Fedora-livecd-list] Revisor is looking for busybox In-Reply-To: <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> Message-ID: <4688B00A.4070403@kanarip.com> Tim Wood wrote: > When I ran yum update revisor, no luck, but I was able to download the > latest off of download.fedora.redhat.com/.../development/... and it seems > to be happily working. > > Once I can generate one via the defaults I'm going to (scary) try one off > of the repository I'm running here. Should I add some or all of the > development rpms to my repository? > Revisor doesn't really depend on what is in the repositories other then with the lists of required packages it has for installation and live media. Given that the latest version of Revisor is released to rawhide you would need to do the occasional # yum --enablerepo=development update revisor to see if there is a more up-to-date version. Other then that, you don't need to enable the latest and greatest of software for Revisor to use composing your media. In fact, on my own development workstation I've been able to do FC5, FC6 and CentOS 5 respins. These obviously don't have the same software as F7 or development ;-) Kind regards, Jeroen van Meeuwen -kanarip From kanarip at kanarip.com Mon Jul 2 07:59:19 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Mon, 02 Jul 2007 09:59:19 +0200 Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <883270.46965.qm@web36506.mail.mud.yahoo.com> References: <883270.46965.qm@web36506.mail.mud.yahoo.com> Message-ID: <4688B057.1080908@kanarip.com> Varoon wrote: > > How to compile c programs in fedora 7 KDE? > I need to command to do this thanks > > Your question may be more appropriate and more accurately answered on the fedora-devel list. Kind regards, Jeroen van Meeuwen -kanarip From tim.wood at datawranglers.com Mon Jul 2 12:35:10 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Mon, 2 Jul 2007 06:35:10 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor is looking for busybox In-Reply-To: <1183337867.20961.4.camel@damaestro> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <1183337867.20961.4.camel@damaestro> Message-ID: <25367.70.59.215.121.1183379710.squirrel@hputz.datawranglers.com> > For future reference, you can download Revisor builds directly from Koji Thanks. I've added that to my notes > DVD. We currently list "busybox" as a required package. Make sure you > have busybox in the repo you point Revisor to. Thanks. Will do. > What version are you using? (rpm -q revisor) 2.0.4.0 is the latest. Hehehe conveniently rpm -qa | grep revisor was the last thing in the open terminal window. 2.0.4.0 it is. thanks again for the help - Tim From tim.wood at datawranglers.com Mon Jul 2 12:42:21 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Mon, 2 Jul 2007 06:42:21 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor is looking for busybox In-Reply-To: <4688B00A.4070403@kanarip.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> Message-ID: <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> > been able to do FC5, FC6 and CentOS 5 respins. These obviously don't > have the same software as F7 or development ;-) Right now I'm doing a respin for an actual project. I'm looking forward to doing one with CentOS 5 just to see what happens. It would be fun to eventually retire most of the LiveCDs I carry around because they have this tool or that tool that's useful once every 4 months... regards, Tim Wood From tim.wood at datawranglers.com Mon Jul 2 16:46:50 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Mon, 2 Jul 2007 10:46:50 -0600 (MDT) Subject: [Fedora-livecd-list] config file question/suggestion In-Reply-To: <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> Message-ID: <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> I added a section to one of the Revisor configs with the short name development (e.g. [development]). No matter what I did, it was ignored until I changed that to something else (e.g. [extras]). Is there a config file that governs what short names are sanctioned or is there a suggestion/bug reporting tool for Revisor? From kanarip at kanarip.com Mon Jul 2 17:42:30 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Mon, 02 Jul 2007 19:42:30 +0200 Subject: [Fedora-livecd-list] config file question/suggestion In-Reply-To: <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> Message-ID: <46893906.10801@kanarip.com> Tim Wood wrote: > I added a section to one of the Revisor configs with the short name > development (e.g. [development]). No matter what I did, it was ignored > until I changed that to something else (e.g. [extras]). Is there a config > file that governs what short names are sanctioned or is there a > suggestion/bug reporting tool for Revisor? > Hi Tim, to enable development (same with testing, debug and source) repositories in the Revisor configuration, you'll need to set repos_enabledevelopment=1 because we prevent these repositories from being displayed in the GUI on purpose. Check the rawhide models for examples. Bugs and suggestions you can just sent to our mailing list: http://lists.fedoraunity.org/mailman/listinfo/revisor-users Kind regards, Jeroen van Meeuwen -kanarip From tim.wood at datawranglers.com Mon Jul 2 21:21:37 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Mon, 2 Jul 2007 15:21:37 -0600 (MDT) Subject: [Fedora-livecd-list] config file question/suggestion In-Reply-To: <46893906.10801@kanarip.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> Message-ID: <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> Jeroen, thanks. From zeter90 at yahoo.com Mon Jul 2 23:16:12 2007 From: zeter90 at yahoo.com (Varoon) Date: Mon, 2 Jul 2007 16:16:12 -0700 (PDT) Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <4688B057.1080908@kanarip.com> Message-ID: <229122.95608.qm@web36511.mail.mud.yahoo.com> May be i did not phrase my question right.... I would like to know if the live of fedora 7 has a c++ compiler.If so how can i use it to compile an already written code? Jeroen van Meeuwen wrote: Varoon wrote: > > How to compile c programs in fedora 7 KDE? > I need to command to do this thanks > > Your question may be more appropriate and more accurately answered on the fedora-devel list. Kind regards, Jeroen van Meeuwen -kanarip -- Fedora-livecd-list mailing list Fedora-livecd-list at redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list --------------------------------- 8:00? 8:25? 8:40? Find a flick in no time with theYahoo! Search movie showtime shortcut. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sundaram at fedoraproject.org Tue Jul 3 09:33:10 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Tue, 03 Jul 2007 15:03:10 +0530 Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <229122.95608.qm@web36511.mail.mud.yahoo.com> References: <229122.95608.qm@web36511.mail.mud.yahoo.com> Message-ID: <468A17D6.3040802@fedoraproject.org> Varoon wrote: > May be i did not phrase my question right.... > I would like to know if the live of fedora 7 has a c++ compiler.If so > how can i use it to compile an already written code? > No, the live cd by default does not have any compiler. If you have enough ram you can yum install gcc and related packages. It might be slow to do compilations on a live cd. Why do you want to compile packages in a live cd? Rahul From kanarip at kanarip.com Tue Jul 3 10:18:27 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Tue, 03 Jul 2007 12:18:27 +0200 Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <468A17D6.3040802@fedoraproject.org> References: <229122.95608.qm@web36511.mail.mud.yahoo.com> <468A17D6.3040802@fedoraproject.org> Message-ID: <468A2273.2060309@kanarip.com> Rahul Sundaram wrote: > Varoon wrote: >> May be i did not phrase my question right.... >> I would like to know if the live of fedora 7 has a c++ compiler.If so >> how can i use it to compile an already written code? >> > > No, the live cd by default does not have any compiler. If you have > enough ram you can yum install gcc and related packages. It might be > slow to do compilations on a live cd. Why do you want to compile > packages in a live cd? > > Rahul > It doesn't really matter why he wants it, he can create a LiveCD or DVD himself using the livecd-tools kickstart files and the package customization stage in Revisor: yum install revisor Kind regards, Jeroen van Meeuwen -kanarip From sundaram at fedoraproject.org Tue Jul 3 12:49:05 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Tue, 03 Jul 2007 18:19:05 +0530 Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <468A2273.2060309@kanarip.com> References: <229122.95608.qm@web36511.mail.mud.yahoo.com> <468A17D6.3040802@fedoraproject.org> <468A2273.2060309@kanarip.com> Message-ID: <468A45C1.8060209@fedoraproject.org> Jeroen van Meeuwen wrote: > Rahul Sundaram wrote: >> Varoon wrote: >>> May be i did not phrase my question right.... >>> I would like to know if the live of fedora 7 has a c++ compiler.If so >>> how can i use it to compile an already written code? >>> >> No, the live cd by default does not have any compiler. If you have >> enough ram you can yum install gcc and related packages. It might be >> slow to do compilations on a live cd. Why do you want to compile >> packages in a live cd? >> >> Rahul >> > > It doesn't really matter why he wants it, he can create a LiveCD or DVD > himself using the livecd-tools kickstart files and the package > customization stage in Revisor: Sure he can but it is does matter why some things are being done and it is useful to understand what users are doing to see if there is anything that can be made easier. Rahul From zeter90 at yahoo.com Tue Jul 3 15:28:42 2007 From: zeter90 at yahoo.com (Varoon) Date: Tue, 3 Jul 2007 08:28:42 -0700 (PDT) Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <468A45C1.8060209@fedoraproject.org> Message-ID: <828629.31481.qm@web36501.mail.mud.yahoo.com> I have already installed the Fedora 7 on my HD off the livecd .. However i would like to be able to install a compiler is that possible? if so what is the name of the compiler and download link. Thanks Rahul Sundaram wrote: Jeroen van Meeuwen wrote: > Rahul Sundaram wrote: >> Varoon wrote: >>> May be i did not phrase my question right.... >>> I would like to know if the live of fedora 7 has a c++ compiler.If so >>> how can i use it to compile an already written code? >>> >> No, the live cd by default does not have any compiler. If you have >> enough ram you can yum install gcc and related packages. It might be >> slow to do compilations on a live cd. Why do you want to compile >> packages in a live cd? >> >> Rahul >> > > It doesn't really matter why he wants it, he can create a LiveCD or DVD > himself using the livecd-tools kickstart files and the package > customization stage in Revisor: Sure he can but it is does matter why some things are being done and it is useful to understand what users are doing to see if there is anything that can be made easier. Rahul -- Fedora-livecd-list mailing list Fedora-livecd-list at redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list --------------------------------- We won't tell. Get more on shows you hate to love (and love to hate): Yahoo! TV's Guilty Pleasures list. -------------- next part -------------- An HTML attachment was scrubbed... URL: From asmith11 at cox.net Wed Jul 4 07:33:06 2007 From: asmith11 at cox.net (Andrew Smith) Date: Wed, 04 Jul 2007 03:33:06 -0400 Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <468A45C1.8060209@fedoraproject.org> References: <229122.95608.qm@web36511.mail.mud.yahoo.com> <468A17D6.3040802@fedoraproject.org> <468A2273.2060309@kanarip.com> <468A45C1.8060209@fedoraproject.org> Message-ID: <468B4D32.3070600@cox.net> Rahul Sundaram wrote: > Jeroen van Meeuwen wrote: >> Rahul Sundaram wrote: >>> Varoon wrote: >>>> May be i did not phrase my question right.... >>>> I would like to know if the live of fedora 7 has a c++ compiler.If so >>>> how can i use it to compile an already written code? >>>> >>> No, the live cd by default does not have any compiler. If you have >>> enough ram you can yum install gcc and related packages. It might be >>> slow to do compilations on a live cd. Why do you want to compile >>> packages in a live cd? >>> >>> Rahul >>> >> >> It doesn't really matter why he wants it, he can create a LiveCD or DVD >> himself using the livecd-tools kickstart files and the package >> customization stage in Revisor: > > Sure he can but it is does matter why some things are being done and > it is useful to understand what users are doing to see if there is > anything that can be made easier. > > Rahul > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > Along those lines, we are using the LiveCD tools to develop customized application releases. We have a Live Runtime and a Live Development build and the Development build includes compilers, debuggers, profilers, etc... This is tremendously useful for troubleshooting and debugging our application. Obviously compiled changes won't stick around after a reboot but we still find the ability to compile extremely useful to our overall process. Varoon: you can probably get the compilers you need with a command like: yum -y install gcc gcc-c++ This of course assumes you are connected to the internet. Good luck, -Andy From sundaram at fedoraproject.org Tue Jul 3 16:15:04 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Tue, 03 Jul 2007 21:45:04 +0530 Subject: [Fedora-livecd-list] How to compile c programs in fedora 7 KDE? In-Reply-To: <828629.31481.qm@web36501.mail.mud.yahoo.com> References: <828629.31481.qm@web36501.mail.mud.yahoo.com> Message-ID: <468A7608.8060706@fedoraproject.org> Varoon wrote: > I have already installed the Fedora 7 on my HD off the livecd .. > However i would like to be able to install a compiler is that possible? > if so what is the name of the compiler and download link. > Thanks > If you installed to your hard disk, it is just like any regular Fedora installation. # yum install gcc-c++ Rahul From jdogalt at yahoo.com Tue Jul 3 19:02:45 2007 From: jdogalt at yahoo.com (Jane Dogalt) Date: Tue, 3 Jul 2007 12:02:45 -0700 (PDT) Subject: [Fedora-livecd-list] livecd install usage case, was Re: How to compile c programs in fedora 7 KDE? In-Reply-To: <468A45C1.8060209@fedoraproject.org> Message-ID: <397139.65498.qm@web56913.mail.re3.yahoo.com> --- Rahul Sundaram wrote: > Jeroen van Meeuwen wrote: > > > > It doesn't really matter why he wants it, he can create a LiveCD or > DVD > > himself using the livecd-tools kickstart files and the package > > customization stage in Revisor: > > Sure he can but it is does matter why some things are being done and > it > is useful to understand what users are doing to see if there is > anything > that can be made easier. In that vein, here is a usage case that I think might be common- It seems to me there should be an easy way to upgrade the official livecd installs to the official/traditional install classes. I.e. yum groupinstall Workstation (or Server or KDE-Workstation, etc...) And of course find some way to expose this nicely to the user. Maybe it already exists, or just works as above. The example that comes to my mind was working from a livecd installed system, and wondering what I had to do to get "man 2 socket" to work. It took me longer than I had hoped to figure out I needed to do "yum install man-pages" (despite how obvious that sounds). When really all I wanted to express to my system was "I used the livecd to install because livecd's are cool, but now that I don't have any storage space limitations, I just want my system to be 'normal'". -dmc/jdog ____________________________________________________________________________________ Now that's room service! Choose from over 150,000 hotels in 45,000 destinations on Yahoo! Travel to find your fit. http://farechase.yahoo.com/promo-generic-14795097 From jkeating at redhat.com Tue Jul 3 19:11:11 2007 From: jkeating at redhat.com (Jesse Keating) Date: Tue, 3 Jul 2007 15:11:11 -0400 Subject: [Fedora-livecd-list] livecd install usage case, was Re: How to compile c programs in fedora 7 KDE? In-Reply-To: <397139.65498.qm@web56913.mail.re3.yahoo.com> References: <397139.65498.qm@web56913.mail.re3.yahoo.com> Message-ID: <200707031511.11811.jkeating@redhat.com> On Tuesday 03 July 2007 15:02:45 Jane Dogalt wrote: > yum groupinstall Workstation (or Server or KDE-Workstation, etc...) yum grouplist Pick your groups and go. We no longer have things defined like "Workstation" or "Server". -- Jesse Keating Release Engineer: Fedora -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From jdogalt at yahoo.com Tue Jul 3 19:35:22 2007 From: jdogalt at yahoo.com (Jane Dogalt) Date: Tue, 3 Jul 2007 12:35:22 -0700 (PDT) Subject: [Fedora-livecd-list] livecd install usage case, was Re: How to compile c programs in fedora 7 KDE? In-Reply-To: <200707031511.11811.jkeating@redhat.com> Message-ID: <112997.86865.qm@web56908.mail.re3.yahoo.com> --- Jesse Keating wrote: > On Tuesday 03 July 2007 15:02:45 Jane Dogalt wrote: > > yum groupinstall Workstation (or Server or KDE-Workstation, etc...) > > yum grouplist > > Pick your groups and go. We no longer have things defined like > "Workstation" > or "Server". I guess that is sort of what I was criticizing. My point was that I didn't want to spend energy thinking and making choices. Yes, I could look at that list of a couple dozen things and ponder which of them I want and which of them might get me the socket manpage (which as I look at it now, doesn't seem at all obvious). But what I really wanted was to not have to think at all, and with one simple standard command, get the system to be 'normal', or rather have all the little obscure things that one would expect to be there in a simple DVD install (where for FC6 I would just blindly check the 2 unchecked group options). The key issue here is that fairly obscure decisions had to be made to get the livecd to fit in 600MB, and it seems like it would save a lot of people a lot of mental energy if there were a simple red button which says "now that I have a network and plenty of disk space, please undo all those fairly obscure decisions without making me waste time thinking and choosing, and figuring out how to make those choices" -dmc/jdog ____________________________________________________________________________________ Be a better Heartthrob. Get better relationship answers from someone who knows. Yahoo! Answers - Check it out. http://answers.yahoo.com/dir/?link=list&sid=396545433 From jdogalt at yahoo.com Tue Jul 3 20:04:28 2007 From: jdogalt at yahoo.com (Jane Dogalt) Date: Tue, 3 Jul 2007 13:04:28 -0700 (PDT) Subject: [Fedora-livecd-list] livecd install usage case, was Re: How to compile c programs in fedora 7 KDE? In-Reply-To: <200707031511.11811.jkeating@redhat.com> Message-ID: <837041.24897.qm@web56906.mail.re3.yahoo.com> --- Jesse Keating wrote: > On Tuesday 03 July 2007 15:02:45 Jane Dogalt wrote: > > yum groupinstall Workstation (or Server or KDE-Workstation, etc...) > > yum grouplist > > Pick your groups and go. We no longer have things defined like > "Workstation" > or "Server". I guess what I really want is this- yum spinlist which would return things like Fedora-LiveCD-Desktop Fedora-LiveCD-KDE Fedora-LiveDVD-Desktop Fedora-LiveDVD-KDE And then, having run this on a system that was installed from the Fedora-LiveCD-Desktop, I would run yum spininstall Fedora-LiveDVD-Desktop which would in one brainless fell swoop get me all the things that you smart fedora folks decided to package on a DVD which didn't have the storage limitations of the CD I downloaded (because it was smaller, more compatable with more systems, etc...) (and yes, clearly this could all be done within the yum group framework...) -dmc/jdog ____________________________________________________________________________________ Get the free Yahoo! toolbar and rest assured with the added security of spyware protection. http://new.toolbar.yahoo.com/toolbar/features/norton/index.php From kanarip at kanarip.com Tue Jul 3 20:25:01 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Tue, 03 Jul 2007 22:25:01 +0200 Subject: [Fedora-livecd-list] livecd install usage case, was Re: How to compile c programs in fedora 7 KDE? In-Reply-To: <837041.24897.qm@web56906.mail.re3.yahoo.com> References: <837041.24897.qm@web56906.mail.re3.yahoo.com> Message-ID: <468AB09D.5090005@kanarip.com> Jane Dogalt wrote: > I guess what I really want is this- > > yum spinlist > > which would return things like > > Fedora-LiveCD-Desktop > Fedora-LiveCD-KDE > Fedora-LiveDVD-Desktop > Fedora-LiveDVD-KDE > > And then, having run this on a system that was installed from the > Fedora-LiveCD-Desktop, I would run > > yum spininstall Fedora-LiveDVD-Desktop > > which would in one brainless fell swoop get me all the things that you > smart fedora folks decided to package on a DVD which didn't have the > storage limitations of the CD I downloaded (because it was smaller, > more compatable with more systems, etc...) > > (and yes, clearly this could all be done within the yum group > framework...) > > -dmc/jdog > We've tried it to make it as simple as it gets with Revisor. Any troubles with that, and you can come knocking on our door. Kind regards, Jeroen van Meeuwen -kanarip From jdogalt at yahoo.com Tue Jul 3 20:40:14 2007 From: jdogalt at yahoo.com (Jane Dogalt) Date: Tue, 3 Jul 2007 13:40:14 -0700 (PDT) Subject: [Fedora-livecd-list] livecd install usage case, was Re: How to compile c programs in fedora 7 KDE? In-Reply-To: <468AB09D.5090005@kanarip.com> Message-ID: <120183.94167.qm@web56911.mail.re3.yahoo.com> --- Jeroen van Meeuwen wrote: > Jane Dogalt wrote: > > I guess what I really want is this- > > > > yum spinlist > > > > which would return things like > > > > Fedora-LiveCD-Desktop > > Fedora-LiveCD-KDE > > Fedora-LiveDVD-Desktop > > Fedora-LiveDVD-KDE > > > > And then, having run this on a system that was installed from the > > Fedora-LiveCD-Desktop, I would run > > > > yum spininstall Fedora-LiveDVD-Desktop > > > > which would in one brainless fell swoop get me all the things that > you > > smart fedora folks decided to package on a DVD which didn't have > the > > storage limitations of the CD I downloaded (because it was smaller, > > more compatable with more systems, etc...) > > > > (and yes, clearly this could all be done within the yum group > > framework...) > > > > -dmc/jdog > > > > We've tried it to make it as simple as it gets with Revisor. Any > troubles with that, and you can come knocking on our door. I don't think we're on the same page here. The usage case I'm talking about has nothing to do with a user making a custom spin. (and I assume thats entirely what revisor is about). My usage case involves a user installing a standard LiveCD spin, but wanting - with the least effort possible - to upgrade that installed system to the the package list of a similar, but not space-constrained _official_ spin. Let me know if I'm confused, and if revisor is also about revising installed systems. -dmc/jdog ____________________________________________________________________________________ Moody friends. Drama queens. Your life? Nope! - their life, your story. Play Sims Stories at Yahoo! Games. http://sims.yahoo.com/ From kanarip at kanarip.com Tue Jul 3 20:59:46 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Tue, 03 Jul 2007 22:59:46 +0200 Subject: [Fedora-livecd-list] livecd install usage case In-Reply-To: <120183.94167.qm@web56911.mail.re3.yahoo.com> References: <120183.94167.qm@web56911.mail.re3.yahoo.com> Message-ID: <468AB8C2.2050800@kanarip.com> Jane Dogalt wrote: > --- Jeroen van Meeuwen wrote: > >> We've tried it to make it as simple as it gets with Revisor. Any >> troubles with that, and you can come knocking on our door. > > I don't think we're on the same page here. The usage case I'm talking > about has nothing to do with a user making a custom spin. (and I > assume thats entirely what revisor is about). > > My usage case involves a user installing a standard LiveCD spin, but > wanting - with the least effort possible - to upgrade that installed > system to the the package list of a similar, but not space-constrained > _official_ spin. > > Let me know if I'm confused, and if revisor is also about revising > installed systems. > We're not yet revising running sytems, but we're getting there ;-) Different approach though. I guess we could say "Install from LiveCD and give me more then you have given me so far" -same goes for DVD. That be effort-less enough, right? Kind regards, Jeroen van Meeuwen -kanarip From tim.wood at datawranglers.com Wed Jul 4 13:35:14 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Wed, 4 Jul 2007 07:35:14 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> Message-ID: <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> Last night I started running Revisor. I thought I'd finally gotten all the config files convinced to play nice. I'm using a repository that existing on another machine on the same network. I haven't tested this repository extensively (it's a replacement for a machine that died), but have used it to do a couple of recent installs/updates on 1 machine and everything seems to work properly. It got to the "Building your media" screen, went through checking dependencies and then sat there on downloading packages. I check it with ps and it was taking 10+% CPU and 23% Memory. This morning, it's still in the same stage (downloading media) and the task progress bar has not moved. CPU usage has dropped to 1%. What I'm assuming is the cache log (/var/tmp/revisorrevisor-yum.log) is empty. The cache directory (/var/tmp/revisor-yumcache) has been created and there's a subdirectory for each of the entries that were on the repo screen (development, fedora, updates) and each has various items including an empty packages directory. The log (/var/log/revisor.log) ends with a copy of the warning I received about /srv/revisor already existing (from a previous failed run). Any suggestions on troubleshooting this? Maybe rip Revisor out and beat it with a baseball bat? From jonathansteffan at gmail.com Wed Jul 4 20:55:27 2007 From: jonathansteffan at gmail.com (Jonathan Steffan) Date: Wed, 04 Jul 2007 14:55:27 -0600 Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> Message-ID: <1183582527.31225.3.camel@damaestro> On Wed, 2007-07-04 at 07:35 -0600, Tim Wood wrote: > Last night I started running Revisor. What type of media are you creating? > I thought I'd finally gotten all > the config files convinced to play nice. May we see? > I'm using a repository that > existing on another machine on the same network. I haven't tested this > repository extensively (it's a replacement for a machine that died), but > have used it to do a couple of recent installs/updates on 1 machine and > everything seems to work properly. > > It got to the "Building your media" screen, went through checking > dependencies and then sat there on downloading packages. I check it with > ps and it was taking 10+% CPU and 23% Memory. This morning, it's still in > the same stage (downloading media) and the task progress bar has not > moved. CPU usage has dropped to 1%. It sounds like you hit a bug. If you're willing, run Revisor from a terminal so you can get the traceback. > What I'm assuming is the cache log > (/var/tmp/revisorrevisor-yum.log) is empty. The cache directory > (/var/tmp/revisor-yumcache) has been created and there's a subdirectory > for each of the entries that were on the repo screen (development, fedora, > updates) and each has various items including an empty packages directory. > The log (/var/log/revisor.log) ends with a copy of the warning I received > about /srv/revisor already existing (from a previous failed run). > > Any suggestions on troubleshooting this? What version of Revisor are you using? (rpm -q revisor) > Maybe rip Revisor out and beat it with a baseball bat? Sure. Just not the face ;-) Jonathan Steffan daMaestro From tim.wood at datawranglers.com Thu Jul 5 13:47:27 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Thu, 5 Jul 2007 07:47:27 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <1183582527.31225.3.camel@damaestro> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> Message-ID: <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> > On Wed, 2007-07-04 at 07:35 -0600, Tim Wood wrote: >> Last night I started running Revisor. > > What type of media are you creating? A live cd. >> I thought I'd finally gotten all >> the config files convinced to play nice. > > May we see? I've attached the three I've modified. sw-gateway-ks-0.0.1.cfg is my kickstart. The 192.168... address is my local repository >> I'm using a repository that >> existing on another machine on the same network. I haven't tested this >> repository extensively (it's a replacement for a machine that died), but >> have used it to do a couple of recent installs/updates on 1 machine and >> everything seems to work properly. >> >> It got to the "Building your media" screen, went through checking >> dependencies and then sat there on downloading packages. I check it >> with >> ps and it was taking 10+% CPU and 23% Memory. This morning, it's still >> in >> the same stage (downloading media) and the task progress bar has not >> moved. CPU usage has dropped to 1%. > > It sounds like you hit a bug. If you're willing, run Revisor from a > terminal so you can get the traceback. I'd be happy to. After doing the command (revisor), how do I do a traceback? >> What I'm assuming is the cache log >> (/var/tmp/revisorrevisor-yum.log) is empty. The cache directory >> (/var/tmp/revisor-yumcache) has been created and there's a subdirectory >> for each of the entries that were on the repo screen (development, >> fedora, >> updates) and each has various items including an empty packages >> directory. >> The log (/var/log/revisor.log) ends with a copy of the warning I >> received >> about /srv/revisor already existing (from a previous failed run). >> >> Any suggestions on troubleshooting this? > > What version of Revisor are you using? (rpm -q revisor) 2.0.4 >> Maybe rip Revisor out and beat it with a baseball bat? > > Sure. Just not the face ;-) Hehehe. Yeah. (Almost) anything to avoid going back to customizing knoppix. -------------- next part -------------- A non-text attachment was scrubbed... Name: revisor-f7-i386.conf Type: application/octet-stream Size: 3024 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: revisor.conf Type: application/octet-stream Size: 3126 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: sw-gateway-ks-0.0.1.cfg Type: application/octet-stream Size: 1321 bytes Desc: not available URL: From kanarip at kanarip.com Thu Jul 5 22:09:42 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Fri, 06 Jul 2007 00:09:42 +0200 Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> Message-ID: <468D6C26.1090902@kanarip.com> Tim Wood wrote: >> On Wed, 2007-07-04 at 07:35 -0600, Tim Wood wrote: >>> Last night I started running Revisor. >> What type of media are you creating? > > A live cd. > >>> I thought I'd finally gotten all >>> the config files convinced to play nice. >> May we see? > > I've attached the three I've modified. sw-gateway-ks-0.0.1.cfg is my > kickstart. The 192.168... address is my local repository > > >>> I'm using a repository that >>> existing on another machine on the same network. I haven't tested this >>> repository extensively (it's a replacement for a machine that died), but >>> have used it to do a couple of recent installs/updates on 1 machine and >>> everything seems to work properly. >>> I'm concerned with that, as Revisor (or actually the yum object Revisor creates for doing dependency resolving and downloading packages), really, really depends on good repositories. I see there's a disc1/ directory which I can only hope is not the DVD of what the Fedora Project released, rather then the Everything tree. >>> It got to the "Building your media" screen, went through checking >>> dependencies and then sat there on downloading packages. I check it >>> with >>> ps and it was taking 10+% CPU and 23% Memory. This morning, it's still >>> in >>> the same stage (downloading media) and the task progress bar has not >>> moved. CPU usage has dropped to 1%. >> It sounds like you hit a bug. If you're willing, run Revisor from a >> terminal so you can get the traceback. > > I'd be happy to. After doing the command (revisor), how do I do a traceback? > If you run from a gnome-terminal -like prompt, you can run: revisor --debug > ~/revisor.log That'd ensure everything is in /a/ logfile, that we can use to track down the issue here. Mind that our logging facility is... erhm... b0rked. Hence the original revisor logfile not showing as much as we'd want it to. >>> Maybe rip Revisor out and beat it with a baseball bat? >> Sure. Just not the face ;-) > > Hehehe. Yeah. (Almost) anything to avoid going back to customizing knoppix. > Gheghe, you know revisor started beating /some-other-application/ with a baseball bat, and *in* the face, right? ;-) Kind regards, Jeroen van Meeuwen -kanarip From jdogalt at yahoo.com Thu Jul 5 22:35:38 2007 From: jdogalt at yahoo.com (Jane Dogalt) Date: Thu, 5 Jul 2007 15:35:38 -0700 (PDT) Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <468D6C26.1090902@kanarip.com> Message-ID: <608060.64659.qm@web56909.mail.re3.yahoo.com> --- Jeroen van Meeuwen wrote: > >>> Maybe rip Revisor out and beat it with a baseball bat? > >> Sure. Just not the face ;-) > > > > Hehehe. Yeah. (Almost) anything to avoid going back to > customizing knoppix. > > > > Gheghe, you know revisor started beating /some-other-application/ > with a > baseball bat, and *in* the face, right? ;-) What are you talking about? -dmc/jdog ____________________________________________________________________________________ Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow From kanarip at kanarip.com Thu Jul 5 22:40:26 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Fri, 06 Jul 2007 00:40:26 +0200 Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <608060.64659.qm@web56909.mail.re3.yahoo.com> References: <608060.64659.qm@web56909.mail.re3.yahoo.com> Message-ID: <468D735A.1070907@kanarip.com> Jane Dogalt wrote: > --- Jeroen van Meeuwen wrote: > >>>>> Maybe rip Revisor out and beat it with a baseball bat? >>>> Sure. Just not the face ;-) >>> Hehehe. Yeah. (Almost) anything to avoid going back to >> customizing knoppix. >> Gheghe, you know revisor started beating /some-other-application/ >> with a >> baseball bat, and *in* the face, right? ;-) > > What are you talking about? > > -dmc/jdog > Inside joke, never mind. Kind regards, Jeroen van Meeuwen -kanarip From tim.wood at datawranglers.com Thu Jul 5 22:07:39 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Thu, 5 Jul 2007 16:07:39 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <468D6C26.1090902@kanarip.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> <468D6C26.1090902@kanarip.com> Message-ID: <25423.70.57.175.112.1183673259.squirrel@hputz.datawranglers.com> [snip] >>> What type of media are you creating? >> >> A live cd. >> [snip] >> I've attached the three I've modified. sw-gateway-ks-0.0.1.cfg is my >> kickstart. The 192.168... address is my local repository >> >> >>>> I'm using a repository that >>>> existing on another machine on the same network. I haven't tested > [snip] > > I'm concerned with that, as Revisor (or actually the yum object Revisor > creates for doing dependency resolving and downloading packages), > really, really depends on good repositories. I see there's a disc1/ > directory which I can only hope is not the DVD of what the Fedora > Project released, rather then the Everything tree. I repointed that to RPMS.os. All of a sudden I'm getting a 'no groups' error right before resolving dependencies. But, it's downloading packages. What's the problem with the install DVD? > If you run from a gnome-terminal -like prompt, you can run: > > revisor --debug > ~/revisor.log I didn't realize that's all that term means... Interesting. >>>> Maybe rip Revisor out and beat it with a baseball bat? >>> Sure. Just not the face ;-) >> >> Hehehe. Yeah. (Almost) anything to avoid going back to customizing >> knoppix. > > Gheghe, you know revisor started beating /some-other-application/ with a > baseball bat, and *in* the face, right? ;-) Jeroen, it feels like that ;-) I suspect I'm missing an inside joke here... 42, fully ladden swallows and Gheghe perhaps... Ah well, 2 out of 3 ain't bad... thanks and regards, Tim From tim.wood at datawranglers.com Thu Jul 5 22:09:19 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Thu, 5 Jul 2007 16:09:19 -0600 (MDT) Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <608060.64659.qm@web56909.mail.re3.yahoo.com> References: <608060.64659.qm@web56909.mail.re3.yahoo.com> Message-ID: <25439.70.57.175.112.1183673359.squirrel@hputz.datawranglers.com> >> Gheghe, you know revisor started beating /some-other-application/ >> with a >> baseball bat, and *in* the face, right? ;-) > > What are you talking about? It _is_ an inside joke! :-) From kanarip at kanarip.com Thu Jul 5 22:56:06 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Fri, 06 Jul 2007 00:56:06 +0200 Subject: [Fedora-livecd-list] Revisor stalls while downloading packages In-Reply-To: <25423.70.57.175.112.1183673259.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> <468D6C26.1090902@kanarip.com> <25423.70.57.175.112.1183673259.squirrel@hputz.datawranglers.com> Message-ID: <468D7706.1000509@kanarip.com> Tim Wood wrote: > [snip] >>>> What type of media are you creating? >>> A live cd. >>> > [snip] >>> I've attached the three I've modified. sw-gateway-ks-0.0.1.cfg is my >>> kickstart. The 192.168... address is my local repository >>> >>> >>>>> I'm using a repository that >>>>> existing on another machine on the same network. I haven't tested > [snip] >> I'm concerned with that, as Revisor (or actually the yum object Revisor >> creates for doing dependency resolving and downloading packages), >> really, really depends on good repositories. I see there's a disc1/ >> directory which I can only hope is not the DVD of what the Fedora >> Project released, rather then the Everything tree. > > I repointed that to RPMS.os. All of a sudden I'm getting a 'no groups' > error right before resolving dependencies. But, it's downloading > packages. What's the problem with the install DVD? > Nothing in particular actually. But as it showed 'disc1' and no other discs being released I was confused with what exactly was in that repository ;-). The no groups error usually comes from lacking comps.xml in the repository metadata. Revisor uses that comps to create the list of Categories and Groups during package selection. Kind regards, Jeroen van Meeuwen -kanarip From wallsweb91 at yahoo.com Fri Jul 6 12:25:06 2007 From: wallsweb91 at yahoo.com (Dan Wallace) Date: Fri, 6 Jul 2007 05:25:06 -0700 (PDT) Subject: [Fedora-livecd-list] livecd install to hd fails Message-ID: <286849.3622.qm@web34712.mail.mud.yahoo.com> I have used the livecd to install fedora on a HP DL360 dual 800Mhz processor raid enabled 1U rack mount server with 768M RAM. I have tried this 3 times, the last choosing all defaults for disk partitioning, and I get the same result each time. It seems to be complaining about 'Volume group "VolGroup00" not found', then it goes into kernel panic. The installer is very pretty, but there is little I can go on to diagnose this problem. This is an install directly to the hardware - no Xen, no vmware etc. Any help would be appreciated. Here is standard error from running liveinst: =========================================== [fedora at localhost ~]$ liveinst & [1] 3499 [fedora at localhost ~]$ FATAL: Module md not found. [fedora at localhost ~]$ Probing for video card: ATI Technologies Inc 3D Rage IIC 215IIC [Mach64 GT IIC] Starting graphical installation... Loading /lib/kbd/keymaps/i386/qwerty/us.map.gz Couldn't find rules file (xfree86) =========================================== Here is output from the console at boot time =========================================== Booting 'Fedora (2.6.21-1.3194.fc7)' root (hd0,0) Filesystem type is ext2fs, partition type 0x83 kernel /vmlinux-2.6.21-1.3194.fc7 no root=LABEL=/ rhgb quiet [Linux-bzImage, setup=0x1e00, size=0x1c9dd4] initrd /initrd-2.6.21-1.3194.fc7.img [Linux-initrd @ 0x2fca8000, 0x343303 bytes ] Uncompressing Linux. . . Ok, booting the kernel. ACPI: Resource is not an IRQ entry agpgart: unable to determine aperture size. agpgart: unable to determine aperture size. Red Hat nash version 6.0.9 starting Reading all physical volumes. This may take a while... No volume groups found Volume group "VolGroup00" not found Unable to access resume device (/dev/VolGroup00/LogVol101) mount: could not find filesystem '/dev/root' setuproot: moving /dev failed: No such file or directory setuproot: error mounting /proc: No such file or directory setuproot: error mounting /sys: No such file or directory switchroot: mount failed: No such file or directory Kernel panic - not syncing: Attempted to kill init! =========================================== ____________________________________________________________________________________ Need Mail bonding? Go to the Yahoo! Mail Q&A for great tips from Yahoo! Answers users. http://answers.yahoo.com/dir/?link=list&sid=396546091 From tim.wood at datawranglers.com Sat Jul 7 02:42:31 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Fri, 6 Jul 2007 20:42:31 -0600 (MDT) Subject: [Fedora-livecd-list] How to fix Resolver not seeing your comps.xml file In-Reply-To: <468D7706.1000509@kanarip.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> <468D6C26.1090902@kanarip.com> <25423.70.57.175.112.1183673259.squirrel@hputz.datawranglers.com> <468D7706.1000509@kanarip.com> Message-ID: <25292.70.57.175.112.1183776151.squirrel@hputz.datawranglers.com> > The no groups error usually comes from lacking comps.xml in the > repository metadata. Revisor uses that comps to create the list of > Categories and Groups during package selection. Grumble. I'm using mrepo to build my repository. It's great except that it seems to ignore the comps.xml file on the source repository. In futzing around, I found out (FWIW) that Resolver (or anything else hitting your repo) finds the comps.xml file by looking in the repomd.xml file. I was able to solve the problem by copying the comparable section out of the repomd.xml on the official fedora repository into my repomd.xml. Then, it was just a matter of making sure the path, etc., in that section were correct for my repo. regards, Tim From sundaram at fedoraproject.org Sat Jul 7 09:07:35 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Sat, 07 Jul 2007 14:37:35 +0530 Subject: [Fedora-livecd-list] How to fix Resolver not seeing your comps.xml file In-Reply-To: <25292.70.57.175.112.1183776151.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> <468D6C26.1090902@kanarip.com> <25423.70.57.175.112.1183673259.squirrel@hputz.datawranglers.com> <468D7706.1000509@kanarip.com> <25292.70.57.175.112.1183776151.squirrel@hputz.datawranglers.com> Message-ID: <468F57D7.8030301@fedoraproject.org> Tim Wood wrote: >> The no groups error usually comes from lacking comps.xml in the >> repository metadata. Revisor uses that comps to create the list of >> Categories and Groups during package selection. > > Grumble. I'm using mrepo to build my repository. It's great except that > it seems to ignore the comps.xml file on the source repository. You should report this issue to Dag then. Rahul From tim.wood at datawranglers.com Sat Jul 7 17:26:49 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Sat, 7 Jul 2007 11:26:49 -0600 (MDT) Subject: [Fedora-livecd-list] How to fix Resolver not seeing your comps.xml file In-Reply-To: <468F57D7.8030301@fedoraproject.org> References: <466ED035.1060009@nanotechnologies.qc.ca> <9d2c731f0706121134l7074c34ek654909ce667bf839@mail.gmail.com> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> <468D6C26.1090902@kanarip.com> <25423.70.57.175.112.1183673259.squirrel@hputz.datawranglers.com> <468D7706.1000509@kanarip.com> <25292.70.57.175.112.1183776151.squirrel@hputz.datawranglers.com> <468F57D7.8030301@fedoraproject.org> Message-ID: <25449.70.57.175.112.1183829209.squirrel@hputz.datawranglers.com> >> Grumble. I'm using mrepo to build my repository. It's great except >> that >> it seems to ignore the comps.xml file on the source repository. > > You should report this issue to Dag then. Yes... haven't had time to locate his bug reporting page/tool/whatever... regards, Tim From tim.wood at datawranglers.com Sat Jul 7 18:11:56 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Sat, 7 Jul 2007 12:11:56 -0600 (MDT) Subject: [Fedora-livecd-list] How to fix Resolver not seeing your comps.xml file In-Reply-To: <25449.70.57.175.112.1183829209.squirrel@hputz.datawranglers.com> References: <466ED035.1060009@nanotechnologies.qc.ca> <25390.71.34.155.75.1183322712.squirrel@hputz.datawranglers.com> <46881B95.7030907@kanarip.com> <25283.71.34.155.75.1183328247.squirrel@hputz.datawranglers.com> <4688B00A.4070403@kanarip.com> <25432.70.59.215.121.1183380141.squirrel@hputz.datawranglers.com> <25437.70.59.215.121.1183394810.squirrel@hputz.datawranglers.com> <46893906.10801@kanarip.com> <25423.70.59.215.121.1183411297.squirrel@hputz.datawranglers.com> <25284.70.59.215.121.1183556114.squirrel@hputz.datawranglers.com> <1183582527.31225.3.camel@damaestro> <25283.70.57.175.112.1183643247.squirrel@hputz.datawranglers.com> <468D6C26.1090902@kanarip.com> <25423.70.57.175.112.1183673259.squirrel@hputz.datawranglers.com> <468D7706.1000509@kanarip.com> <25292.70.57.175.112.1183776151.squirrel@hputz.datawranglers.com> <468F57D7.8030301@fedoraproject.org> <25449.70.57.175.112.1183829209.squirrel@hputz.datawranglers.com> Message-ID: <25425.70.57.175.112.1183831916.squirrel@hputz.datawranglers.com> When I create a traditional kickstart, I'm able to preface a package name with a dash to not install it. For instance: @base-x -linuxwacom -synaptics I can't seem to make the same thing work when Revisor is using a kickstart file. Is there a way to make this work or is the only solution to copy each package name I want (from say the comps.xml file)? thanks, Tim From chitlesh at fedoraproject.org Sat Jul 7 23:40:55 2007 From: chitlesh at fedoraproject.org (Chitlesh GOORAH) Date: Sun, 8 Jul 2007 01:40:55 +0200 Subject: [Fedora-livecd-list] Kadischi reached End Of Life Message-ID: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> Hello there, A few days ago, I mentioned[1] there is no one behind kadischi's development team and it will be EOL'ed this saturday. So today is Saturday 07 July 2007 (in a few minutes it will be 08 July 2007), kadischi's bug review[2] was marked and closed as NOTABUG, while the remaining 4 open bugs under kadischi's name : # A first experience ( #222568 ) # Kadischi does not build on VMware hosted OS ( #216151 ) # Kadischi may raise an exception when attempting to copy device files. ( #211692 ) # Processed: Kadischi build fails if dietlibc is installed ( #167220 ) were closed as WONTFIX. Therefore, I'm announcing the official EOF of Kadischi. It is sad, but it is _not_ the end of creation of custom spins at the Fedora Project since Jeroen van Meeuwen and his friends are actively maintaining revisor[3]. If you were a kadischi user in the past, you will probably see revisor as a better alternative. # I'll like to thank everyone who participated in the kadischi project: whether by contributing codes # or by testing, translations, documentation,... We will still be looking forward to see your contributions at the new revisor project. The Fedora Livecd Mailing list[4] is still active and you can still asked your livecd related questions there. However I don't know what will be the state of the #fedora-livecd irc channel on freenode. The average number of people who joined this channel daily is 5 and there's pretty much no discussions. Well, #fedora-devel is always crowded. cheers, Chitlesh [1] http://clunixchit.blogspot.com/2007/07/kadischis-state.html#links [2] https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=236162 [3] http://revisor.fedoraunity.org/ [4] http://www.redhat.com/mailman/listinfo/fedora-livecd-list -- http://clunixchit.blogspot.com From dmc.fedora at filteredperception.org Sun Jul 8 04:58:49 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sat, 07 Jul 2007 23:58:49 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer Message-ID: <46906F09.2030105@filteredperception.org> Community, At the bottom of this lengthy RFC is a patch to livecd-tools mayflower script which adds the interesting ability to install the livecd system to hard disk, in a way that doesn't require a reboot after installation. Given the proof-of-concept nature of this small patch/hack, I just want to get some feedback from the community before really going all out, and adding support to anaconda to be the installation front-end. For now, you would have to perform the installation steps (outlined below) manually. Concept: Rebootless Installer via LiveCD(/DVD/USB/PXE/...) ---------------------------------------------------------- The basic 'trick' is to use raid1 style mechanisms to live-migrate the root filesystem from the install media, to the desitination drive. Note, that I'm not looking for a debate as to whether or not this would be a useful default installer for fedora. Rather I think that it would *definitely* be a nice checkbox option for people to add to their own customized fedora-derived spins. History ------- In 2001 I created an unpublished livecd spin tool for Mandrake. The resulting images (optionally) possessed the interesting feature that during the initrd boot sequence, all hard drives would be scanned for mountable partitions. If a partition was found with suitable free space (e.g. windows C drive(vfat)), the initrd would copy the livecd to an image file there, and use it instead of the cd drive. (and optionally setting up loadlin/lilo for future usage sans livecd). But this wasn't as useful as what I'm currently talking about, because the installed system would still be a mix of read-only and tmpfs filesystems, and you would have to wait for the cd to copy before seeing anything. (of course I did have X+mozilla+xmms in 135MB...) Then, as I discovered a couple days ago, in 2003, Goswin Brederlow more or less outlined the attached method. http://www.mail-archive.com/debian-boot at lists.debian.org/msg28182.html I haven't yet pinged him, so I don't know if debix is the 'modern' version of the proof of concept iso he referred to there. http://debix.alioth.debian.org/ (stale since 2004?) Then, when I first started getting involved with fedora-livecd-list, someone, I think a redhat employee, mentioned that I might be able to do that iso-caching-to-disk thing mentioned above using raid1 mirror tricks. I.e. in initrd, set up the image from iso as a 1-disk raid1 array, and use it as the root fs. Then after boot, use space on hard disk as a hot-added 2nd disk for the array, and after the raid sync finishes, hot-remove the cdrom based device, so that the cd can be ejected. It wasn't long after this that I managed to re-outline what Goswin had outlined years earlier http://www.redhat.com/archives/fedora-livecd-list/2006-August/msg00000.html Current Mechanism ----------------- The mechanism I've implemented in the attached patch is really dirt simple. - mdadm is added to the initramfs - after normal creation of the device mapper root filesystem, /dev/md7 is created as a raid1 with a single device- the normal /dev/mapper/live-rw root device. Note, I opted with a metadataless device, so that on subsequent boots there is no raid1 involved (? I may have an incomplete understanding of my options that led to this choice ?) - the root filesystem (sysroot) is then mounted from /dev/md7 instead of /dev/mapper/live-rw Current Rebootless Installation Procedure ----------------------------------------- - once booted into the live environment, choose your destination device. (for this exampe, I'll assume /dev/sda1) - add the device to the raid. mdadm /dev/md7 --add /dev/sda1 mdadm /dev/md7 --grow --raid-disks=2 - wait for the resync to finish (watch /proc/mdstat) - remove the original devices from the raid mdadm /dev/md7 --fail /dev/mapper/live-rw mdadm /dev/md7 --remove /dev/mapper/live-rw mdadm /dev/md7 --grow --raid-disks=1 // (perhaps redundant) dmsetup remove live-rw losetup -d /dev/loop121 losetup -d /dev/loop120 losetup -d /dev/loop119 - now you should be able to eject the livecd - install and configure grub in the usual way, with the assumption that the root device on subsquent boots will be /dev/sda1 as normal, and not the current /dev/md7. - now grow the filesystem - CRASH/BURN/DIE... segway to... Current Problem With Less Than Fully Satisfying Workaround ---------------------------------------------------------- The current problem with the attached patch is that it is not possible to resize the small livecd ext3 filesystem to the maximum size of the destination partition _before_ rebooting. (after reboot, no sweat, as the raid1 layer is gone) Specifically, when I tried mdadm /dev/md7 --grow --size=max /proc/mdstat showed the number of blocks go from 3.5G to 0, though strangely the system did not fall over dead. And when I tried mdadm /dev/md7 --grow --size=[ just a bit bigger than mdstat reports ] I get mdadm: Cannot set device size for /dev/md7: No space left on device Now, googling yielded someone with a similar message for raid5 which turned out to be a legitimate bug. My next step will be to post to the linux-raid mailinglist and see what they say. From my perusing of the userspace and kernel code, I have a hunch that I may just be trying for a feature that hasn't been implemented, and hasn't been adequetly documented as such. Perhaps my choice of raid1 options is at issue. But, there are of course many different ways to try and accomplish the same thing, so... Workaround #1 ------------- Create the original livecd ext3 fsimage on a huge (say 777G) sparse file, but only make the filesytem size be the normal container size (~3G). (or alternately, in initramfs, create a device mapper device that is a linear addition of the original fsimage, and a huge device mapper zero device) Then, instead of setting /dev/sda1 as the other half of the mirror, create another devicemapper device which is the linear addition of it and a huge device mapper zero device. Because there is now no need to grow the raid1, you are golden, and can resize the ext3 fs to the max size of sda1. CONS: The main con here is that the installation process will require the zero-ing out of all the unused parts of sda1, AND a whole lot of null work as the raid syncs the tailing imaginary parts of the devices. Admittedly the latter is pretty fast, though if you want to support installing to a 750G disk, you'll need to make the containers 777G and suffer quite a bit of time waiting for the md driver to finish doing a whole lot of nothing. Theoritical Workaround #2 ------------------------- Like WA#1, but instead of the other half of the raid1 being a linear addition of the _entire_ destination device and a big zero device, make it a linear addition of the _beggining_(size of livecd fsimage) of the destination device, and a big zero device. But, to make it work, make that be a multipath md device. Then after the faster raid1 sync finishes, create a new fake device that includes all of the destination device, and add it to the multipath, take out the other, and bingo. I haven't tried this yet to see if it will work. Theoretical Workaround #3 ------------------------- Hack the md raid1 driver so that you can force the sync to believe it is done after reaching a certain point. Together with WA#1, this would achieve relatively best-case overall performance. I've looked into the md sysfs interface, and there doesn't yet appear to be a way to accomplish this. Theoretical Workaround #4 ------------------------- Understand the device mapper mirror target, which lacks Documentation/device-mapper/mirror.txt http://web.glandium.org/blog/?p=141 is the best documentation I've found so far. Though I did run accross a Goswin post somewhere where he claimed that it was really simple (cough cough). Or perhaps redo everything with pvmove (which I haven't researched much), and/or logical volumes as requirements. Or perhaps this request for comments will yield some other sort of answer. Anybody? Anybody? Patch ----- Note, this is against mayflower from livecd-tools-009-1.fc7, and you will also need to add raid1 to the MODULES list in livecd-creator where it generates mayflower.conf on the fly. 80a81,82 > # dmc/jdog-ri-poch > cp /sbin/mdadm sbin 568c570,589 < ln -s /dev/mapper/live-rw /dev/root --- > ## dmc/jdog-ri-poch > #ln -s /dev/mapper/live-rw /dev/root > # create broken mirror > # and hope it doesn't cause another 7 years of bush in the whitehouse :( > modprobe raid1 > mdadm --build /dev/md7 --level=1 --raid-devices=1 /dev/mapper/live-rw > > ln -s /dev/md7 /dev/root > > ## dmc/jdog-ri-poch > # mount the broken mirror instead of the live-rw > #mount -n -t ext3 /dev/mapper/live-rw /sysroot > mount -n -t ext3 /dev/md7 /sysroot > > cat< /sysroot/usr/sbin/rliveinst > #!/bin/bash > /sbin/mdadm /dev/md7 --add \\\$1 ; while ( grep -q recovery /proc/mdstat ); d\ o sleep 7; done; /sbin/mdadm /dev/md7 --fail /dev/mapper/live-rw ; /sbin/mdadm\ /dev/md7 --remove /dev/mapper/live-rw > # todo: configure and install grub and run mkinitrd, and free loop devices > JDRIEOF > chmod +x /sysroot/usr/sbin/rliveinst 570d590 < mount -n -t ext3 /dev/mapper/live-rw /sysroot 682a703 > From dmc.fedora at filteredperception.org Sun Jul 8 08:34:56 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 03:34:56 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <46906F09.2030105@filteredperception.org> References: <46906F09.2030105@filteredperception.org> Message-ID: <4690A1B0.3060305@filteredperception.org> Douglas McClendon wrote: > > Theoretical Workaround #4 > ------------------------- > > Understand the device mapper mirror target, which lacks > Documentation/device-mapper/mirror.txt ... Theoretical Workaround #5 ------------------------- Use a linear device mapper device for the sysroot. Initially it will be composed of the /dev/md7 as par the patch, and a 1.5TB dm-zero device. Then migrate the /dev/md7 from live-rw to /dev/sda1(partial). Then free live-rw components. Then reload the tables so that the sysroot dm device consists of /dev/md7(which is a single partial sda1 usage), + /dev/sda1(the rest of it) + dm-zero (up to 1.5T). Then resize2fs should work and everything is exactly right... Hmm... I'll have to try that tomorrow... (this is because I just noticed someone say that you can play around with the dm tables of an active device. We'll see if I'm allowed to use /dev/sda(1) in two places at once (with or without possibly adding yet another layer of abstraction in there somewhere. And of course, hoping that the mdadm grow problem is just a bug that can be fixed, is still the first preference) -dmc/jdog From ddmbox2000 at yahoo.co.uk Sun Jul 8 10:40:50 2007 From: ddmbox2000 at yahoo.co.uk (dexter) Date: Sun, 8 Jul 2007 11:40:50 +0100 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> Message-ID: <200707081140.51500.ddmbox2000@yahoo.co.uk> On Sun July 8 2007 00:40:55 Chitlesh GOORAH wrote: > Therefore, I'm announcing the official EOF of Kadischi. It is sad, but > it is _not_ the end of creation of custom spins at the Fedora Project Kadischi, flew to close to the bright lights which is the fedoraproject a no-no if your not "insert-big-name at redhat.com" so as the winds and whims changed @redhat kadischi dropped like a stone. Revisor, was not developed anywhere near this list and hence stands on its own feet, a lesson perhaps. ...dex --- "In the FOSS world code is currency" dexter-2007 "If you aint got the green you cant make the scene" the-mask --- From sundaram at fedoraproject.org Sun Jul 8 10:54:44 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Sun, 08 Jul 2007 16:24:44 +0530 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <200707081140.51500.ddmbox2000@yahoo.co.uk> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> Message-ID: <4690C274.3020008@fedoraproject.org> dexter wrote: > On Sun July 8 2007 00:40:55 Chitlesh GOORAH wrote: >> Therefore, I'm announcing the official EOF of Kadischi. It is sad, but >> it is _not_ the end of creation of custom spins at the Fedora Project > > Kadischi, flew to close to the bright lights which is the fedoraproject a > no-no if your not "insert-big-name at redhat.com" so as the winds and whims > changed @redhat kadischi dropped like a stone. > Revisor, was not developed anywhere near this list and hence stands on its own > feet, a lesson perhaps. > What exactly are you trying to imply? Rahul From chitlesh at fedoraproject.org Sun Jul 8 11:23:07 2007 From: chitlesh at fedoraproject.org (Chitlesh GOORAH) Date: Sun, 8 Jul 2007 13:23:07 +0200 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <4690C274.3020008@fedoraproject.org> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> <4690C274.3020008@fedoraproject.org> Message-ID: <13dbfe4f0707080423n25aa3987n3aa4a46d56a4f05d@mail.gmail.com> > dexter wrote: > > Kadischi, flew to close to the bright lights which is the fedoraproject a > > no-no if your not "insert-big-name at redhat.com" so as the winds and whims > > changed @redhat kadischi dropped like a stone. > > Revisor, was not developed anywhere near this list and hence stands on its own > > feet, a lesson perhaps. > > On 7/8/07, Rahul Sundaram wrote: > What exactly are you trying to imply? I didn't understand what you wanted to imply too. However I feel, this is another stupid RH/fedora troll. By the way, perhaps I should make it clear. Kadischi hasn't been dropped, rather no one is working on kadischi. Dexter, if you want to push kadischi forward, feel free. But, you do understand not to divide forces, don't you ? There is no one to blame for kadischi's state. By the way, ask yourself "who is losing when kadischi is EOL'ed?" Believe me, no one. Revisor is already around with several new features which kadischi doesn't have. New features will keep on popping. Revisor is simple to use and installed. There are people who will answer your revisor bug reports. kadischi was just a living dead during the last months and during the last 2 fedora releases we were just doing bug fixing on users' requests and Jasper was coding from time to time. chitlesh PS: I'm @fedoraproject -- http://clunixchit.blogspot.com From jkeating at redhat.com Sun Jul 8 11:34:43 2007 From: jkeating at redhat.com (Jesse Keating) Date: Sun, 8 Jul 2007 07:34:43 -0400 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <200707081140.51500.ddmbox2000@yahoo.co.uk> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> Message-ID: <200707080734.43388.jkeating@redhat.com> On Sunday 08 July 2007 06:40:50 dexter wrote: > Revisor, was not developed anywhere near this list and hence stands on its > own feet, a lesson perhaps. Revisor, which is a wrapper for livecd-tools which /was/ developed and discussed on this list for Fedora, after proving a viable concept in OLPC land. -- Jesse Keating Release Engineer: Fedora -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From kanarip at kanarip.com Sun Jul 8 11:55:48 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Sun, 08 Jul 2007 13:55:48 +0200 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <4690A1B0.3060305@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> Message-ID: <4690D0C4.1010902@kanarip.com> Douglas McClendon wrote: > Douglas McClendon wrote: > >> >> Theoretical Workaround #1-5 >> ------------------------- Just curious; I like what I'm reading, although I do not follow some parts of the theoretical workarounds, but: Is there any particular use case for all of this? I'm asking, because the only thing I can think of, it's all this work being done to prevent a user's "5-min-avg-uptime" desktop from having to reboot after install from a live disc. Kind regards, Jeroen van Meeuwen -kanarip From kanarip at kanarip.com Sun Jul 8 12:17:09 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Sun, 08 Jul 2007 14:17:09 +0200 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <200707080734.43388.jkeating@redhat.com> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> <200707080734.43388.jkeating@redhat.com> Message-ID: <4690D5C5.6010509@kanarip.com> Jesse Keating wrote: > On Sunday 08 July 2007 06:40:50 dexter wrote: >> Revisor, was not developed anywhere near this list and hence stands on its >> own feet, a lesson perhaps. > > Revisor, which is a wrapper for livecd-tools which /was/ developed and > discussed on this list for Fedora, after proving a viable concept in OLPC > land. > +1, all we do is what livecd-tools does, we just provide the packageSack and transaction info -yum instance- after they have been created and populated with info the user provided, and work around some of the issues that not being able to "import livecd-creator" has. Kind regards, Jeroen van Meeuwen -kanarip From ddmbox2000 at yahoo.co.uk Sun Jul 8 12:28:33 2007 From: ddmbox2000 at yahoo.co.uk (dexter) Date: Sun, 8 Jul 2007 13:28:33 +0100 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <13dbfe4f0707080423n25aa3987n3aa4a46d56a4f05d@mail.gmail.com> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <4690C274.3020008@fedoraproject.org> <13dbfe4f0707080423n25aa3987n3aa4a46d56a4f05d@mail.gmail.com> Message-ID: <200707081328.33867.ddmbox2000@yahoo.co.uk> On Sun July 8 2007 12:23:07 Chitlesh GOORAH wrote: > However I feel, this > is another stupid RH/fedora troll. Stupidity -DONT-FEED-THE-TROLLS- ...dex From ddmbox2000 at yahoo.co.uk Sun Jul 8 12:29:20 2007 From: ddmbox2000 at yahoo.co.uk (dexter) Date: Sun, 8 Jul 2007 13:29:20 +0100 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <4690C274.3020008@fedoraproject.org> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> <4690C274.3020008@fedoraproject.org> Message-ID: <200707081329.20810.ddmbox2000@yahoo.co.uk> On Sun July 8 2007 11:54:44 Rahul Sundaram wrote: > What exactly are you trying to imply? > > Rahul Community lead projects should not exist for the sole purpose of getting fedoraproject blessing, which is what happend here fedora picks livecd-tools kadischi dies a slow deaf. ...dex From chitlesh at fedoraproject.org Sun Jul 8 12:39:17 2007 From: chitlesh at fedoraproject.org (Chitlesh GOORAH) Date: Sun, 8 Jul 2007 14:39:17 +0200 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <200707081329.20810.ddmbox2000@yahoo.co.uk> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> <4690C274.3020008@fedoraproject.org> <200707081329.20810.ddmbox2000@yahoo.co.uk> Message-ID: <13dbfe4f0707080539i3c43719cv24a046635d3b8eb7@mail.gmail.com> On 7/8/07, dexter wrote: > Community lead projects should not exist for the sole purpose of getting > fedoraproject blessing, which is what happend here fedora picks livecd-tools > kadischi dies a slow deaf. But you are conscious that a real "community lead project" (as you said) should have: * have enough community contributors * release often release early (under the fedora umbrella) * be ambitious and have a concrete (perhaps long term) goal * (the list goes on) The kadischi project doesn't have any of these, sadly. If you were on this list since december, you should already guess why there are revisor&friends though kadischi was present. Chitlesh -- http://clunixchit.blogspot.com From sundaram at fedoraproject.org Sun Jul 8 14:21:49 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Sun, 08 Jul 2007 19:51:49 +0530 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <200707081329.20810.ddmbox2000@yahoo.co.uk> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> <4690C274.3020008@fedoraproject.org> <200707081329.20810.ddmbox2000@yahoo.co.uk> Message-ID: <4690F2FD.6010803@fedoraproject.org> dexter wrote: > On Sun July 8 2007 11:54:44 Rahul Sundaram wrote: >> What exactly are you trying to imply? >> >> Rahul > > Community lead projects should not exist for the sole purpose of getting > fedoraproject blessing, which is what happend here fedora picks livecd-tools > kadischi dies a slow deaf. There might be a number of tools for any particular purpose and we need to pick one that has active development behind it. Regardless of that, a project with no contributors is going to die a slow death anyway. Rahul From gdk at redhat.com Sun Jul 8 14:18:12 2007 From: gdk at redhat.com (Greg Dekoenigsberg) Date: Sun, 8 Jul 2007 10:18:12 -0400 (EDT) Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <4690F2FD.6010803@fedoraproject.org> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707081140.51500.ddmbox2000@yahoo.co.uk> <4690C274.3020008@fedoraproject.org> <200707081329.20810.ddmbox2000@yahoo.co.uk> <4690F2FD.6010803@fedoraproject.org> Message-ID: On Sun, 8 Jul 2007, Rahul Sundaram wrote: > dexter wrote: >> On Sun July 8 2007 11:54:44 Rahul Sundaram wrote: >>> What exactly are you trying to imply? >>> >>> Rahul >> >> Community lead projects should not exist for the sole purpose of getting >> fedoraproject blessing, which is what happend here fedora picks >> livecd-tools kadischi dies a slow deaf. > > There might be a number of tools for any particular purpose and we need to > pick one that has active development behind it. Regardless of that, a project > with no contributors is going to die a slow death anyway. As the guy who first sponsored Kadischi as a Summer of Code project, I'd like to thank all of the people who worked on Kadischi to make it as good as it was. Sometimes a project can be successful not just from a code perspective, but from what it leads to. Kadischi was the first draft of Live CD tools, and was directly responsible for (a) the realization that we *badly needed* some help from Red Hat hands; and (b) the notion that the Live CD should be a turnkey experience, which led directly to Revisor. Without Kadischi, and the people who worked on it, none of the rest would have happened. We wouldn't have a LiveCD creator, and we wouldn't have a variety of Fedora 7 LiveCDs, and we wouldn't have Revisor. >From my perspective, Kadischi was a dramatic success. But now there's a codebase that's better integrated into the whole Fedora build process, and better maintained by people who can devote a lot of time to it. So the new codebase will be what lives on. That's how open source is supposed to work, isn't it? --g -- Greg DeKoenigsberg Community Development Manager Red Hat, Inc. :: 1-919-754-4255 "To whomsoever much hath been given... ...from him much shall be asked" From dmc.fedora at filteredperception.org Sun Jul 8 14:43:31 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 09:43:31 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <4690D0C4.1010902@kanarip.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> Message-ID: <4690F813.9050904@filteredperception.org> Jeroen van Meeuwen wrote: > Douglas McClendon wrote: >> Douglas McClendon wrote: >> >>> >>> Theoretical Workaround #1-5 >>> ------------------------- > > Just curious; I like what I'm reading, although I do not follow some > parts of the theoretical workarounds, but: Is there any particular use > case for all of this? I'm asking, because the only thing I can think of, > it's all this work being done to prevent a user's "5-min-avg-uptime" > desktop from having to reboot after install from a live disc. Short answer - dunno. I think however "5-min-avg-uptime" doesn't quite describe it. The essence of why liveCDs are cool(==useful?) in the first place, is because they allow users to try out the complete system, without the traditionally complex and problematic process of installing and configuring a linux system. I'm not sure if in 2001 I or Mr. Knopper could really have foreseen the plethora of use-cases for liveCDs as they are used today. In a similar vein, I would say that, perhaps this feature, which I'm just working on out of pure spite for "unnecessary reboots" will spark someone else's imagination, and use-cases will become more evident in the future. Don't even get me started about seeing setroubleshoot suggesting that I reboot my system to fix a problem... Also, as to actual use-cases, clearly persistance-to-USB flash as already described in the livecd-tools wish-list is _vastly_ more useful than all this hackery. But as it turns out, implementing that involves mechanisms of the same nature as these hacks. I'll segway that to... Theoretical Alternate Implementation #1 --------------------------------------- Utilize Mark McCloughlin's dm-snapshot merging patch alluded to here - http://fedoraproject.org/wiki/StatelessLinuxCachedClient That page, along with this one- http://linuxgazette.net/114/kapil.html should tie your brain in even more knots ;) Maybe by the time you've untied, you'll have stumbled across some important use-cases that no one has considered yet, because they didn't know it was possible. -dmc/jdog From jkeating at redhat.com Sun Jul 8 17:45:07 2007 From: jkeating at redhat.com (Jesse Keating) Date: Sun, 8 Jul 2007 13:45:07 -0400 Subject: [Fedora-livecd-list] Kadischi reached End Of Life In-Reply-To: <4690D5C5.6010509@kanarip.com> References: <13dbfe4f0707071640u60d9f0c5vbe17a1f2ad8843f0@mail.gmail.com> <200707080734.43388.jkeating@redhat.com> <4690D5C5.6010509@kanarip.com> Message-ID: <200707081345.10346.jkeating@redhat.com> On Sunday 08 July 2007 08:17:09 Jeroen van Meeuwen wrote: > all we do is what livecd-tools does, we just provide the packageSack and > transaction info -yum instance- after they have been created and > populated with info the user provided, and work around some of the > issues that not being able to "import livecd-creator" has. Which I don't mean to indicate is any small feat (: Putting an easy to use graphical face on livecd-tools is indeed great work and a boon for users. -- Jesse Keating Release Engineer: Fedora -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From dmc.fedora at filteredperception.org Sun Jul 8 19:40:01 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 14:40:01 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <4690D0C4.1010902@kanarip.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> Message-ID: <46913D91.2040803@filteredperception.org> Jeroen van Meeuwen wrote: > Douglas McClendon wrote: >> Douglas McClendon wrote: >> >>> >>> Theoretical Workaround #1-5 >>> ------------------------- > > Just curious; I like what I'm reading, although I do not follow some > parts of the theoretical workarounds, but: Is there any particular use > case for all of this? I'm asking, because the only thing I can think of, > it's all this work being done to prevent a user's "5-min-avg-uptime" > desktop from having to reboot after install from a live disc. More specifically, as par my other reply- One possible implementation of USB-flash persistance involves the same mechanisms vis-a-vis- You might want to make the snapshot overlay device begin its life as a broken mirror or extra-dm-layer device, so that the user may _optionally_ at some point during the life-cycle of the boot session, choose to live-migrate their system changes to a usb-flash device. -dmc/jdog From kanarip at kanarip.com Sun Jul 8 20:15:00 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Sun, 08 Jul 2007 22:15:00 +0200 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <4690F813.9050904@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> Message-ID: <469145C4.1070400@kanarip.com> Douglas McClendon wrote: > Jeroen van Meeuwen wrote: >> Douglas McClendon wrote: >>> Douglas McClendon wrote: >>> >>>> >>>> Theoretical Workaround #1-5 >>>> ------------------------- >> >> Just curious; I like what I'm reading, although I do not follow some >> parts of the theoretical workarounds, but: Is there any particular use >> case for all of this? I'm asking, because the only thing I can think >> of, it's all this work being done to prevent a user's >> "5-min-avg-uptime" desktop from having to reboot after install from a >> live disc. > > Short answer - dunno. I think however "5-min-avg-uptime" doesn't quite > describe it. It wasn't the essence of the point I wanted to make anyway, but I sure hope you get what I mean. User desktops usually do not require an uptime as long as humanly/physically possible -although long uptimes are a feature! it indicates the operating system's stability, including applications that freeze up the system (in which most cases a user would reset the system). The essence of why liveCDs are cool(==useful?) in the > first place, is because they allow users to try out the complete system, > without the traditionally complex and problematic process of installing > and configuring a linux system. I'm not sure if in 2001 I or Mr. > Knopper could really have foreseen the plethora of use-cases for liveCDs > as they are used today. On the other hand, installing Linux isn't that difficult at all, nowadays. I'm sure though this use case (if any) has not seen the day of light in the old knoppix days ;-) > > In a similar vein, I would say that, perhaps this feature, which I'm > just working on out of pure spite for "unnecessary reboots" will spark > someone else's imagination, and use-cases will become more evident in > the future. Don't even get me started about seeing setroubleshoot > suggesting that I reboot my system to fix a problem... > I'm sure there's /a/ use-case for this technical advancement. Besides that, I'm all for any technical advancement -whether it has actual use-cases or not. > Also, as to actual use-cases, clearly persistance-to-USB flash as > already described in the livecd-tools wish-list is _vastly_ more useful > than all this hackery. But as it turns out, implementing that involves > mechanisms of the same nature as these hacks. I'll segway that to... > I"m not sure I understand this one. One of the features of livecd-tools I presume is to just -persistently- commit changes to the USB devices' filesytem. If also committing the running system to the actual hardware is one of it's goals too, it should be a different goal altogether. Either way, +1 to the overall idea. Kind regards, Jeroen van Meeuwen -kanarip From kanarip at kanarip.com Sun Jul 8 20:17:24 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Sun, 08 Jul 2007 22:17:24 +0200 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <46913D91.2040803@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <46913D91.2040803@filteredperception.org> Message-ID: <46914654.3040708@kanarip.com> Douglas McClendon wrote: > Jeroen van Meeuwen wrote: >> Just curious; I like what I'm reading, although I do not follow some >> parts of the theoretical workarounds, but: Is there any particular use >> case for all of this? I'm asking, because the only thing I can think >> of, it's all this work being done to prevent a user's >> "5-min-avg-uptime" desktop from having to reboot after install from a >> live disc. > > More specifically, as par my other reply- > > One possible implementation of USB-flash persistance involves the same > mechanisms vis-a-vis- > > You might want to make the snapshot overlay device begin its life as a > broken > mirror or extra-dm-layer device, so that the user may _optionally_ at > some point > during the life-cycle of the boot session, choose to live-migrate their > system > changes to a usb-flash device. > Nice one! This use-case I recognise. I think, technically, the same techniques could be used, and this is the make-my-system-live feature we (Revisor) want. Great idea! Kind regards, Jeroen van Meeuwen -kanarip From dmc.fedora at filteredperception.org Sun Jul 8 21:03:02 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 16:03:02 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <469145C4.1070400@kanarip.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> Message-ID: <46915106.6090402@filteredperception.org> Jeroen van Meeuwen wrote: > Douglas McClendon wrote: >> Jeroen van Meeuwen wrote: > > The essence of why liveCDs are cool(==useful?) in the >> first place, is because they allow users to try out the complete system, >> without the traditionally complex and problematic process of installing >> and configuring a linux system. I'm not sure if in 2001 I or Mr. >> Knopper could really have foreseen the plethora of use-cases for liveCDs >> as they are used today. > > On the other hand, installing Linux isn't that difficult at all, > nowadays. I'm sure though this use case (if any) has not seen the day of > light in the old knoppix days ;-) I would disagree, though perhaps by making a slightly different point- The key livecd use case is when the user *isn't sure* if they want to install. I.e. installing linux absolutely is still a massive pain in the ass - on some hardware configurations. Perhaps the quintessential livecd use case is verifying that a particular version of a particular distribution fully supports your particular hardware configuration, without guru-level patching/driver-updating/sysadmin knowledge. This has nothing to do with the rebootless installer idea, but I think it counters your criticism of the use case I mentioned. > >> In a similar vein, I would say that, perhaps this feature, which I'm >> just working on out of pure spite for "unnecessary reboots" will spark >> someone else's imagination, and use-cases will become more evident in >> the future. Don't even get me started about seeing setroubleshoot >> suggesting that I reboot my system to fix a problem... >> > > I'm sure there's /a/ use-case for this technical advancement. Besides > that, I'm all for any technical advancement -whether it has actual > use-cases or not. /The/ use case is simple. It does what it does. It is what it is. Whether or not it is well received once it is an available option, remains to be seen. Note, that it is trivial to enhance the existing patch such that things only happen differently than normal if you pass 'support_rebootless' on the kernel commandline. Also note that it is fairly trivial to add an option so that the user can chose whether to live-migrate the cdrom+ram_overlay onto the destination volume, or just the cdrom. In the latter case, the user could still eject the cdrom when installation is complete and continue working in the live system, though on reboot it would forget all the live-session modifications and give you a fresh start. -dmc/jdog From dmc.fedora at filteredperception.org Sun Jul 8 21:13:27 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 16:13:27 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <46915106.6090402@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> Message-ID: <46915377.1000902@filteredperception.org> Douglas McClendon wrote: > Jeroen van Meeuwen wrote: >>> In a similar vein, I would say that, perhaps this feature, which I'm >>> just working on out of pure spite for "unnecessary reboots" will spark >>> someone else's imagination, and use-cases will become more evident in >>> the future. Don't even get me started about seeing setroubleshoot >>> suggesting that I reboot my system to fix a problem... >>> >> >> I'm sure there's /a/ use-case for this technical advancement. Besides >> that, I'm all for any technical advancement -whether it has actual >> use-cases or not. > > > /The/ use case is simple. It does what it does. It is what it is. > Whether or not it is well received once it is an available option, > remains to be seen. Note, that it is trivial to enhance the existing > patch such that things only happen differently than normal if you pass > 'support_rebootless' on the kernel commandline. > > Also note that it is fairly trivial to add an option so that the user > can chose whether to live-migrate the cdrom+ram_overlay onto the > destination volume, or just the cdrom. In the latter case, the user > could still eject the cdrom when installation is complete and continue > working in the live system, though on reboot it would forget all the > live-session modifications and give you a fresh start. Continuing this thought- if MarkMC's dm-snapshot-merging patch was in the kernel, there could be a button in the live session, such that at any point _long after_ the installation, the user could _opt_ to 'fold in' their live-session modification. (i.e. hitting the button triggers a dm-snapshot-merge of the tmpfs overlay into the install-destination-drive). -dmc/jdog From dmc.fedora at filteredperception.org Sun Jul 8 21:22:35 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 16:22:35 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <46915106.6090402@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> Message-ID: <4691559B.5080100@filteredperception.org> Douglas McClendon wrote: > Jeroen van Meeuwen wrote: >> Douglas McClendon wrote: >> I'm sure there's /a/ use-case for this technical advancement. Besides >> that, I'm all for any technical advancement -whether it has actual >> use-cases or not. > > > /The/ use case is simple. It does what it does. It is what it is. > Whether or not it is well received once it is an available option, > remains to be seen. Note, that it is trivial to enhance the existing > patch such that things only happen differently than normal if you pass > 'support_rebootless' on the kernel commandline. > > Also note that it is fairly trivial to add an option so that the user > can chose whether to live-migrate the cdrom+ram_overlay onto the > destination volume, or just the cdrom. In the latter case, the user > could still eject the cdrom when installation is complete and continue > working in the live system, though on reboot it would forget all the > live-session modifications and give you a fresh start. A different continuation of this thought, is the modification to support the iso-caching-to-disk functionality I described in my 2001 project. Though now, instead of vfat, use ntfs-3g. I.e. do your live migration to a file on an ntfs partition, such that after the migration a) the user can eject the cdrom and use the cdrom drive for other reasons b) the system will be much more responsive as data is pulled from the ntfs rather than the cdrom c) with a sufficiently modern (i.e. probably doesn't exist yet) version of grub or loadlin, the live environment could be subsequently booted strait from ntfs, without needing the original livecd. -dmc/jdog From tim.wood at datawranglers.com Sun Jul 8 21:09:58 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Sun, 8 Jul 2007 15:09:58 -0600 (MDT) Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <46915377.1000902@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> Message-ID: <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> > Continuing this thought- if MarkMC's dm-snapshot-merging patch was in the > kernel, there could be a button in the live session, such that at any > point > _long after_ the installation, the user could _opt_ to 'fold in' their > live-session modification. (i.e. hitting the button triggers a > dm-snapshot-merge of the tmpfs overlay into the > install-destination-drive). Hmmm... I'm going to play devil's advocate. The standard linux boot process --at least with Redhat, CentOS and SUSE-- is for a minimal Kernel to start that can just do enough to mount a real filesystem and load the real kernel. It then hands over control to the real kernel and dies. Until this post I was assuming --silly me :-)-- that this discussion was about something similar. Since the cluebat didn't get me on this one, can someone explain the difference? On another front, my two primary use cases for live media are: 1) Handling specific tasks I can't easily handle with one of my existing systems. Insert 10,000 versions of I saved my computer/drive/whatever here. I really don't like Ubuntu _except_ that it handles Mac OS X's hfs filesystem brilliantly. I don't want to run Knoppix on a daily basis but it's impressive what I've done to salvage stuff off a hd/floppy/whatever that someone's Windows box wouldn't recognize. 2) Quick and Dirty dedicated systems. I've created a number of custom live CDs in situations where someone wanted a server to do something fairly specific in a hostile environment. For instance, something to handle basic logins and print serving for a windows computer lab. Because the vast majority of the stuff is on non-modifiable media, it almost doesn't matter what a malicious user does, the staff can reboot. The first case is handled nicely by existing solutions. The second one is a brilliant place for Revisor. Now that I've gotten something of a handle on Revisor, I believe it's going to save me a lot of time fighting Morphix and Knoppix. The thought of using a respin approach to mix and match from previous live cds/cf cards is very very appealing. From dmc.fedora at filteredperception.org Sun Jul 8 22:12:07 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 17:12:07 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> Message-ID: <46916137.7080705@filteredperception.org> Tim Wood wrote: >> Continuing this thought- if MarkMC's dm-snapshot-merging patch was in the >> kernel, there could be a button in the live session, such that at any >> point >> _long after_ the installation, the user could _opt_ to 'fold in' their >> live-session modification. (i.e. hitting the button triggers a >> dm-snapshot-merge of the tmpfs overlay into the >> install-destination-drive). > > Hmmm... I'm going to play devil's advocate. The standard linux boot > process --at least with Redhat, CentOS and SUSE-- is for a minimal Kernel > to start that can just do enough to mount a real filesystem and load the > real kernel. It then hands over control to the real kernel and dies. > Until this post I was assuming --silly me :-)-- that this discussion was > about something similar. Since the cluebat didn't get me on this one, can > someone explain the difference? Maybe someone needs to hit me with a cluebat if you are right. From my understanding, you are confusing root filesystem with kernel. Unless of course grub these days is based on an embedded linux kernel (which for all I know at this instant about grub internals, it might be). I.e. the standard linux boot process, for redhat and centos (I haven't played with suse recently), boots grub, which then pulls /the/ kernel from /boot/vmlinuz...., and starts with an initial root filesystem from data stored in a gzipped compressed cpio (i.e. /boot/initrd-....). That initial root filesystem has an /init or /sbin/init, which these days is a shell script, which handles the process of locating and mounting the real (or shall we just say, the next) root filesystem, and then switching to use that. The switch involves something called pivot_root, which does not involve the kernel dying at any point. There is of course kexec, which could be used to do what you described. If in fact you are correct, that is happening tucked away somewhere that I'm oblivious to. (I kind of doubt it, but I haven't looked at the nash source code recently. nash being the special shell used in internal initrd scripts). > > On another front, my two primary use cases for live media are: > > 1) Handling specific tasks I can't easily handle with one of my existing > systems. Insert 10,000 versions of I saved my computer/drive/whatever > here. I really don't like Ubuntu _except_ that it handles Mac OS X's hfs > filesystem brilliantly. I don't want to run Knoppix on a daily basis but > it's impressive what I've done to salvage stuff off a hd/floppy/whatever > that someone's Windows box wouldn't recognize. > > 2) Quick and Dirty dedicated systems. I've created a number of custom > live CDs in situations where someone wanted a server to do something > fairly specific in a hostile environment. For instance, something to > handle basic logins and print serving for a windows computer lab. Because > the vast majority of the stuff is on non-modifiable media, it almost > doesn't matter what a malicious user does, the staff can reboot. > > The first case is handled nicely by existing solutions. The second one is > a brilliant place for Revisor. Now that I've gotten something of a handle > on Revisor, I believe it's going to save me a lot of time fighting Morphix > and Knoppix. The thought of using a respin approach to mix and match from > previous live cds/cf cards is very very appealing. I completely agree. Once we have a library of dozens if not hundreds of recipes/kickstarts that can be trivially spun to .iso (via one simple commandline or gui command), I would expect all these dozens/hundreds of hand-built non-errata-updated unmaintainable custom livecd distros to totally lose popularity and disappear from significant use. I think it ought to be possible to just walk down the list of 500+ distributions on http://lwn.net/Distributions, and start with the most popular livecds, and make kickstarts/recipes for them (equivalent functionality). Then, they could be automatically respun weekly on headless build servers, and you could always get one with up-to-date security/bugfix updates. Or just spun as needed by the user if they have the livecd-tools/revisor installed and working. Total World(livecd) Domination, BwaHaHa... -dmc/jdog From dmc.fedora at filteredperception.org Sun Jul 8 22:27:31 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 17:27:31 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> Message-ID: <469164D3.2040000@filteredperception.org> Tim Wood wrote: >> Continuing this thought- if MarkMC's dm-snapshot-merging patch was in the >> kernel, there could be a button in the live session, such that at any >> point >> _long after_ the installation, the user could _opt_ to 'fold in' their >> live-session modification. (i.e. hitting the button triggers a >> dm-snapshot-merge of the tmpfs overlay into the >> install-destination-drive). > > Hmmm... I'm going to play devil's advocate. The standard linux boot > process --at least with Redhat, CentOS and SUSE-- is for a minimal Kernel > to start that can just do enough to mount a real filesystem and load the > real kernel. It then hands over control to the real kernel and dies. > Until this post I was assuming --silly me :-)-- that this discussion was > about something similar. Since the cluebat didn't get me on this one, can > someone explain the difference? Actually, what you are describing is the process of booting a _hibernated/suspended_ linux system. Which is really quite orthogonal to everything I have been discussing. (he says with fingers crossed wondering how a rebootlessly installed system will handle hibernating (before itss first reboot)... I think my brain will explode if I try to think about that before just seeing if it works without a problem) -dmc/jdog From tim.wood at datawranglers.com Mon Jul 9 00:34:09 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Sun, 8 Jul 2007 18:34:09 -0600 (MDT) Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <469164D3.2040000@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> <469164D3.2040000@filteredperception.org> Message-ID: <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> Before shooting from the hip anymore, I'll dig into my RHCT/CLP stuff and see what I dig out. Homework... :-( regards, Tim From dmc.fedora at filteredperception.org Mon Jul 9 02:51:34 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 21:51:34 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> <469164D3.2040000@filteredperception.org> <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> Message-ID: <4691A2B6.4030309@filteredperception.org> Tim Wood wrote: > Before shooting from the hip anymore, I'll dig into my RHCT/CLP stuff and > see what I dig out. Homework... :-( Just because I like to talk/type a lot... I'll go ahead and instead of focusing on correcting you, try to actually answer your question. Though first I'll say that my response about hibernated systems was a shot from the hip which is technically wrong and more likely to confuse than clarify. Here is an attempt to outline what is going on with the rebootless installer idea, versus the fedora-7 official livecd, versus a normal installed fedora system. Normal Fedora 7 Installed On Hard Disk System (assume just 1 non lvm partition /dev/sda1) ---------------------------------------------- -The bios loads the grub boot loader from the MBR of /dev/sda -grub knows how to read it's config from /dev/sda1:/boot/grub/grub.conf -grub is configured to boot a specific kernel+ramdisk+appendstring, namely /dev/sda1:/boot/vmlinuz-someversion, /dev/sda1:/boot/initrd-someversion, someargs... -control is thusly passed to the kernel, and the kernel then gunzips and extracts the cpio of the specified initrd (which I think grub copied to a well known place in ram. Only reason I might know this is because qemu's crafty -initrd feature screwed it up for larger initrds recently, though it has been fixed) -the kernel having then extracted the contents into a ram based filesystem, passes control to /init (or maybe /sbin/init, or maybe whatever init= was specified on the cmdline). -now the fun starts. This init is a nash or a bash script, whose job it is to mount the real root filesystem (e.g. the ext3fs on /dev/sda1) and then pivot_root to it. -finally, control is passed to /sbin/init on the real disk-based root filesystem, at which point the contents of the well known /etc/inittab start to matter. Now then, what the Fedora-7 livecd does is along these lines ------------------------------------------------------------ -instead of the bios booting grub loaded on the mbr of a disk, it boots grub(or perhaps isolinux) from the bootsector of the cdrom. -this bootloader behaves much like above, but pulling a kernel+initrd+append_args from some place on the cdrom. -now the fun begins after the initrd is extracted and mounted in a ram based fs as normal. An entirely different /init script within the initrd will go about the business of mounting the 'real root filesystem'. In this case, first the cdrom's iso9660 fs is mounted. Then a squashfs image file is loopback mounted from within the iso9660 fs. Then a sparse ext3fs image file is loopback mounted from within the squashfs. - now the REAL fun begins. A ram based filesystem is created. A sparse file overlay is created within it. Now a device mapper snapshot is created using the read-only ext3fs image, and the read-write overlay file. (I'm skipping some loopback device associations, and in general probably misnaming a few things, as this is unashamedly from the hip, and not suitable to be published). Now, this magic devicemapper snapshotted device appears as /dev/mapper/live-rw, and appears to be a read-write ext3 filesystem, except the writes really get tucked away in ram (which is going to eat away at your ram). - That /dev/mapper/live-rw gets mounted as the 'real root filesystem', and pivot_root is called, and then things progress as normal. Now then- what the rebootless installer patch does differently -------------------------------------------------------------- -Just after the /dev/mapper/live-rw gets set up in the initrd, instead of mounting it as the real root filesystem, it gets used to create a raid1 'mirror'. quotes because in this case the 'mirror' only has 1 device, rather than the usual 2. The mirror is visible as /dev/md7, and THAT gets mounted as the real root filesystem, before pivot_root is called, and everything progresses as normal. But because of /dev/md7 being the 'real root filesystem', long after boot, you can hot-add another device to the mirror, in this case, the target volume that you want to install the system on (e.g. /dev/sda1). After you hot-add, the raid/md driver starts synchronizing the data from /dev/mapper/live-rw to /dev/sda1. When this finishes, you can hot-remove /dev/mapper/live-rw, at which point the system is running from /dev/sda1, just as if you had installed there and rebooted (with the caviat that there is this /dev/md7 layer sitting there until the next reboot). And once /dev/mapper/live-rw is removed from the /dev/md7 array, the resources that constructed it (i.e. the files on the cdrom, and that overlay file in a ram-based fs) can be released/deleted/unmounted. Thus you stop suffering the penalty of that overlay eating up your ram, and you are free to eject the cdrom. Now, for the sake of simplicity, we will assume that "mdadm /dev/md7 --grow --size=max" will actually work, and the 3.5G ext3fs that got migrated to your 100G /dev/sda1 partition, can be grown with resize2fs's ability to online expand a live ext3 filesystem. (technically this does not yet work, so refer to all those nasty workarounds). Ideally, in the coming days/weeks/months/years I'll be able to publish far better patches that far more easily demonstrate the advantage of what is going on here (though the basic concept is simple- no reboot after install). Maybe if people actually start using and liking this feature, I'll turn all of this into a nice whitepaper. -dmc From dmc.fedora at filteredperception.org Mon Jul 9 04:15:00 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sun, 08 Jul 2007 23:15:00 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> <469164D3.2040000@filteredperception.org> <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> Message-ID: <4691B644.9080407@filteredperception.org> Tim Wood wrote: > Before shooting from the hip anymore, I'll dig into my RHCT/CLP stuff and > see what I dig out. Homework... :-( So have you been trolling me? Was the reference to Suse a pointer to the fact that they have already done the rebootless install thing? (I'm too lazy to download to find out?) -dmc From jonathansteffan at gmail.com Mon Jul 9 08:26:06 2007 From: jonathansteffan at gmail.com (Jonathan Steffan) Date: Mon, 09 Jul 2007 02:26:06 -0600 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <4691A2B6.4030309@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> <469164D3.2040000@filteredperception.org> <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> <4691A2B6.4030309@filteredperception.org> Message-ID: <1183969566.5957.18.camel@damaestro> I have a use for something like this. On Sun, 2007-07-08 at 21:51 -0500, Douglas McClendon wrote: > Normal Fedora 7 Installed On Hard Disk System > (assume just 1 non lvm partition /dev/sda1) > ---------------------------------------------- > -The bios loads the grub boot loader from the MBR of /dev/sda > -grub knows how to read it's config from /dev/sda1:/boot/grub/grub.conf > -grub is configured to boot a specific kernel+ramdisk+appendstring, namely > /dev/sda1:/boot/vmlinuz-someversion, /dev/sda1:/boot/initrd-someversion, > someargs... > -control is thusly passed to the kernel, and the kernel then gunzips and > extracts the cpio of the specified initrd (which I think grub copied to a well > known place in ram. Only reason I might know this is because qemu's crafty > -initrd feature screwed it up for larger initrds recently, though it has been fixed) > -the kernel having then extracted the contents into a ram based filesystem, > passes control to /init (or maybe /sbin/init, or maybe whatever init= was > specified on the cmdline). > -now the fun starts. This init is a nash or a bash script, whose job it is to > mount the real root filesystem (e.g. the ext3fs on /dev/sda1) and then > pivot_root to it. > -finally, control is passed to /sbin/init on the real disk-based root > filesystem, at which point the contents of the well known /etc/inittab start to > matter. > > Now then, what the Fedora-7 livecd does is along these lines > ------------------------------------------------------------ > -instead of the bios booting grub loaded on the mbr of a disk, it boots grub(or > perhaps isolinux) from the bootsector of the cdrom. - We boot from a local flash, (random example: http://www.pcengines.ch/cf2g.htm) > -this bootloader behaves much like above, but pulling a > kernel+initrd+append_args from some place on the cdrom. s/cdrom/flash/ > -now the fun begins after the initrd is extracted and mounted in a ram based fs > as normal. An entirely different /init script within the initrd will go about > the business of mounting the 'real root filesystem'. In this case, first the > cdrom's iso9660 fs is mounted. Then a squashfs image file is loopback mounted > from within the iso9660 fs. Then a sparse ext3fs image file is loopback mounted > from within the squashfs. s/iso9660/whatever/ Before the squashfs is loop mounted, we do some checks on it... most likely a checksum of sorts. At this point we test if this is the latest image. If not, we pull the latest image to a ram drive and loop mount that instead of the local squashfs image. I have assumed we would be able to have some sort of network access at this stage. > - now the REAL fun begins. A ram based filesystem is created. A sparse file > overlay is created within it. Now a device mapper snapshot is created using the > read-only ext3fs image, and the read-write overlay file. (I'm skipping some > loopback device associations, and in general probably misnaming a few things, as > this is unashamedly from the hip, and not suitable to be published). Now, this > magic devicemapper snapshotted device appears as /dev/mapper/live-rw, and > appears to be a read-write ext3 filesystem, except the writes really get tucked > away in ram (which is going to eat away at your ram). > - That /dev/mapper/live-rw gets mounted as the 'real root filesystem', and > pivot_root is called, and then things progress as normal. If the squashfs image has not changed, we pivot_root right away. > > Now then- what the rebootless installer patch does differently > -------------------------------------------------------------- > -Just after the /dev/mapper/live-rw gets set up in the initrd, instead of > mounting it as the real root filesystem, it gets used to create a raid1 > 'mirror'. quotes because in this case the 'mirror' only has 1 device, rather > than the usual 2. The mirror is visible as /dev/md7, and THAT gets mounted as > the real root filesystem, before pivot_root is called, and everything progresses > as normal. Neat. Could we create the raid1 from a new loop mounted squashfs that has just been loaded into ram? > > But because of /dev/md7 being the 'real root filesystem', long after boot, you > can hot-add another device to the mirror, in this case, the target volume that > you want to install the system on (e.g. /dev/sda1). After you hot-add, the > raid/md driver starts synchronizing the data from /dev/mapper/live-rw to > /dev/sda1. I would want to sync back to the flash at this point. > When this finishes, you can hot-remove /dev/mapper/live-rw, at which > point the system is running from /dev/sda1, just as if you had installed there > and rebooted (with the caviat that there is this /dev/md7 layer sitting there > until the next reboot). And once /dev/mapper/live-rw is removed from the > /dev/md7 array, the resources that constructed it (i.e. the files on the cdrom, > and that overlay file in a ram-based fs) can be released/deleted/unmounted. > Thus you stop suffering the penalty of that overlay eating up your ram, and you > are free to eject the cdrom. > > Now, for the sake of simplicity, we will assume that "mdadm /dev/md7 --grow > --size=max" will actually work, and the 3.5G ext3fs that got migrated to your > 100G /dev/sda1 partition, can be grown with resize2fs's ability to online expand > a live ext3 filesystem. (technically this does not yet work, so refer to all > those nasty workarounds). > There is my incomplete 2am reply ;-) Jonathan Steffan daMaestro From dmc.fedora at filteredperception.org Mon Jul 9 09:47:05 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Mon, 09 Jul 2007 04:47:05 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <1183969566.5957.18.camel@damaestro> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> <469164D3.2040000@filteredperception.org> <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> <4691A2B6.4030309@filteredperception.org> <1183969566.5957.18.camel@damaestro> Message-ID: <46920419.4090404@filteredperception.org> Jonathan Steffan wrote: > There is my incomplete 2am reply ;-) I might try to reply (or even read your post), but I'm at my 4:44am and 1) theoretical #5 doesn't work, as sda1 can't be both part of /dev/md7 AND part of a devicemapper at the same time (even if both parts are working with different slices). (maybe with a loop partitioning kernel patch/feature?) 2) I think I just figured out the #*(@ undocumented dm mirror target, which if it works, was probably the obvious way to go in the first place... If I don't happen to actually try real usage of it tonight, and someone reads this and wants to comment on my theory of it... that would be nice... my theory for dm mirror migration: (I did do a trivial test which seemed to work) 1) create a linear table with the original device 2) do a reload with a mirror table with the orig device first and the new device second, then resume (this will??? work on a rootfs device?) 3) when sync is complete (oh yeah, use that flag in 2), do a reload with a linear table of the new device. Yeah that sure does sound simple, but could I have ever used a simple Documentation/device-mapper/mirror.txt file that described that example. (yes, if this works, I'll write the file and submit it) I believe that substituting this dm mirror migration for mdadm should work, and of course is just much better period end of story. Though http://sources.redhat.com/lvm2/wiki/RoadMap doesn't give me confidence seeing the entry- "pvmove, snapshot, mirror of root filesystem" -dmc From dmc.fedora at filteredperception.org Tue Jul 10 08:48:31 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 03:48:31 -0500 Subject: [Fedora-livecd-list] rebootless installer - doh! Message-ID: <469347DF.2090300@filteredperception.org> Well... Now that I understand dm-mirror (somewhat at least), it appears the patch to livecd-tools to support rebootless installation is 0 lines long. Hmm... Of course if it actually works, I'm still baffled as to why I haven't seen it as an advertised feature anywhere? Anybody care to hit me with the _metaphorical_ cluebat? Although by my understanding, you still would need to introduce a dm layer for the overlay file to support live-migration to usb-flash. Although maybe I'm just missing something obvious there as well... ? Also, if pvmove can work for this, somebody please let me know. I just don't have enough experience with lvm power usage to know if it is possible (it seems like from what I've read, pvmove is only good for devices which have been pvcreated, but quite possibly my understanding is incomplete). -dmc From dmc.fedora at filteredperception.org Tue Jul 10 09:07:50 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 04:07:50 -0500 Subject: [Fedora-livecd-list] rebootless installer - doh! In-Reply-To: <469347DF.2090300@filteredperception.org> References: <469347DF.2090300@filteredperception.org> Message-ID: <46934C66.9020305@filteredperception.org> Douglas McClendon wrote: > Well... Now that I understand dm-mirror (somewhat at least), it appears > the patch to livecd-tools to support rebootless installation is 0 lines > long. Hmm... Bahh... my brain is in knots. You can't actually nest device mapper entries in the mirror slots of a mirror device mapper entry... AFAIK -dmc From dmc.fedora at filteredperception.org Tue Jul 10 10:57:34 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 05:57:34 -0500 Subject: [Fedora-livecd-list] rebootless installer - BWAHAHAHAHA!!! In-Reply-To: <46934C66.9020305@filteredperception.org> References: <469347DF.2090300@filteredperception.org> <46934C66.9020305@filteredperception.org> Message-ID: <4693661E.9000109@filteredperception.org> Douglas McClendon wrote: > Douglas McClendon wrote: >> Well... Now that I understand dm-mirror (somewhat at least), it >> appears the patch to livecd-tools to support rebootless installation >> is 0 lines long. Hmm... > > Bahh... my brain is in knots. You can't actually nest device mapper > entries in the mirror slots of a mirror device mapper entry... AFAIK DEVICE MAPPER RULEZ!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I just did a REBOOTLESS INSTALL OF THE STOCK FEDORA 7 LIVECD!!!!!!!!!!!!!!!! Ok, so manually performing the mkinitrd that anaconda would have performed is giving me some trouble, but hey, it's 5am..., and I am 99% convinced that I can get it to work after I get some sleep. But hey... it works. Now I'll get to work on writing some serious patches :) -dmc From dmc.fedora at filteredperception.org Tue Jul 10 11:07:37 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 06:07:37 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency Message-ID: <46936879.50009@filteredperception.org> Question: Does the current livecd installer inefficiently write lots of 0's to the destination drive that it doesn't need to? I think it might. The os.img on the F7 livecd is a 4G sparse file with about 2.3G of data. Anaconda's livecdcopy backend uses python's os.read/write. I would guess that that means that 4G of data is getting written, when theoretically only 2.3G needs to. The solution that comes to mind is this- in livecd-tools, create the os.img as a 7G (or 700G??) sparse file. Basically just way big. Then take care to make the ext3fs be the exact correct size for the data (i.e. 2.3G). Then, in the initramfs, just after mounting it (after snapshotting it), do a resize2fs to 7G (or 700G). Then when anaconda does the copy, only copy the first 2.3G of the sparse file. It is however late in the 'day' for me, so maybe someone can chime in with confirmation or refutation of my logic here. -dmc From dmc.fedora at filteredperception.org Tue Jul 10 12:13:46 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 07:13:46 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <46936879.50009@filteredperception.org> References: <46936879.50009@filteredperception.org> Message-ID: <469377FA.2060800@filteredperception.org> Douglas McClendon wrote: > Question: Does the current livecd installer inefficiently write lots of > 0's to the destination drive that it doesn't need to? > > I think it might. The os.img on the F7 livecd is a 4G sparse file with > about 2.3G of data. Anaconda's livecdcopy backend uses python's > os.read/write. I would guess that that means that 4G of data is getting > written, when theoretically only 2.3G needs to. > > The solution that comes to mind is this- > > in livecd-tools, create the os.img as a 7G (or 700G??) sparse file. > Basically just way big. Then take care to make the ext3fs be the exact > correct size for the data (i.e. 2.3G). Then, in the initramfs, just > after mounting it (after snapshotting it), do a resize2fs to 7G (or 700G). To clarify a bit- Clearly the resize2fs should probably happen during boot (long after initramfs). No need to bloat the initramfs with resize2fs. Also, the mechanism that comes to mind for the ext3fs creation is this- Take the existing image built as is, but after final install, resize2fs it to the smallest possible (nearly), then truncate the file, then do the dd seek trick to re-sparsify it vastly larger. Or perhaps just throw in an entire extra tarcopy of the system to a new fs image file created the exact right size from the beginning. This is more work, but will possibly save space on any files that got created and deleted during the installation process. -dmc From dmc.fedora at filteredperception.org Tue Jul 10 12:20:19 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 07:20:19 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <469377FA.2060800@filteredperception.org> References: <46936879.50009@filteredperception.org> <469377FA.2060800@filteredperception.org> Message-ID: <46937983.10601@filteredperception.org> Douglas McClendon wrote: > Douglas McClendon wrote: >> Question: Does the current livecd installer inefficiently write lots >> of 0's to the destination drive that it doesn't need to? >> >> I think it might. The os.img on the F7 livecd is a 4G sparse file >> with about 2.3G of data. Anaconda's livecdcopy backend uses python's >> os.read/write. I would guess that that means that 4G of data is >> getting written, when theoretically only 2.3G needs to. >> >> The solution that comes to mind is this- >> >> in livecd-tools, create the os.img as a 7G (or 700G??) sparse file. >> Basically just way big. Then take care to make the ext3fs be the >> exact correct size for the data (i.e. 2.3G). Then, in the initramfs, >> just after mounting it (after snapshotting it), do a resize2fs to 7G >> (or 700G). > > To clarify a bit- Clearly the resize2fs should probably happen during > boot (long after initramfs). No need to bloat the initramfs with > resize2fs. > Yeah, I really gotta stop posting when I'm sleep deprived. Clearly in initramfs you have access to sysroot thus no bloat. and... > Also, the mechanism that comes to mind for the ext3fs creation is this- > > Take the existing image built as is, but after final install, resize2fs > it to the smallest possible (nearly), then truncate the file, then do > the dd seek trick to re-sparsify it vastly larger. > > Or perhaps just throw in an entire extra tarcopy of the system to a new > fs image file created the exact right size from the beginning. This is > more work, but will possibly save space on any files that got created > and deleted during the installation process. > clearly the resize2fs to minimal will take care of the created/deleted issue. -dmc From gdk at redhat.com Tue Jul 10 12:23:10 2007 From: gdk at redhat.com (Greg Dekoenigsberg) Date: Tue, 10 Jul 2007 08:23:10 -0400 (EDT) Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <46937983.10601@filteredperception.org> References: <46936879.50009@filteredperception.org> <469377FA.2060800@filteredperception.org> <46937983.10601@filteredperception.org> Message-ID: I see someone besides me uses mailing lists to keep public notes for themselves to remember later. :) --g On Tue, 10 Jul 2007, Douglas McClendon wrote: > Douglas McClendon wrote: >> Douglas McClendon wrote: >>> Question: Does the current livecd installer inefficiently write lots of >>> 0's to the destination drive that it doesn't need to? >>> >>> I think it might. The os.img on the F7 livecd is a 4G sparse file with >>> about 2.3G of data. Anaconda's livecdcopy backend uses python's >>> os.read/write. I would guess that that means that 4G of data is getting >>> written, when theoretically only 2.3G needs to. >>> >>> The solution that comes to mind is this- >>> >>> in livecd-tools, create the os.img as a 7G (or 700G??) sparse file. >>> Basically just way big. Then take care to make the ext3fs be the exact >>> correct size for the data (i.e. 2.3G). Then, in the initramfs, just after >>> mounting it (after snapshotting it), do a resize2fs to 7G (or 700G). >> >> To clarify a bit- Clearly the resize2fs should probably happen during boot >> (long after initramfs). No need to bloat the initramfs with resize2fs. >> > > Yeah, I really gotta stop posting when I'm sleep deprived. Clearly in > initramfs you have access to sysroot thus no bloat. > > and... > >> Also, the mechanism that comes to mind for the ext3fs creation is this- >> >> Take the existing image built as is, but after final install, resize2fs it >> to the smallest possible (nearly), then truncate the file, then do the dd >> seek trick to re-sparsify it vastly larger. >> >> Or perhaps just throw in an entire extra tarcopy of the system to a new fs >> image file created the exact right size from the beginning. This is more >> work, but will possibly save space on any files that got created and >> deleted during the installation process. >> > > clearly the resize2fs to minimal will take care of the created/deleted issue. > > -dmc > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > -- Greg DeKoenigsberg Community Development Manager Red Hat, Inc. :: 1-919-754-4255 "To whomsoever much hath been given... ...from him much shall be asked" From dmc.fedora at filteredperception.org Tue Jul 10 14:01:25 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 09:01:25 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <469377FA.2060800@filteredperception.org> References: <46936879.50009@filteredperception.org> <469377FA.2060800@filteredperception.org> Message-ID: <46939135.7040302@filteredperception.org> Douglas McClendon wrote: > Douglas McClendon wrote: >> Question: Does the current livecd installer inefficiently write lots >> of 0's to the destination drive that it doesn't need to? >> >> I think it might. The os.img on the F7 livecd is a 4G sparse file >> with about 2.3G of data. Anaconda's livecdcopy backend uses python's >> os.read/write. I would guess that that means that 4G of data is >> getting written, when theoretically only 2.3G needs to. >> >> The solution that comes to mind is this- >> >> in livecd-tools, create the os.img as a 7G (or 700G??) sparse file. >> Basically just way big. Then take care to make the ext3fs be the >> exact correct size for the data (i.e. 2.3G). Then, in the initramfs, >> just after mounting it (after snapshotting it), do a resize2fs to 7G >> (or 700G). > > To clarify a bit- Clearly the resize2fs should probably happen during > boot (long after initramfs). No need to bloat the initramfs with > resize2fs. Assuming my prior logic is valid, there is another aspect to consider- The above recommendation would add some time to every livecd boot sequence. However long resize2fs would take to run. Also, whatever metadata resize2fs would write to the overlay, would be a permanent ram penalty (albeit small hopefully). An alternate idea I had, is that you could build the minimized ext3fs image as described, but then resize2fs it back to the larger size - during spin composition. Then, at livecd install time, (note people, we are now talking about the normal livecd installer, not my crazy rebootless stuff), you could create a _second_ snapshot of the big read only ext3fs image, and then minimize-resize2fs it, before starting the copy to destination volume. This is much more complicated, but probably the better solution, as it takes the delay and ram hit out of every livecd boot, and adds it only at install time (and the ram hit gets freed after installation completes). The livecd build time resize2fs minimization(and remaximization) might still be a good idea, if it helps minimize the ram and processing time required to do the anaconda install time minimize-resize2fs. -dmc > > Also, the mechanism that comes to mind for the ext3fs creation is this- > > Take the existing image built as is, but after final install, resize2fs > it to the smallest possible (nearly), then truncate the file, then do > the dd seek trick to re-sparsify it vastly larger. > > Or perhaps just throw in an entire extra tarcopy of the system to a new > fs image file created the exact right size from the beginning. This is > more work, but will possibly save space on any files that got created > and deleted during the installation process. > > -dmc > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From dmc.fedora at filteredperception.org Tue Jul 10 14:06:21 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 09:06:21 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: References: <46936879.50009@filteredperception.org> <469377FA.2060800@filteredperception.org> <46937983.10601@filteredperception.org> Message-ID: <4693925D.8070205@filteredperception.org> Greg Dekoenigsberg wrote: > > I see someone besides me uses mailing lists to keep public notes for > themselves to remember later. :) Well, I would hope you do it for the same reason I do. I.e. they aren't just personal notes, but rather technical ideas. And hopefully leveraging exposure to the community might save you time and effort if someone catches your mental/technical errors before either you catch them yourself, or you go implement something that should have been done differently. -dmc From vladimir.shebordaev at gmail.com Tue Jul 10 15:09:34 2007 From: vladimir.shebordaev at gmail.com (Vladimir Shebordaev) Date: Tue, 10 Jul 2007 19:09:34 +0400 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <4693925D.8070205@filteredperception.org> References: <46936879.50009@filteredperception.org> <469377FA.2060800@filteredperception.org> <46937983.10601@filteredperception.org> <4693925D.8070205@filteredperception.org> Message-ID: <4693A12E.2040008@gmail.com> Douglas, so, why don't you try 'em yourself? I guess it would save lots of traffic for you if you send just patch(es) here ;) Regards, Vladimir Douglas McClendon ?????: > Greg Dekoenigsberg wrote: >> >> I see someone besides me uses mailing lists to keep public notes for >> themselves to remember later. :) > > Well, I would hope you do it for the same reason I do. I.e. they aren't > just personal notes, but rather technical ideas. And hopefully > leveraging exposure to the community might save you time and effort if > someone catches your mental/technical errors before either you catch > them yourself, or you go implement something that should have been done > differently. > > -dmc > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > From russdyson at hotmail.com Tue Jul 10 16:01:00 2007 From: russdyson at hotmail.com (russdyson at hotmail.com) Date: Tue, 10 Jul 2007 16:01:00 +0000 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency Message-ID: As a relative noob (only been into Linux for about 1 year - and a lot of what you guys talk about goes over my head), I would just like to add that with my 4GB usb2 stick it does seem to take a disproportionaly long time to copy the boot image files from the livecd iso file to the stick. This might be because it is filling the rest of the drive with 0's as you suggest.Still damn cool though!Russ.> Date: Tue, 10 Jul 2007 06:07:37 -0500> From: dmc.fedora at filteredperception.org> To: fedora-livecd-list at redhat.com> Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency> > Question: Does the current livecd installer inefficiently write lots of 0's to > the destination drive that it doesn't need to?> > I think it might. The os.img on the F7 livecd is a 4G sparse file with about > 2.3G of data. Anaconda's livecdcopy backend uses python's os.read/write. I > would guess that that means that 4G of data is getting written, when > theoretically only 2.3G needs to.> > The solution that comes to mind is this-> > in livecd-tools, create the os.img as a 7G (or 700G??) sparse file. Basically > just way big. Then take care to make the ext3fs be the exact correct size for > the data (i.e. 2.3G). Then, in the initramfs, just after mounting it (after > snapshotting it), do a resize2fs to 7G (or 700G).> > Then when anaconda does the copy, only copy the first 2.3G of the sparse file.> > It is however late in the 'day' for me, so maybe someone can chime in with > confirmation or refutation of my logic here.> > -dmc> > --> Fedora-livecd-list mailing list> Fedora-livecd-list at redhat.com> https://www.redhat.com/mailman/listinfo/fedora-livecd-list _________________________________________________________________ Make every IM count. Download Windows Live Messenger and join the i?m Initiative now. It?s free.? http://im.live.com/messenger/im/home/?source=TAGWL_June07 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmc.fedora at filteredperception.org Tue Jul 10 16:21:12 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 10 Jul 2007 11:21:12 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <4693A12E.2040008@gmail.com> References: <46936879.50009@filteredperception.org> <469377FA.2060800@filteredperception.org> <46937983.10601@filteredperception.org> <4693925D.8070205@filteredperception.org> <4693A12E.2040008@gmail.com> Message-ID: <4693B1F8.8030003@filteredperception.org> Vladimir Shebordaev wrote: > Douglas, > > so, why don't you try 'em yourself? I guess it would save lots of > traffic for you if you send just patch(es) here ;) mail filters (e.g. procmail) are wonderful things. I promise that all patches of mine will have PATCH in the subject line ;) -dmc > > Regards, > Vladimir > > Douglas McClendon ?????: >> Greg Dekoenigsberg wrote: >>> >>> I see someone besides me uses mailing lists to keep public notes for >>> themselves to remember later. :) >> >> Well, I would hope you do it for the same reason I do. I.e. they >> aren't just personal notes, but rather technical ideas. And hopefully >> leveraging exposure to the community might save you time and effort if >> someone catches your mental/technical errors before either you catch >> them yourself, or you go implement something that should have been >> done differently. >> >> -dmc >> >> -- >> Fedora-livecd-list mailing list >> Fedora-livecd-list at redhat.com >> https://www.redhat.com/mailman/listinfo/fedora-livecd-list >> > > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From dmc.fedora at filteredperception.org Wed Jul 11 05:08:56 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 11 Jul 2007 00:08:56 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: References: Message-ID: <469465E8.9080402@filteredperception.org> Russ, If you are referring to the /usr/bin/livecd-iso-to-disk tool, then the problem I described is not the culprit. If however you are talking about the 'install to harddrive' feature run from within the fedora-7-livecd environment, then yes, that would be affecting you. If my theory is correct, the install time might be taking up to 70% longer than it could (if implemented with the resize2fs procedures). -dmc russdyson at hotmail.com wrote: > As a relative noob (only been into Linux for about 1 year - and a lot of what you guys talk about goes over my head), I would just like to add that with my 4GB usb2 stick it does seem to take a disproportionaly long time to copy the boot image files from the livecd iso file to the stick. This might be because it is filling the rest of the drive with 0's as you suggest.Still damn cool though!Russ. From dmc.fedora at filteredperception.org Wed Jul 11 06:12:17 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 11 Jul 2007 01:12:17 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <1183969566.5957.18.camel@damaestro> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org> <469145C4.1070400@kanarip.com> <46915106.6090402@filteredperception.org> <46915377.1000902@filteredperception.org> <25449.70.57.175.112.1183928998.squirrel@hputz.datawranglers.com> <469164D3.2040000@filteredperception.org> <25328.70.57.175.112.1183941249.squirrel@hputz.datawranglers.com> <4691A2B6.4030309@filteredperception.org> <1183969566.5957.18.camel@damaestro> Message-ID: <469474C1.6090303@filteredperception.org> Jonathan Steffan wrote: > I have a use for something like this. > > On Sun, 2007-07-08 at 21:51 -0500, Douglas McClendon wrote: >> Normal Fedora 7 Installed On Hard Disk System >> (assume just 1 non lvm partition /dev/sda1) >> ---------------------------------------------- >> -The bios loads the grub boot loader from the MBR of /dev/sda >> -grub knows how to read it's config from /dev/sda1:/boot/grub/grub.conf >> -grub is configured to boot a specific kernel+ramdisk+appendstring, namely >> /dev/sda1:/boot/vmlinuz-someversion, /dev/sda1:/boot/initrd-someversion, >> someargs... >> -control is thusly passed to the kernel, and the kernel then gunzips and >> extracts the cpio of the specified initrd (which I think grub copied to a well >> known place in ram. Only reason I might know this is because qemu's crafty >> -initrd feature screwed it up for larger initrds recently, though it has been fixed) >> -the kernel having then extracted the contents into a ram based filesystem, >> passes control to /init (or maybe /sbin/init, or maybe whatever init= was >> specified on the cmdline). >> -now the fun starts. This init is a nash or a bash script, whose job it is to >> mount the real root filesystem (e.g. the ext3fs on /dev/sda1) and then >> pivot_root to it. >> -finally, control is passed to /sbin/init on the real disk-based root >> filesystem, at which point the contents of the well known /etc/inittab start to >> matter. >> >> Now then, what the Fedora-7 livecd does is along these lines >> ------------------------------------------------------------ >> -instead of the bios booting grub loaded on the mbr of a disk, it boots grub(or >> perhaps isolinux) from the bootsector of the cdrom. > > - We boot from a local flash, (random example: > http://www.pcengines.ch/cf2g.htm) > >> -this bootloader behaves much like above, but pulling a >> kernel+initrd+append_args from some place on the cdrom. > > s/cdrom/flash/ Yeah, livecd-iso-to-disk pretty much makes everything said about the livecd boot process apply to usb-flash. The interesting forward-looking aspect of writable flash (I guess perhaps even RW cd/dvd media), are things like persistance (storing the read-write changes to the filesystem in an overlay file on the usb-flash, rather than on the overlay file in ram-tmpfs). Or alternately just treating the usb-flash disk like a normal external harddisk. There are of course issues with usb-flash differing from normal hard disks, I.e. why some of the new circular log based filesystems come into play (I really don't know much about those other than skimming the recent articles on lwn.net) > >> -now the fun begins after the initrd is extracted and mounted in a ram based fs >> as normal. An entirely different /init script within the initrd will go about >> the business of mounting the 'real root filesystem'. In this case, first the >> cdrom's iso9660 fs is mounted. Then a squashfs image file is loopback mounted >> from within the iso9660 fs. Then a sparse ext3fs image file is loopback mounted >> from within the squashfs. > > s/iso9660/whatever/ > > Before the squashfs is loop mounted, we do some checks on it... most > likely a checksum of sorts. At this point we test if this is the latest > image. If not, we pull the latest image to a ram drive and loop mount > that instead of the local squashfs image. I have assumed we would be > able to have some sort of network access at this stage. > >> - now the REAL fun begins. A ram based filesystem is created. A sparse file >> overlay is created within it. Now a device mapper snapshot is created using the >> read-only ext3fs image, and the read-write overlay file. (I'm skipping some >> loopback device associations, and in general probably misnaming a few things, as >> this is unashamedly from the hip, and not suitable to be published). Now, this >> magic devicemapper snapshotted device appears as /dev/mapper/live-rw, and >> appears to be a read-write ext3 filesystem, except the writes really get tucked >> away in ram (which is going to eat away at your ram). >> - That /dev/mapper/live-rw gets mounted as the 'real root filesystem', and >> pivot_root is called, and then things progress as normal. > > If the squashfs image has not changed, we pivot_root right away. > >> Now then- what the rebootless installer patch does differently >> -------------------------------------------------------------- >> -Just after the /dev/mapper/live-rw gets set up in the initrd, instead of >> mounting it as the real root filesystem, it gets used to create a raid1 >> 'mirror'. quotes because in this case the 'mirror' only has 1 device, rather >> than the usual 2. The mirror is visible as /dev/md7, and THAT gets mounted as >> the real root filesystem, before pivot_root is called, and everything progresses >> as normal. > > Neat. Could we create the raid1 from a new loop mounted squashfs that > has just been loaded into ram? Absolutely :) Though the fallout from those dozen posts on the rebootless installer idea, is that you'll want to use the device-mapper mirror target 'raid1' rather than the mdadm raid1, or at least until the --grow works better with the latter. Although really, unless the devicemapper mirror target starts to become problematic, I see no downsides with that... Ok, well, currently you can't throttle the sync speed AFAIK with the dm-mirror, while you can with mdadm raid1. > >> But because of /dev/md7 being the 'real root filesystem', long after boot, you >> can hot-add another device to the mirror, in this case, the target volume that >> you want to install the system on (e.g. /dev/sda1). After you hot-add, the >> raid/md driver starts synchronizing the data from /dev/mapper/live-rw to >> /dev/sda1. > > I would want to sync back to the flash at this point. No reason I can think of why it shouldn't work. Live-migration of root filesystems sure is fun :) -dmc > >> When this finishes, you can hot-remove /dev/mapper/live-rw, at which >> point the system is running from /dev/sda1, just as if you had installed there >> and rebooted (with the caviat that there is this /dev/md7 layer sitting there >> until the next reboot). And once /dev/mapper/live-rw is removed from the >> /dev/md7 array, the resources that constructed it (i.e. the files on the cdrom, >> and that overlay file in a ram-based fs) can be released/deleted/unmounted. >> Thus you stop suffering the penalty of that overlay eating up your ram, and you >> are free to eject the cdrom. >> >> Now, for the sake of simplicity, we will assume that "mdadm /dev/md7 --grow >> --size=max" will actually work, and the 3.5G ext3fs that got migrated to your >> 100G /dev/sda1 partition, can be grown with resize2fs's ability to online expand >> a live ext3 filesystem. (technically this does not yet work, so refer to all >> those nasty workarounds). >> > > There is my incomplete 2am reply ;-) > > Jonathan Steffan > daMaestro > > > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From dmc.fedora at filteredperception.org Wed Jul 11 06:39:26 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 11 Jul 2007 01:39:26 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <469465E8.9080402@filteredperception.org> References: <469465E8.9080402@filteredperception.org> Message-ID: <46947B1E.7040802@filteredperception.org> Douglas McClendon wrote: > Russ, > > If you are referring to the /usr/bin/livecd-iso-to-disk tool, then > the problem I described is not the culprit. If however you are talking > about the 'install to harddrive' feature run from within the > fedora-7-livecd > environment, then yes, that would be affecting you. To correct and clarify myself- If you have hacked/patched livecd-iso-to-disk to support something like the output of livecd-tools --uncompressed, then theoretically a similar issue might come into play. (but unless you know for sure that is the case, it's not the case). Also, the issue is still valid for the case of running livecd-iso-to-disk to move a livecd iso onto a usb flash stick, and _then_ booting a live environment from that flash, and _then_ installing to some _other_ disk using the anaconda live install to harddrive feature. I.e. it comes into play only in that last step, when installing fedora-7 from a live booted environment (cd or usb-flash) to hard disk. -dmc > > If my theory is correct, the install time might be taking up to 70% > longer than it could (if implemented with the resize2fs procedures). > > -dmc > > russdyson at hotmail.com wrote: >> As a relative noob (only been into Linux for about 1 year - > > and a lot of what you guys talk about goes over my head), I > > would just like to add that with my 4GB usb2 stick it > > does seem to take a disproportionaly long time to copy the > > boot image files from the livecd iso file to the stick. > > This might be because it is filling the rest of the drive with 0's > > as you suggest.Still damn cool though!Russ. > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From russdyson at hotmail.com Wed Jul 11 08:17:12 2007 From: russdyson at hotmail.com (russdyson at hotmail.com) Date: Wed, 11 Jul 2007 08:17:12 +0000 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency Message-ID: Hi dmc,I was referring to the /usr/bin/livecd-iso-to-disk tool and not the hard disk install.So just ignore me there then.Thanks.> Date: Wed, 11 Jul 2007 00:08:56 -0500> From: dmc.fedora at filteredperception.org> To: fedora-livecd-list at redhat.com> Subject: Re: [Fedora-livecd-list] RFC- improving livecd installer efficiency> > Russ,> > If you are referring to the /usr/bin/livecd-iso-to-disk tool, then> the problem I described is not the culprit. If however you are talking> about the 'install to harddrive' feature run from within the fedora-7-livecd> environment, then yes, that would be affecting you.> > If my theory is correct, the install time might be taking up to 70% longer than > it could (if implemented with the resize2fs procedures).> > -dmc> > russdyson at hotmail.com wrote:> > As a relative noob (only been into Linux for about 1 year -> > and a lot of what you guys talk about goes over my head), I> > would just like to add that with my 4GB usb2 stick it> > does seem to take a disproportionaly long time to copy the> > boot image files from the livecd iso file to the stick.> > This might be because it is filling the rest of the drive with 0's> > as you suggest.Still damn cool though!Russ.> > --> Fedora-livecd-list mailing list> Fedora-livecd-list at redhat.com> https://www.redhat.com/mailman/listinfo/fedora-livecd-list _________________________________________________________________ With Windows Live Hotmail, you can personalize your inbox with your favorite color. www.windowslive-hotmail.com/learnmore/personalize.html?locale=en-us&ocid=TXT_TAGLM_HMWL_reten_addcolor_0607 -------------- next part -------------- An HTML attachment was scrubbed... URL: From markmc at redhat.com Wed Jul 11 09:36:47 2007 From: markmc at redhat.com (Mark McLoughlin) Date: Wed, 11 Jul 2007 10:36:47 +0100 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <46936879.50009@filteredperception.org> References: <46936879.50009@filteredperception.org> Message-ID: <1184146607.2997.139.camel@blaa> Hi, I've lost track of the all the details here but ... On Tue, 2007-07-10 at 06:07 -0500, Douglas McClendon wrote: > Question: Does the current livecd installer inefficiently write lots of 0's to > the destination drive that it doesn't need to? > > I think it might. The os.img on the F7 livecd is a 4G sparse file with about > 2.3G of data. Anaconda's livecdcopy backend uses python's os.read/write. I > would guess that that means that 4G of data is getting written, when > theoretically only 2.3G needs to. ... how about something like this: http://www.gnome.org/~markmc/code/e2cp.c i.e. read the ext3 metadata and copy everything but the unallocated blocks. (One thing from the above code you don't want to do is the check_all_zeros() bit - if an allocated block is all zeros, you *do* want to copy the zeros to the disk) Cheers, Mark. From dmc.fedora at filteredperception.org Wed Jul 11 19:50:23 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 11 Jul 2007 14:50:23 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <1184146607.2997.139.camel@blaa> References: <46936879.50009@filteredperception.org> <1184146607.2997.139.camel@blaa> Message-ID: <4695347F.2010407@filteredperception.org> Mark McLoughlin wrote: > Hi, > I've lost track of the all the details here but ... > > On Tue, 2007-07-10 at 06:07 -0500, Douglas McClendon wrote: >> Question: Does the current livecd installer inefficiently write lots of 0's to >> the destination drive that it doesn't need to? >> >> I think it might. The os.img on the F7 livecd is a 4G sparse file with about >> 2.3G of data. Anaconda's livecdcopy backend uses python's os.read/write. I >> would guess that that means that 4G of data is getting written, when >> theoretically only 2.3G needs to. > > ... how about something like this: > > http://www.gnome.org/~markmc/code/e2cp.c > > i.e. read the ext3 metadata and copy everything but the unallocated > blocks. I'm not as into low level filesystem internals as you are. Tell me if this paraphrasing is accurate- e2cp is sort of like a dd for filesystems, that understands sparseness and/or unused/unallocated blocks, and handles them efficiently (ignores them). The main downside I think it has as a solution to the issue, is that it doesn't fix the case of a destination volume of say 3.0G. I.e. there is no reason why the livecd installer shouldn't be able to install it's 2.3G payload onto a 3.0G destination. The solution I outlined covers that* case, while I don't think using e2cp does. * _IF_ there aren't any flaws with anything I theorized The most recent solution I was proposing, involved taking a second snapshot of the 4.0G (or perhaps now 1T) sparse ext3 os.img file, and resize2fs-ing it down to minimal, and then copying it. The big question which I haven't tested yet, is whether or not such a resize2fs will happen quickly, with very few actual changes written to the image (i.e. the new overlay in ram). If it does take less than a minute and use less than a 1MB of ram, it seems like a good solution to me. Perhaps with your low level knowledge of ext2(/resize2fs?) you can answer that. I.e. if you take an image, resize it to minimal, then resize it to 1TB, then how long and how many changes will it take to resize it back down to minimal? Of course I can just test it myself, and probably will soon enough. -dmc -dmc > > (One thing from the above code you don't want to do is the > check_all_zeros() bit - if an allocated block is all zeros, you *do* > want to copy the zeros to the disk) > > Cheers, > Mark. > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From hunt at m2s.com Wed Jul 11 20:00:03 2007 From: hunt at m2s.com (Elias Hunt) Date: Wed, 11 Jul 2007 16:00:03 -0400 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <4691559B.5080100@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org><469145C4.1070400@kanarip.com><46915106.6090402@filteredperception.org> <4691559B.5080100@filteredperception.org> Message-ID: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAEC@exchange2.ad.medicalmetrx.com> Douglas, Assuming that iso-caching-to-disk feature is implemented, would it be easy to have the caching occur on an ext3 partition? My company actually has a use case where we create custom livecds for hundreds of systems, using a local drive for data storage only as we don't want the OS to be modified permanently (a reboot will restore to normal). Having the performance gain of actually running the iso from the local disk, while still being able to mount the disk for data storage, and being able to insert other CDs to the running system would provide huge benefits. -Eli -----Original Message----- From: fedora-livecd-list-bounces at redhat.com [mailto:fedora-livecd-list-bounces at redhat.com] On Behalf Of Douglas McClendon Sent: Sunday, July 08, 2007 5:23 PM To: fedora-livecd-list at redhat.com Subject: Re: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer Douglas McClendon wrote: > Jeroen van Meeuwen wrote: >> Douglas McClendon wrote: >> I'm sure there's /a/ use-case for this technical advancement. Besides >> that, I'm all for any technical advancement -whether it has actual >> use-cases or not. > > > /The/ use case is simple. It does what it does. It is what it is. > Whether or not it is well received once it is an available option, > remains to be seen. Note, that it is trivial to enhance the existing > patch such that things only happen differently than normal if you pass > 'support_rebootless' on the kernel commandline. > > Also note that it is fairly trivial to add an option so that the user > can chose whether to live-migrate the cdrom+ram_overlay onto the > destination volume, or just the cdrom. In the latter case, the user > could still eject the cdrom when installation is complete and continue > working in the live system, though on reboot it would forget all the > live-session modifications and give you a fresh start. A different continuation of this thought, is the modification to support the iso-caching-to-disk functionality I described in my 2001 project. Though now, instead of vfat, use ntfs-3g. I.e. do your live migration to a file on an ntfs partition, such that after the migration a) the user can eject the cdrom and use the cdrom drive for other reasons b) the system will be much more responsive as data is pulled from the ntfs rather than the cdrom c) with a sufficiently modern (i.e. probably doesn't exist yet) version of grub or loadlin, the live environment could be subsequently booted strait from ntfs, without needing the original livecd. -dmc/jdog -- Fedora-livecd-list mailing list Fedora-livecd-list at redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list From dmc.fedora at filteredperception.org Wed Jul 11 20:17:21 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 11 Jul 2007 15:17:21 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAEC@exchange2.ad.medicalmetrx.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org><469145C4.1070400@kanarip.com><46915106.6090402@filteredperception.org> <4691559B.5080100@filteredperception.org> <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAEC@exchange2.ad.medicalmetrx.com> Message-ID: <46953AD1.8060208@filteredperception.org> Elias Hunt wrote: > Douglas, > > Assuming that iso-caching-to-disk feature is implemented, would it be > easy to have the caching occur on an ext3 partition? Yup Yup Yup :) My 2001 project would scan for all partitions it could blindly mount (mount -t auto), and use any that it found available with at least 2X the free space needed. I'll take your use-case as added motivation to work on a proof of concept implementation of that feature... So much code to write... so little time ;) -dmc > > My company actually has a use case where we create custom livecds for > hundreds of systems, using a local drive for data storage only as we > don't want the OS to be modified permanently (a reboot will restore to > normal). Having the performance gain of actually running the iso from > the local disk, while still being able to mount the disk for data > storage, and being able to insert other CDs to the running system would > provide huge benefits. > > -Eli > > -----Original Message----- > From: fedora-livecd-list-bounces at redhat.com > [mailto:fedora-livecd-list-bounces at redhat.com] On Behalf Of Douglas > McClendon > Sent: Sunday, July 08, 2007 5:23 PM > To: fedora-livecd-list at redhat.com > Subject: Re: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless > installer > > Douglas McClendon wrote: >> Jeroen van Meeuwen wrote: >>> Douglas McClendon wrote: > >>> I'm sure there's /a/ use-case for this technical advancement. Besides >>> that, I'm all for any technical advancement -whether it has actual >>> use-cases or not. >> >> /The/ use case is simple. It does what it does. It is what it is. >> Whether or not it is well received once it is an available option, >> remains to be seen. Note, that it is trivial to enhance the existing >> patch such that things only happen differently than normal if you pass > >> 'support_rebootless' on the kernel commandline. >> >> Also note that it is fairly trivial to add an option so that the user >> can chose whether to live-migrate the cdrom+ram_overlay onto the >> destination volume, or just the cdrom. In the latter case, the user >> could still eject the cdrom when installation is complete and continue > >> working in the live system, though on reboot it would forget all the >> live-session modifications and give you a fresh start. > > A different continuation of this thought, is the modification to support > the > iso-caching-to-disk functionality I described in my 2001 project. > Though now, > instead of vfat, use ntfs-3g. > > I.e. do your live migration to a file on an ntfs partition, such that > after the > migration > > a) the user can eject the cdrom and use the cdrom drive for other > reasons > b) the system will be much more responsive as data is pulled from the > ntfs > rather than the cdrom > c) with a sufficiently modern (i.e. probably doesn't exist yet) version > of grub > or loadlin, the live environment could be subsequently booted strait > from ntfs, > without needing the original livecd. > > -dmc/jdog > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From hunt at m2s.com Wed Jul 11 20:27:36 2007 From: hunt at m2s.com (Elias Hunt) Date: Wed, 11 Jul 2007 16:27:36 -0400 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <46953AD1.8060208@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org><469145C4.1070400@kanarip.com><46915106.6090402@filteredperception.org> <4691559B.5080100@filteredperception.org><50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAEC@exchange2.ad.medicalmetrx.com> <46953AD1.8060208@filteredperception.org> Message-ID: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAFE@exchange2.ad.medicalmetrx.com> Perfect, I'll keep a watch out for anything traveling down the pipe. If you would like, feel free to pester me directly for assistance testing (when you get that far), and I'll happily spin some test CDs based on our setup using any new/patched code and provide feedback. -E -----Original Message----- Elias Hunt wrote: > Douglas, > > Assuming that iso-caching-to-disk feature is implemented, would it be > easy to have the caching occur on an ext3 partition? Yup Yup Yup :) My 2001 project would scan for all partitions it could blindly mount (mount -t auto), and use any that it found available with at least 2X the free space needed. I'll take your use-case as added motivation to work on a proof of concept implementation of that feature... So much code to write... so little time ;) -dmc > > My company actually has a use case where we create custom livecds for > hundreds of systems, using a local drive for data storage only as we > don't want the OS to be modified permanently (a reboot will restore to > normal). Having the performance gain of actually running the iso from > the local disk, while still being able to mount the disk for data > storage, and being able to insert other CDs to the running system would > provide huge benefits. > > -Eli From dmc.fedora at filteredperception.org Wed Jul 11 20:35:27 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 11 Jul 2007 15:35:27 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <46953AD1.8060208@filteredperception.org> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org><469145C4.1070400@kanarip.com><46915106.6090402@filteredperception.org> <4691559B.5080100@filteredperception.org> <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAEC@exchange2.ad.medicalmetrx.com> <46953AD1.8060208@filteredperception.org> Message-ID: <46953F0F.6050707@filteredperception.org> Douglas McClendon wrote: > Elias Hunt wrote: >> Douglas, >> >> Assuming that iso-caching-to-disk feature is implemented, would it be >> easy to have the caching occur on an ext3 partition? > Note, it is even trivial to have an option of whether you want to migrate the compressed squashfs base, or the uncompressed ext3fs base. With the former, you would still suffer from decompression during normal use, but the latter would take much more disk space. Live migration of the root fs is just so cool. Almost makes me want to create a fedora derivative where the root fs is _always_ an extra dm linear device layer, so that at any point in the future, you can live migrate the root filesystem. Use-case: You have a hard disk failing in a truly mission critical system. Now, you can live-migrate the root-fs to an external USB-drive, and then reboot to remove the physically failed drive at your leisure. Have I mentioned that I am a recent BS-CoE grad from KU actively seeking employment- http://douglas.mcclendon.org/resume (may the mailinglist protocol lords forgive me, yet again ;) -dmc > > Yup Yup Yup :) My 2001 project would scan for all partitions it could > blindly mount (mount -t auto), and use any that it found available with > at least 2X the free space needed. > > I'll take your use-case as added motivation to work on a proof of > concept implementation of that feature... So much code to write... so > little time ;) > > -dmc > > >> >> My company actually has a use case where we create custom livecds for >> hundreds of systems, using a local drive for data storage only as we >> don't want the OS to be modified permanently (a reboot will restore to >> normal). Having the performance gain of actually running the iso from >> the local disk, while still being able to mount the disk for data >> storage, and being able to insert other CDs to the running system would >> provide huge benefits. >> >> -Eli >> >> -----Original Message----- >> From: fedora-livecd-list-bounces at redhat.com >> [mailto:fedora-livecd-list-bounces at redhat.com] On Behalf Of Douglas >> McClendon >> Sent: Sunday, July 08, 2007 5:23 PM >> To: fedora-livecd-list at redhat.com >> Subject: Re: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless >> installer >> >> Douglas McClendon wrote: >>> Jeroen van Meeuwen wrote: >>>> Douglas McClendon wrote: >> >>>> I'm sure there's /a/ use-case for this technical advancement. Besides >>>> that, I'm all for any technical advancement -whether it has actual >>>> use-cases or not. >>> >>> /The/ use case is simple. It does what it does. It is what it is. >>> Whether or not it is well received once it is an available option, >>> remains to be seen. Note, that it is trivial to enhance the existing >>> patch such that things only happen differently than normal if you pass >> >>> 'support_rebootless' on the kernel commandline. >>> >>> Also note that it is fairly trivial to add an option so that the user >>> can chose whether to live-migrate the cdrom+ram_overlay onto the >>> destination volume, or just the cdrom. In the latter case, the user >>> could still eject the cdrom when installation is complete and continue >> >>> working in the live system, though on reboot it would forget all the >>> live-session modifications and give you a fresh start. >> >> A different continuation of this thought, is the modification to support >> the iso-caching-to-disk functionality I described in my 2001 project. >> Though now, instead of vfat, use ntfs-3g. >> >> I.e. do your live migration to a file on an ntfs partition, such that >> after the migration >> >> a) the user can eject the cdrom and use the cdrom drive for other >> reasons >> b) the system will be much more responsive as data is pulled from the >> ntfs rather than the cdrom >> c) with a sufficiently modern (i.e. probably doesn't exist yet) version >> of grub or loadlin, the live environment could be subsequently booted >> strait >> from ntfs, without needing the original livecd. >> >> -dmc/jdog >> >> -- >> Fedora-livecd-list mailing list >> Fedora-livecd-list at redhat.com >> https://www.redhat.com/mailman/listinfo/fedora-livecd-list >> >> -- >> Fedora-livecd-list mailing list >> Fedora-livecd-list at redhat.com >> https://www.redhat.com/mailman/listinfo/fedora-livecd-list > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From dmc.fedora at filteredperception.org Wed Jul 11 21:38:33 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 11 Jul 2007 16:38:33 -0500 Subject: [Fedora-livecd-list] [RFC/PATCH] livecd rebootless installer In-Reply-To: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAFE@exchange2.ad.medicalmetrx.com> References: <46906F09.2030105@filteredperception.org> <4690A1B0.3060305@filteredperception.org> <4690D0C4.1010902@kanarip.com> <4690F813.9050904@filteredperception.org><469145C4.1070400@kanarip.com><46915106.6090402@filteredperception.org> <4691559B.5080100@filteredperception.org><50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAEC@exchange2.ad.medicalmetrx.com> <46953AD1.8060208@filteredperception.org> <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AAFE@exchange2.ad.medicalmetrx.com> Message-ID: <46954DD9.2010809@filteredperception.org> Elias Hunt wrote: > Perfect, I'll keep a watch out for anything traveling down the pipe. > > If you would like, feel free to pester me directly for assistance > testing (when you get that far), and I'll happily spin some test CDs > based on our setup using any new/patched code and provide feedback. Definitely. And to spam this list with yet more technical ideas... in this case specific to your use case... The live snapshot device, normally located in ram, could also be live migrated to a different file on the cache-destination drive, such that changes to the root-fs from the live-session (i.e. /var/log/messages ....) don't eat up your ram. Then, since you don't want changes to survive subsequent reboots, when rebooting from the livecd (or perhaps grub harddrive mbr), it would detect that the livecd's ext3 image file has already been cached to disk, and use it, but also utilize a fresh overlay file on the disk, so that no changes from the live-session are remembered. (also, to correct/clarify myself yet again, I do realize that using lvm for rootfs is basically 'always using a dm linear device layer for the rootfs'. But as I said earlier, I'm not yet a power-user of lvm. And by limiting yourself to device-mapper, you can support the more general use case of non-lvm, OR lvm root filesystems) -dmc > -E > > -----Original Message----- > Elias Hunt wrote: >> Douglas, >> >> Assuming that iso-caching-to-disk feature is implemented, would it be >> easy to have the caching occur on an ext3 partition? > > > Yup Yup Yup :) My 2001 project would scan for all partitions it could > blindly > mount (mount -t auto), and use any that it found available with at least > 2X the > free space needed. > > I'll take your use-case as added motivation to work on a proof of > concept > implementation of that feature... So much code to write... so little > time ;) > > -dmc > > >> My company actually has a use case where we create custom livecds for >> hundreds of systems, using a local drive for data storage only as we >> don't want the OS to be modified permanently (a reboot will restore to >> normal). Having the performance gain of actually running the iso from >> the local disk, while still being able to mount the disk for data >> storage, and being able to insert other CDs to the running system > would >> provide huge benefits. >> >> -Eli > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From markmc at redhat.com Thu Jul 12 07:25:12 2007 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 12 Jul 2007 08:25:12 +0100 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <4695347F.2010407@filteredperception.org> References: <46936879.50009@filteredperception.org> <1184146607.2997.139.camel@blaa> <4695347F.2010407@filteredperception.org> Message-ID: <1184225112.2820.27.camel@blaa> On Wed, 2007-07-11 at 14:50 -0500, Douglas McClendon wrote: > The main downside I think it has as a solution to the issue, is that it doesn't > fix the case of a destination volume of say 3.0G. I.e. there is no reason why > the livecd installer shouldn't be able to install it's 2.3G payload onto a 3.0G > destination. I thought that was already covered with a resize2fs ? > The most recent solution I was proposing, involved taking a second snapshot of > the 4.0G (or perhaps now 1T) sparse ext3 os.img file, and resize2fs-ing it down > to minimal, and then copying it. This sounds like a way of doing the same thing as e2cp ... by resizing it down to its minimal size, you'd be removing all unallocated data blocks and so e2cp would have nothing to ignore. So, they're different solutions to the "let's not copy unallocated blocks" problem. Not sure why a snapshot is needed ... > I.e. if you take an image, resize it to minimal, then resize it to 1TB, then > how long and how many changes will it take to resize it back down to minimal? > Of course I can just test it myself, and probably will soon enough. I would imagine it would be similar to how long it would take to mke2fs a 1TB sparse file. Cheers, Mark. From dmc.fedora at filteredperception.org Thu Jul 12 09:47:17 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 12 Jul 2007 04:47:17 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <1184225112.2820.27.camel@blaa> References: <46936879.50009@filteredperception.org> <1184146607.2997.139.camel@blaa> <4695347F.2010407@filteredperception.org> <1184225112.2820.27.camel@blaa> Message-ID: <4695F8A5.4050305@filteredperception.org> Mark McLoughlin wrote: > On Wed, 2007-07-11 at 14:50 -0500, Douglas McClendon wrote: > >> The main downside I think it has as a solution to the issue, is that it doesn't >> fix the case of a destination volume of say 3.0G. I.e. there is no reason why >> the livecd installer shouldn't be able to install it's 2.3G payload onto a 3.0G >> destination. > > I thought that was already covered with a resize2fs ? If you mean already as in 'exists in the f7livecd' then the answer is no. The resize2fs that the f7livecd anaconda does is _after_ the fs image copy, and thus only effective for expansion, not shrink. the f7livecd anaconda installer will red-error-of-death you if you try to install to a 3G destination volume. > >> The most recent solution I was proposing, involved taking a second snapshot of >> the 4.0G (or perhaps now 1T) sparse ext3 os.img file, and resize2fs-ing it down >> to minimal, and then copying it. > > This sounds like a way of doing the same thing as e2cp ... by resizing > it down to its minimal size, you'd be removing all unallocated data > blocks and so e2cp would have nothing to ignore. > > So, they're different solutions to the "let's not copy unallocated > blocks" problem. > > Not sure why a snapshot is needed ... Because you haven't yet added in-flight-resize2fs support to e2cp. But if you don't care about fixing the 2.4G->3.9G destination volume problem, then e2cp is sufficient. -dmc > >> I.e. if you take an image, resize it to minimal, then resize it to 1TB, then >> how long and how many changes will it take to resize it back down to minimal? >> Of course I can just test it myself, and probably will soon enough. > > I would imagine it would be similar to how long it would take to mke2fs > a 1TB sparse file. > > Cheers, > Mark. > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From markmc at redhat.com Thu Jul 12 10:15:33 2007 From: markmc at redhat.com (Mark McLoughlin) Date: Thu, 12 Jul 2007 11:15:33 +0100 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <4695F8A5.4050305@filteredperception.org> References: <46936879.50009@filteredperception.org> <1184146607.2997.139.camel@blaa> <4695347F.2010407@filteredperception.org> <1184225112.2820.27.camel@blaa> <4695F8A5.4050305@filteredperception.org> Message-ID: <1184235333.7592.12.camel@blaa> Hey, Okay, I follow you now. Doing: - Create snapshot of os.img - Resize it down to the smallest possible size - Copy it to disk - Resize it back up to the size of the disk has the dual advantages of being able to install to the small possible disk size and not copying unallocated blocks. Sounds like a reasonable plan ... go ahead and give it a shot. The reservation I'd have is if you're resizing down from 4G to 3G, the 1G of data blocks which have to be moved could potentially be at the end of the image. In that case, you'd need a 1G COW area for the snapshot. It'd be much better if you could resize filesystem as you're copying it, and that should be possible, but you'd have to e.g. add support for copying to resize2fs. Cheers, Mark. From dmc.fedora at filteredperception.org Thu Jul 12 11:00:36 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 12 Jul 2007 06:00:36 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <1184235333.7592.12.camel@blaa> References: <46936879.50009@filteredperception.org> <1184146607.2997.139.camel@blaa> <4695347F.2010407@filteredperception.org> <1184225112.2820.27.camel@blaa> <4695F8A5.4050305@filteredperception.org> <1184235333.7592.12.camel@blaa> Message-ID: <469609D4.5010404@filteredperception.org> Mark McLoughlin wrote: > Hey, > Okay, I follow you now. Doing: > > - Create snapshot of os.img > - Resize it down to the smallest possible size > - Copy it to disk > - Resize it back up to the size of the disk > > has the dual advantages of being able to install to the small possible > disk size and not copying unallocated blocks. > > Sounds like a reasonable plan ... go ahead and give it a shot. > > The reservation I'd have is if you're resizing down from 4G to 3G, the > 1G of data blocks which have to be moved could potentially be at the end > of the image. In that case, you'd need a 1G COW area for the snapshot. The part of my theory that is supposed to cover that, is an extra minimize/truncate/sparsify_expand/maximize cycle on the image just before it gets burned in the squashfs. Fingers crossed... > > It'd be much better if you could resize filesystem as you're copying > it, and that should be possible, but you'd have to e.g. add support for > copying to resize2fs. Sort of another way of saying "add in-flight-resize2fs support to e2cp" ;) -dmc > > Cheers, > Mark. > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From dmc.fedora at filteredperception.org Thu Jul 12 11:32:23 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 12 Jul 2007 06:32:23 -0500 Subject: [Fedora-livecd-list] [PATCH] rebootless installer / live migration support for mayflower Message-ID: <46961147.7050202@filteredperception.org> In the name of release-early-release-often, here is an untested patch to livecd-tools mayflower script, which I believe adds support for rebootless installation. A proof of concept rebootless installer should hopefully be forthcoming in a day or so, and an anaconda based one in a week or so. Note, that the prior patch I had, was actually missing a --force flag on the mdadm build command. But don't worry, mdadm is out of the picture since it's --grow feature wouldn't work for me. This patch is far superior, and is a successful implementation of the theoretical workaround #4. I leave it as an excercise for the reader how to perform a rebootless installation manually ;) -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: mayflower.lms.patch Type: text/x-patch Size: 1690 bytes Desc: not available URL: From dmc.fedora at filteredperception.org Thu Jul 12 18:38:49 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 12 Jul 2007 13:38:49 -0500 Subject: [Fedora-livecd-list] RFC- improving livecd installer efficiency In-Reply-To: <469609D4.5010404@filteredperception.org> References: <46936879.50009@filteredperception.org> <1184146607.2997.139.camel@blaa> <4695347F.2010407@filteredperception.org> <1184225112.2820.27.camel@blaa> <4695F8A5.4050305@filteredperception.org> <1184235333.7592.12.camel@blaa> <469609D4.5010404@filteredperception.org> Message-ID: <46967539.4040700@filteredperception.org> Douglas McClendon wrote: > Mark McLoughlin wrote: >> Hey, >> Okay, I follow you now. Doing: >> >> - Create snapshot of os.img >> - Resize it down to the smallest possible size >> - Copy it to disk >> - Resize it back up to the size of the disk >> >> has the dual advantages of being able to install to the small >> possible >> disk size and not copying unallocated blocks. >> >> Sounds like a reasonable plan ... go ahead and give it a shot. >> >> The reservation I'd have is if you're resizing down from 4G to 3G, >> the >> 1G of data blocks which have to be moved could potentially be at the end >> of the image. In that case, you'd need a 1G COW area for the snapshot. > > > The part of my theory that is supposed to cover that, is an extra > minimize/truncate/sparsify_expand/maximize cycle on the image just > before it gets burned in the squashfs. Fingers crossed... Actually even before testing this, I'm 90% sure it won't work... But I'm 99% sure I figured out something that will. You'll love it, trust me ;) -dmc From sundaram at fedoraproject.org Thu Jul 12 23:09:41 2007 From: sundaram at fedoraproject.org (Rahul Sundaram) Date: Fri, 13 Jul 2007 04:39:41 +0530 Subject: [Fedora-livecd-list] livecd install to hd fails In-Reply-To: <286849.3622.qm@web34712.mail.mud.yahoo.com> References: <286849.3622.qm@web34712.mail.mud.yahoo.com> Message-ID: <4696B4B5.6060000@fedoraproject.org> Dan Wallace wrote: > I have used the livecd to install fedora on a HP DL360 > dual 800Mhz processor raid enabled 1U rack mount > server with 768M RAM. I have tried this 3 times, the > last choosing all defaults for disk partitioning, and > I get the same result each time. It seems to be > complaining about 'Volume group "VolGroup00" not > found', then it goes into kernel panic. The installer > is very pretty, but there is little I can go on to > diagnose this problem. This is an install directly to > the hardware - no Xen, no vmware etc. Any help would > be appreciated. > > Here is standard error from running liveinst: > =========================================== > [fedora at localhost ~]$ liveinst & > [1] 3499 > [fedora at localhost ~]$ FATAL: Module md not found. > [fedora at localhost ~]$ Probing for video card: ATI > Technologies Inc 3D Rage IIC 215IIC [Mach64 GT IIC] > Starting graphical installation... > Loading /lib/kbd/keymaps/i386/qwerty/us.map.gz > Couldn't find rules file (xfree86) I remember reading a similar report. Check and file a new bug report with the hardware information. Rahul From dalestubblefield at gmail.com Fri Jul 13 12:20:00 2007 From: dalestubblefield at gmail.com (Dale Stubblefield) Date: Fri, 13 Jul 2007 07:20:00 -0500 Subject: [Fedora-livecd-list] Repository Names Message-ID: <13a07b140707130520k1d4df302w76b98ab08a4828be@mail.gmail.com> Concerning the default "/usr/share/livecd-tools/livecd-fedora-desktop.ks... The default repositories in the file are: repo --name=d7 --baseurl= http://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/os repo --name=e7 --baseurl= http://download.fedora.redhat.com/pub/fedora/linux/extras/development/i386 These are not valid links; however, I know that this one it: http://download.fedora.redhat.com/pub/fedora/linux/releases/7/Everything/i386/os/ However, if I change repo --name=d7 --baseurl= http://download.fedora.redhat.com/pub/fedora/linux/core/development/i386/os to repo --name=d7 --baseurl= http://download.fedora.redhat.com/pub/fedora/linux/releases/7/Everything/i386/os/ I still get the same error - "No Repositories Available to Set Up" and no livecd is created. How can I get this to point to a repository correct? Do I need to have the correct "name" of the repo? How do I determine that? Thanks, Dale -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.wood at datawranglers.com Fri Jul 13 12:42:06 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Fri, 13 Jul 2007 06:42:06 -0600 Subject: [Fedora-livecd-list] Repository Names In-Reply-To: <13a07b140707130520k1d4df302w76b98ab08a4828be@mail.gmail.com> References: <13a07b140707130520k1d4df302w76b98ab08a4828be@mail.gmail.com> Message-ID: <3DF69614-268F-4C6F-95CB-2714120BACB1@datawranglers.com> Dale, this isn't exactly an answer, but a tool to deal with part of what you're running into. You can pop those paths into a browser to check them and (often) figure out what's wrong. For instance: > http://download.fedora.redhat.com/pub/fedora/linux/core/development/ > i386/os I copied the part through 'fedora' into my browser and started clicking down. It turns out 'development' is empty _but_ there's a development directory several levels up that (if memory serves) has replaced it. In something slightly closer to an answer, Revisor works quite nicely with everything I've thrown at it kickstart-wise (well... except that I can't use the -package syntax to delete an optional package from a group **grumble**). Several of the defaults work quite well. regards, Tim Wood On Jul 13, 2007, at 6:20 AM, Dale Stubblefield wrote: > Concerning the default "/usr/share/livecd-tools/livecd-fedora- > desktop.ks... > > The default repositories in the file are: > repo --name=d7 --baseurl= http://download.fedora.redhat.com/pub/ > fedora/linux/core/development/i386/os > repo --name=e7 --baseurl=http://download.fedora.redhat.com/pub/ > fedora/linux/extras/development/i386 > > These are not valid links; however, I know that this one it: > http://download.fedora.redhat.com/pub/fedora/linux/releases/7/ > Everything/i386/os/ > > However, if I change > repo --name=d7 --baseurl= http://download.fedora.redhat.com/pub/ > fedora/linux/core/development/i386/os > to > repo --name=d7 --baseurl= http://download.fedora.redhat.com/pub/ > fedora/linux/releases/7/Everything/i386/os/ > > I still get the same error - "No Repositories Available to Set Up" > and no > livecd is created. > > How can I get this to point to a repository correct? Do I need to > have the correct "name" of the repo? How do I determine that? > > Thanks, > Dale > > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list -------------- next part -------------- An HTML attachment was scrubbed... URL: From zeter90 at yahoo.com Sat Jul 14 15:31:58 2007 From: zeter90 at yahoo.com (Varoon) Date: Sat, 14 Jul 2007 08:31:58 -0700 (PDT) Subject: [Fedora-livecd-list] installing gcc Message-ID: <921841.10664.qm@web36515.mail.mud.yahoo.com> i have install fedora 7live cd on my hard drive.However i am tring to install gcc 3.2 rpm file i used tihs commnad yum install *****.rpm However i get an error during install that yum can access a sayin that i need to acess a url ( fedora.mirror... the url was somthin glike that)to update..and i did not set up any network options --------------------------------- Expecting? Get great news right away with email Auto-Check. Try the Yahoo! Mail Beta. -------------- next part -------------- An HTML attachment was scrubbed... URL: From herlo1 at gmail.com Sat Jul 14 16:39:44 2007 From: herlo1 at gmail.com (Clint Savage) Date: Sat, 14 Jul 2007 10:39:44 -0600 Subject: [Fedora-livecd-list] installing gcc In-Reply-To: <921841.10664.qm@web36515.mail.mud.yahoo.com> References: <921841.10664.qm@web36515.mail.mud.yahoo.com> Message-ID: On 7/14/07, Varoon wrote: > > i have install fedora 7live cd on my hard drive.However i am tring to > install gcc 3.2 rpm file > i used tihs commnad > yum install *****.rpm > However i get an error during install that yum can access a sayin that i > need to acess a url ( fedora.mirror... the url was somthin glike that)to > update..and i did not set up any network options Varoon, If you are trying to install an rpm from your local filesystem, you cannot use the 'install' option of yum. Try 'yum localinstall ******.rpm' instead. It should make all the difference in the world. Cheers, Clint -------------- next part -------------- An HTML attachment was scrubbed... URL: From zeter90 at yahoo.com Sat Jul 14 18:22:50 2007 From: zeter90 at yahoo.com (Varoon) Date: Sat, 14 Jul 2007 11:22:50 -0700 (PDT) Subject: [Fedora-livecd-list] installing gcc Message-ID: <235251.22089.qm@web36509.mail.mud.yahoo.com> After executing the command u suggested here is the result. __________________________________________________ [root at localhost Desktop]# yum localinstall gcc-3.2-1.i386.rpm Loading "installonlyn" plugin Setting up Local Package Process Examining gcc-3.2-1.i386.rpm: gcc - 2:3.2-1.i386 Marking gcc-3.2-1.i386.rpm to be installed Could not retrieve mirrorlist http://mirrors.fedoraproject.org/mirrorlist?repo=updates-released-f7&arch=i386 error was [Errno 4] IOError: Error: Cannot open/read repomd.xml file for repository: updates --------------------------------- Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. -------------- next part -------------- An HTML attachment was scrubbed... URL: From herlo1 at gmail.com Sun Jul 15 01:26:07 2007 From: herlo1 at gmail.com (Clint Savage) Date: Sat, 14 Jul 2007 19:26:07 -0600 Subject: [Fedora-livecd-list] installing gcc In-Reply-To: <235251.22089.qm@web36509.mail.mud.yahoo.com> References: <235251.22089.qm@web36509.mail.mud.yahoo.com> Message-ID: On 7/14/07, Varoon wrote: > > After executing the command u suggested here is the result. > __________________________________________________ > [root at localhost Desktop]# yum localinstall gcc-3.2-1.i386.rpm > Loading "installonlyn" plugin > Setting up Local Package Process > Examining gcc-3.2-1.i386.rpm: gcc - 2:3.2-1.i386 > Marking gcc-3.2-1.i386.rpm to be installed > Could not retrieve mirrorlist > http://mirrors.fedoraproject.org/mirrorlist?repo=updates-released-f7&arch=i386error was > [Errno 4] IOError: resolution')> > Error: Cannot open/read repomd.xml file for repository: updates > Varoon, Your problem is related to the fact that the repository its trying to connect to either doesn't exist or you have some network problem. Check in /etc/yum.repos.d and find the url you see above (it will be similar but not exact) and try loading it in a web page. If it doesn't work, you'll need to fix the url to where it goes. I don't know the exact fix you need, but it is related to connecting to the url you're specifying. Its possible to disable the repository. use yum --help for info on that. Cheers, Clint -------------- next part -------------- An HTML attachment was scrubbed... URL: From dalestubblefield at gmail.com Mon Jul 16 12:51:52 2007 From: dalestubblefield at gmail.com (Dale Stubblefield) Date: Mon, 16 Jul 2007 07:51:52 -0500 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <20070713160014.B8605734BE@hormel.redhat.com> References: <20070713160014.B8605734BE@hormel.redhat.com> Message-ID: <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> Tim, I am aware of the replacement directory; however, when plugging a different baseurl into the repo line of the kickstart, I have not figured out how to determine the correct name of the repo. I am unable to use revisor because I wish to install a package that does not have a RPM. To do so, I must use the --shell flag with the livecd-creator command so that I can have the chrooted environment in which to do this. The version of Revisor of I have does not have this feature. Can you or anyone else provide me with a sample copy of either your kickstart file for livecd-creator OR what you are putting in the repo line OR what you are entering on the command line (in case you are specifying your repos that way)? Thanks, Dale Dale, this isn't exactly an answer, but a tool to deal with part of what you're running into. You can pop those paths into a browser to check them and (often) figure out what's wrong. For instance: > http://download.fedora.redhat.com/pub/fedora/linux/core/development/ > i386/os I copied the part through 'fedora' into my browser and started clicking down. It turns out 'development' is empty _but_ there's a development directory several levels up that (if memory serves) has replaced it. In something slightly closer to an answer, Revisor works quite nicely with everything I've thrown at it kickstart-wise (well... except that I can't use the -package syntax to delete an optional package from a group **grumble**). Several of the defaults work quite well. regards, Tim Wood -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim.wood at datawranglers.com Mon Jul 16 14:02:05 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Mon, 16 Jul 2007 08:02:05 -0600 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> References: <20070713160014.B8605734BE@hormel.redhat.com> <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> Message-ID: <62C5BA93-0E59-4998-981F-22CA22C653E7@datawranglers.com> Dale, I should have been clearer. Since most of the development has been going into Revisor, I've been using it (essentially) the way you'd use livecd. I point it at my kickstart and tell it to go. I could share my kickstart but it's pointed at a repository with a private IP so you'd crash and burn pretty quickly. But, if you install Revisor, in /etc/revisor/conf.d/ there are several kickstart files that seem to work quite nicely. I'm curious about the difference between the chroot'd environment and the way the post section is run under Revisor. I thought the later was chroot'd. I've got some commands (chkconfig, e.g.) in there that assume I'm chrooted and work. Of course, I probably just had a Willie E Coyote moment and looked down (holds up sign that says Momma) ;-) _________________________________ Tim Wood, CLP, RHCT 719.338.7484 (tel) 719.325.7032 (fax) The Data Wranglers Web, Database & more since since 1994 www.datawranglers.com On Jul 16, 2007, at 6:51 AM, Dale Stubblefield wrote: > Tim, > > I am aware of the replacement directory; however, when plugging a > different baseurl into the repo line of the kickstart, I have not > figured out how to determine the correct name of the repo. > > I am unable to use revisor because I wish to install a package that > does not have a RPM. To do so, I must use the --shell flag with > the livecd-creator command so that I can have the chrooted > environment in which to do this. The version of Revisor of I have > does not have this feature. > > Can you or anyone else provide me with a sample copy of either your > kickstart file for livecd-creator OR what you are putting in the > repo line OR what you are entering on the command line (in case you > are specifying your repos that way)? > > Thanks, > Dale From hunt at m2s.com Mon Jul 16 16:09:13 2007 From: hunt at m2s.com (Elias Hunt) Date: Mon, 16 Jul 2007 12:09:13 -0400 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> References: <20070713160014.B8605734BE@hormel.redhat.com> <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> Message-ID: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AD52@exchange2.ad.medicalmetrx.com> Dale, Here are the repo lines that we use for our livecds. This includes a custom repo that we create internally with rpm packages and then running the createrepo command against the local repository. repo --name=f7_everything --baseurl=http://download.fedora.redhat.com/pub/fedora/linux/releases/7/ Everything/i386/os repo --name=f7_updates --baseurl=http://download.fedora.redhat.com/pub/fedora/linux/updates/7/i 386 repo --name=local --baseurl=file:///usr/local/repo All three of those live in the same kickstart file and the package manager appears to be smart enough to pick up the most recent version of any packages it finds in multiple repositories. Hope this helps. -Eli From: fedora-livecd-list-bounces at redhat.com [mailto:fedora-livecd-list-bounces at redhat.com] On Behalf Of Dale Stubblefield Sent: Monday, July 16, 2007 8:52 AM To: fedora-livecd-list at redhat.com Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 Tim, I am aware of the replacement directory; however, when plugging a different baseurl into the repo line of the kickstart, I have not figured out how to determine the correct name of the repo. I am unable to use revisor because I wish to install a package that does not have a RPM. To do so, I must use the --shell flag with the livecd-creator command so that I can have the chrooted environment in which to do this. The version of Revisor of I have does not have this feature. Can you or anyone else provide me with a sample copy of either your kickstart file for livecd-creator OR what you are putting in the repo line OR what you are entering on the command line (in case you are specifying your repos that way)? Thanks, Dale Dale, this isn't exactly an answer, but a tool to deal with part of what you're running into. You can pop those paths into a browser to check them and (often) figure out what's wrong. For instance: > http://download.fedora.redhat.com/pub/fedora/linux/core/development/ > i386/os I copied the part through 'fedora' into my browser and started clicking down. It turns out 'development' is empty _but_ there's a development directory several levels up that (if memory serves) has replaced it. In something slightly closer to an answer, Revisor works quite nicely with everything I've thrown at it kickstart-wise (well... except that I can't use the -package syntax to delete an optional package from a group **grumble**). Several of the defaults work quite well. regards, Tim Wood -------------- next part -------------- An HTML attachment was scrubbed... URL: From kanarip at kanarip.com Mon Jul 16 18:57:11 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Mon, 16 Jul 2007 20:57:11 +0200 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <62C5BA93-0E59-4998-981F-22CA22C653E7@datawranglers.com> References: <20070713160014.B8605734BE@hormel.redhat.com> <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> <62C5BA93-0E59-4998-981F-22CA22C653E7@datawranglers.com> Message-ID: <469BBF87.70200@kanarip.com> Tim Wood wrote: > Dale, I should have been clearer. Since most of the development has > been going into Revisor, I've been using it (essentially) the way you'd > use livecd. I point it at my kickstart and tell it to go. I could > share my kickstart but it's pointed at a repository with a private IP so > you'd crash and burn pretty quickly. But, if you install Revisor, in > /etc/revisor/conf.d/ there are several kickstart files that seem to work > quite nicely. > > I'm curious about the difference between the chroot'd environment and > the way the post section is run under Revisor. I thought the later was > chroot'd. I've got some commands (chkconfig, e.g.) in there that assume > I'm chrooted and work. Of course, I probably just had a Willie E Coyote > moment and looked down (holds up sign that says Momma) ;-) > As Revisor uses livecd-tools for this job, it's pretty well chroot'ed. Kind regards, Jeroen van Meeuwen -kanarip From ccrayne at crayne.org Mon Jul 16 20:32:31 2007 From: ccrayne at crayne.org (Charles Crayne) Date: Mon, 16 Jul 2007 13:32:31 -0700 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AD52@exchange2.ad.medicalmetrx.com> References: <20070713160014.B8605734BE@hormel.redhat.com> <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AD52@exchange2.ad.medicalmetrx.com> Message-ID: <20070716133231.498b4349@thor.crayne.org> On Mon, 16 Jul 2007 12:09:13 -0400 "Elias Hunt" wrote: > Here are the repo lines that we use for our livecds. This includes a > custom repo that we create internally with rpm packages and then > running the createrepo command against the local repository. When running createrepo, do you use the groupfile option, and, if so, how do you create the group file? -- Chuck From hunt at m2s.com Mon Jul 16 21:00:44 2007 From: hunt at m2s.com (Elias Hunt) Date: Mon, 16 Jul 2007 17:00:44 -0400 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <20070716133231.498b4349@thor.crayne.org> References: <20070713160014.B8605734BE@hormel.redhat.com><13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com><50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AD52@exchange2.ad.medicalmetrx.com> <20070716133231.498b4349@thor.crayne.org> Message-ID: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962ADBC@exchange2.ad.medicalmetrx.com> We don't use the groupfile option. We just have a single directory that we put all in house created rpms into. Then run createrepo specifying that directory. Since all of our packages are in house we don't lump them into the default package groups, and we don't have custom groups defined either. A quick search revealed this to be a sample groupfile: http://mirror.centos.org/centos/4/extras/i386/yumgroups.xml Hopefully that will help you accomplish what you need. -Eli -----Original Message----- From: fedora-livecd-list-bounces at redhat.com [mailto:fedora-livecd-list-bounces at redhat.com] On Behalf Of Charles Crayne Sent: Monday, July 16, 2007 4:33 PM To: fedora-livecd-list at redhat.com Subject: Re: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27,Issue 15 On Mon, 16 Jul 2007 12:09:13 -0400 "Elias Hunt" wrote: > Here are the repo lines that we use for our livecds. This includes a > custom repo that we create internally with rpm packages and then > running the createrepo command against the local repository. When running createrepo, do you use the groupfile option, and, if so, how do you create the group file? -- Chuck -- Fedora-livecd-list mailing list Fedora-livecd-list at redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list From dmc.fedora at filteredperception.org Mon Jul 16 21:03:14 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Mon, 16 Jul 2007 16:03:14 -0500 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? Message-ID: <469BDD12.9080207@filteredperception.org> On the Fedora LiveCD wiki's wishlist, is 'persistance' - http://fedoraproject.org/wiki/FedoraLiveCD " * Persistence; e.g. the ability to save livecd changes (e.g. installed software) to a USB stick instead of tmpfs * LiveCD content + persistence all on a 1GB USB stick image " While I have plenty of ideas for this, and have vague memories of seeing other distributions do it, I'd like to get a feel for what the fedora community actually wants. I.e. lets design this feature. How should it work from the end user perspective? Also, there are a bazillion distros and livecds out there, if you happen to know of any that do this in a way that you like, please mention them. -dmc From kanarip at kanarip.com Mon Jul 16 21:30:12 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Mon, 16 Jul 2007 23:30:12 +0200 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469BDD12.9080207@filteredperception.org> References: <469BDD12.9080207@filteredperception.org> Message-ID: <469BE364.20500@kanarip.com> Douglas McClendon wrote: > On the Fedora LiveCD wiki's wishlist, is 'persistance' - > > http://fedoraproject.org/wiki/FedoraLiveCD > > " > * Persistence; e.g. the ability to save livecd changes (e.g. > installed software) to a USB stick instead of tmpfs > * LiveCD content + persistence all on a 1GB USB stick image > " > > While I have plenty of ideas for this, and have vague memories of seeing > other distributions do it, I'd like to get a feel for what the fedora > community actually wants. I.e. lets design this feature. How should it > work from the end user perspective? > > Also, there are a bazillion distros and livecds out there, if you happen > to know of any that do this in a way that you like, please mention them. > The first feature requests that when booting some media that you can't commit changes to, changes in tmpfs should be saved to a USB stick or something similar that holds a filesystem that we /can/ commit to, so that when you boot it again, you can roll-forward the changes from the 'tmpfs' on that medium and continue like it's a normal system. Does it make sense? The second just wants USB media like the LiveCD but with a real RW filesystem. Kind regards, Jeroen van Meeuwen -kanarip From katzj at redhat.com Mon Jul 16 21:38:03 2007 From: katzj at redhat.com (Jeremy Katz) Date: Mon, 16 Jul 2007 17:38:03 -0400 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469BDD12.9080207@filteredperception.org> References: <469BDD12.9080207@filteredperception.org> Message-ID: <1184621883.2947.8.camel@localhost.localdomain> On Mon, 2007-07-16 at 16:03 -0500, Douglas McClendon wrote: > While I have plenty of ideas for this, and have vague memories of seeing other > distributions do it, I'd like to get a feel for what the fedora community > actually wants. I.e. lets design this feature. How should it work from the end > user perspective? The crux of the idea is that you have a file on some form of read/write device that can store changes that you've made to the system. So, eg, if I boot with the live image (either off of CD or a usb key) and install emacs because I can't use a system without emacs, those changes are saved and the next time I boot, I have my emacs installed already. The simple and obvious way to do this is to be able to look for a file with a certain name (based on a UUID from the live image) located on either a USB key or a hard drive and then layer that as our snapshot dev rather than a sparse file in memory. That then leaves a few questions: 1) Do we then mount every partition we can find in the initramfs to get to the persistence file (with an option to disable, obviously)? Or do we require the device info to be specified in the bootloader. The first has the advantage of more "just working". The second is better for the case of not touching things and thus being a little bit safer. 2) How do we do the initial set up of this file? It's easy enough once they've booted the image to say "set up a persistence file for future use" and be able to prompt for where to keep it, etc. But then do we try to snapshot to it immediately and save any changes they've already made? Obviously this is better from a user perspective, but may be tricky to do sanely 3) When doing an install from the live image, do we keep the changes? If so, how does that interact with some of the hacky things we currently do in the live init script such as user addition and disabling firstboot? I somewhat suspect that the answer is to get together the quick path for each of these (explicit specification of where the persistence is, require manual set up) and just see how things work. Jeremy From katzj at redhat.com Mon Jul 16 21:39:15 2007 From: katzj at redhat.com (Jeremy Katz) Date: Mon, 16 Jul 2007 17:39:15 -0400 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469BE364.20500@kanarip.com> References: <469BDD12.9080207@filteredperception.org> <469BE364.20500@kanarip.com> Message-ID: <1184621955.2947.10.camel@localhost.localdomain> On Mon, 2007-07-16 at 23:30 +0200, Jeroen van Meeuwen wrote: > The first feature requests that when booting some media that you can't > commit changes to, changes in tmpfs should be saved to a USB stick or > something similar that holds a filesystem that we /can/ commit to, so > that when you boot it again, you can roll-forward the changes from the > 'tmpfs' on that medium and continue like it's a normal system. Does it > make sense? This is really the important one, see my other response for more details. > The second just wants USB media like the LiveCD but with a real RW > filesystem. I think that for this, you should just install to the USB stick. That should work these days and if it doesn't, then we should just fix it rather than papering over by allowing something like this. Jeremy From kanarip at kanarip.com Mon Jul 16 22:22:49 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Tue, 17 Jul 2007 00:22:49 +0200 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <1184621955.2947.10.camel@localhost.localdomain> References: <469BDD12.9080207@filteredperception.org> <469BE364.20500@kanarip.com> <1184621955.2947.10.camel@localhost.localdomain> Message-ID: <469BEFB9.8050004@kanarip.com> Jeremy Katz wrote: > On Mon, 2007-07-16 at 23:30 +0200, Jeroen van Meeuwen wrote: >> The first feature requests that when booting some media that you can't >> commit changes to, changes in tmpfs should be saved to a USB stick or >> something similar that holds a filesystem that we /can/ commit to, so >> that when you boot it again, you can roll-forward the changes from the >> 'tmpfs' on that medium and continue like it's a normal system. Does it >> make sense? > > This is really the important one, see my other response for more > details. > >> The second just wants USB media like the LiveCD but with a real RW >> filesystem. > > I think that for this, you should just install to the USB stick. That > should work these days and if it doesn't, then we should just fix it > rather than papering over by allowing something like this. > Right. Mind that it's not me who created the wish-list, I think it was David in the early days after FUDCon in Boston. The second (USB sticks) seems far more important to me then the stateless-but-persistent one. Kind regards, Jeroen van Meeuwen -kanarip From ccrayne at crayne.org Mon Jul 16 22:44:12 2007 From: ccrayne at crayne.org (Charles Crayne) Date: Mon, 16 Jul 2007 15:44:12 -0700 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962ADBC@exchange2.ad.medicalmetrx.com> References: <20070713160014.B8605734BE@hormel.redhat.com> <13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com> <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AD52@exchange2.ad.medicalmetrx.com> <20070716133231.498b4349@thor.crayne.org> <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962ADBC@exchange2.ad.medicalmetrx.com> Message-ID: <20070716154412.5f244a1f@thor.crayne.org> On Mon, 16 Jul 2007 17:00:44 -0400 "Elias Hunt" wrote: > Hopefully that will help you accomplish what you need. Thank you. It is a step in the right direction, but what I really need to do is to work around the fact that my download quota is only about 400MB per day. The Fedora repo is no problem, because I have purchased both the x86_64 and i386 distribution DVDs, but I need to build (and keep updated) local repos for such updates and additional packages which I want to include. Any tips on working under this limitation would be appreciated. -- Chuck From katzj at redhat.com Tue Jul 17 00:34:47 2007 From: katzj at redhat.com (Jeremy Katz) Date: Mon, 16 Jul 2007 20:34:47 -0400 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469BEFB9.8050004@kanarip.com> References: <469BDD12.9080207@filteredperception.org> <469BE364.20500@kanarip.com> <1184621955.2947.10.camel@localhost.localdomain> <469BEFB9.8050004@kanarip.com> Message-ID: <20070716203447.71a036bc@Nokia-N800-26> On Tue, 17 Jul 2007 00:22:49 +0200 Jeroen van Meeuwen wrote: > Jeremy Katz wrote: > > On Mon, 2007-07-16 at 23:30 +0200, Jeroen van Meeuwen wrote: > >> The second just wants USB media like the LiveCD but with a real RW > >> filesystem. > > > > I think that for this, you should just install to the USB stick. > > That should work these days and if it doesn't, then we should just > > fix it rather than papering over by allowing something like this. > > > > Right. Mind that it's not me who created the wish-list, I think it was > David in the early days after FUDCon in Boston. > > The second (USB sticks) seems far more important to me then the > stateless-but-persistent one. USB sticks play a big part in the first and really are in a lot of ways the more interesting case especially if you then do the second step of having an encrypted home dir setup. I ran with the second half of this while waiting on a replacement drive for my laptop; with work to make it easy to setup and to fix the weirdnesses (of which lack of persistence was the biggest as I had to keep reinstalling the packages I had forgotten in my config), this could be really compelling to use on a regular basis Jeremy From dmc.fedora at filteredperception.org Tue Jul 17 05:25:40 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 17 Jul 2007 00:25:40 -0500 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <1184621883.2947.8.camel@localhost.localdomain> References: <469BDD12.9080207@filteredperception.org> <1184621883.2947.8.camel@localhost.localdomain> Message-ID: <469C52D4.5070607@filteredperception.org> Jeremy Katz wrote: > > I somewhat suspect that the answer is to get together the quick path for > each of these (explicit specification of where the persistence is, > require manual set up) and just see how things work. I agree, and was planning on going ahead and trying out some ideas myself. I just want to make sure I take into consideration any particular use-cases that people may already have in mind. And in particular, I'm hoping that if a good end-user implementation already exists somewhere that I'm oblivious to, that someone mentions it. Knoppix, at least from what I see, still uses unionfs, so thats not interesting for us. ubuntu also uses unionfs, and their persistence support looks like a mess right now- https://help.ubuntu.com/community/LiveCDPersistence googling for "[gentoo/suse/mandriava] livecd persistence" didn't yield anything immediately obvious (look I managed to spell it correctly there :) Anyway, I'll see what I can do in the next week or two. -dmc From hunt at m2s.com Tue Jul 17 14:32:00 2007 From: hunt at m2s.com (Elias Hunt) Date: Tue, 17 Jul 2007 10:32:00 -0400 Subject: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27, Issue 15 In-Reply-To: <20070716154412.5f244a1f@thor.crayne.org> References: <20070713160014.B8605734BE@hormel.redhat.com><13a07b140707160551q6c0d3aeelec8a258b32b57a0d@mail.gmail.com><50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AD52@exchange2.ad.medicalmetrx.com><20070716133231.498b4349@thor.crayne.org><50B0D0F07E90AD4A9A5DFB3CC6C7ACA962ADBC@exchange2.ad.medicalmetrx.com> <20070716154412.5f244a1f@thor.crayne.org> Message-ID: <50B0D0F07E90AD4A9A5DFB3CC6C7ACA962AE02@exchange2.ad.medicalmetrx.com> I suspect you could mirror the updates repository locally under the 400MB/day mark, at least after the initial download which might take a couple days. Could be mistaken here, but I doubt it is a regular occurrence for updates to be more than 400MB/day. So I would just do an rsync mirror. This would also provide the benefit of pulling down the repo files, so you would not need to run createrepo on top of this. As for other packages, probably just pulling down what you can into a local mirror each day would work until you've gotten everything you need. You could have a daily script that runs createrepo on that folder to keep everything up-to-date. I used to do a number of script based mirrors like this in the past with bandwidth caps, it would take a couple days to get them setup, but then at worst they might be a day behind if too many updates occurred on the same day. -E -----Original Message----- From: fedora-livecd-list-bounces at redhat.com [mailto:fedora-livecd-list-bounces at redhat.com] On Behalf Of Charles Crayne Sent: Monday, July 16, 2007 6:44 PM To: fedora-livecd-list at redhat.com Subject: Re: [Fedora-livecd-list] Re: Fedora-livecd-list Digest, Vol 27,Issue 15 On Mon, 16 Jul 2007 17:00:44 -0400 "Elias Hunt" wrote: > Hopefully that will help you accomplish what you need. Thank you. It is a step in the right direction, but what I really need to do is to work around the fact that my download quota is only about 400MB per day. The Fedora repo is no problem, because I have purchased both the x86_64 and i386 distribution DVDs, but I need to build (and keep updated) local repos for such updates and additional packages which I want to include. Any tips on working under this limitation would be appreciated. -- Chuck -- Fedora-livecd-list mailing list Fedora-livecd-list at redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list From donvogt2001 at yahoo.com Tue Jul 17 16:50:02 2007 From: donvogt2001 at yahoo.com (don vogt) Date: Tue, 17 Jul 2007 09:50:02 -0700 (PDT) Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? Message-ID: <889324.62453.qm@web84102.mail.mud.yahoo.com> ate: Tue, 17 Jul 2007 00:25:40 -0500 From: Douglas McClendon Subject: Re: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? To: fedora-livecd-list at redhat.com Message-ID: <469C52D4.5070607 at filteredperception.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Jeremy Katz wrote: > > I somewhat suspect that the answer is to get together the quick path for > each of these (explicit specification of where the persistence is, > require manual set up) and just see how things work. >I agree, and was planning on going ahead and trying >out some ideas myself. I >just want to make sure I take into consideration any >particular use-cases that >people may already have in mind. And in particular, >I'm hoping that if a good >end-user implementation already exists somewhere that >I'm oblivious to, that >someone mentions it. >Knoppix, at least from what I see, still uses unionfs, >so thats not interesting >for us. >ubuntu also uses unionfs, and their persistence >support looks like a mess right > now- >https://help.ubuntu.com/community/LiveCDPersistence >googling for "[gentoo/suse/mandriava] livecd >persistence" didn't yield anything >immediately obvious (look I managed to spell it >correctly there :) >Anyway, I'll see what I can do in the next week or >two. -dmc I have been using centos livecd a little bit. I use their Perrsistence feature although I don't think it does all you have proposed here. The way I use it is: Boot the CD Configure it the way I like it, passwords, resolution Cookies and stuff in the browser,etc. When finished I click on a feature called "save local resources" The popup then asks me where to save it. In my case I specify /dev/hda ( I guess it could be a USB stick) The program then creates a directory called CENTOS_LIVECD (or something similar) on dev/hda and tars several files, /home, .etc/ password and so forth. When I boot later the boot process finds that directory and untars the files onto my system. I haven't played with it enough to know all that it does. For example if I installed a program outside of my home directory, I don't know if it would retrieve that. Also I have seen no documentation of this feature. I asked on the cent_os list and didn't get any answer. It seems like a worthy feature for a livecd if one wishes to use it a lot. From kanarip at kanarip.com Tue Jul 17 19:10:53 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Tue, 17 Jul 2007 21:10:53 +0200 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <889324.62453.qm@web84102.mail.mud.yahoo.com> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> Message-ID: <469D143D.9020707@kanarip.com> > It seems like a worthy feature for a livecd if one > wishes to use it a lot. On the other hand, why not compose the live cd or dvd with all that rightaway? Kind regards, Jeroen van Meeuwen -kanarip From dmc.fedora at filteredperception.org Tue Jul 17 19:58:47 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 17 Jul 2007 14:58:47 -0500 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469D143D.9020707@kanarip.com> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> <469D143D.9020707@kanarip.com> Message-ID: <469D1F77.5010405@filteredperception.org> Jeroen van Meeuwen wrote: >> It seems like a worthy feature for a livecd if one >> wishes to use it a lot. > > On the other hand, why not compose the live cd or dvd with all that > rightaway? The point of persistence is to have a nice fat distro on a cute 3" dvd, combined with a nice cute 1G usb stick, and be able to use the combination, as a physically portable linux system. The persistent usb holds _your home directory_. Do you want to be recomposing your livedvd _during every boot in which the contents of your home directory change_ ? -dmc > > Kind regards, > > Jeroen van Meeuwen > -kanarip > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From lbrooks at MIT.EDU Tue Jul 17 22:55:12 2007 From: lbrooks at MIT.EDU (Lane Brooks) Date: Tue, 17 Jul 2007 16:55:12 -0600 Subject: [Fedora-livecd-list] how do I install a local rpm Message-ID: <469D48D0.2030802@mit.edu> I have a few local rpm files that are not part of a yum repo that I would like to install on my live cd. What is the recommended way to do that? Thanks, Lane From patrice.guay at nanotechnologies.qc.ca Tue Jul 17 23:05:03 2007 From: patrice.guay at nanotechnologies.qc.ca (Patrice Guay) Date: Tue, 17 Jul 2007 19:05:03 -0400 Subject: [Fedora-livecd-list] how do I install a local rpm In-Reply-To: <469D48D0.2030802@mit.edu> References: <469D48D0.2030802@mit.edu> Message-ID: <469D4B1F.8080009@nanotechnologies.qc.ca> Lane Brooks wrote: > I have a few local rpm files that are not part of a yum repo that I > would like to install on my live cd. What is the recommended way to > do that? > If you are using the livecd-creator script, use the "--shell" switch. It will let you customize your live CD. -- Patrice From lbrooks at MIT.EDU Tue Jul 17 23:08:59 2007 From: lbrooks at MIT.EDU (Lane Brooks) Date: Tue, 17 Jul 2007 17:08:59 -0600 Subject: [Fedora-livecd-list] how do I install a local rpm In-Reply-To: <469D4B1F.8080009@nanotechnologies.qc.ca> References: <469D48D0.2030802@mit.edu> <469D4B1F.8080009@nanotechnologies.qc.ca> Message-ID: <469D4C0B.4050408@mit.edu> How do I make the rpms visible to the livecd-creator shell though? livecd-creator does a chroot and I do not know how to make the rpms visible since they are sitting on my local drive. Lane Patrice Guay wrote: > Lane Brooks wrote: >> I have a few local rpm files that are not part of a yum repo that I >> would like to install on my live cd. What is the recommended way to >> do that? >> > If you are using the livecd-creator script, use the "--shell" switch. It > will let you customize your live CD. > > -- > Patrice > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From patrice.guay at nanotechnologies.qc.ca Tue Jul 17 23:19:11 2007 From: patrice.guay at nanotechnologies.qc.ca (Patrice Guay) Date: Tue, 17 Jul 2007 19:19:11 -0400 Subject: [Fedora-livecd-list] how do I install a local rpm In-Reply-To: <469D4C0B.4050408@mit.edu> References: <469D48D0.2030802@mit.edu> <469D4B1F.8080009@nanotechnologies.qc.ca> <469D4C0B.4050408@mit.edu> Message-ID: <469D4E6F.1020003@nanotechnologies.qc.ca> Lane Brooks wrote: >>> I have a few local rpm files that are not part of a yum repo that I >>> would like to install on my live cd. What is the recommended way to >>> do that? >>> >> If you are using the livecd-creator script, use the "--shell" switch. It >> will let you customize your live CD. >> > How do I make the rpms visible to the livecd-creator shell though? > livecd-creator does a chroot and I do not know how to make the rpms > visible since they are sitting on my local drive. > You can put the rpms on a server and fetch the files with wget or scp. You can also put the files on a removable media (CD-ROM, usb key, floppy disk). This option will require much effort since you will have to mount the media inside the chroot environment before getting access to the files. -- Patrice From dmc.fedora at filteredperception.org Tue Jul 17 23:17:06 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 17 Jul 2007 18:17:06 -0500 Subject: [Fedora-livecd-list] how do I install a local rpm In-Reply-To: <469D48D0.2030802@mit.edu> References: <469D48D0.2030802@mit.edu> Message-ID: <469D4DF2.2040705@filteredperception.org> Lane Brooks wrote: > I have a few local rpm files that are not part of a yum repo that I > would like to install on my live cd. What is the recommended way to do > that? # on the host build system yum install createrepo mkdir /var/tmp/myrepo cp /path/to/my/*.rpm /var/tmp/myrepo/ createrepo /var/tmp/myrepo then in your kickstart (this is for livecd-creator, probably similar for revisor), add something like repo --name=myrepo --baseurl=file:///var/tmp/myrepo then add your rpm names to the package list. I'm a bit annoyed as to how hard it seems (from what little I've looked so far) to add arbitrary files, given the chroot. But for rpms, the above is pretty simple really. -dmc From lbrooks at MIT.EDU Wed Jul 18 04:12:35 2007 From: lbrooks at MIT.EDU (Lane Brooks) Date: Tue, 17 Jul 2007 22:12:35 -0600 Subject: [Fedora-livecd-list] how do I install a local rpm In-Reply-To: <469D4DF2.2040705@filteredperception.org> References: <469D48D0.2030802@mit.edu> <469D4DF2.2040705@filteredperception.org> Message-ID: <469D9333.6090404@mit.edu> Great. That worked perfectly. Thanks. Lane Douglas McClendon wrote: > Lane Brooks wrote: >> I have a few local rpm files that are not part of a yum repo that I >> would like to install on my live cd. What is the recommended way to >> do that? > > # on the host build system > yum install createrepo > > mkdir /var/tmp/myrepo > cp /path/to/my/*.rpm /var/tmp/myrepo/ > createrepo /var/tmp/myrepo > > then in your kickstart (this is for livecd-creator, probably similar for > revisor), add something like > > > repo --name=myrepo --baseurl=file:///var/tmp/myrepo > > then add your rpm names to the package list. > > I'm a bit annoyed as to how hard it seems (from what little I've looked > so far) to add arbitrary files, given the chroot. But for rpms, the > above is pretty simple really. > > -dmc > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list From alexm at asic.udl.cat Wed Jul 18 06:23:19 2007 From: alexm at asic.udl.cat (=?ISO-8859-1?Q?Alexandre_Magaz_Gra=E7a?=) Date: Wed, 18 Jul 2007 08:23:19 +0200 Subject: [Fedora-livecd-list] livecd-creator fails without mirrorlist repos Message-ID: <469DB1D7.5000302@asic.udl.cat> Hi, Running livecd-creator (latest git version) having all repo lines in ks without mirrorlist gives me this error: Traceback (most recent call last): File "/usr/bin/livecd-creator", line 1120, in sys.exit(main()) File "/usr/bin/livecd-creator", line 1097, in main target.parse(options.kscfg) File "/usr/bin/livecd-creator", line 319, in parse for cmd_name, cmd_url in self.repos: ValueError: too many values to unpack The repo lines in ks are: repo --name=fedora --baseurl=http://mirrors.ircam.fr/pub/fedora/linux/releases/7/Everything/i386/os repo --name=upd --baseurl=http://mirrors.ircam.fr/pub/fedora/linux/updates/7/i386 repo --name=livna --baseurl=http://rpm.livna.org/fedora/7/i386/ repo --name=local --baseurl=file:///var/www/html The attached patch solves the problem. Cheers, Alex -------------- next part -------------- A non-text attachment was scrubbed... Name: repos_without_mirrorlist.patch Type: text/x-patch Size: 546 bytes Desc: not available URL: From katzj at redhat.com Wed Jul 18 14:06:20 2007 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 18 Jul 2007 10:06:20 -0400 Subject: [Fedora-livecd-list] livecd-creator fails without mirrorlist repos In-Reply-To: <469DB1D7.5000302@asic.udl.cat> References: <469DB1D7.5000302@asic.udl.cat> Message-ID: <1184767580.9488.16.camel@localhost.localdomain> On Wed, 2007-07-18 at 08:23 +0200, Alexandre Magaz Gra?a wrote: > Running livecd-creator (latest git version) having all repo lines in ks > without mirrorlist gives me this error: Thanks; applied Jeremy From rnorwood at redhat.com Wed Jul 18 22:43:11 2007 From: rnorwood at redhat.com (Robin Norwood) Date: Wed, 18 Jul 2007 18:43:11 -0400 Subject: [Fedora-livecd-list] LiveCD for Red Hat High Message-ID: Hi, I thought I'd share this with the list, since it is based on your fine work. Last week, we held the second annual Red Hat High here in Raleigh. I helped with the software side of things, and used the livecd tools to do it. If you haven't heard about RHH, here are some articles: http://www.redhatmagazine.com/2007/07/09/red-hat-high-2007-getting-started/ http://www.redhatmagazine.com/2007/07/13/red-hat-high-update/ http://www.blendernation.com/2007/07/16/red-hat-high-2007/ In a nutshell, it's a technology camp for incoming high school freshmen, using all open source software. For the classrooms, we used lab space donated by NCSU. However, a couple of the labs were being used for NCSU classes during the week of the camp, so we couldn't just format the drives and install Fedora. Our solution was to use a live cd. I based the live cd .ks file off of the example ones provided in livecd-tools. We added some packages, made some changes to the %post script, and ended up with a very workable Fedora 7 system. Here's the .iso and .ks file if you're curious: http://people.redhat.com/rnorwood/rhh-livecd/ We used USB keys for persistent storage. If I'd had more time, I would've liked to either: o Make a live USB key with persistent storage. Probably make it so that the fedora user's homedir would be mounted rw off of the USB stick, and thus persist between boots. o Figure out some sort of network storage mechanism. We looked at gmailfs, but couldn't get it to work in time. For some of the other labs, where we were allowed to format the drives, I modified the livecd .ks file (mostly removed parts of the %post and added partitioning options), and made a kickstart iso to use that file. This worked perfectly. With some %include magic, I could've minimized the duplication between the two .ks files. I did run into a few technical glitches - one lab had systems with a very incompatible ATI video chipset that system-config-display would not configure properly automatically. A bit of manual tweaking produced an xorg.conf file that would boot the machine into an acceptable resolution. I took that file and produced a livecd which replaced the default xorg.conf with the tweaked one in the %post section. Some of the machine's bioses were not configured to allow booting from cdrom, but the NCSU lab folks were helpful there. Anyway, many thanks to the livecd folks, and I hope you enjoyed the success story. -RN -- Robin Norwood Red Hat, Inc. "The Sage does nothing, yet nothing remains undone." -Lao Tzu, Te Tao Ching From kanarip at kanarip.com Wed Jul 18 22:59:55 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Thu, 19 Jul 2007 00:59:55 +0200 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469D1F77.5010405@filteredperception.org> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> <469D143D.9020707@kanarip.com> <469D1F77.5010405@filteredperception.org> Message-ID: <469E9B6B.20609@kanarip.com> Douglas McClendon wrote: > Jeroen van Meeuwen wrote: >>> It seems like a worthy feature for a livecd if one >>> wishes to use it a lot. >> >> On the other hand, why not compose the live cd or dvd with all that >> rightaway? > > The point of persistence is to have a nice fat distro on a cute 3" dvd, > combined with a nice cute 1G usb stick, and be able to use the > combination, as a physically portable linux system. > > The persistent usb holds _your home directory_. Do you want to be > recomposing your livedvd _during every boot in which the contents of > your home directory change_ ? > For the portable Linux system use case, I'd take a larger USB stick or a USB hard drive and run from there. Kind regards, Jeroen van Meeuwen -kanarip From dmc.fedora at filteredperception.org Thu Jul 19 03:11:53 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 18 Jul 2007 22:11:53 -0500 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469E9B6B.20609@kanarip.com> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> <469D143D.9020707@kanarip.com> <469D1F77.5010405@filteredperception.org> <469E9B6B.20609@kanarip.com> Message-ID: <469ED679.7050402@filteredperception.org> Jeroen van Meeuwen wrote: > Douglas McClendon wrote: >> Jeroen van Meeuwen wrote: >>>> It seems like a worthy feature for a livecd if one >>>> wishes to use it a lot. >>> On the other hand, why not compose the live cd or dvd with all that >>> rightaway? >> The point of persistence is to have a nice fat distro on a cute 3" dvd, >> combined with a nice cute 1G usb stick, and be able to use the >> combination, as a physically portable linux system. >> >> The persistent usb holds _your home directory_. Do you want to be >> recomposing your livedvd _during every boot in which the contents of >> your home directory change_ ? >> > > For the portable Linux system use case, I'd take a larger USB stick or a > USB hard drive and run from there. Your preferred use case is certainly as valid or even moreso than the one I presented. But having both options seems ideal. Also, there may be an issue with usbflash data, and that in some instances it might be better to have it be mostly preburned as squashfs, rather than treated as a normal ext3fs. I.e. the whole jffs2 thing. Personally I rather like the idea of having my personal core system on read-only media, with just my homedir on flash. Admittedly, flash and portable disks will no doubt get cheaper and cheaper, but its still kind of nice to have the mass quanity of data preburned on cd/dvd/bluray... -peace... -dmc From dmc.fedora at filteredperception.org Thu Jul 19 03:40:13 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 18 Jul 2007 22:40:13 -0500 Subject: [Fedora-livecd-list] [PATCH] cleanupDeleted and container_size - saves 5.5% output size on f7livecd-i686 Message-ID: <469EDD1D.7060908@filteredperception.org> Attached is a patch. It could be broken into two patches for clarity. Please review and apply if appropriate. patch#1: add --container-size flag Using this flag, one can cause the os.img that sits in the squashfs.img, to be a larger sparse file than the filesystem contained in it. By default, the old behavior happens. One somewhat unrealistic, but real justification for this, would be booting a livecd on a system with 16G of ram. Without this flag, you would be limited to only writing 2G of data (given the existing f7-livecd-i686 choice of a 4G uncompressed-size, and about 2.1G of data). With this flag and a container of say 1T or 100G, there would be no negative consequences, but the livecd user would be able to online resize2fs their root filesystem upward if they liked. The more realistic usefulness comes if through a persistence implementation, the overlay file is not a 512M file stored in tmpfs, but an 8G file stored on an ipod. Everything mentioned above applies. patch#2: add --cleanup-deleted flag Using this flag, about 25% is added to the build time, but any files that were created and deleted in the install_root during installation, will not result in wasted space on the resulting iso. This happens because of the nature of the filesystem living on a sparse file. Anecdotally, this could be used to respin the fedora7-livecd-i686, from its existing 700MB, to 664MB. Or more likely, add another 100MB or so of uncompressed software onto it. This is of even greater importance as people spin their own livecds, and do intensive things during the installation, that might cause even more wasted space. Also, this somewhat lays the foundation for a 'turboinstaller' that improves current liveinst efficiency by 10-25% (for cdrom vs usbflash respectively), as mentioned in bug 248082 (pending efficiency tests with extra device mapper layers) Please review. Any comments, suggestions, or criticisms are welcome. I still consider myself a relative novice when it comes to python. peace... -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: livecd-creator.cleanupDeleted_and_containersize.patch Type: text/x-patch Size: 10693 bytes Desc: not available URL: From kanarip at kanarip.com Thu Jul 19 09:22:35 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Thu, 19 Jul 2007 11:22:35 +0200 Subject: [Fedora-livecd-list] [PATCH] cleanupDeleted and container_size - saves 5.5% output size on f7livecd-i686 In-Reply-To: <469EDD1D.7060908@filteredperception.org> References: <469EDD1D.7060908@filteredperception.org> Message-ID: <469F2D5B.8020006@kanarip.com> Douglas McClendon wrote: > Attached is a patch. > > It could be broken into two patches for clarity. Please review and > apply if appropriate. > > patch#1: add --container-size flag > Nice one. I'll wait and see if livecd-tools accepts it, and if they don't, I'll add it to Revisor. > > patch#2: add --cleanup-deleted flag > > Using this flag, about 25% is added to the build time, but any files > that were created and deleted in the install_root during installation, > will not result in wasted space on the resulting iso. This happens > because of the nature of the filesystem living on a sparse file. > Anecdotally, this could be used to respin the fedora7-livecd-i686, from > its existing 700MB, to 664MB. Or more likely, add another 100MB or so > of uncompressed software onto it. > Same as above. Kind regards, Jeroen van Meeuwen -kanarip From kanarip at kanarip.com Thu Jul 19 09:25:56 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Thu, 19 Jul 2007 11:25:56 +0200 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469ED679.7050402@filteredperception.org> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> <469D143D.9020707@kanarip.com> <469D1F77.5010405@filteredperception.org> <469E9B6B.20609@kanarip.com> <469ED679.7050402@filteredperception.org> Message-ID: <469F2E24.2030105@kanarip.com> Douglas McClendon wrote: > Your preferred use case is certainly as valid or even moreso than the > one I presented. But having both options seems ideal. Also, there may > be an issue with usbflash data, and that in some instances it might be > better to have it be mostly preburned as squashfs, rather than treated > as a normal ext3fs. I.e. the whole jffs2 thing. > Well, I'm not against anything here, I'm sorry if it looked that way. It just doesn't look like it's worth the effort to me personally. > Personally I rather like the idea of having my personal core system on > read-only media, with just my homedir on flash. > Right, that makes sense. A home directory from flash would be nice, but it wouldn't be really 'system persistence' would it? yum install foo and yum remove bar will not have foo and will have bar after a reboot, right? Kind regards, Jeroen van Meeuwen -kanarip From Matt_Domsch at dell.com Thu Jul 19 15:12:45 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Thu, 19 Jul 2007 10:12:45 -0500 Subject: [Fedora-livecd-list] [PATCH] alternate syslinux menus Message-ID: <20070719151245.GA10497@auslistsprd01.us.dell.com> livecd-creator look for multiple syslinux menus livecd-creator doesn't work on CentOS5 because the copy of syslinux in CentOS 5 doesn't include vesamenu.c32, but instead has menu.c32. Patch below has it look for both in priority order. Thanks, Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux diff --git a/creator/livecd-creator b/creator/livecd-creator index e227612..10cabce 100755 --- a/creator/livecd-creator +++ b/creator/livecd-creator @@ -823,9 +823,21 @@ class InstallationTarget: os.unlink("%s/install_root/boot/livecd-initramfs.img" %(self.build_dir,)) - for p in ["isolinux.bin", "vesamenu.c32"]: - path = "%s/install_root/usr/lib/syslinux/%s" % (self.build_dir, p) + syslinuxfiles = ["isolinux.bin"] + menus = ["vesamenu.c32", "menu.c32"] + syslinuxMenu = None + + for m in menus: + path = "%s/install_root/usr/lib/syslinux/%s" % (self.build_dir, m) + if os.path.isfile(path): + menus.append(m) + syslinuxMenu=m + break + if syslinuxMenu is None: + raise InstallationError("syslinux not installed : no suitable *menu.c32 found") + for p in syslinuxfiles: + path = "%s/install_root/usr/lib/syslinux/%s" % (self.build_dir, p) if not os.path.isfile(path): raise InstallationError("syslinux not installed : %s not found" % path) @@ -839,7 +851,7 @@ class InstallationTarget: have_background = "" cfg = """ -default vesamenu.c32 +default %(menu)s timeout 600 %(background)s @@ -851,7 +863,7 @@ menu color tabmsg 0 #ffffffff #00000000 menu color unsel 0 #ffffffff #00000000 menu color hotsel 0 #ff000000 #ffffffff menu color hotkey 7 #ffffffff #ff000000 -""" %{"label": self.fs_label, "background" : have_background} +""" %{"menu" : syslinuxMenu, "label": self.fs_label, "background" : have_background} stanzas = [("linux", "Run from image", ""), ("runfromram", "Run from RAM - requires 1 GB+", "live_ram")] From tim.wood at datawranglers.com Thu Jul 19 17:34:25 2007 From: tim.wood at datawranglers.com (Tim Wood) Date: Thu, 19 Jul 2007 11:34:25 -0600 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist- what does it =?UTF-8?Q?mean=3F?= In-Reply-To: <469F2E24.2030105@kanarip.com> References: <469F2E24.2030105@kanarip.com> Message-ID: <7c3243204cf8e419807dba4e806a51af@hputz.datawranglers.com> Ideally, for me, there'd be a persistance configuration governing what is persistant. Examples: 1) User wants to carry their data with them but not install packages 2) Someone wants package persistance 3) Someone wants a way to do custom configuration and then lock it down So, maybe you can do the following types of things: * specify paths that are persistant * specify whether those paths are modifiable (e.g. lock it down) * specify package persistance Maybe looking like this ... using a psuedo syntax that just hit me: [persistant paths] /home/user(rw) /etc(r) [persistant packages] * [persistant options] I guess the config would have to exist as say /etc/persistance.conf and be part of the persistant archive. Then joe user could go in, and change the /etc entry to (rw), reboot and then update a config file and then lock it down again and reboot. FWIW, this is somewhat similar to something I hacked together for a customized Knoppix disk. Timf Wood On Thu, 19 Jul 2007 11:25:56 +0200, Jeroen van Meeuwen wrote: > Douglas McClendon wrote: >> Your preferred use case is certainly as valid or even moreso than the >> one I presented. But having both options seems ideal. Also, there may >> be an issue with usbflash data, and that in some instances it might be >> better to have it be mostly preburned as squashfs, rather than treated >> as a normal ext3fs. I.e. the whole jffs2 thing. >> > > Well, I'm not against anything here, I'm sorry if it looked that way. It > just doesn't look like it's worth the effort to me personally. > >> Personally I rather like the idea of having my personal core system on >> read-only media, with just my homedir on flash. >> > > Right, that makes sense. A home directory from flash would be nice, but > it wouldn't be really 'system persistence' would it? yum install foo and > yum remove bar will not have foo and will have bar after a reboot, right? > > Kind regards, > > Jeroen van Meeuwen > -kanarip > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list -- _________________________________ Tim Wood, CLP, RHCT 719.338.7484 (tel) 719.325.7032 (fax) The Data Wranglers Web, Database & more since since 1994 www.datawranglers.com From dmc.fedora at filteredperception.org Thu Jul 19 18:11:49 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 19 Jul 2007 13:11:49 -0500 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469F2E24.2030105@kanarip.com> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> <469D143D.9020707@kanarip.com> <469D1F77.5010405@filteredperception.org> <469E9B6B.20609@kanarip.com> <469ED679.7050402@filteredperception.org> <469F2E24.2030105@kanarip.com> Message-ID: <469FA965.6010103@filteredperception.org> Jeroen van Meeuwen wrote: > Douglas McClendon wrote: >> Your preferred use case is certainly as valid or even moreso than the >> one I presented. But having both options seems ideal. Also, there may >> be an issue with usbflash data, and that in some instances it might be >> better to have it be mostly preburned as squashfs, rather than treated >> as a normal ext3fs. I.e. the whole jffs2 thing. >> > > Well, I'm not against anything here, I'm sorry if it looked that way. It > just doesn't look like it's worth the effort to me personally. Which is why I'm the one implementing it ;) > >> Personally I rather like the idea of having my personal core system on >> read-only media, with just my homedir on flash. >> > > Right, that makes sense. A home directory from flash would be nice, but > it wouldn't be really 'system persistence' would it? yum install foo and > yum remove bar will not have foo and will have bar after a reboot, right? Actually, what I said was a bit confusing, and made it sound more like I was going in the direction of what Don Vogt was saying that CentOS5-livecd did. In truth I really just meant doing the rootfs snapshot overlay as persistence, for as you say full 'system persistence', just that in practice, the majority of it would be used for my homedir. OTOH, this is another situation of where I can see it possibly being useful implementing both strategies, i.e. you can have the rootfs overlay on your usbstick, and/or a specially named .tgz which will get automatically untarred to / upon boot if it exists. I think I ought to be able to implement both by monday before the devel freeze... (he says ambitiously :) I'm in the process of attempting to follow the fedora8 feature proposal policy. Right now I'm stuck at figuring out how to make my first fedoraforum wiki page with the account I created last night. (whenever I click the link 'create new page' it throws up an error saying 'you are not allowed to edit this page'. This is after logging on, on _my_ empty user-homepage) -dmc From dmc.fedora at filteredperception.org Thu Jul 19 18:22:48 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 19 Jul 2007 13:22:48 -0500 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist- what does it mean? In-Reply-To: <7c3243204cf8e419807dba4e806a51af@hputz.datawranglers.com> References: <469F2E24.2030105@kanarip.com> <7c3243204cf8e419807dba4e806a51af@hputz.datawranglers.com> Message-ID: <469FABF8.5000802@filteredperception.org> Tim Wood wrote: > Ideally, for me, there'd be a persistance configuration governing what is > persistant. Examples: > > 1) User wants to carry their data with them but not install packages > 2) Someone wants package persistance > 3) Someone wants a way to do custom configuration and then lock it down > > > So, maybe you can do the following types of things: > > * specify paths that are persistant > * specify whether those paths are modifiable (e.g. lock it down) > * specify package persistance > This is actually not as doable for fedora since it uses dm-snapshot overlay rather than unionfs for its cow magic. OTOH, it is doable utilizing the alternate persistence method that I just alluded to in the reply to JvM I just sent. The downside, is that unlike using dm-snapshot overlay for persistence, implementing what you described would require more user involvement. I.e. if you go create a file in your homedir, then yank the plug on the computer, you would lose the file. Unlike with dm-snapshot-overlay-persistence, in which the file would be there. Admittedly, I've only given in 2-5 minutes of real thought, but I can't yet think of any way to provide that fine grain level of control with dm-snapshot-overlay as the persistence mechanism. Though honestly I'm the sort of person that would say "hey, lets invite unionfs back to the party for even more options" :) (though not for fedora8 timeframe... I'm not that crazy ;) peace... -dmc > > Maybe looking like this ... using a psuedo syntax that just hit me: > > [persistant paths] > /home/user(rw) > /etc(r) > > [persistant packages] > * > > [persistant options] > > > I guess the config would have to exist as say /etc/persistance.conf and be > part of the persistant archive. Then joe user could go in, and change the > /etc entry to (rw), reboot and then update a config file and then lock it > down again and reboot. > > FWIW, this is somewhat similar to something I hacked together for a > customized Knoppix disk. > > Timf Wood > > > > > > On Thu, 19 Jul 2007 11:25:56 +0200, Jeroen van Meeuwen > wrote: >> Douglas McClendon wrote: >>> Your preferred use case is certainly as valid or even moreso than the >>> one I presented. But having both options seems ideal. Also, there may >>> be an issue with usbflash data, and that in some instances it might be >>> better to have it be mostly preburned as squashfs, rather than treated >>> as a normal ext3fs. I.e. the whole jffs2 thing. >>> >> Well, I'm not against anything here, I'm sorry if it looked that way. It >> just doesn't look like it's worth the effort to me personally. >> >>> Personally I rather like the idea of having my personal core system on >>> read-only media, with just my homedir on flash. >>> >> Right, that makes sense. A home directory from flash would be nice, but >> it wouldn't be really 'system persistence' would it? yum install foo and >> yum remove bar will not have foo and will have bar after a reboot, right? >> >> Kind regards, >> >> Jeroen van Meeuwen >> -kanarip >> >> -- >> Fedora-livecd-list mailing list >> Fedora-livecd-list at redhat.com >> https://www.redhat.com/mailman/listinfo/fedora-livecd-list From Matt_Domsch at dell.com Thu Jul 19 19:49:26 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Thu, 19 Jul 2007 14:49:26 -0500 Subject: [Fedora-livecd-list] [PATCH] alternate syslinux menus (fixed) Message-ID: <20070719194926.GA4863@auslistsprd01.us.dell.com> livecd-creator look for multiple syslinux menus livecd-creator doesn't work on CentOS5 because the copy of syslinux in CentOS 5 doesn't include vesamenu.c32, but instead has menu.c32. Patch below has it look for both in priority order. Fixed to copy the right list of files into isolinux/. Thanks, Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux diff --git a/creator/livecd-creator b/creator/livecd-creator index e227612..10cabce 100755 --- a/creator/livecd-creator +++ b/creator/livecd-creator @@ -823,9 +823,21 @@ class InstallationTarget: os.unlink("%s/install_root/boot/livecd-initramfs.img" %(self.build_dir,)) - for p in ["isolinux.bin", "vesamenu.c32"]: - path = "%s/install_root/usr/lib/syslinux/%s" % (self.build_dir, p) + syslinuxfiles = ["isolinux.bin"] + menus = ["vesamenu.c32", "menu.c32"] + syslinuxMenu = None + + for m in menus: + path = "%s/install_root/usr/lib/syslinux/%s" % (self.build_dir, m) + if os.path.isfile(path): + syslinuxfiles.append(m) + syslinuxMenu=m + break + if syslinuxMenu is None: + raise InstallationError("syslinux not installed : no suitable *menu.c32 found") + for p in syslinuxfiles: + path = "%s/install_root/usr/lib/syslinux/%s" % (self.build_dir, p) if not os.path.isfile(path): raise InstallationError("syslinux not installed : %s not found" % path) @@ -839,7 +851,7 @@ class InstallationTarget: have_background = "" cfg = """ -default vesamenu.c32 +default %(menu)s timeout 600 %(background)s @@ -851,7 +863,7 @@ menu color tabmsg 0 #ffffffff #00000000 menu color unsel 0 #ffffffff #00000000 menu color hotsel 0 #ff000000 #ffffffff menu color hotkey 7 #ffffffff #ff000000 -""" %{"label": self.fs_label, "background" : have_background} +""" %{"menu" : syslinuxMenu, "label": self.fs_label, "background" : have_background} stanzas = [("linux", "Run from image", ""), ("runfromram", "Run from RAM - requires 1 GB+", "live_ram")] From jsteer at bitscout.com Thu Jul 19 20:37:18 2007 From: jsteer at bitscout.com (Jon Steer) Date: Thu, 19 Jul 2007 16:37:18 -0400 Subject: Fwd: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <74e6f65d0707191153s5449c239k1452ec5d19d5a21b@mail.gmail.com> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> <469D143D.9020707@kanarip.com> <469D1F77.5010405@filteredperception.org> <469E9B6B.20609@kanarip.com> <469ED679.7050402@filteredperception.org> <469F2E24.2030105@kanarip.com> <469FA965.6010103@filteredperception.org> <74e6f65d0707191153s5449c239k1452ec5d19d5a21b@mail.gmail.com> Message-ID: <74e6f65d0707191337v6cf9be78m70c0ae4d2dbc4c42@mail.gmail.com> I have used a livecd distro called devil-linux for some time. They have a "save-config-to-disk" that saves much of /etc and /var to disk. When the distro boots up, it looks for a tar file to unwrap all of /etc and /var from. If you add /home to the list of directories that are save, it works for me. jon On 7/19/07, Douglas McClendon wrote: > Jeroen van Meeuwen wrote: > > Douglas McClendon wrote: > >> Your preferred use case is certainly as valid or even moreso than the > >> one I presented. But having both options seems ideal. Also, there may > >> be an issue with usbflash data, and that in some instances it might be > >> better to have it be mostly preburned as squashfs, rather than treated > >> as a normal ext3fs. I.e. the whole jffs2 thing. > >> > > > > Well, I'm not against anything here, I'm sorry if it looked that way. It > > just doesn't look like it's worth the effort to me personally. > > Which is why I'm the one implementing it ;) > > > > >> Personally I rather like the idea of having my personal core system on > >> read-only media, with just my homedir on flash. > >> > > > > Right, that makes sense. A home directory from flash would be nice, but > > it wouldn't be really 'system persistence' would it? yum install foo and > > yum remove bar will not have foo and will have bar after a reboot, right? > Actually, what I said was a bit confusing, and made it sound more like I was > going in the direction of what Don Vogt was saying that CentOS5-livecd did. In > truth I really just meant doing the rootfs snapshot overlay as persistence, for > as you say full 'system persistence', just that in practice, the majority of it > would be used for my homedir. > > OTOH, this is another situation of where I can see it possibly being useful > implementing both strategies, i.e. you can have the rootfs overlay on your > usbstick, and/or a specially named .tgz which will get automatically untarred to > / upon boot if it exists. > > I think I ought to be able to implement both by monday before the devel > freeze... (he says ambitiously :) I'm in the process of attempting to follow > the fedora8 feature proposal policy. Right now I'm stuck at figuring out how to > make my first fedoraforum wiki page with the account I created last night. > (whenever I click the link 'create new page' it throws up an error saying 'you > are not allowed to edit this page'. This is after logging on, on _my_ empty > user-homepage) > > -dmc > > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > -- "Whereever you go, there you are" -- "Whereever you go, there you are" From jsteer at bitscout.com Thu Jul 19 20:40:11 2007 From: jsteer at bitscout.com (Jon Steer) Date: Thu, 19 Jul 2007 16:40:11 -0400 Subject: [Fedora-livecd-list] How to save to USB/Disk with FC6 LiveCd setup.. Message-ID: <74e6f65d0707191340j3df2e53j47600d4a64ce0433@mail.gmail.com> Hi, I am constrained to building liveCD's with the FC6 liveCD toolset right now. I have been searching through the archives for the scripts that will allow me to either save a FC6-based liveCD to disk or to USB. Did those ever exist? or did those tools only exist for F7? thanks, jon -- "Whereever you go, there you are" From vladimir.shebordaev at gmail.com Thu Jul 19 21:19:00 2007 From: vladimir.shebordaev at gmail.com (Vladimir Shebordaev) Date: Fri, 20 Jul 2007 01:19:00 +0400 Subject: [Fedora-livecd-list] How to save to USB/Disk with FC6 LiveCd setup.. In-Reply-To: <74e6f65d0707191340j3df2e53j47600d4a64ce0433@mail.gmail.com> References: <74e6f65d0707191340j3df2e53j47600d4a64ce0433@mail.gmail.com> Message-ID: <469FD544.8030208@gmail.com> Hi, Jon! Current LiveCD tools were much re-written, not to say, redesigned during last 5 months, so I guess they are not expected to support anything elder than FC7. It seems to be possible to backport 'em, but it's going to take some time for development and testing. Also it does not make much sense in general since FC7 released... In the hope it helps. Regards, Vladimir Jon Steer ?????: > Hi, > > I am constrained to building liveCD's with the FC6 liveCD toolset right > now. > > I have been searching through the archives for the scripts that will > allow me to either save a FC6-based liveCD to disk or to USB. > > Did those ever exist? or did those tools only exist for F7? > > thanks, > jon > > > > From dmc.fedora at filteredperception.org Thu Jul 19 21:20:24 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 19 Jul 2007 16:20:24 -0500 Subject: [Fedora-livecd-list] How to save to USB/Disk with FC6 LiveCd setup.. In-Reply-To: <74e6f65d0707191340j3df2e53j47600d4a64ce0433@mail.gmail.com> References: <74e6f65d0707191340j3df2e53j47600d4a64ce0433@mail.gmail.com> Message-ID: <469FD598.7030405@filteredperception.org> Jon Steer wrote: > Hi, > > I am constrained to building liveCD's with the FC6 liveCD toolset right > now. > > I have been searching through the archives for the scripts that will > allow me to either save a FC6-based liveCD to disk or to USB. I can't imagine why livecd-iso-to-disk wouldn't work for putting a livecd-creator based iso to disk (and livecd-creator did work in the fc6 timeframe... though I never actually used it myself). I.e. I don't think anything has happened with livecd-creator since fc6 that would cause the current version of livecd-iso-to-disk not to work. But if you try and it doesn't, post your experiment results and maybe something will come to mind. peace... -dmc > > Did those ever exist? or did those tools only exist for F7? > > thanks, > jon > > > > From cha.gascon at gmail.com Fri Jul 20 02:31:36 2007 From: cha.gascon at gmail.com (Cha Gascon) Date: Fri, 20 Jul 2007 10:31:36 +0800 Subject: [Fedora-livecd-list] RFC- 'persistance' is on LiveCD wishlist - what does it mean? In-Reply-To: <469F2E24.2030105@kanarip.com> References: <889324.62453.qm@web84102.mail.mud.yahoo.com> <469D143D.9020707@kanarip.com> <469D1F77.5010405@filteredperception.org> <469E9B6B.20609@kanarip.com> <469ED679.7050402@filteredperception.org> <469F2E24.2030105@kanarip.com> Message-ID: happy birthday to Jeroen van Meeuwen! :) sorry for the OT :p On 7/19/07, Jeroen van Meeuwen wrote: > Douglas McClendon wrote: > > Your preferred use case is certainly as valid or even moreso than the > > one I presented. But having both options seems ideal. Also, there may > > be an issue with usbflash data, and that in some instances it might be > > better to have it be mostly preburned as squashfs, rather than treated > > as a normal ext3fs. I.e. the whole jffs2 thing. > > > > Well, I'm not against anything here, I'm sorry if it looked that way. It > just doesn't look like it's worth the effort to me personally. > > > Personally I rather like the idea of having my personal core system on > > read-only media, with just my homedir on flash. > > > > Right, that makes sense. A home directory from flash would be nice, but > it wouldn't be really 'system persistence' would it? yum install foo and > yum remove bar will not have foo and will have bar after a reboot, right? > > Kind regards, > > Jeroen van Meeuwen > -kanarip > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > -- Marie Charisse L. Gascon http://www.chasys.net From patrice.guay at nanotechnologies.qc.ca Fri Jul 20 13:00:55 2007 From: patrice.guay at nanotechnologies.qc.ca (Patrice Guay) Date: Fri, 20 Jul 2007 09:00:55 -0400 Subject: [Fedora-livecd-list] [PATCH] cleanupDeleted and container_size - saves 5.5% output size on f7livecd-i686 In-Reply-To: <469EDD1D.7060908@filteredperception.org> References: <469EDD1D.7060908@filteredperception.org> Message-ID: <46A0B207.2@nanotechnologies.qc.ca> Douglas McClendon wrote: > patch#2: add --cleanup-deleted flag > > Using this flag, about 25% is added to the build time, but any files > that were created and deleted in the install_root during installation, > will not result in wasted space on the resulting iso. This happens > because of the nature of the filesystem living on a sparse file. > Anecdotally, this could be used to respin the fedora7-livecd-i686, > from its existing 700MB, to 664MB. Or more likely, add another 100MB > or so of uncompressed software onto it. > > This is of even greater importance as people spin their own livecds, > and do intensive things during the installation, that might cause even > more wasted space. > > Also, this somewhat lays the foundation for a 'turboinstaller' that > improves current liveinst efficiency by 10-25% (for cdrom vs usbflash > respectively), as mentioned in bug 248082 (pending efficiency tests > with extra device mapper layers) > > Please review. Any comments, suggestions, or criticisms are welcome. > I still consider myself a relative novice when it comes to python. > My current spin of a live CD went from 731MB to 615MB using the --cleanup-deleted flag. This allow me to put more stuff on the live CD while I was previously thinking about what to remove. Thanks, -- Patrice Guay From dmc.fedora at filteredperception.org Sat Jul 21 09:46:03 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sat, 21 Jul 2007 04:46:03 -0500 Subject: [Fedora-livecd-list] [PATCH] cleanupDeleted and container_size - saves 5.5% output size on f7livecd-i686 In-Reply-To: <469EDD1D.7060908@filteredperception.org> References: <469EDD1D.7060908@filteredperception.org> Message-ID: <46A1D5DB.30808@filteredperception.org> Douglas McClendon wrote: > Attached is a patch. > ... > Please review. Any comments, suggestions, or criticisms are welcome. I > still consider myself a relative novice when it comes to python. Just a heads up, I noticed one minor bug, in that the e2fsck invoked by the patch will fail if you have redirected the output of livecd-creator. Thus a -y (or -p?) needs to be added to that call. Since I have done more testing, and figured out that I can implement the 'turboinstaller' without even impacting mayflower, I'll probably post a new version of the patch this weekend sometime. Amongst my thoughts are a) is there really any reason to not make the container-size be 1T by default, and not even have the option? I guess I'll leave it as is for now, but suggest that f8t1 ship with 1T if nobody can suggest any reason not to. Likewise for the overlay which is currently 512M. b) the cp --sparse is a fairly time consuming step for very little return, so it seems like it might best be housed by a --resparse flag. c) the ugly heuristic for determining minimal resize amount, can be replaced by one which starts at 110% of calculated data size, and then homes in the target and even optomized down the smallest 4K amount. Thus I suspect it can be reduced from a process that takes a couple minutes, down to one which takes a couple seconds. d) given (b) and (c), and given that the benefits of cleanup-deleted are so immense (5.5% on stock f7i686-livecd, 20+% on Patrice Guay's use case), I think it should happen by default. e) I'll add a --turbo-liveinst flag (see bug 248082), which will cause a generation of the minimized overlay to sit alongside os.img, and then I'll create a patch to anaconda which just checks to see if it exists, and uses it if it does. I've run some tests, and the penalty appears to be about half a meg for the overlay file, and while I can't nail down the overhead for the devicemapper layer, it seems to be rather small, which means that the mechanism should provide a significant (? 5%-50% ?) benefit for installation speed, as well as removing the artificial limitation on installation volume size. (or maybe I'm wrong, we'll see...) peace... -dmc From dmc.fedora at filteredperception.org Sat Jul 21 10:29:56 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sat, 21 Jul 2007 05:29:56 -0500 Subject: [Fedora-livecd-list] [PATCH] cleanupDeleted and container_size - saves 5.5% output size on f7livecd-i686 In-Reply-To: <46A1D5DB.30808@filteredperception.org> References: <469EDD1D.7060908@filteredperception.org> <46A1D5DB.30808@filteredperception.org> Message-ID: <46A1E024.4040402@filteredperception.org> Douglas McClendon wrote: > Douglas McClendon wrote: >> Attached is a patch. >> > > ... > >> Please review. Any comments, suggestions, or criticisms are welcome. >> I still consider myself a relative novice when it comes to python. > ... > a) is there really any reason to not make the container-size be 1T by > default, and not even have the option? I guess I'll leave it as is for > now, but suggest that f8t1 ship with 1T if nobody can suggest any reason > not to. Likewise for the overlay which is currently 512M. Ok, the answer to this is that squashfs doesn't appear to be too brilliant about how it handles sparse files. So 1T as default is out of the question. Though making the mayflower initramfs overlay match the container still seems like a good idea. > > d) given (b) and (c), and given that the benefits of cleanup-deleted are > so immense (5.5% on stock f7i686-livecd, 20+% on Patrice Guay's use > case), I think it should happen by default. > make that 15%+, for those of you who know math :) -dmc From jsteer at bitscout.com Sat Jul 21 21:09:22 2007 From: jsteer at bitscout.com (Jon Steer) Date: Sat, 21 Jul 2007 17:09:22 -0400 Subject: [Fedora-livecd-list] How to save to USB/Disk with FC6 LiveCd setup.. In-Reply-To: <469FD598.7030405@filteredperception.org> References: <74e6f65d0707191340j3df2e53j47600d4a64ce0433@mail.gmail.com> <469FD598.7030405@filteredperception.org> Message-ID: <74e6f65d0707211409p16dbef9at29c9997d5e91e120@mail.gmail.com> Hi, I attempted to use isotostick.sh from the .8 F7 release. What I found was that it failed in this step. And any step that expected isolinux.cfg cp $CDMNT/isolinux/* $USBMNT/$SYSLINUXPATH sed -i -e "s/CDLABEL=[^ ]*/$USBLABEL/" -e "s/rootfstype=[^ ]*/rootfstype=$USBFS/" $USBMNT/$SYSLINUXPATH/isolinux.cfg mv $USBMNT/$SYSLINUXPATH/isolinux.cfg $USBM $SYSLINUXPATH/syslinux.cfg The ISO I created did not have isolinux on it. I saw the note about using a later version of syslinux on the box, but I'm not sure whether that is supposed to be on the ISO image? or on the system creating the USB key? thanks, jon On 7/19/07, Douglas McClendon wrote: > Jon Steer wrote: > > Hi, > > > > I am constrained to building liveCD's with the FC6 liveCD toolset right > > now. > > > > I have been searching through the archives for the scripts that will > > allow me to either save a FC6-based liveCD to disk or to USB. > > > I can't imagine why livecd-iso-to-disk wouldn't work for putting a > livecd-creator based iso to disk (and livecd-creator did work in the fc6 > timeframe... though I never actually used it myself). > > I.e. I don't think anything has happened with livecd-creator since fc6 that > would cause the current version of livecd-iso-to-disk not to work. But if you > try and it doesn't, post your experiment results and maybe something will come > to mind. > > peace... > > -dmc > > > > > > > Did those ever exist? or did those tools only exist for F7? > > > > thanks, > > jon > > > > > > > > > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > -- "Whereever you go, there you are" From dmc.fedora at filteredperception.org Sat Jul 21 21:27:30 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sat, 21 Jul 2007 16:27:30 -0500 Subject: [Fedora-livecd-list] [PATCH] cleanupDeleted and container_size - saves 5.5% output size on f7livecd-i686 In-Reply-To: <469EDD1D.7060908@filteredperception.org> References: <469EDD1D.7060908@filteredperception.org> Message-ID: <46A27A42.2040900@filteredperception.org> Douglas McClendon wrote: > patch#1: add --container-size flag > > Using this flag, one can cause the os.img that sits in the squashfs.img, > to be a larger sparse file than the filesystem contained in it. By > default, the old behavior happens. One somewhat unrealistic, but real > justification for this, would be booting a livecd on a system with 16G > of ram. Without this flag, you would be limited to only writing 2G of > data (given the existing f7-livecd-i686 choice of a 4G > uncompressed-size, and about 2.1G of data). With this flag and a > container of say 1T or 100G, there would be no negative consequences, > but the livecd user would be able to online resize2fs their root > filesystem upward if they liked. > > The more realistic usefulness comes if through a persistence > implementation, the overlay file is not a 512M file stored in tmpfs, but > an 8G file stored on an ipod. Everything mentioned above applies. > Ok, so there is a negative consequence of a large container. As I alluded to before, mksquashfs apparently does not give any special consideration to sparse files. Naturally it can compress a file containing huge blocks of 0s pretty well, but it still appears to be going to an aweful lot of work to do that. Short of adding more intelligent handling of sparse files to squashfs (feasible?), another alternative which satisfies the above resizing scenarios would be this- Instead of having a larger container, at runtime before the resize, reload the device mapper table for live-rw. Instead of the os.img loop device being the base of the snapshot device, create a new device which is a dm linear addition of os.img and a dm-zero device. This will add a penalty for the extra dm-layer, but given that this use-case is pretty obscure, it should be sufficient. Given this line of reasoning, I can no longer think of any need for the --container-size patch. New patch coming soon, with the other changes as well. -dmc From dmc.fedora at filteredperception.org Sun Jul 22 03:34:38 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Sat, 21 Jul 2007 22:34:38 -0500 Subject: [Fedora-livecd-list] Revised: [PATCH] cleanupDeleted Message-ID: <46A2D04E.90909@filteredperception.org> Changes from prior patch- - removed --container-size option, not really worthwhile - made cleanupDeleted default, revert to old behavior with --ignore-deleted - find minimal resize2fs size using a binary search accurate to the smallest possible blocksize - removed cp --sparse=always pass. not really worthwhile - added -y to e2fsck so that redirecting output didn't cause it to fail -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: livecd-creator.cleanup_deleted.patch Type: text/x-patch Size: 6022 bytes Desc: not available URL: From dmc.fedora at filteredperception.org Mon Jul 23 08:06:37 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Mon, 23 Jul 2007 03:06:37 -0500 Subject: [Fedora-livecd-list] [PATCH] --turbo-liveinst improves livecd installer speed by about 20% Message-ID: <46A4618D.6040008@filteredperception.org> Attached are a couple of rough alpha quality patches. They implement what I have described rather verbosely on fedora-livecd-list and in bug 248082. The short story is the fedora7 livecd installer works by copying a 4G ext3 image to the destination rootfs, and then resizing to maximal size. The attached patches improve the speed of this step by 10-30% (for cdrom vs fast livecd-iso-to-disk'd usbflash install media respectively). This is accomplished because the 4G image actually only holds 2G of data. The patch to livecd-creator, when invoked with --turbo-liveinst, will create a small (25kb) binary delta file on the livecd. The patch to anaconda, will detect the presence of the file, and if it is there, use it with devicemapper to create a 2G ext3 image, which can naturally be copied to the destination volume more quickly. (I.e. 2G of zeros don't get written to disk). In addition to improving installation speed, this also results in rootfs volumes of 2.1G->3.9G being supported. This patch is less than polished and elegant. But I'm hoping it might make it into f8t1. Please review, and give suggestions for improvements. I'll plan on trying to clean it up in whatever ways seem best over the next couple of days. These are patches against current livecd git and anaconda cvs snaps. I think they will work (very limited testing so far), but mainly I post them to get eyeballs and brains looking at the code and the idea. No doubt within 24-48 hours I will post a more respectable looking patchset that I hope more people might be willing to test. (note, the cleanupDeleted patch I just posted to livecd-tools is included here, as it lays the foundation for this... as well as reclaiming lots (5-15+%) of wasted space on the livecd) -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: livecd.turboliveinst.patch Type: text/x-patch Size: 13249 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: anaconda.turboliveinst.patch Type: text/x-patch Size: 2572 bytes Desc: not available URL: From patrice.guay at nanotechnologies.qc.ca Mon Jul 23 12:21:59 2007 From: patrice.guay at nanotechnologies.qc.ca (Patrice Guay) Date: Mon, 23 Jul 2007 08:21:59 -0400 Subject: [Fedora-livecd-list] [PATCH] alternate syslinux menus (fixed) In-Reply-To: <20070719194926.GA4863@auslistsprd01.us.dell.com> References: <20070719194926.GA4863@auslistsprd01.us.dell.com> Message-ID: <46A49D67.7030807@nanotechnologies.qc.ca> Matt Domsch wrote: > livecd-creator look for multiple syslinux menus > > livecd-creator doesn't work on CentOS5 because the copy of syslinux in > CentOS 5 doesn't include vesamenu.c32, but instead has menu.c32. > Patch below has it look for both in priority order. > > Fixed to copy the right list of files into isolinux/. > > Thanks, > Matt > I'm currently rebuilding a CentOS 5 Live CD with the livecd-creator script. I tried your patch with syslinux-3.11-4 installed both on the build host and the live CD. The resulting CD is usable. However, the timeout delay does not work. I had the same issue when I was using syslinux-3.36-2 (includes vesamenu.c32) with the livecd-creator script unpatched. The timeout countdown does not start. Did you notice the same behavior? -- Patrice Guay From katzj at redhat.com Mon Jul 23 18:32:50 2007 From: katzj at redhat.com (Jeremy Katz) Date: Mon, 23 Jul 2007 14:32:50 -0400 Subject: [Fedora-livecd-list] [PATCH] alternate syslinux menus (fixed) In-Reply-To: <46A49D67.7030807@nanotechnologies.qc.ca> References: <20070719194926.GA4863@auslistsprd01.us.dell.com> <46A49D67.7030807@nanotechnologies.qc.ca> Message-ID: <1185215570.15644.11.camel@localhost.localdomain> On Mon, 2007-07-23 at 08:21 -0400, Patrice Guay wrote: > However, the timeout delay does not work. I had the same issue when I > was using syslinux-3.36-2 (includes vesamenu.c32) with the > livecd-creator script unpatched. The timeout countdown does not start. > Did you notice the same behavior? Some syslinux versions have this bug. I don't remember when it was introduced, but based on looking at the changelog, I fixed it in 3.36-3. The fix should be able to be applied to any earlier version with the problem as well IIRC Jeremy From katzj at redhat.com Mon Jul 23 18:32:53 2007 From: katzj at redhat.com (Jeremy Katz) Date: Mon, 23 Jul 2007 14:32:53 -0400 Subject: [Fedora-livecd-list] [PATCH] alternate syslinux menus (fixed) In-Reply-To: <20070719194926.GA4863@auslistsprd01.us.dell.com> References: <20070719194926.GA4863@auslistsprd01.us.dell.com> Message-ID: <1185215573.15644.13.camel@localhost.localdomain> On Thu, 2007-07-19 at 14:49 -0500, Matt Domsch wrote: > livecd-creator look for multiple syslinux menus > > livecd-creator doesn't work on CentOS5 because the copy of syslinux in > CentOS 5 doesn't include vesamenu.c32, but instead has menu.c32. > Patch below has it look for both in priority order. > > Fixed to copy the right list of files into isolinux/. Thanks Matt; applied. Jeremy From dmc.fedora at filteredperception.org Tue Jul 24 08:43:57 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 24 Jul 2007 03:43:57 -0500 Subject: [Fedora-livecd-list] [PATCH] persistence - VERY ROUGH first pass Message-ID: <46A5BBCD.9000705@filteredperception.org> Attached is a _first pass_ patch that enables LiveOS persistence for livecd-tools. Before you read this and get _too_ excited, note that I'm really only posting this because of the f8t1 devel freeze. If I get any encouragement, I am very willing to work my ass off in the next few days to get this cleaned up into respectable shape for a potential f8t1 feature. But I also realize it is probably just too late for that. (OTOH the code is already structured so that it is obvious that in the absence of a persistence bootarg, the code has no impact) The Principals Of OPeration are- - unless a boot argument of "persistence[=[devspec][:pathspec]]" is seen, nothing will behave differently, except for the existence of a new tool /usr/sbin/livetool - after booting normally (no persistence bootarg), if you invoke "/usr/sbin/livetool persistence initialize [devspec][:pathspec]" then a persistence file will be initialized. By tomorrow I should have an option that allows the existing in-ram-overlay to be live migrated to the new file (thus freeing the ram by the time the command completes). I have done this manually. Currently this tool is not implemented. Creating a sparse or zeros file of any size (say 512M) should work fine. e.g. dd if=/dev/zero of=/mnt/stick/LiveOS/Persistence-mylabel \ bs=1k count=1 seek=$(( 1024 * 512) - to use the persistence file on a subsequent boot, add the bootarg "persistence" which is equivalent to "persistence=auto" - devspec can be in the following forms (should be obvious what they mean), and defaults to auto, which means to search all mountable partitions. (the search process mounts readonly (with paranoid blockdev --setro as well) sda1 sdb /dev/sdc3 LABEL=MyLabel UUID=MyUUID - pathspec defaults to /LiveOS/Persistence-LIVEOS_LABEL (e.g. /LiveOS/Persistence-Fedora-7-Live-i386) if pathspec does not start with a /, it prepends the above default. (e.g. persistence=:dmc would mean to search all partitions for /LiveOS/Persistence-Fedora-7-Live-i386-dmc) if pathspec starts with a / then it is used as is. e.g. persistence=:/home/dmc/testpers would search all partitions(and whole devices) for /home/dmc/testpers ) To round out the examples, persistence=LABEL="my usb stick":/my/path/to/file Finally: Known bugs: 1) I had to comment out the mayflower generated init's trap and set -e 2) ext3's inability to mounted readonly is annoying (yeah, I know norecovery or ext2 will probably get it for me, still annoying) 3) choice handling is broken. Don't use auto when it will find more than one entry. Maybe just don't use auto at all. 4) livetool doesn't do anything. For now, just create a sparse file of any interesting length (512M is fine) as mentioned above. Well, that about covers it for now. As always, comments, suggestions, and criticisms will be greatly appreciated. (though feel free to wait 48 hours for plenty of obvious improvements, such as fixing those known bugs). "release early, release often"... -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: livecd.persistence.patch Type: text/x-patch Size: 11656 bytes Desc: not available URL: From Parminder.Lehal at us.bosch.com Tue Jul 24 16:54:18 2007 From: Parminder.Lehal at us.bosch.com (Lehal Parminder (GS-FI/ENG3-NA)) Date: Tue, 24 Jul 2007 12:54:18 -0400 Subject: [Fedora-livecd-list] RE: Fedora-livecd-list Digest, Vol 27, Issue 25 In-Reply-To: <20070724160013.CF13073764@hormel.redhat.com> Message-ID: <2BA27942E9DB854993481A5BCAA99F6901CFC8AE@sbdmail14.us.bosch.com> Hello, How can we create a live CD containing files which are not in a RPM? More specifically, I need to create a live CD containing tomcat and a custom web application(jsp). The custom web application is in war format. Is it possible to do it without creating a rpm for the custom app? Regards, Parminder S. Lehal -----Original Message----- From: fedora-livecd-list-bounces at redhat.com [mailto:fedora-livecd-list-bounces at redhat.com] On Behalf Of fedora-livecd-list-request at redhat.com Sent: Tuesday, July 24, 2007 12:00 PM To: fedora-livecd-list at redhat.com Subject: Fedora-livecd-list Digest, Vol 27, Issue 25 Send Fedora-livecd-list mailing list submissions to fedora-livecd-list at redhat.com To subscribe or unsubscribe via the World Wide Web, visit https://www.redhat.com/mailman/listinfo/fedora-livecd-list or, via email, send a message with subject or body 'help' to fedora-livecd-list-request at redhat.com You can reach the person managing the list at fedora-livecd-list-owner at redhat.com When replying, please edit your Subject line so it is more specific than "Re: Contents of Fedora-livecd-list digest..." Today's Topics: 1. Re: [PATCH] alternate syslinux menus (fixed) (Jeremy Katz) 2. Re: [PATCH] alternate syslinux menus (fixed) (Jeremy Katz) 3. [PATCH] persistence - VERY ROUGH first pass (Douglas McClendon) ---------------------------------------------------------------------- Message: 1 Date: Mon, 23 Jul 2007 14:32:50 -0400 From: Jeremy Katz Subject: Re: [Fedora-livecd-list] [PATCH] alternate syslinux menus (fixed) To: fedora-livecd-list at redhat.com Message-ID: <1185215570.15644.11.camel at localhost.localdomain> Content-Type: text/plain On Mon, 2007-07-23 at 08:21 -0400, Patrice Guay wrote: > However, the timeout delay does not work. I had the same issue when I > was using syslinux-3.36-2 (includes vesamenu.c32) with the > livecd-creator script unpatched. The timeout countdown does not start. > Did you notice the same behavior? Some syslinux versions have this bug. I don't remember when it was introduced, but based on looking at the changelog, I fixed it in 3.36-3. The fix should be able to be applied to any earlier version with the problem as well IIRC Jeremy ------------------------------ Message: 2 Date: Mon, 23 Jul 2007 14:32:53 -0400 From: Jeremy Katz Subject: Re: [Fedora-livecd-list] [PATCH] alternate syslinux menus (fixed) To: fedora-livecd-list at redhat.com Message-ID: <1185215573.15644.13.camel at localhost.localdomain> Content-Type: text/plain On Thu, 2007-07-19 at 14:49 -0500, Matt Domsch wrote: > livecd-creator look for multiple syslinux menus > > livecd-creator doesn't work on CentOS5 because the copy of syslinux in > CentOS 5 doesn't include vesamenu.c32, but instead has menu.c32. > Patch below has it look for both in priority order. > > Fixed to copy the right list of files into isolinux/. Thanks Matt; applied. Jeremy ------------------------------ Message: 3 Date: Tue, 24 Jul 2007 03:43:57 -0500 From: Douglas McClendon Subject: [Fedora-livecd-list] [PATCH] persistence - VERY ROUGH first pass To: fedora-livecd-list at redhat.com Message-ID: <46A5BBCD.9000705 at filteredperception.org> Content-Type: text/plain; charset="iso-8859-1" Attached is a _first pass_ patch that enables LiveOS persistence for livecd-tools. Before you read this and get _too_ excited, note that I'm really only posting this because of the f8t1 devel freeze. If I get any encouragement, I am very willing to work my ass off in the next few days to get this cleaned up into respectable shape for a potential f8t1 feature. But I also realize it is probably just too late for that. (OTOH the code is already structured so that it is obvious that in the absence of a persistence bootarg, the code has no impact) The Principals Of OPeration are- - unless a boot argument of "persistence[=[devspec][:pathspec]]" is seen, nothing will behave differently, except for the existence of a new tool /usr/sbin/livetool - after booting normally (no persistence bootarg), if you invoke "/usr/sbin/livetool persistence initialize [devspec][:pathspec]" then a persistence file will be initialized. By tomorrow I should have an option that allows the existing in-ram-overlay to be live migrated to the new file (thus freeing the ram by the time the command completes). I have done this manually. Currently this tool is not implemented. Creating a sparse or zeros file of any size (say 512M) should work fine. e.g. dd if=/dev/zero of=/mnt/stick/LiveOS/Persistence-mylabel \ bs=1k count=1 seek=$(( 1024 * 512) - to use the persistence file on a subsequent boot, add the bootarg "persistence" which is equivalent to "persistence=auto" - devspec can be in the following forms (should be obvious what they mean), and defaults to auto, which means to search all mountable partitions. (the search process mounts readonly (with paranoid blockdev --setro as well) sda1 sdb /dev/sdc3 LABEL=MyLabel UUID=MyUUID - pathspec defaults to /LiveOS/Persistence-LIVEOS_LABEL (e.g. /LiveOS/Persistence-Fedora-7-Live-i386) if pathspec does not start with a /, it prepends the above default. (e.g. persistence=:dmc would mean to search all partitions for /LiveOS/Persistence-Fedora-7-Live-i386-dmc) if pathspec starts with a / then it is used as is. e.g. persistence=:/home/dmc/testpers would search all partitions(and whole devices) for /home/dmc/testpers ) To round out the examples, persistence=LABEL="my usb stick":/my/path/to/file Finally: Known bugs: 1) I had to comment out the mayflower generated init's trap and set -e 2) ext3's inability to mounted readonly is annoying (yeah, I know norecovery or ext2 will probably get it for me, still annoying) 3) choice handling is broken. Don't use auto when it will find more than one entry. Maybe just don't use auto at all. 4) livetool doesn't do anything. For now, just create a sparse file of any interesting length (512M is fine) as mentioned above. Well, that about covers it for now. As always, comments, suggestions, and criticisms will be greatly appreciated. (though feel free to wait 48 hours for plenty of obvious improvements, such as fixing those known bugs). "release early, release often"... -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: livecd.persistence.patch Type: text/x-patch Size: 11656 bytes Desc: not available Url : https://www.redhat.com/archives/fedora-livecd-list/attachments/20070724/ fc38d308/livecd.persistence.bin ------------------------------ -- Fedora-livecd-list mailing list Fedora-livecd-list at redhat.com https://www.redhat.com/mailman/listinfo/fedora-livecd-list End of Fedora-livecd-list Digest, Vol 27, Issue 25 ************************************************** From katzj at redhat.com Tue Jul 24 17:06:44 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 24 Jul 2007 13:06:44 -0400 Subject: [Fedora-livecd-list] Revised: [PATCH] cleanupDeleted In-Reply-To: <46A2D04E.90909@filteredperception.org> References: <46A2D04E.90909@filteredperception.org> Message-ID: <1185296804.7093.11.camel@localhost.localdomain> On Sat, 2007-07-21 at 22:34 -0500, Douglas McClendon wrote: > Changes from prior patch- > > - removed --container-size option, not really worthwhile > - made cleanupDeleted default, revert to old behavior with --ignore-deleted > - find minimal resize2fs size using a binary search accurate to the smallest > possible blocksize > - removed cp --sparse=always pass. not really worthwhile > - added -y to e2fsck so that redirecting output didn't cause it to fail Okay, this looks good at this point and doing it by default definitely feels like the right thing. Would be nice if the filesystem were smarter, though, as this is kind of ugly to have to do. Or at least if resize2fs had a minimal mode. Can you file something in bugzilla against e2fsprogs for the latter (feel free to cc me)? Jeremy From fedoracore.lists at gmail.com Tue Jul 24 17:14:12 2007 From: fedoracore.lists at gmail.com (jhon choptieso) Date: Tue, 24 Jul 2007 13:14:12 -0400 Subject: [Fedora-livecd-list] boot problem Message-ID: <9bc2adb00707241014p99c9516j3ef3178e1d34ec28@mail.gmail.com> greetings. I have boot problems. I used kadischi to build my liveCD. Here the log: #qemu -m 256 -cdrom fedora-live.iso ... losetup: failed to open /cdrom/kadischi.sqsh: No such file or directory. ... kernel panic. No such file or directory... some idea about why it passes that? -- jhon choptieso From katzj at redhat.com Tue Jul 24 18:18:12 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 24 Jul 2007 14:18:12 -0400 Subject: [Fedora-livecd-list] [PATCH] --turbo-liveinst improves livecd installer speed by about 20% In-Reply-To: <46A4618D.6040008@filteredperception.org> References: <46A4618D.6040008@filteredperception.org> Message-ID: <1185301092.7093.24.camel@localhost.localdomain> On Mon, 2007-07-23 at 03:06 -0500, Douglas McClendon wrote: > They implement what I have described rather verbosely on fedora-livecd-list and > in bug 248082. > > The short story is the fedora7 livecd installer works by copying a 4G ext3 image > to the destination rootfs, and then resizing to maximal size. > > The attached patches improve the speed of this step by 10-30% (for cdrom vs fast > livecd-iso-to-disk'd usbflash install media respectively). > > This is accomplished because the 4G image actually only holds 2G of data. The > patch to livecd-creator, when invoked with --turbo-liveinst, will create a small > (25kb) binary delta file on the livecd. The patch to anaconda, will detect the > presence of the file, and if it is there, use it with devicemapper to create a > 2G ext3 image, which can naturally be copied to the destination volume more > quickly. (I.e. 2G of zeros don't get written to disk). So I'm still not convinced that the wins here are really substantial enough given the additional contortions that we have to go through to get things going. Or that even if the win is worthwhile, that it's really the best path to take. Continuing to layer more levels of device-mapper indirection just leads to hitting ... interesting cases within the kernel as well as making it a lot harder to tell what's going on. For avoiding the writes and speeding things up, could we instead not be smarter in the copying we use to actually take the sparseness into account? As for being able to get to a smaller rootfs, I think that if you really want to get to requiring a small root, then we have to entirely change the copying to be more of a "copy the bits from the filesystem" as opposed to a block-level copy. That would also avoid the duplicated copies if you have, eg, a separate /usr (and also avoid the overhead of needing that space on the rootfs). This would also end up avoiding the copying overhead as well as the filesystem resizing, although you'd then have to format the entire fs. I think I'd be slightly more interested in doing that, although care has to be taken to ensure that we really are preserving everything then. > This patch is less than polished and elegant. But I'm hoping it might make it > into f8t1. Please review, and give suggestions for improvements. I'll plan on > trying to clean it up in whatever ways seem best over the next couple of days. I think trying to do this path for test1 is a little optimistic, although now we've got a good start to be able to be doing some experimentation immediately post-test1. > (note, the cleanupDeleted patch I just posted to livecd-tools is included here, > as it lays the foundation for this... as well as reclaiming lots (5-15+%) of > wasted space on the livecd) For future purposes, note that it's a lot easier if you don't do this. I mean, interdiff works but it just makes it harder to do a direct review. Saying that it depends on an earlier patch is simple to deal with. Jeremy From katzj at redhat.com Tue Jul 24 18:33:09 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 24 Jul 2007 14:33:09 -0400 Subject: [Fedora-livecd-list] [PATCH] persistence - VERY ROUGH first pass In-Reply-To: <46A5BBCD.9000705@filteredperception.org> References: <46A5BBCD.9000705@filteredperception.org> Message-ID: <1185301989.7093.35.camel@localhost.localdomain> On Tue, 2007-07-24 at 03:43 -0500, Douglas McClendon wrote: > Attached is a _first pass_ patch that enables LiveOS persistence for > livecd-tools. Before you read this and get _too_ excited, note that I'm really > only posting this because of the f8t1 devel freeze. If I get any encouragement, > I am very willing to work my ass off in the next few days to get this cleaned up > into respectable shape for a potential f8t1 feature. But I also realize it is > probably just too late for that. (OTOH the code is already structured so that it > is obvious that in the absence of a persistence bootarg, the code has no impact) Given where a few things stand[1], I'd prefer to hold off until post-test1. Although this definitely looks like a good start > The Principals Of OPeration are- > > - unless a boot argument of "persistence[=[devspec][:pathspec]]" is seen, > nothing will behave differently, except for the existence of a new tool > /usr/sbin/livetool Sounds good for now; we'll almost certainly want to make it default to automatic eventually with the option being to not do it. > - after booting normally (no persistence bootarg), if you invoke As long as nothing has been found, it should be fine to do this, no? Also, instead of using the label as the key, we might want to go for something better and UUID based. I'm just a little worried that labels aren't unique enough. > - devspec can be in the following forms (should be obvious what they mean), > and defaults to auto, which means to search all mountable partitions. (the > search process mounts readonly (with paranoid blockdev --setro as well) blockdev --setro is an awesome idea. Having to use fuse for ntfs is annoying at best, fragile if we go further. How much is it really worth to be able to have it on ntfs? > - pathspec defaults to /LiveOS/Persistence-LIVEOS_LABEL > (e.g. /LiveOS/Persistence-Fedora-7-Live-i386) > > if pathspec does not start with a /, it prepends the above default. > (e.g. persistence=:dmc would mean to search all partitions for > /LiveOS/Persistence-Fedora-7-Live-i386-dmc) Per above, I think that we want to have it tied to the UUID of the live image. At which point, specifying it becomes more for * Putting it buried somewhere on the filesystem other than /LiveOS * Handling multiple persistence on the same media The first I can sort of see... the second seems like it's probably less useful. Especially once the label isn't the important part > Finally: Known bugs: > > 1) I had to comment out the mayflower generated init's trap and set -e It might be worth integrating directly into the mayflower init. I'm not opposed to keeping it separate, though. Keeping it separate might actually help a little as I try to spend some time to get us to a more stock initramfs > 2) ext3's inability to mounted readonly is annoying (yeah, I know norecovery or > ext2 will probably get it for me, still annoying) Yeah -- what happens when you try to mount it with the block device read-only and a dirty journal? > 3) choice handling is broken. Don't use auto when it will find more than one > entry. Maybe just don't use auto at all. We definitely want auto. With more than one entry, maybe we should just bail? Asking for input is bound to be tricky given things like USB keyboards > Well, that about covers it for now. As always, comments, suggestions, and > criticisms will be greatly appreciated. (though feel free to wait 48 hours for > plenty of obvious improvements, such as fixing those known bugs). > > "release early, release often"... Thanks for getting a good start on this! Jeremy [1] Having to fight entirely unrelated things to get working images spit out just due to distro flux From dmc.fedora at filteredperception.org Tue Jul 24 21:34:35 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 24 Jul 2007 16:34:35 -0500 Subject: [Fedora-livecd-list] RE: Fedora-livecd-list Digest, Vol 27, Issue 25 In-Reply-To: <2BA27942E9DB854993481A5BCAA99F6901CFC8AE@sbdmail14.us.bosch.com> References: <2BA27942E9DB854993481A5BCAA99F6901CFC8AE@sbdmail14.us.bosch.com> Message-ID: <46A6706B.3050507@filteredperception.org> Lehal Parminder (GS-FI/ENG3-NA) wrote: > Hello, > > How can we create a live CD containing files which are not in a RPM? > More specifically, I need to create a live CD containing tomcat and a > custom web application(jsp). The custom web application is in war > format. Is it possible to do it without creating a rpm for the custom > app? I mentioned in a prior mail that I'm a bit annoyed by this as well. I may whip up a simple patch allowing args to livecd-creator to throw in extra directories. For now, I think the best way is to use the --shell option, and then do something like 'ls -altr /var/tmp/livecd-creator*' to find the installation root to copy your files into. But NOTE- you would have to do this in ANOTHER terminal, because in the shell livecd-creator gives you, you are stuck in the chroot. I haven't tested the above, but I think it should work. -dmc From katzj at redhat.com Tue Jul 24 21:49:48 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 24 Jul 2007 17:49:48 -0400 Subject: [Fedora-livecd-list] RE: Fedora-livecd-list Digest, Vol 27, Issue 25 In-Reply-To: <46A6706B.3050507@filteredperception.org> References: <2BA27942E9DB854993481A5BCAA99F6901CFC8AE@sbdmail14.us.bosch.com> <46A6706B.3050507@filteredperception.org> Message-ID: <1185313788.3705.0.camel@localhost.localdomain> On Tue, 2007-07-24 at 16:34 -0500, Douglas McClendon wrote: > Lehal Parminder (GS-FI/ENG3-NA) wrote: > > How can we create a live CD containing files which are not in a RPM? > > More specifically, I need to create a live CD containing tomcat and a > > custom web application(jsp). The custom web application is in war > > format. Is it possible to do it without creating a rpm for the custom > > app? > > I mentioned in a prior mail that I'm a bit annoyed by this as well. I may whip > up a simple patch allowing args to livecd-creator to throw in extra directories. > > For now, I think the best way is to use the --shell option, and then do > something like 'ls -altr /var/tmp/livecd-creator*' to find the installation root > to copy your files into. But NOTE- you would have to do this in ANOTHER > terminal, because in the shell livecd-creator gives you, you are stuck in the > chroot. You can also grab things in your config's %post section. That's what I've done in the past to accomplish this. I stuck files up on a webserver and just grabbed them with wget Jeremy From tchung at fedoraproject.org Tue Jul 24 21:52:06 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Tue, 24 Jul 2007 14:52:06 -0700 Subject: [Fedora-livecd-list] Digest Mode for fedora-livecd-list Message-ID: <369bce3b0707241452w63cc7026qf1e8b3baebafab9f@mail.gmail.com> All, I'd like to propose to turn off "digest mode" in the list. This means, users subscribing to this list with "digest mode", *will not* receive any mail from this list unless you turn off "digest mode" in your list preferences. Please let me know your opinion and "+1" if you agree. Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From Matt_Domsch at dell.com Tue Jul 24 22:00:51 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Tue, 24 Jul 2007 17:00:51 -0500 Subject: [Fedora-livecd-list] Digest Mode for fedora-livecd-list In-Reply-To: <369bce3b0707241452w63cc7026qf1e8b3baebafab9f@mail.gmail.com> References: <369bce3b0707241452w63cc7026qf1e8b3baebafab9f@mail.gmail.com> Message-ID: <20070724220051.GD5058@auslistsprd01.us.dell.com> On Tue, Jul 24, 2007 at 02:52:06PM -0700, Thomas Chung wrote: > All, > > I'd like to propose to turn off "digest mode" in the list. > This means, users subscribing to this list with "digest mode", *will > not* receive any mail from this list unless you turn off "digest mode" > in your list preferences. Why? To avoid thread/subject line breakage? -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From dmc.fedora at filteredperception.org Tue Jul 24 21:57:06 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 24 Jul 2007 16:57:06 -0500 Subject: [Fedora-livecd-list] Revised: [PATCH] cleanupDeleted In-Reply-To: <1185296804.7093.11.camel@localhost.localdomain> References: <46A2D04E.90909@filteredperception.org> <1185296804.7093.11.camel@localhost.localdomain> Message-ID: <46A675B2.4090400@filteredperception.org> Jeremy Katz wrote: > On Sat, 2007-07-21 at 22:34 -0500, Douglas McClendon wrote: >> Changes from prior patch- >> >> - removed --container-size option, not really worthwhile >> - made cleanupDeleted default, revert to old behavior with --ignore-deleted >> - find minimal resize2fs size using a binary search accurate to the smallest >> possible blocksize >> - removed cp --sparse=always pass. not really worthwhile >> - added -y to e2fsck so that redirecting output didn't cause it to fail > > Okay, this looks good at this point and doing it by default definitely > feels like the right thing. Would be nice if the filesystem were > smarter, though, as this is kind of ugly to have to do. Or at least if > resize2fs had a minimal mode. Can you file something in bugzilla > against e2fsprogs for the latter (feel free to cc me)? The thought had occurred to me. Actually I went so far as to look at the e2fsprogs code, to see if it wasn't really simple to add. What I discovered was that there are several places (not just one) where it discovers that it is too small (for varying reasons) and throws up an error. But yeah, I'll go ahead and file a bug. -dmc From tchung at fedoraproject.org Tue Jul 24 22:07:20 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Tue, 24 Jul 2007 15:07:20 -0700 Subject: [Fedora-livecd-list] Digest Mode for fedora-livecd-list In-Reply-To: <20070724220051.GD5058@auslistsprd01.us.dell.com> References: <369bce3b0707241452w63cc7026qf1e8b3baebafab9f@mail.gmail.com> <20070724220051.GD5058@auslistsprd01.us.dell.com> Message-ID: <369bce3b0707241507s543170d4x90e3e1561d360fe1@mail.gmail.com> On 7/24/07, Matt Domsch wrote: > On Tue, Jul 24, 2007 at 02:52:06PM -0700, Thomas Chung wrote: > > All, > > > > I'd like to propose to turn off "digest mode" in the list. > > This means, users subscribing to this list with "digest mode", *will > > not* receive any mail from this list unless you turn off "digest mode" > > in your list preferences. > > > Why? To avoid thread/subject line breakage? With Digest Mode on, users will receive email only once a month. This will cause nothing but problem with communication within a Project. Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From Matt_Domsch at dell.com Tue Jul 24 22:09:40 2007 From: Matt_Domsch at dell.com (Matt Domsch) Date: Tue, 24 Jul 2007 17:09:40 -0500 Subject: [Fedora-livecd-list] Digest Mode for fedora-livecd-list In-Reply-To: <369bce3b0707241507s543170d4x90e3e1561d360fe1@mail.gmail.com> References: <369bce3b0707241452w63cc7026qf1e8b3baebafab9f@mail.gmail.com> <20070724220051.GD5058@auslistsprd01.us.dell.com> <369bce3b0707241507s543170d4x90e3e1561d360fe1@mail.gmail.com> Message-ID: <20070724220940.GE5058@auslistsprd01.us.dell.com> On Tue, Jul 24, 2007 at 03:07:20PM -0700, Thomas Chung wrote: > On 7/24/07, Matt Domsch wrote: > >On Tue, Jul 24, 2007 at 02:52:06PM -0700, Thomas Chung wrote: > >> All, > >> > >> I'd like to propose to turn off "digest mode" in the list. > >> This means, users subscribing to this list with "digest mode", *will > >> not* receive any mail from this list unless you turn off "digest mode" > >> in your list preferences. > > > > > >Why? To avoid thread/subject line breakage? > > With Digest Mode on, users will receive email only once a month. > This will cause nothing but problem with communication within a > Project. This can be configured per-list to send it daily. How big in Kb should a digest be before it gets sent out? 30 Should a digest be dispatched daily when the size threshold isn't reached? No Yes -Matt -- Matt Domsch Linux Technology Strategist, Dell Office of the CTO linux.dell.com & www.dell.com/linux From tchung at fedoraproject.org Tue Jul 24 22:16:45 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Tue, 24 Jul 2007 15:16:45 -0700 Subject: [Fedora-livecd-list] Digest Mode for fedora-livecd-list In-Reply-To: <20070724220940.GE5058@auslistsprd01.us.dell.com> References: <369bce3b0707241452w63cc7026qf1e8b3baebafab9f@mail.gmail.com> <20070724220051.GD5058@auslistsprd01.us.dell.com> <369bce3b0707241507s543170d4x90e3e1561d360fe1@mail.gmail.com> <20070724220940.GE5058@auslistsprd01.us.dell.com> Message-ID: <369bce3b0707241516h44e5e471y2d47c4f24019311c@mail.gmail.com> On 7/24/07, Matt Domsch wrote: > This can be configured per-list to send it daily. > > > How big in Kb should a digest be before it gets sent out? > 30 > > Should a digest be dispatched daily when the size threshold isn't > reached? > No Yes Currently, it's configured as following: digest_size_threshhold: 60 kb digest_send_periodic: Yes. digest_volume_frequency: Monthly. Should we keep the current configuration and continue to allow "digest mode"? Regards, -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From dmc.fedora at filteredperception.org Tue Jul 24 22:30:14 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 24 Jul 2007 17:30:14 -0500 Subject: [Fedora-livecd-list] [PATCH] persistence - VERY ROUGH first pass In-Reply-To: <1185301989.7093.35.camel@localhost.localdomain> References: <46A5BBCD.9000705@filteredperception.org> <1185301989.7093.35.camel@localhost.localdomain> Message-ID: <46A67D76.8010406@filteredperception.org> Jeremy Katz wrote: > On Tue, 2007-07-24 at 03:43 -0500, Douglas McClendon wrote: >> Attached is a _first pass_ patch that enables LiveOS persistence for >> livecd-tools. Before you read this and get _too_ excited, note that I'm really >> only posting this because of the f8t1 devel freeze. If I get any encouragement, >> I am very willing to work my ass off in the next few days to get this cleaned up >> into respectable shape for a potential f8t1 feature. But I also realize it is >> probably just too late for that. (OTOH the code is already structured so that it >> is obvious that in the absence of a persistence bootarg, the code has no impact) > > Given where a few things stand[1], I'd prefer to hold off until > post-test1. Although this definitely looks like a good start agreed. Definitely would have felt more test1 worthy if it had been polished and tested and used for at least a couple weeks. > >> The Principals Of OPeration are- >> >> - unless a boot argument of "persistence[=[devspec][:pathspec]]" is seen, >> nothing will behave differently, except for the existence of a new tool >> /usr/sbin/livetool > > Sounds good for now; we'll almost certainly want to make it default to > automatic eventually with the option being to not do it. The only reason I would put up front as to not do it by default, would be adding boot time to _every_ livecd boot. But it may well be that the mount scanning only takes <2seconds, in which case, making it default might seem ok. And actually, I've got a different feature I've been working on for Elias Hunt, that could work well with the mount scan as well. (caching the iso/sqfs/ext3fsimg to disk) > >> - after booting normally (no persistence bootarg), if you invoke > > As long as nothing has been found, it should be fine to do this, no? > Also, instead of using the label as the key, we might want to go for > something better and UUID based. I'm just a little worried that labels > aren't unique enough. But UUIDs are just SO UGLY ;) But you are technically right. For my next iteration, I'll use both, i.e. default becomes /LiveOS/Persistence-LABEL-UUID Mainly having the label makes an ls in the directory actually somewhat intelligible to the user. > >> - devspec can be in the following forms (should be obvious what they mean), >> and defaults to auto, which means to search all mountable partitions. (the >> search process mounts readonly (with paranoid blockdev --setro as well) > > blockdev --setro is an awesome idea. Having to use fuse for ntfs is > annoying at best, fragile if we go further. How much is it really worth > to be able to have it on ntfs? Honestly it was entirely frustrating last night trying to use it. But for the next iteration at least, I'll try to keep in there. I admit this is a very debatable point. Speaking of which, there is a HUGE ISSUE that maybe people can chime in with advice on- How do I detect that a filesystem has not been cleanly unmounted? (ext3? ntfs? vfat??) It seems like the right thing to do is to consider not cleanly unmounted filesystems as offlimits (especially because they may be part of a hibernated system). > >> - pathspec defaults to /LiveOS/Persistence-LIVEOS_LABEL >> (e.g. /LiveOS/Persistence-Fedora-7-Live-i386) >> >> if pathspec does not start with a /, it prepends the above default. >> (e.g. persistence=:dmc would mean to search all partitions for >> /LiveOS/Persistence-Fedora-7-Live-i386-dmc) > > Per above, I think that we want to have it tied to the UUID of the live > image. At which point, specifying it becomes more for > * Putting it buried somewhere on the filesystem other than /LiveOS > * Handling multiple persistence on the same media > The first I can sort of see... the second seems like it's probably less > useful. Especially once the label isn't the important part See above. I agree with the why you presented. But I can see more of a use-case on the second. We'll see what people on this list think and do once the patch is actually polished and tested a bit. > >> Finally: Known bugs: >> >> 1) I had to comment out the mayflower generated init's trap and set -e > > It might be worth integrating directly into the mayflower init. I'm not > opposed to keeping it separate, though. Keeping it separate might > actually help a little as I try to spend some time to get us to a more > stock initramfs I'll keep it seperate for now. Once it starts getting to the point of merge worthy you can let me know what you prefer. > >> 2) ext3's inability to mounted readonly is annoying (yeah, I know norecovery or >> ext2 will probably get it for me, still annoying) > > Yeah -- what happens when you try to mount it with the block device > read-only and a dirty journal? "EXT3-fs write access unavailable cannot proceed" As mentioned, ext2 or norecovery will probably get around this though. It's just going to make the mount scanning code ugly to have to massively special case each filesystem type. Like for instance mount.ntfs not supporting the -n flag. It _seems_ like "mount -n -o ro -t auto" ought to work for the generic case (without blockdev --setro). But I guess thats asking too much at the moment :) > >> 3) choice handling is broken. Don't use auto when it will find more than one >> entry. Maybe just don't use auto at all. > > We definitely want auto. With more than one entry, maybe we should just > bail? Asking for input is bound to be tricky given things like USB > keyboards Agreed. This was only because I got frustrated coding in a hurry last night. If the USB keyboard thing is an issue, I can go back to my original thought, which was to have a timeout on the choice, and bail (=overlay-in-ram-as-normal) if no option is given. Anybody have any ideas on how to implement a ticking counter on a bash read prompt? ('read -t 60 choice' will work, but I'd like to have a ticking counter if possible) > >> Well, that about covers it for now. As always, comments, suggestions, and >> criticisms will be greatly appreciated. (though feel free to wait 48 hours for >> plenty of obvious improvements, such as fixing those known bugs). >> >> "release early, release often"... > > Thanks for getting a good start on this! > > Jeremy > > [1] Having to fight entirely unrelated things to get working images spit > out just due to distro flux Yeah, I know you and jkeating have lots on your plate. This is definitely nowhere near a candidate for f8t1. But if the deadline helped motivate me to get something done, all the better ;) -dmc From tchung at fedoraproject.org Tue Jul 24 22:49:43 2007 From: tchung at fedoraproject.org (Thomas Chung) Date: Tue, 24 Jul 2007 15:49:43 -0700 Subject: [Fedora-livecd-list] Digest Mode for fedora-livecd-list In-Reply-To: <369bce3b0707241516h44e5e471y2d47c4f24019311c@mail.gmail.com> References: <369bce3b0707241452w63cc7026qf1e8b3baebafab9f@mail.gmail.com> <20070724220051.GD5058@auslistsprd01.us.dell.com> <369bce3b0707241507s543170d4x90e3e1561d360fe1@mail.gmail.com> <20070724220940.GE5058@auslistsprd01.us.dell.com> <369bce3b0707241516h44e5e471y2d47c4f24019311c@mail.gmail.com> Message-ID: <369bce3b0707241549y1e88922cqb5597c300595eb87@mail.gmail.com> On 7/24/07, Thomas Chung wrote: > Currently, it's configured as following: > > digest_size_threshhold: 60 kb > digest_send_periodic: Yes. > digest_volume_frequency: Monthly. I was wrong about the list sending out digest monthly. Apparently, volume = monthly and issue = 60 kb. So it sends out every 60 kb for digest. Still, I would like to suggest everyone to turn off "digest mode" to avoid thread/subject line breakage. :) If you don't know how to turn it off, please contact me in private. Regards. -- Thomas Chung http://fedoraproject.org/wiki/ThomasChung From katzj at redhat.com Tue Jul 24 23:31:51 2007 From: katzj at redhat.com (Jeremy Katz) Date: Tue, 24 Jul 2007 19:31:51 -0400 Subject: [Fedora-livecd-list] [PATCH] persistence - VERY ROUGH first pass In-Reply-To: <46A67D76.8010406@filteredperception.org> References: <46A5BBCD.9000705@filteredperception.org> <1185301989.7093.35.camel@localhost.localdomain> <46A67D76.8010406@filteredperception.org> Message-ID: <1185319911.4950.7.camel@localhost.localdomain> On Tue, 2007-07-24 at 17:30 -0500, Douglas McClendon wrote: > Jeremy Katz wrote: > > On Tue, 2007-07-24 at 03:43 -0500, Douglas McClendon wrote: > >> The Principals Of OPeration are- > >> > >> - unless a boot argument of "persistence[=[devspec][:pathspec]]" is seen, > >> nothing will behave differently, except for the existence of a new tool > >> /usr/sbin/livetool > > > > Sounds good for now; we'll almost certainly want to make it default to > > automatic eventually with the option being to not do it. > > The only reason I would put up front as to not do it by default, would be adding > boot time to _every_ livecd boot. But it may well be that the mount scanning > only takes <2seconds, in which case, making it default might seem ok. The time will end up depending on how many devices you have. I suspect the benefits of being able to have it just work will outweigh the increase in boot time. We just then have to find up ways to counter the increase by making other things take less time ;-) > >> - devspec can be in the following forms (should be obvious what they mean), > >> and defaults to auto, which means to search all mountable partitions. (the > >> search process mounts readonly (with paranoid blockdev --setro as well) > > > > blockdev --setro is an awesome idea. Having to use fuse for ntfs is > > annoying at best, fragile if we go further. How much is it really worth > > to be able to have it on ntfs? > > Honestly it was entirely frustrating last night trying to use it. But for the > next iteration at least, I'll try to keep in there. I admit this is a very > debatable point. Speaking of which, there is a HUGE ISSUE that maybe people can > chime in with advice on- > > How do I detect that a filesystem has not been cleanly unmounted? (ext3? ntfs? > vfat??) It's filesystem dependent and I don't know of any utilities that just generally expose it. > It seems like the right thing to do is to consider not cleanly unmounted > filesystems as offlimits (especially because they may be part of a hibernated > system). Hmm, hibernation is a good point. We can detect if the system has been hibernated (from Linux) at least -- look for a swap with a signature of S2SUSPEND. And then we could not do so. > >> 2) ext3's inability to mounted readonly is annoying (yeah, I know norecovery or > >> ext2 will probably get it for me, still annoying) > > > > Yeah -- what happens when you try to mount it with the block device > > read-only and a dirty journal? > > "EXT3-fs write access unavailable cannot proceed" > > As mentioned, ext2 or norecovery will probably get around this though. It's > just going to make the mount scanning code ugly to have to massively special > case each filesystem type. Like for instance mount.ntfs not supporting the -n > flag. It _seems_ like "mount -n -o ro -t auto" ought to work for the generic > case (without blockdev --setro). But I guess thats asking too much at the moment :) kzak is trying to build up some momentum around a revived util-linux upstream, so it might be worth talking with him about it. It probably will come down to having to special case things, though :( > >> 3) choice handling is broken. Don't use auto when it will find more than one > >> entry. Maybe just don't use auto at all. > > > > We definitely want auto. With more than one entry, maybe we should just > > bail? Asking for input is bound to be tricky given things like USB > > keyboards > > Agreed. This was only because I got frustrated coding in a hurry last night. > If the USB keyboard thing is an issue, I can go back to my original thought, > which was to have a timeout on the choice, and bail (=overlay-in-ram-as-normal) > if no option is given. If there's a choice, then falling back to no persistence is definitely the sane thing to do > Anybody have any ideas on how to implement a ticking counter on a bash read > prompt? ('read -t 60 choice' will work, but I'd like to have a ticking counter > if possible) There's some countdown code in mayflower -- search for COUNTDOWN Jeremy From dmc.fedora at filteredperception.org Tue Jul 24 23:57:52 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Tue, 24 Jul 2007 18:57:52 -0500 Subject: [Fedora-livecd-list] [PATCH] --turbo-liveinst improves livecd installer speed by about 20% In-Reply-To: <1185301092.7093.24.camel@localhost.localdomain> References: <46A4618D.6040008@filteredperception.org> <1185301092.7093.24.camel@localhost.localdomain> Message-ID: <46A69200.3000101@filteredperception.org> Jeremy Katz wrote: > On Mon, 2007-07-23 at 03:06 -0500, Douglas McClendon wrote: >> They implement what I have described rather verbosely on fedora-livecd-list and >> in bug 248082. >> >> The short story is the fedora7 livecd installer works by copying a 4G ext3 image >> to the destination rootfs, and then resizing to maximal size. >> >> The attached patches improve the speed of this step by 10-30% (for cdrom vs fast >> livecd-iso-to-disk'd usbflash install media respectively). >> >> This is accomplished because the 4G image actually only holds 2G of data. The >> patch to livecd-creator, when invoked with --turbo-liveinst, will create a small >> (25kb) binary delta file on the livecd. The patch to anaconda, will detect the >> presence of the file, and if it is there, use it with devicemapper to create a >> 2G ext3 image, which can naturally be copied to the destination volume more >> quickly. (I.e. 2G of zeros don't get written to disk). > > So I'm still not convinced that the wins here are really substantial > enough given the additional contortions that we have to go through to > get things going. I understand your attitude. I will however do another round of polishing and performance testing in an attempt to convince you. Ultimately, I hope to at least convince you to do your own side-by-side taste test and confirm my results. :) Or that even if the win is worthwhile, that it's > really the best path to take. Continuing to layer more levels of > device-mapper indirection just leads to hitting ... interesting cases > within the kernel as well as making it a lot harder to tell what's going > on. For the sake of argument (i.e. I'm not fanatically attached to this patch), I would say that the interesting kernel cases, are probably useful ones to get the bugs out of sooner rather than later. As for making it harder to tell whats going on, I think after another round of polishing, it should actually not be terribly complicated. Actually, after you see some of the patches I'll be submitting in the coming months with other device mapper tricks, you might look fondly on the days when you thought this was too much complexity :) :) :) > For avoiding the writes and speeding things up, could we instead not be > smarter in the copying we use to actually take the sparseness into > account? Definitely a valid and interesting option. In the same vein, as I alluded to elsewhere, I think it might be interesting to see if squashfs could be made to support sparse files. I _think_ it currently knows nothing of sparseness. And the way I think it breaks up things into chunks, I suspect that the amount of space the 2G of sparseness in os.img, takes up in the squashed image, is significant. (I may test that one of these days and post my findings). Alas, my specialty is in throwing together commandline tools in devious ways. I could probably become knowledgable about ext3 and squashfs internals, but it's not really what I want to do in the near future. The fact that I can work around the absence of cool fs features with device mapper tricks... I just find very amusing, in a brain twisting sort of way. > > As for being able to get to a smaller rootfs, I think that if you really > want to get to requiring a small root, then we have to entirely change > the copying to be more of a "copy the bits from the filesystem" as > opposed to a block-level copy. That would also avoid the duplicated > copies if you have, eg, a separate /usr (and also avoid the overhead of > needing that space on the rootfs). This would also end up avoiding the > copying overhead as well as the filesystem resizing, although you'd then > have to format the entire fs. I think I'd be slightly more interested > in doing that, although care has to be taken to ensure that we really > are preserving everything then. I agree with all of this. Still, I go back to the existing patch, which hopefully should be a bit more elegant and polished soon, and suggest that it is at least worth seriously considering for F8. > >> This patch is less than polished and elegant. But I'm hoping it might make it >> into f8t1. Please review, and give suggestions for improvements. I'll plan on >> trying to clean it up in whatever ways seem best over the next couple of days. > > I think trying to do this path for test1 is a little optimistic, > although now we've got a good start to be able to be doing some > experimentation immediately post-test1. agreed. definitely not enough burn in testing for test1. But as with persistence, if the test1 devfreeze contributed to my motivation to get something out the door, all the better. > >> (note, the cleanupDeleted patch I just posted to livecd-tools is included here, >> as it lays the foundation for this... as well as reclaiming lots (5-15+%) of >> wasted space on the livecd) > > For future purposes, note that it's a lot easier if you don't do this. > I mean, interdiff works but it just makes it harder to do a direct > review. Saying that it depends on an earlier patch is simple to deal > with. Yeah, it was just laziness on my part. Not entirely related, but I still need to learn more about git, and where the heck 'diff --git' comes from. -dmc From kanarip at kanarip.com Wed Jul 25 12:12:15 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Wed, 25 Jul 2007 14:12:15 +0200 Subject: [Fedora-livecd-list] RE: Fedora-livecd-list Digest, Vol 27, Issue 25 In-Reply-To: <2BA27942E9DB854993481A5BCAA99F6901CFC8AE@sbdmail14.us.bosch.com> References: <2BA27942E9DB854993481A5BCAA99F6901CFC8AE@sbdmail14.us.bosch.com> Message-ID: <46A73E1F.4090804@kanarip.com> Lehal Parminder (GS-FI/ENG3-NA) wrote: > Hello, > > How can we create a live CD containing files which are not in a RPM? > More specifically, I need to create a live CD containing tomcat and a > custom web application(jsp). The custom web application is in war > format. Is it possible to do it without creating a rpm for the custom > app? > > Regards, > > Parminder S. Lehal > You could use the copy_dir and strip_copy_dir configuration directives in Revisor, too. Kind regards, Jeroen van Meeuwen -kanarip From kanarip at kanarip.com Wed Jul 25 12:30:54 2007 From: kanarip at kanarip.com (Jeroen van Meeuwen) Date: Wed, 25 Jul 2007 14:30:54 +0200 Subject: [Fedora-livecd-list] boot problem In-Reply-To: <9bc2adb00707241014p99c9516j3ef3178e1d34ec28@mail.gmail.com> References: <9bc2adb00707241014p99c9516j3ef3178e1d34ec28@mail.gmail.com> Message-ID: <46A7427E.5020902@kanarip.com> jhon choptieso wrote: > greetings. > I have boot problems. I used kadischi to build my liveCD. > > Here the log: > #qemu -m 256 -cdrom fedora-live.iso > ... > losetup: failed to open /cdrom/kadischi.sqsh: No such file or directory. > ... > kernel panic. > > No such file or directory... some idea about why it passes that? > Kadischi is End-Of-Life and is no longer supported. I suggest you use one of the other wonderful tools around, such as livecd-tools or revisor. Kind regards, Jeroen van Meeuwen -kanarip From katzj at redhat.com Wed Jul 25 14:33:03 2007 From: katzj at redhat.com (Jeremy Katz) Date: Wed, 25 Jul 2007 10:33:03 -0400 Subject: [Fedora-livecd-list] [PATCH] --turbo-liveinst improves livecd installer speed by about 20% In-Reply-To: <46A69200.3000101@filteredperception.org> References: <46A4618D.6040008@filteredperception.org> <1185301092.7093.24.camel@localhost.localdomain> <46A69200.3000101@filteredperception.org> Message-ID: <1185373983.11828.17.camel@localhost.localdomain> On Tue, 2007-07-24 at 18:57 -0500, Douglas McClendon wrote: > Jeremy Katz wrote: > > On Mon, 2007-07-23 at 03:06 -0500, Douglas McClendon wrote: > >> They implement what I have described rather verbosely on fedora-livecd-list and > >> in bug 248082. > >> > >> The short story is the fedora7 livecd installer works by copying a 4G ext3 image > >> to the destination rootfs, and then resizing to maximal size. > >> > >> The attached patches improve the speed of this step by 10-30% (for cdrom vs fast > >> livecd-iso-to-disk'd usbflash install media respectively). > >> > >> This is accomplished because the 4G image actually only holds 2G of data. The > >> patch to livecd-creator, when invoked with --turbo-liveinst, will create a small > >> (25kb) binary delta file on the livecd. The patch to anaconda, will detect the > >> presence of the file, and if it is there, use it with devicemapper to create a > >> 2G ext3 image, which can naturally be copied to the destination volume more > >> quickly. (I.e. 2G of zeros don't get written to disk). > > > > So I'm still not convinced that the wins here are really substantial > > enough given the additional contortions that we have to go through to > > get things going. > > I understand your attitude. I will however do another round of polishing and > performance testing in an attempt to convince you. Ultimately, I hope to at > least convince you to do your own side-by-side taste test and confirm my > results. :) *grin* Sounds good. > > As for being able to get to a smaller rootfs, I think that if you really > > want to get to requiring a small root, then we have to entirely change > > the copying to be more of a "copy the bits from the filesystem" as > > opposed to a block-level copy. That would also avoid the duplicated > > copies if you have, eg, a separate /usr (and also avoid the overhead of > > needing that space on the rootfs). This would also end up avoiding the > > copying overhead as well as the filesystem resizing, although you'd then > > have to format the entire fs. I think I'd be slightly more interested > > in doing that, although care has to be taken to ensure that we really > > are preserving everything then. > > I agree with all of this. Still, I go back to the existing patch, which > hopefully should be a bit more elegant and polished soon, and suggest that it is > at least worth seriously considering for F8. Doing a file level copy rather than the block copy currently being done, though, should be pretty straight-forward also. It'd just be changing the copy method in the livecd backend of anaconda to use the copytree stuff that's already there. Maybe I'll try to throw together a quick test to time later and see how it fares. I know it'll be a win for if you have a separate /usr, the question is just for the "base" case. Jeremy From nd_stew at yahoo.com Wed Jul 25 19:55:25 2007 From: nd_stew at yahoo.com (C S) Date: Wed, 25 Jul 2007 12:55:25 -0700 (PDT) Subject: [Fedora-livecd-list] LiveUSB to LiveUSB with F7? Message-ID: <284110.79530.qm@web32010.mail.mud.yahoo.com> All references I can find for spinning have a running host(with rpm, yum, etc) as a requirement to build live's upon. But is it possible to have livecd-creator spin a new CD off of F7's Live itself? I assume Revisor would only build upon this. Being a newbie I didn't know if it's in the design, plan or not possible at all... cs ____________________________________________________________________________________ Be a better Heartthrob. Get better relationship answers from someone who knows. Yahoo! Answers - Check it out. http://answers.yahoo.com/dir/?link=list&sid=396545433 From dmc.fedora at filteredperception.org Wed Jul 25 20:22:22 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 25 Jul 2007 15:22:22 -0500 Subject: [Fedora-livecd-list] LiveUSB to LiveUSB with F7? In-Reply-To: <284110.79530.qm@web32010.mail.mud.yahoo.com> References: <284110.79530.qm@web32010.mail.mud.yahoo.com> Message-ID: <46A7B0FE.30304@filteredperception.org> C S wrote: > All references I can find for spinning have a running > host(with rpm, yum, etc) as a requirement to build > live's upon. But is it possible to have > livecd-creator spin a new CD off of F7's Live itself? > I assume Revisor would only build upon this. Wow. Must be the collective unconsciousness. I was just thinking about this today. Of course, what you described, I think can be done pretty trivially. I.e. just get the livecd-tools /usr/bin/livecd-iso-to-disk (either by spinning your own livecd with the livecd-tools rpm, or wget/urlgrabbing a copy of the script from somewhere, etc...) and then doing livecd-iso-to-disk /dev/root /dev/ That is F7-LiveCD-Itself-to-LiveUSB. For F7-LiveCD-Itself-to-spin-new-cd you would do something like livecd-creator --base-on=/dev/root For your subject line of LiveUSB-to-LiveUSB, no easy script already does that that I know of (though dd might be sufficient for some cases). But of course, I have some devious added functionality via devicemapper in mind ;) -dmc > Being a newbie I didn't know if it's in the design, > plan or not possible at all... > > > cs From dmc.fedora at filteredperception.org Wed Jul 25 20:36:35 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 25 Jul 2007 15:36:35 -0500 Subject: [Fedora-livecd-list] LiveUSB to LiveUSB with F7? In-Reply-To: <46A7B0FE.30304@filteredperception.org> References: <284110.79530.qm@web32010.mail.mud.yahoo.com> <46A7B0FE.30304@filteredperception.org> Message-ID: <46A7B453.7020901@filteredperception.org> Douglas McClendon wrote: > C S wrote: >> All references I can find for spinning have a running >> host(with rpm, yum, etc) as a requirement to build >> live's upon. But is it possible to have >> livecd-creator spin a new CD off of F7's Live itself? I assume Revisor >> would only build upon this. > > Wow. Must be the collective unconsciousness. I was just thinking about > this today. > > Of course, what you described, I think can be done pretty trivially. > I.e. just get the livecd-tools /usr/bin/livecd-iso-to-disk (either by > spinning your own livecd with the livecd-tools rpm, or wget/urlgrabbing > a copy of the script from somewhere, etc...) and then doing > > livecd-iso-to-disk /dev/root /dev/ > err... make that /dev/live instead of /dev/root > That is F7-LiveCD-Itself-to-LiveUSB. For > F7-LiveCD-Itself-to-spin-new-cd you would do something like > livecd-creator --base-on=/dev/root > /dev/live here as well. Though you will need to get livecd-creator and dependencies installed. (yum install livecd-creator from the livecd will work if you have enough ram and a net connection). And you'll no doubt want to use --tmpdir pointing to some place with lots of space mounted (i.e. not use the default tmpdir of /var/tmp which would be in ram usually on a livecd) -dmc From nd_stew at yahoo.com Wed Jul 25 22:07:27 2007 From: nd_stew at yahoo.com (C S) Date: Wed, 25 Jul 2007 15:07:27 -0700 (PDT) Subject: [Fedora-livecd-list] LiveUSB to LiveUSB with F7? In-Reply-To: <46A7B453.7020901@filteredperception.org> Message-ID: <376370.30581.qm@web32010.mail.mud.yahoo.com> Gee, I was thinking the same, should be possible in theory, wanted to find out if I was the only one actually trying it... I put the repo off on a partition with livecd-creator, booted with a LiveUSB and setup the prereq's. I've tried it with and without the image in RAM with --tmpdir reassigned. However, during the install it seems to always fail usually in the vicinity of the kernel: ... localhost kernel: journal commit I/O errorrpmdb: fsync Input/output error error: db4 error(5) from db->sync: Input/output error Installing: python-libs ##################### [ 98/112] Ideas!? --- Douglas McClendon wrote: > Douglas McClendon wrote: > > C S wrote: > >> All references I can find for spinning have a > running > >> host(with rpm, yum, etc) as a requirement to > build > >> live's upon. But is it possible to have > >> livecd-creator spin a new CD off of F7's Live > itself? I assume Revisor > >> would only build upon this. > > > > Wow. Must be the collective unconsciousness. I > was just thinking about > > this today. > > > > Of course, what you described, I think can be done > pretty trivially. > > I.e. just get the livecd-tools > /usr/bin/livecd-iso-to-disk (either by > > spinning your own livecd with the livecd-tools > rpm, or wget/urlgrabbing > > a copy of the script from somewhere, etc...) and > then doing > > > > livecd-iso-to-disk /dev/root /dev/ partition here> > > > > err... make that /dev/live instead of /dev/root > > > > That is F7-LiveCD-Itself-to-LiveUSB. For > > F7-LiveCD-Itself-to-spin-new-cd you would do > something like > > livecd-creator --base-on=/dev/root > > > > /dev/live here as well. Though you will need to get > livecd-creator and > dependencies installed. (yum install livecd-creator > from the livecd will work > if you have enough ram and a net connection). And > you'll no doubt want to use > --tmpdir pointing to some place with lots of space > mounted (i.e. not use the > default tmpdir of /var/tmp which would be in ram > usually on a livecd) > > -dmc > > -- > Fedora-livecd-list mailing list > Fedora-livecd-list at redhat.com > https://www.redhat.com/mailman/listinfo/fedora-livecd-list > ____________________________________________________________________________________ Luggage? GPS? Comic books? Check out fitting gifts for grads at Yahoo! Search http://search.yahoo.com/search?fr=oni_on_mail&p=graduation+gifts&cs=bz From dmc.fedora at filteredperception.org Wed Jul 25 22:44:46 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 25 Jul 2007 17:44:46 -0500 Subject: [Fedora-livecd-list] LiveUSB to LiveUSB with F7? In-Reply-To: <376370.30581.qm@web32010.mail.mud.yahoo.com> References: <376370.30581.qm@web32010.mail.mud.yahoo.com> Message-ID: <46A7D25E.3050103@filteredperception.org> C S wrote: > Gee, I was thinking the same, should be possible in > theory, wanted to find out if I was the only one > actually trying it... > > I put the repo off on a partition with livecd-creator, > booted with a LiveUSB and setup the prereq's. I've > tried it with and without the image in RAM with > --tmpdir reassigned. > > However, during the install it seems to always fail > usually in the vicinity of the kernel: > > ... > > localhost kernel: journal commit I/O errorrpmdb: fsync > Input/output error > error: db4 error(5) from db->sync: Input/output error > Installing: python-libs > ##################### [ 98/112] > > > Ideas!? My first guess would be that you are running out of ram. Maybe another 256M would do it, or maybe David or Jeremy know of some aspect of the install that might be creating files outside of tmpdir (i.e. eating up your ram). If you've got a big liveusb, maybe once my persistence patch is ready for primetime, that would help (moving the RW space from ram to usb/disk). -dmc From dmc.fedora at filteredperception.org Wed Jul 25 23:06:16 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Wed, 25 Jul 2007 18:06:16 -0500 Subject: [Fedora-livecd-list] [PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs Message-ID: <46A7D768.9020900@filteredperception.org> attached is a patch which gaging from the history of this list, will be welcomed. I know there may already be some similar functionality already in revisor, this patch adds it to livecd-tools The two new options are --addidir= and --addsdir= Both may be invoked any number of times. The contents of the directories specified, will be copied to the target livecd. addsdir adds the contents to the rootfs of the target, while addidir adds the contents to the iso9660fs of the target. An example might be to use this to add an index.html and autorun.bat to the iso so that when the iso is popped in a winblowz machine, a webpage from the cdrom will automatically launch. The addidir copy happens just before mkisofs, so this can be used to easily replace /isolinux/splash.jpg The addsdir copy happens after installation, but before the selinux relabel and optional prelink. These semantics are up for debate. Test and enjoy... As always, comments, criticisms, suggestions are welcome. -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: livecd.addxdirs.patch Type: text/x-patch Size: 5640 bytes Desc: not available URL: From katzj at redhat.com Thu Jul 26 14:09:34 2007 From: katzj at redhat.com (Jeremy Katz) Date: Thu, 26 Jul 2007 10:09:34 -0400 Subject: [Fedora-livecd-list] [PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs In-Reply-To: <46A7D768.9020900@filteredperception.org> References: <46A7D768.9020900@filteredperception.org> Message-ID: <1185458974.6960.18.camel@localhost.localdomain> A few thoughts... On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote: > I know there may already be some similar functionality already in revisor, this > patch adds it to livecd-tools > > The two new options are > > --addidir= > and > --addsdir= First off, at a glance, the options just look like gobbledy-gook to me :-) In cases like this, it's almost always better to spell out what is meant rather than an abbreviation that isn't immediately obvious. The second thing is that having this as command-line options leads to an increased chance of the live image build not being reproducible. As much as possible, the movement has been to specifying things within the kickstart config rather than having command line options for specifying things. In fact, I'd like to overall entirely deprecate the use of most of the command-line arguments in favor of having them specified in the kickstart config. And at the same time, I also want to look at making sure that we include the kickstart config on the image that's created so that people can more easily see how the image was created, modify it, and then create their own. What do other people think of this? The things which really "matter" which aren't currently expressed in the config are prelink and the uncompressed size of the rootfs. The latter is easy (we just add interpretation of part / --size ...). Adding the former is probably not terrible to do either. > Both may be invoked any number of times. The contents of the directories > specified, will be copied to the target livecd. addsdir adds the contents to > the rootfs of the target, while addidir adds the contents to the iso9660fs of > the target. Then when and what of these occurring seems fine. Although I wonder if we'd be better off just making it so that you could do %post --nochroot and also moving %post to be run later Jeremy From dmc.fedora at filteredperception.org Thu Jul 26 15:22:44 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 26 Jul 2007 10:22:44 -0500 Subject: [Fedora-livecd-list] [PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs In-Reply-To: <1185458974.6960.18.camel@localhost.localdomain> References: <46A7D768.9020900@filteredperception.org> <1185458974.6960.18.camel@localhost.localdomain> Message-ID: <46A8BC44.9030700@filteredperception.org> Jeremy Katz wrote: > A few thoughts... > > On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote: >> I know there may already be some similar functionality already in revisor, this >> patch adds it to livecd-tools >> >> The two new options are >> >> --addidir= >> and >> --addsdir= > > First off, at a glance, the options just look like gobbledy-gook to > me :-) In cases like this, it's almost always better to spell out what > is meant rather than an abbreviation that isn't immediately obvious. aesthetics. rename however you like. > The second thing is that having this as command-line options leads to an > increased chance of the live image build not being reproducible. As > much as possible, the movement has been to specifying things within the > kickstart config rather than having command line options for specifying > things. I think this is where ideals meet practicality. But to work with you, would you be happier with a patch, where say, instead of --addsdir, there was --add-user-system-data, which behaved exactly the same, but the implementation mechanism included rolling the directory into a dirt simple noarch rpm with almost no data in the specfile? I.e. do the rpmbuild -bs, rpmbuild --rebuild, and createrepo for the user? I suppose it really isn't that hard (though it will be months before such a thing makes it to the top of my priorities). > > In fact, I'd like to overall entirely deprecate the use of most of the > command-line arguments in favor of having them specified in the > kickstart config. And at the same time, I also want to look at making > sure that we include the kickstart config on the image that's created so > that people can more easily see how the image was created, modify it, > and then create their own. What do other people think of this? The > things which really "matter" which aren't currently expressed in the > config are prelink and the uncompressed size of the rootfs. The latter > is easy (we just add interpretation of part / --size ...). Adding the > former is probably not terrible to do either. > >> Both may be invoked any number of times. The contents of the directories >> specified, will be copied to the target livecd. addsdir adds the contents to >> the rootfs of the target, while addidir adds the contents to the iso9660fs of >> the target. > > Then when and what of these occurring seems fine. Although I wonder if > we'd be better off just making it so that you could do %post --nochroot > and also moving %post to be run later I just saw a mention on kickstart-list, about using just such a scenario, and wondered if it was really possible. Of course, as far as reproducability goes, it is a bit wonky, as %post --nochroot seems to me to be just as much a supreme breaking of reproducability as the --addsdir thing. I.e. the environment outside the chroot, is going to be entirely different for every build (especially traditional anaconda kickstart usage, versus livecd-creator usage). Also, in my opinion, adding things to the isodir really has /nothing/ to do with the system being defined in the kickstart, and really should be separate (maybe even separate tool, 'livecd-repacker'?). I mean, suppose you want to roll your own mp3/ogg library on your CD. Not the sort of thing that seems appropriate for a kickstart. -dmc From katzj at redhat.com Thu Jul 26 18:51:35 2007 From: katzj at redhat.com (Jeremy Katz) Date: Thu, 26 Jul 2007 14:51:35 -0400 Subject: [Fedora-livecd-list] [PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs In-Reply-To: <46A8BC44.9030700@filteredperception.org> References: <46A7D768.9020900@filteredperception.org> <1185458974.6960.18.camel@localhost.localdomain> <46A8BC44.9030700@filteredperception.org> Message-ID: <1185475895.16236.20.camel@localhost.localdomain> On Thu, 2007-07-26 at 10:22 -0500, Douglas McClendon wrote: > Jeremy Katz wrote: > > A few thoughts... > > > > On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote: > >> I know there may already be some similar functionality already in revisor, this > >> patch adds it to livecd-tools > >> > >> The two new options are > >> > >> --addidir= > >> and > >> --addsdir= > > > > First off, at a glance, the options just look like gobbledy-gook to > > me :-) In cases like this, it's almost always better to spell out what > > is meant rather than an abbreviation that isn't immediately obvious. > > aesthetics. rename however you like. Yeah, definitely a minor quibble that's easy to resolve. > > The second thing is that having this as command-line options leads to an > > increased chance of the live image build not being reproducible. As > > much as possible, the movement has been to specifying things within the > > kickstart config rather than having command line options for specifying > > things. > > I think this is where ideals meet practicality. But to work with you, would you > be happier with a patch, where say, instead of --addsdir, there was > --add-user-system-data, which behaved exactly the same, but the implementation > mechanism included rolling the directory into a dirt simple noarch rpm with > almost no data in the specfile? I.e. do the rpmbuild -bs, rpmbuild --rebuild, > and createrepo for the user? That doesn't really change anything, though. It still fundamentally means that the image can't be reproducibly created from its config. > > In fact, I'd like to overall entirely deprecate the use of most of the > > command-line arguments in favor of having them specified in the > > kickstart config. And at the same time, I also want to look at making > > sure that we include the kickstart config on the image that's created so > > that people can more easily see how the image was created, modify it, > > and then create their own. What do other people think of this? The > > things which really "matter" which aren't currently expressed in the > > config are prelink and the uncompressed size of the rootfs. The latter > > is easy (we just add interpretation of part / --size ...). Adding the > > former is probably not terrible to do either. > > > >> Both may be invoked any number of times. The contents of the directories > >> specified, will be copied to the target livecd. addsdir adds the contents to > >> the rootfs of the target, while addidir adds the contents to the iso9660fs of > >> the target. > > > > Then when and what of these occurring seems fine. Although I wonder if > > we'd be better off just making it so that you could do %post --nochroot > > and also moving %post to be run later > > I just saw a mention on kickstart-list, about using just such a scenario, and > wondered if it was really possible. Of course, as far as reproducability goes, > it is a bit wonky, as %post --nochroot seems to me to be just as much a supreme > breaking of reproducability as the --addsdir thing. I.e. the environment > outside the chroot, is going to be entirely different for every build > (especially traditional anaconda kickstart usage, versus livecd-creator usage). The difference is that you can at least keep it in mind and be doing, eg, a wget to grab files. Sure, it doesn't prevent you from doing something like using a local file, but there's nothing now that stops you from using a file:// repo. > Also, in my opinion, adding things to the isodir really has /nothing/ to do with > the system being defined in the kickstart, and really should be separate (maybe > even separate tool, 'livecd-repacker'?). I mean, suppose you want to roll your > own mp3/ogg library on your CD. Not the sort of thing that seems appropriate > for a kickstart. But if I want to include a library of demos or videos about Fedora to help educate a new user, I do want them there. So yes, there are cases why reproducibility might not matter. But as a general case, it really does matter quite a bit Jeremy From dmc.fedora at filteredperception.org Thu Jul 26 20:19:03 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 26 Jul 2007 15:19:03 -0500 Subject: [Fedora-livecd-list] [PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs In-Reply-To: <1185475895.16236.20.camel@localhost.localdomain> References: <46A7D768.9020900@filteredperception.org> <1185458974.6960.18.camel@localhost.localdomain> <46A8BC44.9030700@filteredperception.org> <1185475895.16236.20.camel@localhost.localdomain> Message-ID: <46A901B7.2070608@filteredperception.org> Jeremy Katz wrote: > On Thu, 2007-07-26 at 10:22 -0500, Douglas McClendon wrote: >> Jeremy Katz wrote: >>> A few thoughts... >>> >>> On Wed, 2007-07-25 at 18:06 -0500, Douglas McClendon wrote: >>>> I know there may already be some similar functionality already in revisor, this >>>> patch adds it to livecd-tools >>>> >>>> The two new options are >>>> >>>> --addidir= >>>> and >>>> --addsdir= >>> The second thing is that having this as command-line options leads to an >>> increased chance of the live image build not being reproducible. As >>> much as possible, the movement has been to specifying things within the >>> kickstart config rather than having command line options for specifying >>> things. >> I think this is where ideals meet practicality. But to work with you, would you >> be happier with a patch, where say, instead of --addsdir, there was >> --add-user-system-data, which behaved exactly the same, but the implementation >> mechanism included rolling the directory into a dirt simple noarch rpm with >> almost no data in the specfile? I.e. do the rpmbuild -bs, rpmbuild --rebuild, >> and createrepo for the user? > > That doesn't really change anything, though. It still fundamentally > means that the image can't be reproducibly created from its config. If reproducibility from a single text file config is that important to you, then you are correct. Short of uuencoding tarballs, there is no way to get there from here. -dmc From katzj at redhat.com Thu Jul 26 21:07:20 2007 From: katzj at redhat.com (Jeremy Katz) Date: Thu, 26 Jul 2007 17:07:20 -0400 Subject: [Fedora-livecd-list] [PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs In-Reply-To: <46A901B7.2070608@filteredperception.org> References: <46A7D768.9020900@filteredperception.org> <1185458974.6960.18.camel@localhost.localdomain> <46A8BC44.9030700@filteredperception.org> <1185475895.16236.20.camel@localhost.localdomain> <46A901B7.2070608@filteredperception.org> Message-ID: <1185484040.16236.31.camel@localhost.localdomain> On Thu, 2007-07-26 at 15:19 -0500, Douglas McClendon wrote: > Jeremy Katz wrote: > > That doesn't really change anything, though. It still fundamentally > > means that the image can't be reproducibly created from its config. > > If reproducibility from a single text file config is that important to you, then > you are correct. Short of uuencoding tarballs, there is no way to get there > from here. We're already dependent on being able to get to the repo -- being dependent on grabbing things from a web server or similar also for "added" pieces isn't crazy I don't think. Although doing it in a way that's flexible enough to _allow_ local files is fine, I don't want to be showing that as the "one true way" Jeremy From dmc.fedora at filteredperception.org Thu Jul 26 21:57:15 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Thu, 26 Jul 2007 16:57:15 -0500 Subject: [Fedora-livecd-list] [PATCH] --addXdir , facility to easily add external directories to target sysrootfs OR isofs In-Reply-To: <1185484040.16236.31.camel@localhost.localdomain> References: <46A7D768.9020900@filteredperception.org> <1185458974.6960.18.camel@localhost.localdomain> <46A8BC44.9030700@filteredperception.org> <1185475895.16236.20.camel@localhost.localdomain> <46A901B7.2070608@filteredperception.org> <1185484040.16236.31.camel@localhost.localdomain> Message-ID: <46A918BB.2090502@filteredperception.org> Jeremy Katz wrote: > On Thu, 2007-07-26 at 15:19 -0500, Douglas McClendon wrote: >> Jeremy Katz wrote: >>> That doesn't really change anything, though. It still fundamentally >>> means that the image can't be reproducibly created from its config. >> If reproducibility from a single text file config is that important to you, then >> you are correct. Short of uuencoding tarballs, there is no way to get there >> from here. > > We're already dependent on being able to get to the repo -- being > dependent on grabbing things from a web server or similar also for > "added" pieces isn't crazy I don't think. Although doing it in a way > that's flexible enough to _allow_ local files is fine, I don't want to > be showing that as the "one true way" I'd honestly by perfectly happy if they were completely undocumented options, except at the bottom of the existing /usr/share/doc/livecd-tools-version/HACKING file. -dmc From dmc.fedora at filteredperception.org Fri Jul 27 08:30:01 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Fri, 27 Jul 2007 03:30:01 -0500 Subject: [Fedora-livecd-list] Revised: [PATCH] turboLiveInst - improves livecd/usb installer speed by 15-20+% Message-ID: <46A9AD09.7040309@filteredperception.org> Attached is a revised version of my turboLiveInst patch to livecd-tools and anaconda. This version is more polished. I.e. bugs have been fixed, complexity removed, and therefore should be easier to review. I performed some anecdotal performance tests, on a sony vaio vgn-n250e. I used a 30G destination volume for all tests, and when using usbstick media, it was media that reported 8.5MB/s from hdparm -t. I did have selinux disabled, and did not use the prelink option. I'd love to hear performance numbers from differing test rigs. The performance results I got were- install from cdrom without turboLiveInst: copy: 250s postinst: 86s install from cdrom with turboLiveInst: copy: 299s postinst: 84s install from usb without turboLiveInst: copy: 226s postinst: 72s install from usb with turboLiveInst: copy: 175s postinst: 72s Conclusions: On this testrig, installing from cdrom, turboLiveInst yielded a 20% speedup in copy, which resulted in an end to end install speedup of 15%. Installing from usb, turboLiveInst yielded a 29% speedup in copy, which resulted in an end to end install speedup of 20%. I did test copy-to-ram mode, and the resulting benefits were laughably huge. But this is only because this laptop has 1G of ram, and very strange behaviour occurs this near the threshold of having too little ram to use this feature. Though this is still an argument in favor of turboLiveInst, in that somehow it found itself on the better side of the threshold. I would expect that with 2G of ram, the benefit would be on the order of 35-50% speedup, as the main thing masking the benefit in the cdrom case, is the slow access to install media, which hides the benefit of cutting the needed disk writes nearly in half. The secondary benefit of turboLiveInst is that it removes the artificial limitation that the target rootfs must be greater than 4.0G, instead of the 2.1G actual uncompressed size of the contents of the LiveCD. Jeremy has pushed back against this patch because of complexity. Hopefully this round of polishing will make the patch much easier to read and understand. In addition, since the cleanupDeleted patch which this one depends on has already been merged, that should also make this a bit more palatable. Jeremy also brought up the idea of doing a file level copy installation, rather than the current block level mechanism. This _is_ a good idea, in my opinion, in that it will more intelligently support situations such as a separate /usr filesystem. In addition there is no way that turboLiveInst or the existing block level mechanism can be made to support xfs or other non-ext3 destination filesystems chosen by the user (should those options return). But- I would argue that file level installations may suffer badly due to cdrom seeking. I would also argue that there is no reason why turboLiveInst, could not be the first choice for installation technique, with fallbacks to file level copies for the seperate /usr and xfs type scenarios. Ultimately I just hope that turboLiveInst gets serious consideration for F8, via performance comparison with whatever other options may exist. Finally, here are some notes on the architecture, which may help you to understand the code- As I mentioned in the original patch, the basic idea is to simply, at livecd-creator build time, use a device mapper snapshot to generate a delta file between the 4.0G filesystem housing 2.1G of data, and a 2.1G filesystem holding the same data. Then at anaconda/liveinst time, that delta file is used to recreate a virtual image of the 2.1G filesystem, so that it can be copied to the installation target, rather than the 4.0G filesystem which includes 1.9G of zeros that needn't be written to disk. I ended up using /dev/loop118 in the initramfs init (mayflower generated) to expose the delta file. /dev/loop118 was mknod'd already by mayflower, but not actually used for anything. I used it to expose the delta file (osmin.tgz), because /dev/loop121 was being used to expose the 4.0G os.img, and it seemed simplest to use an identical mechanism to expose that data. Because by the time anaconda runs, the original cdrom and squashfs filesystems have been lazy unmounted, a simple cp at that time was not an option(?). I chose to extract the delta file (osmin, a 16MB sparse file containing 1.2M of data, compressed to 25kb on cdrom) into /dev/shm (i.e. ram) so that reads from it would not try to go to the cdrom. This may not have been necessary. I chose to calculate the size of the filesystem at livecd-creator time, and include it with osmin as osmin.size(together forming osmin.tgz). This is less complex than what I did in the first pass, which was to use dumpe2fs to calculate it at ananconda time. As always, questions, comments, criticisms, and especially testers are more than welcome. peace... -dmc -------------- next part -------------- A non-text attachment was scrubbed... Name: anaconda.turboliveinst.3.patch Type: text/x-patch Size: 2672 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: livecd.turboliveinst.3.patch Type: text/x-patch Size: 10711 bytes Desc: not available URL: From katzj at redhat.com Fri Jul 27 20:28:10 2007 From: katzj at redhat.com (Jeremy Katz) Date: Fri, 27 Jul 2007 16:28:10 -0400 Subject: [Fedora-livecd-list] Pushed a few changes Message-ID: <1185568090.29846.7.camel@localhost.localdomain> I threw together a few little changes today and pushed them into the repo. You can now use $basearch as your repo URL (or in the mirrorlist URL) and the right thing should happen. Also, if you include memtest86+ in your manifest, it should land in the boot loader config. Finally, if you put kernel-xen in your manifest, that should now work. And with that, I'm out for the next week. dmg -- I'll look some more at the turboinst stuff when I get back. I also want to try to get the persistence and other arch bits in pretty soon then. Jeremy From dmc.fedora at filteredperception.org Fri Jul 27 22:08:52 2007 From: dmc.fedora at filteredperception.org (Douglas McClendon) Date: Fri, 27 Jul 2007 17:08:52 -0500 Subject: [Fedora-livecd-list] minor turboLiveInst notes In-Reply-To: <1185568090.29846.7.camel@localhost.localdomain> References: <1185568090.29846.7.camel@localhost.localdomain> Message-ID: <46AA6CF4.3080900@filteredperception.org> Jeremy Katz wrote: > And with that, I'm out for the next week. dmg -- I'll look some more at > the turboinst stuff when I get back. I also want to try to get the > persistence and other arch bits in pretty soon then. Cool. I'll now be working exclusively on persistence, and hope to have something decent ready by the time you get back. A couple minor notes on turbo-liveinst, which I strongly encourage you to ignore while off the clock- 1) to understand the code, look at the patches in the order that events happen. I.e. livecd-creator -> mayflower -> liveinst.sh -> livecd.py (and isotostick.sh is orthogonal) 2) obviously I typo-swapped the performance numbers on cdrom install with/out turbo-liveinst. I.e. the 250s is _with_ turbo, and the 299s, is _without_. 3) the weird dd padding of 512 zero bytes to osmin.tgz is because I discovered loop devices ignore the last N mod 512 bytes of files of size N bytes. Probably would be better to do this in livecd-creator than in mayflower-init. 4) it might be more technically accurate to rm the osmin.tgz after losetup-ing it, so that if you ever remove the loop device, that the 25kb gets freed (?). 5) the time impact on livecd-creation is negligible. Because cleanupDeleted has already basically packed all the data into the front of the sparse file, resizing it back to that size happens very quickly. -dmc