From mgoldman at redhat.com Mon Jan 30 09:15:56 2012 From: mgoldman at redhat.com (Marek Goldmann) Date: Mon, 30 Jan 2012 10:15:56 +0100 Subject: [Thincrust-devel] [Overview] appliance-tools patches on LVM support In-Reply-To: <13B81B9DDD69EC4DB30F266136B6832501FBC8D5@CDCEXMAIL02.tw.trendnet.org> References: <13B81B9DDD69EC4DB30F266136B6832501FBC8D5@CDCEXMAIL02.tw.trendnet.org> Message-ID: <4F265FCC.7050906@redhat.com> Hi all, First of all - thanks for the patch! It would be very nice to expose kickstart's LVM support. Now the hard part - sadly I am not the right guy to review that patch. Would anyone be willing to review it instead of me? --Marek On 01/12/2012 06:44 AM, henry_huang at trendmicro.com.cn wrote: > Hi, Marek > > Days ago, my colleague (Yongjun) has sent the four patches on help appliance-tools support LVM. > As you requested, I will add more description on these patches here. > > [Background] > The Appliance Creation Tool is a nice tool to create Appliance Images from a redhat-series kickstart file. This tool uses the liveCD creator API as well as patches to the LIVECD API to create a multi-partitioned disk image, contained in the applianace-tools package. > These disk images can then be booted in a virtual container such as Xen, KVM, and VMware. Currently, BoxGrinder (BG) based on the appliance-tools is a popular project to grind out appliances for multiple virtualization and Cloud providers, like Amazon EC2. > Actually we (TrendMicro) have many projects working on these fields, and we'd like to share what we have now in these fields, like LVM support in appliance-tools here. > > [Problem] > However, as we know, there is no current support on logical volume management (LVM) in appliance-tools. LVM has long been a valuable technique on managing disk drives and similar mass-storage devices. > We can get much benefit from LVM: > 1. Managing large hard disk farms by letting you add disks, replace disks, copy and share contents from one disk to another without disrupting service (hot swapping). > 2. Making backups by taking "snapshots." > 3. Creating single logical volumes of multiple physical volumes or entire hard disks, allowing for dynamic volume resizing. > Therefore, our patches are working on this problem, to make appliance-tools support LVM. > > [Solution] > We have provided four patches on this problem. > [PATCH 1/4] Add LVM physical volume partition type support > [PATCH 2/4] Add Logical Volume Management(LVM) group interface to partitionedfs > [PATCH 3/4] Add Logical Volume interface to partitionedfs > [PATCH 4/4] Add Logical Volume Management to ApplianceImageCreator > > 1. Overview on appliance-tools components. > Appliance-tools is mainly based on "imgcreate" library in LiveCD tools. > Appliance-tools: __init__.py + appliance.py + partitionedfs.py > * __init__.py: => appcreate, support for creating appliance images > * appliance.py: Appliance Image Creator class > * partitionedfs.py: partitioned files system class, extends fs.py from imgcreate/fs.py > > 2. Key problems in partitionedfs.py components. > a) imgcreate/kickstart.py does not support LVM configuration on kickstart files. > b) imgcreate/fs.py does not support LVM interfaces > > For the first problem: > However, patches on imgcreate/kickstart.py to support LVM configuration are much more difficult and it also will affect upstream live-tools. > As "pykickstart" library (already used in appliance-tools) has appropriate LVM options, we decide to reuse it. > @@ -120,6 +128,8 @@ class ApplianceImageCreator(ImageCreator): > > #list of partitions from kickstart file > parts = kickstart.get_partitions(self.ks) > + groups = self.ks.handler.volgroup.vgList > + volumes = self.ks.handler.logvol.lvList > > #list of disks where a disk is an dict with name: and size > disks = [] > More information can be found in [PATCH 4/4]. > > For the second problem: > We have to add a new interface on LVM support at "partitionedfs.py". > Here are three important functions added to support LVM in "partitionedfs.py". > self.__format_volumes() > self.__map_volumes() > self.__unmap_volumes() > More information can be found in [PATCH 1/4], [PATCH 2/4], [PATCH 3/4] > > [TODO] > 1. Our patches currently do not support for grub2. > 2. We will support to calculate LVM size to fit the size of pv., as --grow support for "logvol" options. > > If you have any comments, please let me know. > Thanks& Best wishes, > > Henry Huang | QA Engineer, CDC Enterprise Segment > Building B, Soho International Plaza, No.48 > Ning Nan Avenue, Nanjing 210012, P. R. China > Office: 86-25-52386680 > TREND MICRO EMAIL NOTICE > The information contained in this email and any attachments is confidential and may be subject to copyright or other intellectual property protection. If you are not the intended recipient, you are not authorized to use or disclose this information, and we request that you notify us by reply mail or telephone and delete the original message from your mail system. > From jboggs at redhat.com Mon Jan 30 16:39:18 2012 From: jboggs at redhat.com (Joey Boggs) Date: Mon, 30 Jan 2012 11:39:18 -0500 Subject: [Thincrust-devel] [Overview] appliance-tools patches on LVM support In-Reply-To: <4F265FCC.7050906@redhat.com> References: <13B81B9DDD69EC4DB30F266136B6832501FBC8D5@CDCEXMAIL02.tw.trendnet.org> <4F265FCC.7050906@redhat.com> Message-ID: <4F26C7B6.1050007@redhat.com> On 01/30/2012 04:15 AM, Marek Goldmann wrote: > Hi all, > > First of all - thanks for the patch! It would be very nice to expose > kickstart's LVM support. > > Now the hard part - sadly I am not the right guy to review that patch. > Would anyone be willing to review it instead of me? > > --Marek > > On 01/12/2012 06:44 AM, henry_huang at trendmicro.com.cn wrote: >> Hi, Marek >> >> Days ago, my colleague (Yongjun) has sent the four patches on help >> appliance-tools support LVM. >> As you requested, I will add more description on these patches here. >> >> [Background] >> The Appliance Creation Tool is a nice tool to create Appliance Images >> from a redhat-series kickstart file. This tool uses the liveCD >> creator API as well as patches to the LIVECD API to create a >> multi-partitioned disk image, contained in the applianace-tools package. >> These disk images can then be booted in a virtual container such as >> Xen, KVM, and VMware. Currently, BoxGrinder (BG) based on the >> appliance-tools is a popular project to grind out appliances for >> multiple virtualization and Cloud providers, like Amazon EC2. >> Actually we (TrendMicro) have many projects working on these fields, >> and we'd like to share what we have now in these fields, like LVM >> support in appliance-tools here. >> >> [Problem] >> However, as we know, there is no current support on logical volume >> management (LVM) in appliance-tools. LVM has long been a valuable >> technique on managing disk drives and similar mass-storage devices. >> We can get much benefit from LVM: >> 1. Managing large hard disk farms by letting you add disks, replace >> disks, copy and share contents from one disk to another without >> disrupting service (hot swapping). >> 2. Making backups by taking "snapshots." >> 3. Creating single logical volumes of multiple physical volumes or >> entire hard disks, allowing for dynamic volume resizing. >> Therefore, our patches are working on this problem, to make >> appliance-tools support LVM. >> >> [Solution] >> We have provided four patches on this problem. >> [PATCH 1/4] Add LVM physical volume partition type support >> [PATCH 2/4] Add Logical Volume Management(LVM) group interface to >> partitionedfs >> [PATCH 3/4] Add Logical Volume interface to partitionedfs >> [PATCH 4/4] Add Logical Volume Management to ApplianceImageCreator >> >> 1. Overview on appliance-tools components. >> Appliance-tools is mainly based on "imgcreate" library in LiveCD tools. >> Appliance-tools: __init__.py + appliance.py + partitionedfs.py >> * __init__.py: => appcreate, support for creating appliance >> images >> * appliance.py: Appliance Image Creator class >> * partitionedfs.py: partitioned files system class, extends fs.py >> from imgcreate/fs.py >> >> 2. Key problems in partitionedfs.py components. >> a) imgcreate/kickstart.py does not support LVM configuration on >> kickstart files. >> b) imgcreate/fs.py does not support LVM interfaces >> >> For the first problem: >> However, patches on imgcreate/kickstart.py to support LVM >> configuration are much more difficult and it also will affect >> upstream live-tools. >> As "pykickstart" library (already used in appliance-tools) has >> appropriate LVM options, we decide to reuse it. >> @@ -120,6 +128,8 @@ class ApplianceImageCreator(ImageCreator): >> >> #list of partitions from kickstart file >> parts = kickstart.get_partitions(self.ks) >> + groups = self.ks.handler.volgroup.vgList >> + volumes = self.ks.handler.logvol.lvList >> >> #list of disks where a disk is an dict with name: and size >> disks = [] >> More information can be found in [PATCH 4/4]. >> >> For the second problem: >> We have to add a new interface on LVM support at "partitionedfs.py". >> Here are three important functions added to support LVM in >> "partitionedfs.py". >> self.__format_volumes() >> self.__map_volumes() >> self.__unmap_volumes() >> More information can be found in [PATCH 1/4], [PATCH 2/4], [PATCH 3/4] >> >> [TODO] >> 1. Our patches currently do not support for grub2. >> 2. We will support to calculate LVM size to fit the size of pv., >> as --grow support for "logvol" options. >> >> If you have any comments, please let me know. >> Thanks& Best wishes, >> >> Henry Huang | QA Engineer, CDC Enterprise Segment >> Building B, Soho International Plaza, No.48 >> Ning Nan Avenue, Nanjing 210012, P. R. China >> Office: 86-25-52386680 >> TREND MICRO EMAIL NOTICE >> The information contained in this email and any attachments is >> confidential and may be subject to copyright or other intellectual >> property protection. If you are not the intended recipient, you are >> not authorized to use or disclose this information, and we request >> that you notify us by reply mail or telephone and delete the original >> message from your mail system. >> > > _______________________________________________ > Thincrust-devel mailing list > Thincrust-devel at redhat.com > https://www.redhat.com/mailman/listinfo/thincrust-devel I'll try and get to it this afternoon EST, if not tomorrow