[PATCH 1/3] mock cvs 2007-03: cross-compile support
Michael E Brown
Michael_E_Brown at dell.com
Wed Jun 13 04:49:41 UTC 2007
On Tue, Jun 12, 2007 at 09:15:13PM -0500, Ken MacLeod wrote:
> This patch adds support for building RPMs for cross-compile targets.
> It adds an option '--toolkit' to specify a cross-development toolkit
> (and version) config for toolkits like DENX ELDK, TimeSys, MontaVista,
> or Wind River.
>
> The chroot (-r) configuration needs to supply the 'target_arch' and
> cross-distribution-provided host tools yum archive (those packages
> that normally install in the host system). [These should move over to
> the --toolkit config to allow unmodified host mock cfgs.]
Looks like toolkit config file has same format and general layout as the
normal mock config file. Why not use the normal mock config file
handling? It would eliminate a small portion of the cross patches in
that we no longer need another command line opt, nor processing for
another config file.
Your toolkit config file could easily import the base os config file of
your choice and go from there adding options. You would be exploiting
the fact that the config files are executable python code (which you
already appear to be doing).
For example:
# mock -r my_cross_config ... specfile
# cat /etc/mock/my_cross_config.cfg
#!/usr/bin/python
exec("base_config_file")
# extend with new options:
config_opts["my_new_opt"] = "foo"
> The --toolkit configuration file supplies the following variables [to
> be added to a doc somewhere]:
>
> base_arch (ppc, arm, mips)
> for the given 'target_arch' (eg. ppc_74xx, ppc_85xx)
>
> macros
> additional rpmbuild macros
>
> tk_rpmmacros
> points to where to put macros if the toolkit needs that (ELDK)
>
> tk_env
> set in the environment before calling rpmbuild (or the toolkit's
> equivalent) to set PATH, CROSS_COMPILE, etc.
>
> tk_fixup_cmd
> yum command (RPM to install) in the host (/) chroot after all else
> to fix things up for cross-building [should go away when everyone
> clean-builds, right?]
Can you give an overview for the ignorant (ie. me) about the difference
between /cross and /target? Why is /cross its own chroot-like thing?
Couldnt you just say, "install this one additional cross-buildsys-rpm,
which pulls in the cross-toolchain?"
>
> tk_cross_chroot_setup_cmd
> 'buildsys-build' to install for the /cross root (complete yum
> command)
If you do the above, could this be eliminated?
>
> tk_cross_yum.conf
> yum.conf for the /cross root
>
> tk_target_chroot_chroot_cmd
> 'buildsys-build' to install for the /target root (complete yum
> command)
Reasonable.
>
> tk_target_yum.conf
> yum.conf for the /target root
Reasonable.
>
>
> ---
>
> mock.py | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----------
> 1 files changed, 87 insertions(+), 19 deletions(-)
>
> diff --git a/mock.py b/mock.py
> index fc1b6aa..f072d78 100644
> --- a/mock.py
> +++ b/mock.py
> @@ -246,15 +246,29 @@ class Root:
>
> self._prep_install()
> self.yum(cmd)
> +
> + cross_cmd = cmd
> + for cross_fs in ('cross', 'target'):
> + if self.config.has_key('tk_' + cross_fs + '_yum.conf'):
> + if cmd[0:7] == 'install':
Dont like parsing previous cmd value for this. too fragile. Just
integrate this into the previous test.
At the same time, cache stuff could probably go into a separate
function: (cmd, cross_cmd, create_cache) = self._unpack_cache()
and move the stuff slightly above this function that sets cmd into said
function.
> + cross_cmd = self.config['tk_' + cross_fs + '_chroot_setup_cmd']
> + self.yum(cross_cmd, cross_fs)
> +
> + if self.config.has_key('tk_fixup_cmd') and cmd[0:7] == 'install':
> + self.yum(self.config['tk_fixup_cmd'])
> self._prep_build()
>
> if create_cache:
> self.pack()
>
> - def yum(self, cmd):
> + def yum(self, cmd, crossdir=None):
> """use yum to install packages/package groups into the chroot"""
> # mock-helper yum --installroot=rootdir cmd
> - basecmd = '%s --installroot %s' % (self.config['yum'], self.rootdir)
> + yumrootdir = self.rootdir
> + if crossdir is not None:
> + yumrootdir = os.path.join(yumrootdir, crossdir)
> + basecmd = '%s --installroot %s' % (self.config['yum'], yumrootdir)
I dont like the implicit join in the function. I'd rather see:
def yum(self, cmd, rootDirOverride=None):
yumrootdir=self.rootdir
if rootDirOverride is not None:
yumrootdir=rootDirOverride
basecmd = '%s --installroot %s' % (self.config['yum'], yumrootdir)
>
> self._mount() # check it again
> command = '%s %s' % (basecmd, cmd)
> @@ -282,8 +296,9 @@ class Root:
> shutil.copy2(srpm, dest)
> rootdest = os.path.join(self.builddir, 'originals', srpmfn)
>
> - cmd = "%s -c 'rpm -Uvh --nodeps %s' %s" % (self.config['runuser'],
> - rootdest, self.config['chrootuser'])
> + cmd = "%s -c '%s rpm -Uvh --nodeps %s' %s" \
> + % (self.config['runuser'], self.config['tk_env'],
> + rootdest, self.config['chrootuser'])
> (retval, output) = self.do_chroot(cmd)
Definitely need to rebase the patch, as the line numbers skew.
This doesnt seem like a specific toolkit thing, as tk_env is used for
every rpm invokation. How about make it generic? "rpm_addtl_env" or some
such. You can continue to set it in your override config.
>
> if retval != 0:
> @@ -303,8 +318,10 @@ class Root:
> chrootspec = spec.replace(self.rootdir, '') # get rid of rootdir prefix
> # grab the .spec file from the specdir
> # run rpmbuild -bs --nodeps specfile
> - cmd = "%s -c 'rpmbuild -bs --target %s --nodeps %s' %s" % (self.config['runuser'],
> - self.target_arch, chrootspec, self.config['chrootuser'])
> + cmd = "%s -c '%s rpmbuild -bs %s --nodeps %s' %s" \
> + % (self.config['runuser'], self.config['tk_env'],
> + self.config['tk_target_option'], chrootspec,
> + self.config['chrootuser'])
Again, generic. "rpmbuild_addtl_cmdline"
>
> (retval, output) = self.do_chroot(cmd)
> if retval != 0:
> @@ -355,9 +372,10 @@ class Root:
> srpmfn = os.path.basename(srpm_in)
> # run with --nodeps b/c of the check above we know we have our build
> # deps satisfied.
> - cmd = "cd /;%s -c 'rpmbuild --rebuild --target %s --nodeps %s' %s" % (
> - self.config['runuser'], self.target_arch, srpm_in,
> - self.config['chrootuser'])
> + cmd = "cd /;%s -c '%s rpmbuild --rebuild %s --nodeps %s' %s" \
> + % (self.config['runuser'], self.config['tk_env'],
> + self.config['tk_target_option'], srpm_in,
> + self.config['chrootuser'])
Generic.
>
> self.state("build")
>
> @@ -566,6 +584,17 @@ class Root:
>
> return rpmUtils.miscutils.unique(reqlist)
>
> + def write_yum_conf(self, yum_conf, crossdir=''):
> + if os.path.exists( os.path.join(self.rootdir, crossdir, 'etc', 'yum.conf')):
> + yum_conf_path = os.path.join('/', crossdir, 'etc', 'yum.conf')
> + cmd = "chown %s.%s %s" % (self.config['chrootuid'],
> + self.config['chrootgid'], yum_conf_path)
> + self.do_chroot(cmd, fatal = True)
> + yumconf = os.path.join(self.rootdir, crossdir, 'etc', 'yum.conf')
> + yumconf_fo = open(yumconf, 'w')
> + yumconf_content = yum_conf
> + yumconf_fo.write(yumconf_content)
Overall, I like this function. I dont like crossdir=''. Should use same
idiom I went over above. That makes this not-crossdir specific at all.
> +
> def _prep_install(self):
> """prep chroot for installation"""
> # make chroot dir
> @@ -581,6 +610,12 @@ class Root:
> os.path.join(self.rootdir, 'var/lock/rpm'),
> os.path.join(self.rootdir, 'etc/yum.repos.d')]:
> self._ensure_dir(item)
> + if self.config.has_key('tk_cross_yum.conf'):
> + self._ensure_dir(os.path.join(self.rootdir, 'cross/etc'))
> + self._ensure_dir(os.path.join(self.rootdir, 'cross/var/lock/rpm'))
> + if self.config.has_key('tk_target_yum.conf'):
> + self._ensure_dir(os.path.join(self.rootdir, 'target/etc'))
> + self._ensure_dir(os.path.join(self.rootdir, 'target/var/lock/rpm'))
Hmm.. surely there is a way to make this generic?
config_opts["addtl_root_dirs_list"] = ( ... )
or something like that.
>
> self._mount()
>
> @@ -623,15 +658,12 @@ class Root:
> fo.close()
>
> # write in yum.conf into chroot
> - if os.path.exists( os.path.join(self.rootdir, 'etc', 'yum.conf')):
> - cmd = "chown %s.%s /etc/yum.conf" % (self.config['chrootuid'],
> - self.config['chrootgid'])
> - self.do_chroot(cmd, fatal = True)
> - yumconf = os.path.join(self.rootdir, 'etc', 'yum.conf')
> - yumconf_fo = open(yumconf, 'w')
> - yumconf_content = self.config['yum.conf']
> - yumconf_fo.write(yumconf_content)
> -
> + self.write_yum_conf(self.config['yum.conf'])
> + if self.config.has_key('tk_cross_yum.conf'):
> + self.write_yum_conf(self.config['tk_cross_yum.conf'], 'cross')
> + if self.config.has_key('tk_target_yum.conf'):
> + self.write_yum_conf(self.config['tk_target_yum.conf'], 'target')
How about:
config file:
yum_cross_conf = """ ... yum config here ... """
yum_target_conf = """ ... yum config here ... """
config_opts['additional_yum_configs'] = [ (yum_cross_conf, "cross"), (yum_target_conf, "target")]
code:
if self.config.has_key('additional_yum_configs'):
for (yum_conf, yum_path) in self.config['additional_yum_configs']:
self.write_yum_conf( yum_conf, yum_path )
> +
> # files in /etc that need doing
> filedict = self.config['files']
> for key in filedict:
> @@ -703,7 +735,8 @@ class Root:
> self.do_chroot(cmd, fatal = True)
>
> # rpmmacros default
> - macrofile_out = '%s%s/.rpmmacros' % (self.rootdir, self.homedir)
> + macrofile_out = '%s%s/%s' % (self.rootdir, self.homedir,
> + self.config['tk_rpmmacros'])
> if not os.path.exists(macrofile_out):
> rpmmacros = open(macrofile_out, 'w')
> rpmmacros.write(self.config['macros'])
> @@ -740,6 +773,8 @@ def command_parse():
> help="do not clean chroot before building", default=True)
> parser.add_option("--arch", action ="store", dest="arch",
> default=None, help="target build arch")
> + parser.add_option("--toolkit", action ="store", dest="toolkit",
> + default=None, help="cross-development toolkit config file name")
not needed.
> parser.add_option("--debug", action ="store_true", dest="debug",
> default=False, help="Output copious debugging information")
> parser.add_option("--resultdir", action="store", type="string",
> @@ -778,6 +813,7 @@ def setup_default_config_opts(config_opts):
> config_opts['debug'] = False
> config_opts['quiet'] = False
> config_opts['target_arch'] = 'i386'
> + config_opts['toolkit'] = None
ditto
> config_opts['files'] = {}
> config_opts['yum.conf'] = ''
> config_opts['macros'] = """
> @@ -790,6 +826,13 @@ def setup_default_config_opts(config_opts):
> config_opts['files']['/etc/resolv.conf'] = "nameserver 192.168.1.1\n"
> config_opts['files']['/etc/hosts'] = "127.0.0.1 localhost localhost.localdomain\n"
>
> + # cross-development config options, default is native
> + config_opts['tk_env'] = ''
> + config_opts['tk_rpmmacros'] = '.rpmmacros'
start...
> + # --target is late-bound in main() to allow command line or config
> + # override
> + # config_opts['tk_target_option'] = '--target %s' % config_opts['target_arch']
> +
> # caching-related config options
> config_opts['rebuild_cache'] = False
> config_opts['use_cache'] = False
> @@ -821,6 +864,22 @@ def set_config_opts_per_cmdline(config_opts, options):
> if options.uniqueext:
> config_opts['unique-ext'] = options.uniqueext
>
> + if options.toolkit:
> + config_opts['toolkit'] = options.toolkit
> +
> +def toolkit(config_path, config_opts, toolkit):
> + # read in toolkit config file by toolkit name
> + if toolkit.endswith('.cfg'):
> + cfg = '%s/%s' % (config_path, toolkit)
> + else:
> + cfg = '%s/%s.cfg' % (config_path, toolkit)
> +
> + if os.path.exists(cfg):
> + execfile(cfg)
> + else:
> + error("Could not find config file %s for toolkit %s" % (cfg, toolkit))
> + sys.exit(1)
> +
> def do_clean(config_opts, init=0):
> my = None
> try:
> @@ -946,7 +1005,16 @@ def main():
>
> # cmdline options override config options
> set_config_opts_per_cmdline(config_opts, options)
> +
> + if config_opts['toolkit'] is not None:
> + toolkit(config_path, config_opts, config_opts['toolkit'])
end...
I think all this (start...end) code goes away if you fold the config
files together.
>
> + # cmdline options override toolkit config options too
> + set_config_opts_per_cmdline(config_opts, options)
> +
> + if not config_opts.has_key('tk_target_option'):
> + config_opts['tk_target_option'] = '--target %s' % config_opts['target_arch']
Make more generic?
Overall, I like the concept. I think we can do a lot of it in a more
generic way so that the words "cross" or "target" dont appear quite so
many times. I think we can support this being included if you make the
suggested changes.
Clark? Any additional comments? How does this fit with what you were
doing?
--
Michael
More information about the Fedora-buildsys-list
mailing list