[virt-tools-list] [virt-manager PATCH][v2][RFC] Introduction of cloud-init configuration in virt-install

Tue Jun 25 10:17:51 UTC 2019

On Mon, Jun 24, 2019 at 01:53:38PM -0400, Cole Robinson wrote:
> On 6/24/19 12:40 PM, Daniel P. Berrangé wrote:
> > On Mon, Jun 24, 2019 at 12:12:16PM -0400, Cole Robinson wrote:
> >> On 6/21/19 1:18 PM, Athina Plaskasoviti wrote:
> >>> I am sending a v2 of the patch for cloud init basic configuration during cloud image installation, applying previous suggestions.
> >>>
> >>> Fixed: 
> >>>
> >>> -the way the installation is triggered (no_install is no longer required):
> >>> 	--install is_cloud=yes ... --import
> >>>
> >>> -added tmpfile for the files that are being generated in the installation process 
> >>> (meta-data, user-data, config.iso)
> >>>
> >>> -changed condition in installerinject.py from label='cloud' to cloudinit=True
> >>>
> >>
> >>
> >> I did some digging into cloud-init behavior to try and get a better idea
> >> of what the minimal viable config is, and get ideas for future work.
> >>
> >>
> >> cloud-init behavior
> >> ===================
> >>
> >> The patch generates an .iso with volume label 'cidata' which is the
> >> magic to make cloud-init search it for config data. cloud-init must find
> >> a 'meta-data' and 'user-data' file in order to consider the iso a valid
> >> data source. There isn't any requirements on the contents though. They
> >> can both basically be empty AFAICT.
> >>
> >> This patch is setting meta-data 'instance-id' and 'hostname' fields but
> >> neither are required: instance-id seems to only be needed on real cloud
> >> providers (and cloud-init fills in a 'nocloud' value for us by default).
> >> hostname will predictably set the VM hostname but that's not something
> >> we need to set by default. So in fact meta-data is fine to leave empty
> >> and I think that should be our default behavior.
> >>
> >> Every distro cloud image has some default config in
> >> /etc/cloud/cloud.cfg. This is generated from cloud-init.git
> >> tools/render-cloudcfg which generates cloud.cfg from
> >> config/cloud.cfg.tmpl. Basically the only input to the template is the
> >> distro name. For example fedora generates in in the RPM spec with simply:
> >>
> >> python3 tools/render-cloudcfg --variant fedora >
> >> $RPM_BUILD_ROOT/%{_sysconfdir}/cloud/cloud.cfg
> >>
> >> cloud images seem to have this default behavior:
> >> - ssh is enabled and running
> >> - root account is locked
> >> - cloud.cfg will explicitly disallow ssh to the root account, via
> >> disable_root. Attempts to ssh to root will print an error message
> >> pointing to the default account
> >> - cloud.cfg will create a default account, locked, with full sudo access.
> >>
> >> The name of the default account depends on the value passed to the
> >> tools/render-cloudcfg script. For Fedora it's 'fedora', Ubuntu is
> >> 'ubuntu'. The openstack page here documents the common ones:
> >> https://docs.openstack.org/image-guide/obtain-images.html
> >>
> >> An extra note: I was wrong when I said in other threads that fedora
> >> images have a default password, they just have a default account name
> >> which is locked by default.
> > 
> > That good, so I was in fact right in remembering that.
> > 
> 
> Yeah, sorry for the confusion.
> 
> >> virt-install cli
> >> ================
> >>
> >> I think the virt-install entry point for requesting cloud-init stuff
> >> should actually be its own top level option, like --unattended, because
> >> we may want to add suboptions and I don't want to mix those in with
> >> --install. So I'm thinking:
> >>
> >> # does whatever we choose for default behavior, like --install is_cloud
> >> virt-install --cloud-init ...
> >>
> >> # User provides their own user-data file. Maybe for meta-data too?
> >> virt-install --cloud-init user-data=/path/to/user-data
> >>
> >> # Specify password for the default user. Yeah I know this isn't
> >> # a good pattern to put a password on the cli but we could combine it
> >> # with immediate password expiration so the user has to change it
> >> # on first log in
> >> virt-install --cloud-init password=FOOBAR
> >>
> >> # Something like this to inject an ssh key? But honestly I'm not
> >> # convinced we should go down that route and instead just make users
> >> # manually set this themselves like they would for any other VM
> >> virt-install --cloud-init sshkey=/path/to/key
> > 
> > I think ssh keys is the preferred way to setup cloud images in many
> > cases as no one sane should ever be using passwords for ssh auth these
> > days given the continual brute force attacks they face.
> > 
> > Saying users must set a password and then manually copy their ssh key
> > in afterwards is crazy when cloud init supports getting the ssh key
> > in straight away without needing passwords.
> > 
> > I would not be surprised to even find cloud images which have
> > explicitly disabled SSH passwords entirely leaving SSH keys as the
> > only option. Some organizations will have a complete ban on use of
> > passwords for login purposes too.
> > 
> 
> My impression is that most people trying to use these images are just
> trying to get quick access to a distro VM and think a small
> pre-installed image is the way to go about it. They expect it to 'just
> work' like say a virt-builder image but don't realize that cloud-init is
> going to get in their way. They see the VM hang for a while then drop to
> a login prompt and they don't know how to log in.
> 
> I think the goal of this work should be to make the images work like a
> typical virt-install/virt-manager import, not to mirror the expected
> cloud image interaction model. If a user is coming to these images with
> knowledge of cloud-init they can probably google their way out of the
> problem. We can give them the tools like an sshkey= option to make it
> work like they want

I think the opposite actually. If I'm running a cloud-image locally I
want to experiance it in exactly the same way I'll experiance it when
running it on OpenStack / AWS later. This means I'll want to interact
with it via SSH using public keys.  Traditional text or graphical
consoles are not something I'd ever care about for cloud images except
when things go badly wrong.

I do accept though that we have a bit of a mis-match in paradigms for
virt-manager though, since that tool is inherantly tailored around the
idea of graphical console access.  So for virt-manager I think it does
make sense to have a stronger focus on password based access.

virt-install though I think should be aiming to facilitate usage in
the same way as with normal cloud software like OpenStack/AWS since
it is a console based tool.

> Trying to facilitate ssh by default opens this whole can of worms. You
> won't be able to login by default with virt-viewer/virt-manager or virsh
> console. We have to have some logic for finding user ssh keys, figure
> out how to deal with multiple keys, and error if the user doesn't have a
> key. If we want to autolaunch ssh from virt-install, that means the user
> sees no boot output, we need to wait for the guest IP address which
> means validating that's possible with the selected network method, it
> would only work remotely if we used an ssh tunnel. There's probably more
> issues. Honestly I don't think it's worth trying to figure out this
> whole new paradigm for a reasonably niche case like this.

I don't think any of those problems need solving.

This is a different usage from normal installs since you already have
the guest OS provisioned. virt-install in this case is really just being
used as a convenient tool to write the libvirt XML & create the cloud-init
config. Thus I don't see any need to interact with the console when
importing a pre-built image, so there's no need for virt-install to
autolaunch SSH client, and thus need need to worry about network access
methods, etc.

If I was using this then I'd just be runnig SSH myself afterwards, relying
on the libvirt NSS module for the hostname eg

$  virt-install --name fedora30 \
                --import /var/lib/libvirt/images/fedora30.qcow
                --cloud-init ssh-key=$HOME/.ssh/id_rsa.pub
		--no-auto-console
....wait 10 seconds...	       
$  ssh fedora30

Rather than pointing to my id_rsa.pub most commonly though I would need
to use the keys known to my SSH agent, because the id_rsa.pub is on my
laptop, but my VMs are all luanched on remote servers. This would want
support for

   --cloud-init ssh-key=agent

We can implement this by using "ssh-add -L" which displays the public
keys the agent has.

In fact the question of "default" behaviour doesn't need solving for
virt-install as we would always require something to be specified with
the --cloud-init arg.

ie, we have a range of possible options

   --cloud-init passwd=generate
   --cloud-init passwd=/path/to/file
   --cloud-init ssh-key=agent
   --cloud-init ssh-key=/path/to/file
   --cloud-init root-passwd=generate
   --cloud-init root-passwd=/path/to/file
   --cloud-init root-ssh-key=agent
   --cloud-init root-ssh-key=/path/to/file

They're not exclusive - we can provide passwords + keys at the same
time, both for the default user account and for root.

> >> The simplest: use the guest name as the default password. It's not
> >> hardcoded, it's simple to type. Less safe though because the VM is
> >> exposed to ssh for that first bootup with a possibly predictable password.
> > 
> > I would consider any use of a predictable default password to be a
> > security flaw. Guest name is predictable and visible to any process
> > on the host so is not safe.
> > 
> 
> That's fair. But I'm really not into the idea of trying to make ssh work
> automagically. Providing explicit options to use cloud-init ssh key
> injection is fine with me though, for the cli at least. UI I'm unsure
> 
> But for the default, I think that leaves generating a random password
> for root, similar to what is done with virt-builder. root is not
> accessible over ssh, so we drop them into 'virsh console' and they can
> copy/paste it if they want. Then the password is expired and they set
> their own on first login.

As above I don't htink there's any notion of "default" behaviour for
virt-install as the user would always have to pass some values to the
--cloud-init arg to specify what todo.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|