[libvirt] [PATCH v2] Make LXC container startup/shutdown/I/O more robust

Daniel P. Berrange berrange at redhat.com
Thu Mar 10 13:46:13 UTC 2011


On Wed, Mar 09, 2011 at 05:02:24PM +0100, Jiri Denemark wrote:
> On Thu, Feb 24, 2011 at 13:42:18 +0000, Daniel P. Berrange wrote:
> > The current LXC I/O controller looks for HUP to detect
> > when a guest has quit. This isn't reliable as during
> > initial bootup it is possible that 'init' will close
> > the console and let mingetty re-open it. The shutdown
> > of containers was also flakey because it only killed
> > the libvirt I/O controller and expected container
> > processes to gracefully follow.
> > 
> > Change the I/O controller such that when it see HUP
> > or an I/O error, it uses kill($PID, 0) to see if the
> > process has really quit.
> > 
> > Change the container shutdown sequence to use the
> > virCgroupKillPainfully function to ensure every
> > really goes away
> > 
> > This change makes the use of the 'cpu', 'cpuacct'
> > and 'memory' cgroups controllers compulsory with
> > LXC
> 
> The documentation in drvlxc.html says cpuacct, memory, and devices are the
> mandatory controllers and the code lxcVmStart() agrees with that so this
> commit message needs to be corrected.

Ah yes.

> > @@ -341,7 +352,8 @@ ignorable_epoll_accept_errno(int errnum)
> >  static int lxcControllerMain(int monitor,
> >                               int client,
> >                               int appPty,
> > -                             int contPty)
> > +                             int contPty,
> > +                             pid_t container)
> >  {
> >      int rc = -1;
> >      int epollFd;
> > @@ -447,7 +459,13 @@ static int lxcControllerMain(int monitor,
> >                          ++numActive;
> >                      }
> >                  } else if (epollEvent.events & EPOLLHUP) {
> > -                    VIR_DEBUG("EPOLLHUP from fd %d", epollEvent.data.fd);
> > +                    if (lxcPidGone(container))
> > +                        goto cleanup;
> > +                    curFdOff = epollEvent.data.fd == appPty ? 0 : 1;
> > +                    if (fdArray[curFdOff].active) {
> > +                        fdArray[curFdOff].active = 0;
> > +                        --numActive;
> > +                    }
> >                      continue;
> 
> Heh, thanks for the opportunity to learn about epoll. This might be a trivial
> question but... what if we get EPOLLIN event immediately followed by EPOLLHUP
> on the same fd? Do we end up leaving the data unread until another EPOLLIN
> arrives? Although it shouldn't be a big deal since we will just read the data
> from init after the console gets reopened by mingetty.

The previous branch in this 'if' will have handled the EPOLLIN event
so we shouldn't delay data.


> ...
> > @@ -2844,7 +2811,7 @@ static virDriver lxcDriver = {
> >      lxcDomainLookupByName, /* domainLookupByName */
> >      lxcDomainSuspend, /* domainSuspend */
> >      lxcDomainResume, /* domainResume */
> > -    lxcDomainShutdown, /* domainShutdown */
> > +    NULL, /* domainShutdown */
> >      NULL, /* domainReboot */
> >      lxcDomainDestroy, /* domainDestroy */
> >      lxcGetOSType, /* domainGetOSType */
> 
> So we lost virDomainShutdown for LXC. I guess that's because it didn't really
> work anyway, right? I wonder if there is a way to do graceful shutdown for a
> container without having a special deamon in it.

Yes & no. Thinking a little more, it depends on whats running in the
container. If doing full OS virtualization though, with a real 'init',
just sending SIGTERM isn't going to work. We likely need to actually
look inside the container and see if there's a UNIX socket somewhere
we can feed 'init' a controlled shutdown sequence.  And fallback to
a plain SIGTERM if none is found.

I'll work on that as a later patch...

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|




More information about the libvir-list mailing list