[Container-tools] Nulecule/atomicapp validation and next step ideas

[Brian (bex) Exelbierd]

I met with a production user of docker recently and over coffee I had a 
chance to talk to them about Nulecule.  They are running Debian and 
using Marathon (a Mesos framework) for orchestration.

For the most part they validated our direction, granted it is a sample 
size of 1.

Here are some specifics:

Things that are interesting:
  - Today they have the configuration files for Mesos-Marathon stored 
and deployed separately from the images that make up the application. 
Being able to bundle up this information would be a big win.

  - If the DevAssistant plugin can help drive best practices in 
application development/framework usage that would be a huge win.

  - Today there is a parameterization problem in Mesos-Marathon. 
Mesos-Marathon cannot launch 5 instances of the same container that vary 
only by a parm, say an ID, from the same configuration file.  To do this 
you have to write 5 different configuration files, each with the ID 
hardcoded.  This obviously creates a change management concern. 
Nulecule looks like it could solve this issue, especially is some of the 
answers can be provided via an API or otherwise programatically.

  - There is a kubernetes framework on top of Mesos that should just 
work. (However, see below)

Things that are concerns:
  - How tied is Nulecule to Kubernetes?  Can it support concepts outside 
of this thought process. [I said I didn't know of any reason it 
couldn't.] Specifically, in Mesos-Marathon:

    - There is one json configruation file that drives orchestration, 
container linking (containers are linked via the cluster network, not 
docker link private networks), and networking.

    - There is no tieing containers that are part of the same app to the 
same host.  This is not guaranteed or enforcable. Open question: Does a 
pod tie things to same host? [He wasn't sure, I didn't think so.]

    - Container connections are always done via service discovery. There 
are two kinds of service discovery, a light-weight one built into 
Marathon and etcd.

  - They are a debian shop today.  While switching to a RHEL-family 
architecture is not necessarily a show stopper, it is concerning that 
the only application available to use the spec's reference output, 
atomic, has no instructions or packaging for other distributions. Ditto 
for atomicapp as building successfully on other distros.

  - While there is a kubernetes framework on top of Mesos, they chose 
Marathon because it is simpler and not supporting this is a show stopper.

All in all I think this is a good showing and starts to give some on the 
ground validation to the effort.

Some possible outcomes that came to mind are:
   - Create a parameterized example that shows the same app launched 
with 5 different minor variable changes.

   - Test atomicapp/atomic with Mesos-Kubernetes

   - Write a Mesos-Marathon provider

   - Verify that atomic/atomicapp can run on an alternative distribution

   - (Quasi-related) Verify that atomic can run on an alternative 
distribution

To support us moving forward, I plan to begin working on making sure 
atomic and the atomicapp/Nulecule examples run without issue on Debian. 
  If anyone wants to help, reach out to me please.

regards,

bex
-- 
Brian (bex) Exelbierd
Associate Software Engineer
Open Source and Standards
Red Hat, Inc., Brno, Czech Republic
P: +420 606 055 877 (CZ) | E: bexelbie at redhat.com (preferred)
IRC: bexelbie | IM: Google: bcexelbi at gmail.com