[EnMasse] Proposal: fewer repositories

[Ulf Lilleengen]

On Fri, Jun 30, 2017 at 1:36 PM, Gordon Sim <gsim at redhat.com> wrote:

> I agree with the view that we should evolve towards fewer repositories
> than we have now.
>
> To my mind, breaking things down by the deployment or address-space type
> doesn't feel right.
>
> While I can certainly see that a single repo would simplify the CI, I
> instinctively dislike that also. There would seem to me to be a whole range
> of fairly unrelated things all bundled together. In a way it is avoiding
> the effort of trying to find the right division. Had we started off that
> way, it might have made sense, however while I think this is an important
> conversation to have, I wouldn't want to rush in to that decision.
>
> I could see a couple of smaller changes that would reduce the number of
> repos, without making any radical change. e.g.
>
> * combine ragent and routilities
>
> * combine router-image and router-metrics
>
> * combine artemis-image and topic-forwarder
>
> * combine mqtt-gateway and mqtt-lwt
>
> I think putting subserv into routilities would make sense as a short term
> step also, as there is some code that can be shared. However longer term I
> think we need to rethink the implementation of that component and it could
> end up in a very different form.
>
> If we did all that, we could eliminate 5 repos from the current set.
>
>

I agree it's a work-around in that sense, and I don't think we'll find the
right division between the components in the near future, as it is likely
to evolve. For instance, we've discussed things like a) combining
functionality of router agent and queue scheduler in a 'address manager',
b) making the console work standalone as well as in enmasse, c) creating a
'standard-controller' that handles the 'standard' address space, and d)
documenting and testing the API between components.

There will probably be more of these kinds of consolidation and moving of
functionality, and I argue that having the components in the same
repository would ease that work significantly. We can then define certain
requirements to be met by a component before we split it into its own repo
again. For instance, it could be that the "level of independence should be
at level of qpid dispatch or artemis" in terms of:

   * having its own release cycle
   * a versioned and documented API of protocols and configuration tunables
   * being packaged and bundled as a 'standalone' application that can be
easily put in a container

We'll have to solve the CI 'problems' when splitting out repos eventually,
but I think that a single repo will help the developer experience short
term as well. I'd really like for the developer experience to be this:

   * start local openshift cluster
   * check out enmasse (and create your branch)
   * modify source code of component X
   * run a single command, i.e. 'gradle check pack deploy systemtest'
   * git commit && git push && git pull-request

And, the CI should run the exact same commands so that builds are
reproducible. Today, the list is much longer for doing more than a build
and unit test:

   * start local openshift cluster
   * checkout component X repo
   * modify source code of component X
   * follow component X build instructions and push docker (push docker can
be skipped if you can run it directly against OpenShift)
   * checkout enmasse repo
   * modify enmasse config to include your custom component (may be skipped
if you run it outside of OpenShift, but you may have to undeploy the one
running already)
   * deploy enmasse from modified config (or original component if you run
outside OpenShift)
   * checkout systemtests
   * follow systemtests build instructions and run
   * git commit && git push && git pull-request

And thats if your work involves modifying 1 component only.  When starting
to split out repos, we can focus on keeping a good developer experience,
which I think involves using a common build tool (i.e. gradle plugin to
deal with the hard stuff) for instance.


Ulf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/enmasse/attachments/20170703/ad74cd91/attachment.htm>