[almighty] Monorepo

Tue Sep 27 17:08:47 UTC 2016

Perhaps this is out of scope for Almighty, I was first contributing to the
discussion merely based on
thoughts I had coming from the Catapult idea.
Catapult demonstrated how a process of configuring tools, CI/CD pipeline,
etc. for a new project
could be reduced from days (or more) to minutes. One way for me to consider
the potential benefits
of this capability is to compare it to Maven. To list some of the benefits
that Maven brought to
Java in a more-digestible convention-over-configuration way:
- archetypes for easy kickstart
- coordination of build dependencies between modules (local and remote) and
their versions
- standard configuration format to enable better 3rd party tooling
- possibility of performing build, executing all tests, and even deploying
to environment in a single command

Relate that to microservices today. It would be nice to have
- an efficient way to create a new repo for language X that includes a
default build/test/deploy
- a simple, standard way to describe relationships between repos and
external services
- language agnostic builds (e.g., via Jenkins) that trigger downstream
builds based on repo dependencies
- repeat for language Y

This is more complex than Java since rather than a native API call the
communication between modules may be
RESTful or messaging-based or other, but tools like Arquillian allowed us
to perform the same even within a single
"mvn install" build: spin up an environment and perform integration testing.

Certainly the same tooling could be applied to a mono-repo but without an
approach to reduce the start-up time for
the build/test/deploy pipeline and to simplify the inter-repo dependencies,
it seems too costly to scale many repo.

This would be most valuable for loosely coupled services with well defined
APIs since you must give up the
ACIDity that's inherent in a hg or git mono repo.

Regards, Travis

On Mon, Sep 26, 2016 at 4:33 AM, Max Rydahl Andersen <manderse at redhat.com>
wrote:

> Hi Travis!
> I agree that today many-repo are problematic at a large scale.
> I'm not sure the issue can be "fixed" at Almighty level alone since I
> think
> the main issue around this is the limits of git (does not have good
> composition/submodule support)
> and the whole infrastructure around the "one project=one repo" idea.
> But I'm all ears for suggestions on how we could help improve it - it
> sounds
> like you have some thoughts on it. Care to share ?
> /max
>
> I think my point was missed (probably not worded clearly). I am not mixing
> microservices and many-repo but relating them.
>
> I don't think it's feasible with open source tools today to efficiently
> scale many microservices in a many-repo approach. In ALR's Push It Real
> Good presentation, the complexity and startup costs for a new idea were
> listed, and these are prohibitive (or at least inefficient) for you to take
> on for each small project.
>
> I was hoping ALMighty would solve this problem so that many-repo could
> become a better option. And if it does solve the problem, then possibly the
> arguments in favor of mono-repo are not as strong since I assume ALMighty
> will eventually be maintained inside of ALMighty.
>
>
> Regards,
> Travis Brown
>
>
>
> On Thu, Sep 22, 2016 at 12:41 AM, Baiju Muthukadan <bmuthuka at redhat.com>
> wrote:
>
>> Hi Travis,
>>
>> On Thu, Sep 22, 2016 at 10:45 AM, Travis Brown <tkbrown at gmail.com> wrote:
>> [...snip...]
>> > If the ALMighty project claims to make it easy to go from new
>> > project (and usually new repo) to production microservice but was in
>> > turn developed with a model that does not fit the goal it is
>> > enabling, then we are not eating our own dog food.
>>
>> I think it's not a good idea to mix Monorepo and Microservices.
>> Monorepo is all about how we are going to organize source code in a
>> single repository.  Whereas Microservices is an architectural style.
>>
>> Martin Fowler describe it this way [1]:
>>
>>   The microservice architectural style is an approach to developing a
>>   single application as a suite of small services, each running in its
>>   own process and communicating with lightweight mechanisms, often an
>>   HTTP resource API.
>>
>>
>> > I think the problems favouring a monorepo are possibly the same
>> > problems this project should help solve.
>>
>> That's a good thought!
>>
>> I would like to quote the entire comment by my friend Sajith in FB
>> post [2]:
>>
>>   Found Conway's Law to be true in almost all the cases -
>>   http://www.melconway.com/research/committees.html - "Organizations
>>   which design systems are constrained to produce designs which are
>>   copies of the communication structures of these organizations"
>>
>>   Would one team be responsible for all the microservices or would you
>>   have multiple small teams, each one with responsibility for a small
>>   part of the overall system?  Also how would your delivery pipleline
>>   look like? Would each service have it's own build/test/deploy
>>   pipeline?
>>
>>   If you have independent teams owning the services like
>>   https://gotocon.com/.../KevinGoldsmith..., Netlfix,
>>   https://sudo.hailoapp.com/.../journey-into-a.../ etc - organizing
>>   each service as a folder or git submodule brings with it a lot of
>>   other questions (versioning together, tight coupling)
>>
>>   The definition of a service, microservice, macroservice,
>>   mediumservice vary wildly depending on who you talk to these days :)
>>
>>   Guess it depends a lot on how you slice your services up and how
>>   that reflects your team structure - The complexity could be either
>>   in modules or in the interstitial glue.
>>
>>   If you want to build, test, tag/version and deploy services
>>   independently, how would a monorepo work?
>>
>>   Independently deployable services brings with it further complexity
>>   (discovery, increased surface area for failure etc) - but if your
>>   team structure supports that, you could rely on tooling to make it
>>   less painful
>>
>>
>> [1] http://martinfowler.com/articles/microservices.html
>> [2] https://www.facebook.com/baijum/posts/10154503962859641
>>
>> Regards,
>> Baiju M
>>
>> _______________________________________________
>> almighty-public mailing list
>> almighty-public at redhat.com
>> https://www.redhat.com/mailman/listinfo/almighty-public
>>
>
> _______________________________________________
> almighty-public mailing list
> almighty-public at redhat.com
> https://www.redhat.com/mailman/listinfo/almighty-public
>
> /max
> http://about.me/maxandersen
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/almighty-public/attachments/20160927/f32eeb83/attachment.htm>