[katello-devel] Organization deletion bug, orchestration, testing

[Lukas Zapletal]

Sure,

if you dont mind putting this on this page:

https://fedorahosted.org/katello/wiki/OrchestrationNG

I would be happy.

LZ

On Mon, Dec 17, 2012 at 01:45:59PM +0000, Dmitri Dolguikh wrote:
> On 17/12/12 11:36 AM, Lukas Zapletal wrote:
> >On Mon, Dec 17, 2012 at 10:52:33AM +0100, Martin Bacovsky wrote:
> >>to handle communication between services. Today I came across
> >>article about messaging [1] and to me it seems a way to go. I know
> >>it would be a *big* change, but I think it is worth to at least
> >>consider it, especially when we have messaging broker (QPID) already
> >>set up in katello.
> >Well, messaging is not the only way to orchestrate things, but it looks
> >like a good option since developers trying to achieve orchestration tend
> >to write state machines or queue-based processors.
> >
> >I see the biggest advantage in easy de-coupling. We need to get
> >orchestration out of the model, out of Katello completely. With
> >messaging between Katello and OrchestrationNG, it will be possible to
> >run many lightweight Ruby processes/threads for orchestration without
> >Rails in the memory.
> >
> >Having standalone broker also allows services to communicate directly to
> >each other - for request-reply read-only lookups.
> >
> >>- consistency - support for transactions, proven messaging patterns
> >>would help us to keep data consistent across subsystems. Like DB
> >>server is good at keeping data healthy (FKs, constraints,
> >>transactions), similarly  message broker is good at keeping
> >>communication between system parts healthy. (lets learn from our
> >>previous mistakes)
> >While messaging systems (including AMQP-based) supports both local and
> >distributed transactions, I'd rather stick with transaction-compensation
> >support. We would need to have distributed transaction support to be
> >implemented in all three backend engines which is not feasible right
> >now. Transaction compensation can be easier to implement if we strictly
> >divide messaging exchanges.
> >
> >There are many other advantages like performance, durability, better
> >error recovery and other things. The ultimate goal should be to drop
> >REST via HTTP and integrate using REST via AMQP. But for the first phase,
> >Katello still can have simple bridge between AMQP and HTTP for each
> >backend engine.
> >
> 
> I agree with the general sentiment expressed so far - I think we
> need to revise our approach to orchestration.
> 
> Let's avoid architectural discussions in this thread, but rather
> concentrate on what we need from the new engine, stuff like:
> 
> Crash-only semantics (this is pretty big - as things stand right now
> we won't recover from any crash should it happen in the middle of an
> orchestration)
> Simpler definition of orchestrations
> Ability to orchestrate any operation, whether it includes AR model or not
> Ability to recover from certain types of failures (500, 503 type of errors)
> Easier testability of orchestrations
> 
> As a side note, I think we need to dispose of the idea of
> synchronous orchestrations altogether (it makes error recovery
> possible without breaking the semantics of the call among other
> things), and aim for eventual consistency between Katello (primary)
> and its subsystems.
> 
> -d
> 
> 
> 
> _______________________________________________
> katello-devel mailing list
> katello-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/katello-devel

-- 
Later,

 Lukas "lzap" Zapletal
 #katello #systemengine