[katello-devel] Organization deletion bug, orchestration, testing

[Lukas Zapletal]

On Mon, Dec 17, 2012 at 10:52:33AM +0100, Martin Bacovsky wrote:
> to handle communication between services. Today I came across
> article about messaging [1] and to me it seems a way to go. I know
> it would be a *big* change, but I think it is worth to at least
> consider it, especially when we have messaging broker (QPID) already
> set up in katello.

Well, messaging is not the only way to orchestrate things, but it looks
like a good option since developers trying to achieve orchestration tend
to write state machines or queue-based processors.

I see the biggest advantage in easy de-coupling. We need to get
orchestration out of the model, out of Katello completely. With
messaging between Katello and OrchestrationNG, it will be possible to
run many lightweight Ruby processes/threads for orchestration without
Rails in the memory.

Having standalone broker also allows services to communicate directly to
each other - for request-reply read-only lookups.

> - consistency - support for transactions, proven messaging patterns
> would help us to keep data consistent across subsystems. Like DB
> server is good at keeping data healthy (FKs, constraints,
> transactions), similarly  message broker is good at keeping
> communication between system parts healthy. (lets learn from our
> previous mistakes)

While messaging systems (including AMQP-based) supports both local and
distributed transactions, I'd rather stick with transaction-compensation
support. We would need to have distributed transaction support to be
implemented in all three backend engines which is not feasible right
now. Transaction compensation can be easier to implement if we strictly
divide messaging exchanges.

There are many other advantages like performance, durability, better
error recovery and other things. The ultimate goal should be to drop
REST via HTTP and integrate using REST via AMQP. But for the first phase,
Katello still can have simple bridge between AMQP and HTTP for each
backend engine.

-- 
Later,

 Lukas "lzap" Zapletal
 #katello #systemengine