[katello-devel] Organization deletion bug, orchestration, testing

[Lukas Zapletal]

On Fri, Dec 14, 2012 at 03:06:33AM -0500, Ivan Necas wrote:
> With JRuby, reusing this tools might be feasible, right?

It does not depend on which platform are we running Katello on,
integration with such a service is usually external - via an API like
web-service, REST service or messaging. Engine is a standalone process
usually.

I don't thing this is acceptable from the management point of view. But
I think it is feasible to refactor this.

> We also need better testing for edge cases. E.g. what happens when I delete
> an organization and restart all the services in the mean time. How can I recover?
> Trying to solve this issues can point us the the weak points (which will be probably
> very similar to those described in this thread.)
> 
> I agree neither black box testing nor unit tests will help us much here. Unit test because
> of the isolation, black box because everything might seem working at the moment and
> knowledge of the code base can help finding the issue. 
> 
> 
> Stories:
> 
> As a integrator, I want to have the edge cases automated.
> As a integrator, I want to provide standardized ways for recovering from various non-standard situations.
> 
> When talking about keeping consistency between the systems, the first obvious step, as Mirek pointed out, is to finally introduce foreign keys
> to keep the consistency in our own database. This should be #1 priority, if we don't want to spend the rest of our lives fixing bugs like this
> one.

Well, with decent integration approach you don't really need this - data
changes should be one-way only. If any process changing our database
gets stuck, you get notified and you can easily make it finish (e.g.
after you restart services involved).

The main advantage of this approach is you (as a caller) are not
responsible error recovery anymore. Then you don't need to focus so much
on testing edge (error) cases.

But if we stick with our current "web server process rules them all"
approach, then we would need that. Many more tests. I'd start with
implementing integration oriented tests (that should come with PulpV2
and minitest/webmock/facorygirl/vcr merge).


-- 
Later,

 Lukas "lzap" Zapletal
 #katello #systemengine