[Ansible-service-broker] Bootstrap recap

[jesusr at redhat.com]

All,

I met with Erik & Alay to discuss the bootstrap. The main issue is
deleting ALL specs causes many bugs, primarily in the cases where they
are going to be added again.

Each of the 3 bugs identified in the proposal was case where we
deleted the specs, before we were able to process the specs from the
registry. Either, we didn't add the spec yet or there was an error
reaching the registry. Both scenarios exhibited the same behavior
because our process was to delete ALL specs first.

Given the above, we came to the conclusion that the simplest solution
to this problem is to simply not delete all of the specs, but to
process them inline. Basically back to approach #1 as outlined by:

https://github.com/openshift/ansible-service-broker/pull/997#discussion
_r198575908

  1. Fetch the specs from a registry

  2. Fetch the specs from the datastore

  3. Delete any specs in the datastore currently marked for
     deletion, assuming there is no instance that has a reference
     to this spec which is marked for deletion.

     i. if there is a reference, leave the spec alone
     ii. if there is no reference, delete the spec

  4. For each spec from the registry, get the spec from the datastore

     i. if found, update it, and save to datastore
     ii. if not found, add it to the datastore

  5. For each spec in the datastore, NOT in registry list,
     mark it for deletion.

This removes all of the concurrency issues we were trying to "solve"
in all of the calls. It's easy to understand. Because we will be doing
the handling of the delete (#3 above) in-place this will cause the
bootstrap to take longer.

We can make the bootstrap endpoint async so that it won't be affected
by how long the bootstrap takes to reconcile. The topic of adding a
last_operation like endpoint for bootstrap came up, but for now, we are
forgoing that portion of the solution. The last_operation is only
of interest to clients that interact with the broker directly, for
instance, sbcli. We can implement the same polling mechanism that
that catalog uses for the standard OSB methods.

The above will fix the original bugs we outlined, without too much
intrusion. We didn't go too deep into backwards compatibility with
existing clients, which seems to only affect apb tool. Apb tool is
being replaced by sbcli anyway.

The plan is as follows:

1) send out this email
2) add the above solution as a comment to the proposal PR
3) close proposal PR
4) create trello card with above steps
5) Alay will implement the solution from the trello card

If anyone has any strong objections please speak up now.

Sincerely,
jesus