[Pulp-list] Repo delete + CDS = confusion

[Jeff Ortel]

On 04/10/2011 10:28 AM, Jay Dobies wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I need some other opinions on how repo delete should function for repos
> that exist on a CDS.
>
> Currently, if a repo is deployed on a CDS, it cannot be deleted from the
> Pulp server until it's unassociated from the CDS. Until recently, the
> behavior was that unassociate was strictly a server operation; the CDS
> would delete the repo the next time it syncced and saw the repo was no
> longer in its list. That's an important note; the CDS never tracked a
> list of its repositories, it simply syncced what it was told to by the
> server and deleted whatever was left.
>
> At the time it was sufficient. It also made CDS instances easier to deal
> with.
>
> Then we added repo auth, which complicated things. The CDS now needed to
> keep some track of what repos it had deployed, or at least the
> authorized ones. During unassociate, the CDS is sent a message saying
> the auth credentials for that repo were None. That stuff all works and
> is fine.
>
> The problem is what happens if the CDS is down and the repo is deleted
> on the server. The unassociate call is still required before the delete
> can occur. But that unassociate call is no longer server-side only and
> requires talking to the CDS to have it remove the repo auth settings for
> that repo. That means if the CDS is offline, you can't unassociate.
> Therefore, if the CDS is offline, you can't delete a repo that is
> deployed to it.
>
> Obviously that's not what we want.
>
> When I added global repo auth I introduced the REST idea of "partial
> content", meaning that the server successfully updated the global repo
> auth but not all of the CDSes were updated. I guess I could use that
> here as well. It feels bad to allow the delete if the repo is still on a
> CDS, but I suppose that's no more bad than allowing a auth update but
> not having all CDSes pick up that change. Arguably, it's safer than the
> auth case since the delete will still occur on the next CDS sync.
>
> So my ultimate question is if we're comfortable with the partial content
> approach. I need to get this fixed ASAP for RHUI and will probably have
> to do it in a few other CDS APIs as well (QE is gonna have a field day
> doing stuff while CDS instances aren't running and filing bugs).

I agree with the 'partial content' approach.  Since CDS(s) can be offline (down), I think 
we have no choice than to support the concept of them being temporarily out-of-sync from 
both the content and Command & Control standpoint.

>
> If we are comfortable with it, anyone want to volunteer to review the
> CDS and repo APIs for where this should be applied? The CLIs will have
> to be updated as well to look at the returned result (ok v. partial
> content) and display to the user which CDS instances failed to be
> updated. But it's gotta happen and I don't know when I'd have time to do it.

An alternative approach:

Do we dispatch these remote requests through the Task subsystem?  If so and these tasks 
are durable[1], then perhaps we can assume the CDSs are guaranteed to be updated (and back 
in sync) the next time it comes back up.  In that case, we can delete the repo in pulp 
after successfully enqueueing the tasks to update the CDS(s).

[1] A "durable" task is persistent and retries RMI until successful.  Basically, tries 
until the remote command is executed or the CDS is unregistered.

>
> - --
> Jay Dobies
> RHCE# 805008743336126
> Freenode: jdob
> http://pulpproject.org
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.14 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/
>
> iQEcBAEBAgAGBQJNocyBAAoJEOMmcTqOSQHCWrUIAI6sQlsG9HeiZfNLmAgC8jJs
> S/WWrRSeLxtkUQxP4HPe8GKzHOImp6VIViaDuHQ6On1jg5PDyj7nyxaW2yOgNRis
> 33HWwnk00HfNRzUb0gSFvdxzOALvTedZUF6w6/Ooa4FFPJjOuuIwAh3jT4+9+l/V
> yLXwgpd8vdPaxomWE1c2MZFacO/NLiOjGrTQNzBIsErJ59+ixC8qj4KSJXAxvzcY
> UjN2/K39A9NSM5+KF4joRQ4walFN5bfHy9ZeEsgJEgjOX0VIOHfjSthvgKJP4Usk
> /GwP1kZsZ6o4yFP67MDL+vBppGddNy1duKJujKrN16zHGa6IJM4R2DGoF5ibp9k=
> =5Wxj
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list