[Pulp-dev] Publish API for Plugin Writers (Pulp3)

[Brian Bouterse]

I was thinking about the design here and I wanted to share some thoughts.

For the MVP, I think a publisher implemented by a plugin developer would
write all files into the working directory and the platform will
"atomically publish" that data into the location configured by the
repository. The "atomic publish" aspect would copy/stage the files in a
permanent location but would use a single symlink to the top level folder
to go live with the data. This would make atomic publication the default
behavior. This runs after the publish() implemented by the plugin developer
returns, after it has written all of its data to the working dir.

Note that ^ allows for the plugin writer to write the actual contents of
files in the working directory instead of symlinks, causing Pulp to
duplicate all content on disk with every publish. That would be a
incredibly inefficient way to write a plugin but it's something the
platform would not prevent in any explicit way. I'm not sure if this is
something we should improve on or not.

At a later point, we could add in the incremental publish maybe as a method
on a Publisher called incremental_publish() which would only be called if
the previous publish only had units added.



On Mon, Apr 17, 2017 at 4:22 PM, Brian Bouterse <bbouters at redhat.com> wrote:

> For plugin writers who are writing a publisher for Pulp3, what do they
> need to handle during publishing versus platform? To make a comparison
> against sync, the "Download API" and "Changesets" [0] allows the plugin
> writer to tell platform about a remote piece of content. Then platform
> handles creating the unit, fetching it, and saving it. Will there be a
> similar API to support publishing to ease the burden of a plugin writer?
> Also will this allow platform to have a structured knowledge of a
> publication with Pulp3?
>
> I wanted to try to characterize the problem statement as two separate
> questions:
>
> 1) How will units be recorded to allow platform to know which units
> comprise a specific publish?
> 2) What are plugin writer's needs at publish time, and what repetitive
> tasks could be moved to platform?
>
> As a quick recalling of how Pulp2 works. Each publisher would write files
> into the working directory and then they would get moved into their
> permanent home. Also there is the incrementalPublisher base machinery which
> allowed for an additive publication which would use the previous publish
> and was "faster". Finally in Pulp2, the only record of a publication are
> the symlinks on the filesystem.
>
> I have some of my own ideas on these things, but I'll start the
> conversation.
>
> [0]: https://github.com/pulp/pulp/pull/2876
>
> -Brian
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170419/b2c7d9cf/attachment.htm>