[Pulp-dev] pulp3: Publishing Proposal

[Jeff Ortel]

On 06/28/2017 10:37 PM, Michael Hrivnak wrote:
> 
> On Wed, Jun 28, 2017 at 4:52 PM, Jeff Ortel <jortel at redhat.com <mailto:jortel at redhat.com>> wrote:
> 
> 
>     I considered storing the base path in the Publication. But I don't see how the query using the /path/
>     component of the URL could be indexed if the path is split between the Publication and the LinkedArtifact.
> 
> 
> Ah yes! When I was hurriedly writing earlier, I knew there was some algorithmic problem related to paths but
> couldn't remember what it was. This is it. It's solvable, but at some point you need all the base paths in a
> tree structure that the serving app can use to find the correct match to a full path, similar to traversing
> directories in a filesystem. Or you need some other equivalent algorithm.
> 
> Quickly brainstorming on that though...
> 
> Assume we have a restriction that for any given base path, it must not reside within any other base path. We
> have that restriction today with yum repos.
> 
> Given a path with n segments, for example /a/b/c/foo.rpm would have 3 segments, the serving app could break it
> up into n possible base paths: /a/, /a/b/, /a/b/c/. It can do one database search on publication base paths
> for all n possible paths, and should only get one or zero results. If that field is indexed, this could be a
> quick way to find the right publication. Something like:
> 
> Publication.objects.get(base_path__in=['/a/', '/a/b/', '/a/b/c/'])
> 
> Granted, then you must do a second DB query for the published artifact within that publication. But this
> should all be a very small portion of the total time it takes to actually transmit the file itself, and I
> could easily believe this is as fast as reading a symlink off of an NFS mount.

This seems very promising.  I really do like normalizing the base path by storing in the publication.  I am
also not concerned by 2 queries vs 1.

> 
> -- 
> 
> Michael Hrivnak
> 
> Principal Software Engineer, RHCE 
> 
> Red Hat
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 847 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170629/92a25650/attachment.sig>