[Pulp-list] Restructuring proposal for /var/lib/pulp

[Dennis Gregorovic]

On Tue, 2010-08-31 at 11:47 -0400, Pradeep Kilambi wrote:
> I'm currently working on a task to make our packages globally unique
> across the pulp server. Meaning, we'll have one and only one package
> with same NVREA + checksum at a time and linked to multiple repos. For
> this I need to restructure our content directory a bit. This is my
> proposal,
> 
> Current Structure,
> 
> /var/lib/pulp/
>              |_ ReopA/
>              |
>              |_ RepoB/
> 
> Proposed New Structure,
> 
> /var/lib/pulp/
>              |_Packages/checksum[:3]/*.rpm
>              |
>              |_Repos/
>                     |_RepoA
>                     |_RepoB
> 
> Packages in RepoA and RepoB will be symlinks to Packages directory.
> Apache will now only expose /var/lib/pulp/Repos and not include
> Packages directory. 
> 
> Considering this is kinda major change I'm  working off a branch and
> will see how stable it is before I merge it into this sprint. Overall
> change is gonna be to both grinder and pulp. 
> 
> Lemme know if you have any concerns with this change,

What's the rationale for making NVREA+checksum globally unique?  I would
think either NVRA+signature or just NVRA would be unique.  Also, note
that I've dropped the epoch from the clause.  Brew, koji, and RHN all
prevent multiple RPMs with the same NVRA and different epoch.  I think
that's a good practice to continue.

The directory structure that we are using for the CDN is:

/<root>/
        |_origin/
        |       |
        |        <name>/
        |              |_<version>/
        |                         |_<release>/
        |                                    |_<signature key>/*.rpm
        |_content/<...>/<symlinks>

This path makes it quick and easy to find a package.  The one issue I
have is that the <name> directory has around 8k subdirectories.  So,
opening that directory takes some time, but it's an operation that is
rarely performed.

Also, I would suggest lowercase directory names.

Cheers
-- Dennis