[Pulp-list] Missing packages in clones

[Pradeep Kilambi]

On 02/28/2012 08:39 AM, Chris St. Pierre wrote:
> We've been tracking a Pulp issue for a while now, but we haven't found
> a replication scenario for it just yet.  What happens is that a repo
> -- usually a clone or a clone-of-a-clone -- simply starts missing a
> few (or a great many) package files, although the packages are listed
> in the repo metadata.  There's nothing in either the Pulp or the
> Grinder logs to indicate why the packages are missing.
>
> For instance:
>
> % pulp-admin repo content --id=lustre-prod-rhel-6-x86_64-updates | grep audit
> audit-viewer-0.5.1-3.el6.x86_64.rpm
> audit-2.1.3-3.el6.x86_64.rpm
> audit-libs-2.1.3-3.el6.i686.rpm
> audit-libs-devel-2.1.3-3.el6.x86_64.rpm
> audit-libs-python-2.1.3-3.el6.x86_64.rpm
> audit-libs-2.1.3-3.el6.x86_64.rpm
> audit-libs-devel-2.1.3-3.el6.i686.rpm
>
> % gunzip -c repodata/primary.xml.gz | grep '<name>audit'
>    <name>audit</name>
>    <name>audit-libs</name>
>    <name>audit-libs-devel</name>
>    <name>audit-libs-devel</name>
>    <name>audit-libs-python</name>
>    <name>audit-viewer</name>
>    <name>audit-libs</name>
>
> % ls audit*
> audit-viewer-0.5.1-3.el6.x86_64.rpm
>
> It's like audit and audit-libs* have been removed.  They're present in
> the parent repo, though:
>
> % pulp-admin repo info --id=lustre-prod-rhel-6-x86_64-updates
> [...snip...]
> Feed URL                file:///var/lib/pulp//repos/rhel-6-x86_64-updates
> [...snip...]
>
> % pulp-admin repo content --id=rhel-6-x86_64-updates | grep audit
> audit-viewer-0.5.1-3.el6.x86_64.rpm
> audit-2.1.3-3.el6.x86_64.rpm
> audit-libs-2.1.3-3.el6.i686.rpm
> audit-libs-devel-2.1.3-3.el6.x86_64.rpm
> audit-libs-python-2.1.3-3.el6.x86_64.rpm
> audit-libs-2.1.3-3.el6.x86_64.rpm
> audit-libs-devel-2.1.3-3.el6.i686.rpm
>
> % cd /var/lib/pulp//repos/rhel-6-x86_64-updates
> % gunzip -c repodata/primary.xml.gz | grep '<name>audit'
>    <name>audit-viewer</name>
>    <name>audit</name>
>    <name>audit-libs</name>
>    <name>audit-libs-devel</name>
>    <name>audit-libs-devel</name>
>    <name>audit-libs-python</name>
>    <name>audit-libs</name>
>
> % ls audit*
> audit-2.1.3-3.el6.x86_64.rpm
> audit-libs-devel-2.1.3-3.el6.i686.rpm
> audit-viewer-0.5.1-3.el6.x86_64.rpm
> audit-libs-2.1.3-3.el6.i686.rpm    audit-libs-devel-2.1.3-3.el6.x86_64.rpm
> audit-libs-2.1.3-3.el6.x86_64.rpm  audit-libs-python-2.1.3-3.el6.x86_64.rpm
>
> A sync solves the problem -- temporarily.  Without fail, within a few
> days packages go missing again.  There doesn't seem to be any rhyme or
> reason to which packages go missing; next time it's unlikely that
> audit-libs* will be gone again.
>
> It's not always just a few packages, either:
>
> % pulp-admin repo info --id=lustre-prod-rhel-6-x86_64-optional
> [...snip...]
> Packages                2726
> [...snip...]
>
> % ls *.rpm | wc -l
> 8
>
> This does not seem to happen during a sync; for instance, the
> lustre-prod-rhel-6-x86_64-optional repo was synced
> 2012-02-26T16:35:16-05:00, was working on the 27th of February, and
> was broken on the morning of the 28th.  We don't have a good idea of
> when exactly the symlinks were removed.
>
> Has anyone else seen this?  Any ideas or suggestions of where to go
> next?  Thanks!
>

  I'll try to reproduce this above scenario and see if i can find 
anything. There was a similar issue with resyncs we addressed few weeks 
ago. I haven't seen any missing packages since then.

~ Prad