Legacy mirror structure

[Warren Togami]

Matthias Saou wrote:

> Jesse Keating wrote :
> 
> 
>>Yes, difficult thing to think of.  Perhaps it would be feasable for the 
>>master mirror to hold the correct headers/ file that has SRPMS and debug 
>>stuff, but have a mirror.headers/ directory that is the result of
>>ignoring the SRPMS and the debug stuff.  I personally don't wish to
>>include debug stuff.  If the end user wants debug, they can download the
>>SRPM and build it themselves.  But a headers/ and a headers.nosrpms/ 
>>would be useful for mirror folks.
> 
> 
> Well, currently, the (yum) headers for SRPMS are not used for anything
> AFAIK. And as for having them included on a single config line along with
> the binary packages, it may be better to have exactly the opposite. With
> apt for instance, you're much better off not having "rpm-src" lines enabled
> in your configuration if you don't plan on using "apt-get source" very
> often. You save bandwidth as well as execution time.

I totally agree here.  "apt-get" source would also potentially fail on 
many of the RH packages due to the prevalent missing BuildRequires. 
Legacy's target users are NOT developers who use the SRPMS often.

For this reason I would not include SRPMS for the base distribution nor 
the almost as numerous RH supplied updates.  I would include only SRPMS 
of Legacy supplied updates, as Legacy QA and buildsystem necessitates 
fixing BuildRequires, so "apt-get source" at least has a chance of 
working properly (but it still is not very useful at all).

You make mirroring faster by leaving out those THOUSANDS of SRPMS that 
0.001% of the users would use.  You make client usage faster by leaving 
it out.  You make it far less of a burden in mirror administration by 
making Legacy as small as possible by default.

> 
> Anyway, wouldn't it be better to directly concentrate on the integration of
> the new common repodata files? The logical evolution is that it'll become
> the main supported server-side metadata format, so IMHO it's the most
> important to start thinking of now.

+1 exactly

> How will it handle binary vs. source sets of packages? AFAICT, it doesn't
> make any special difference between both, but as I said above, it may be
> interesting to decide to separate them if it can lower to amount of
> transfered data when minor changes occur (e.g. one new update).
> 
> I think the solution I like most is :
> 
> [...]/SRPMS/repodata/
> [...]/SRPMS/*.rpm
> [...]/$basearch/repodata/
> [...]/$basearch/*.rpm
> 

Jesse was earlier grappling with the problem of the ugly symlink for a 
unified SRPMS for all archs.  Using a structure like this would work 
wonderfully, while very conveniently placing all archs and the SRPMS at 
the same level.

Something like:
$distnumber/$repository/$basearch/*.rpm

Example:
7.2/updates/SRPMS/repodata/ <--- XML metadata
7.2/updates/SRPMS/headers/  <--- yum native headers
7.2/updates/i386/repodata/  <--- XML metadata
7.2/updates/i386/headers/   <--- yum native headers
7.2/updates/base/           <--- non-flat apt genbasedir location (*)
7.2/legacy/SRPMS/repodata/
7.2/legacy/SRPMS/headers/
7.2/legacy/i386/repodata/
7.2/legacy/i386/headers/
7.2/legacy/base/
  . . .
1/updates/SRPMS/repodata/
1/updates/SRPMS/headers/
1/updates/i386/repodata/
1/updates/i386/headers/
1/updates/x86_64/repodata/
1/updates/x86_64/headers/
1/updates/base/

Would something like this work?

> With a default configuration which only fetches metadata for $basearch
> packages but with the ability to easily enable fetching it for source
> packages too. Just as you said concerning the debug packages, that very few
> people actually need them, the same applies (in a different proportion) to
> source packages IMHO.
> 
> Matthias
> 

I totally agree with everything Matthias has stated here.

Warren

* I am not sure if apt works this way or not.  I have never used 
genbasedir without the --flat option.  Anyone know if anything like this 
tree structure is possible for the native apt metadata?