Yum, Proxy Cache Safety, Storage Backend

Nicolas Mailhot nicolas.mailhot at laposte.net
Thu Jan 24 19:23:59 UTC 2008


Le jeudi 24 janvier 2008 à 12:49 -0600, Les Mikesell a écrit :

> I'm missing something here.  A cache is only going to resend what it got 
> the first time.  If the first user got the right thing, so will the 
> second.

No it won't. Common failures are:

- T0: user A updates foo and in the course of doing it populates the
proxy cache with yum T0 metadata and foo rpm only.
- T1: upstream updates repo changing bar rpm version. Metadata is
regenerated with the same filenames as before. For the proxy its T0 copy
is still valid till the expiration delay  it got in http headers the
first time.
- T2: user B wants to update bar. The proxy happily serves yum the
cached T0 metadata (since yum didn't request any special treatment for
it). It tells yum the T0 bar is available. Unfortunately it's not - user
A didn't download it so it's not in the cache, and upstream moved to a
new version

- T0: user A updates foo and in the course of doing it populates the
proxy cache with yum T0 metadata and foo rpm 
- T1: upstream signs foo rpm and regenerates the repo. Metadata is
regenerated with the same filenames as before. For the proxy its T0 copy
is still valid till the expiration delay  it got in http headers the
first time.
- T2: proxy cache is contended, it evicts foo rpm since it takes a lot
of room. T0 metadata is small so it's retained for now
- T3: user B wants to update foo. The proxy happily serves yum the
cached T0 metadata (since yum didn't request any special treatment for
it). It tells yum foo is available with T0 checksum. Unfortunately it's
not - T0 foo rpm was evicted from the cache, when user B requests the
same URL as user A it gets T1 signed foo with a new checksum.

(you can invert this case with a proxy which strategy is to keep big
files which are expensive to download, and drop small metadata)

In all those cases the proxy worked as designed and was fooled by
yum/createrepo using the same URLs for different objects.

A proxy will always serve you a set of files as they existed on the
original location at some time. But there is zero insurance they were
taken from this location at the same time, so you *must* take care new
content is created with new filenames, or your web server tells the
proxy when old data will be replaced by new one (via expires), or your
system is able to cope with a mix of files from different times.

-- 
Nicolas Mailhot
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: Ceci est une partie de message num?riquement sign?e
URL: <http://listman.redhat.com/archives/fedora-devel-list/attachments/20080124/ca596c4f/attachment.sig>


More information about the fedora-devel-list mailing list