Mock and squid

[Michael Thomas]

Michael Thomas wrote:
> Paul Howarth wrote:
> 
>>Anybody have squid working well with mock?
>>
>>I followed the guide on the wiki (Extras/MockTricks) and have the
>>following non-default config options:
>>
>>cache_swap_low 95
>>cache_swap_high 98
>>maximum_object_size 150000 KB
>>maximum_object_size_in_memory 8 MB
>>cache_replacement_policy heap LFUDA
>>memory_replacement_policy heap LFUDA
>>cache_dir ufs /var/spool/squid 4400 16 256
>>acl localnet src 192.168.2.0/24
>>http_access allow localnet
>>memory_pools_limit 64 MB
>>
>>/var/spool/squid is a separate 5G partition but squid seems very
>>reluctant to fill it. After several mock builds, it's using only a small
>>percentage of the available space:
>>
>>/dev/mapper/VgBuildSys-squid
>>                       5160576    489928   4408504  11% /var/spool/squid
>>
>>The store.log indicates that it's not saving much at all (most RPM
>>packages are RELEASE-d, and only the repodata XML files appear to be
>>being stored).
>>
>>Any suggestions as to why this might be, and/or how to debug it?
> 
> 
> Are you sure the RELEASE lines correspond to the actual rpm files and
> not the rpm headers?  Check for a HTTP status code of 200 for the actual
> rpm files, vs. a status code of 206 (partial content) for the rpm
> headers.  The squid configuration on this page does not cache partial
> downloads (will squid even do that?) and will RELEASE them immediately:
> 
> Partial download (rpm header) not stored:
> 1156979634.763 RELEASE -1 FFFFFFFF 831BB6E09AECB2516F9279A7271237F9  206
> 1156979626        -1        -1 application/x-rpm 26711/26711 GET
> http://newmirror.linux.duke.edu/pub/fedora/linux/core/updates/5/i386/selinux-policy-targeted-2.3.7-2.fc5.noarch.rpm
> 
> Full rpm stored:
> 1156979716.933 SWAPOUT 00 00000076 E8B3B83D4E58A2EA421A5B88B72B8C8C  200
> 1156979642 1156785034        -1 application/x-rpm 532860/532860 GET
> http://newmirror.linux.duke.edu/pub/fedora/linux/core/updates/5/i386/selinux-policy-targeted-2.3.7-2.fc5.noarch.rpm
> 
> You might be able to use squidclient to tell you directly if something
> is in the cache, but I haven't found decent docuementation for this tool
> yet.  Instead you can use it to download individual packages and look in
> access.log for the TCP_HIT vs. TCP_MISS messages, where TCP_HIT
> indicates that a cached object is returned and TCP_MISS indicates that
> the object was not cached:
> 
> # squidclient -s -v -m GET
> http://newmirror.linux.duke.edu/pub/fedora/linux/core/updates/5/i386/selinux-policy-targeted-2.3.7-2.fc5.noarch.rpm
> # tail -1 access.log
> # tail -1 access.log
> 1156980363.449      5 127.0.0.1 TCP_HIT/200 533210 GET
> http://newmirror.linux.duke.edu/pub/fedora/linux/core/updates/5/i386/selinux-policy-targeted-2.3.7-2.fc5.noarch.rpm
> - NONE/- application/x-rpm

To follow up to my own post:

You can use squidclient to download the file, then look at the X-Cache
headers to see if it was retrieved from the cache.  The X-Cache header
will be missing if it did not come from the cache.  This avoids having
to look in access.log for TCP_HIT and TCP_MISS messages:

# squidclient -m GET
http://newmirror.linux.duke.edu/pub/fedora/linux/core/updates/5/i386/selinux-policy-targeted-2.3.7-2.fc5.noarch.rpm
| grep -a X-Cache
X-Cache: HIT from owl.kobold.org
X-Cache-Lookup: HIT from owl.kobold.org:3128


Unfortunately, it looks like squidclient only supports HTTP/1.0, so you
won't be able to request a partial download of the file just to verify
that it's in the cache; you'll have to download the entire thing.

--Wart
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3820 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/fedora-extras-list/attachments/20060830/c4626b62/attachment.bin>