Improving availability and guaranteeing integrity in ISO - internal sha1sums
David Timms
dtimms at iinet.net.au
Sat Jun 9 02:20:14 UTC 2007
Jesse Keating wrote:
> On Friday 08 June 2007 14:24:47 Anthony Bryan wrote:
>> I was hoping Fedora could investigate using Metalinks for their ISO
>> downloads. Metalink is an XML format for listing all the ways you can
>> get a file or collection of files (mirrors + their location, rsync,
>> p2p) along with checksums to automatically repair a file in case of
>> error, signatures, language, OS/arch, and other metadata. It's mainly
>> used for large files like ISOs, where errors can be very frustrating.
>>
>> It's supported by about 20 programs on unix, mac, and win, including
>> aria2 (already in the Fedora repos). It's used by openSUSE,
>> OpenOffice.org, cURL, and many other distributions.
>>
>> Here's a screenshot of a Metalink download in the DownThemAll Firefox
>> extension (nightly build). What you don't see are all the mirrors and
>> checksums.
>> http://code.downthemall.net/maierman/metaselect4.png
>>
>> http://en.wikipedia.org/wiki/Metalink
>
> This is something interesting, and I wonder if we could make use of
> MirrorManager ( https://hosted.fedoraproject.org/projects/mirrormanager ) to
> have dynamic .metalink files created with updated mirror readiness info.
> Certainly something that looks worth looking into.
I recently worked on a tiny tool to use already downloaded files to
reconstruct a full iso-image {eg Fedora test to Fedora release iso}. I
could have made this a lot more sure that it was doing the right thing
by comparing sha1sum of the individual files with reference sha1sum
files. {for interest see:
http://www.redhat.com/archives/fedora-test-list/2007-June/msg00018.html }
The simplest way to do this would be for the iso spin system to perform
an sha1sum * > SHA1SUM within each directory of an iso spin, and have
each result inserted into the corresponding directory.
I could see other uses for embedding checksums within the iso archive :
- live test the files written within an iso image:
- insert media {gets automounted}
- cd /media/whatever
- sha1sum -c SHA1SUM
- actually proves the disc/drive/PC can properly read the individual
files.
This could be extended to a simple tool in the root folder that can
verify that all files in all folders on the cd/dvd are readable and
correct. {python would be my weapon of choice, but }.
- Enable fast download method like above/jigdo/bittorrent to work on per
file contained within the {iso} archive, rather than the iso file {whole
or in fixed sized pieces that span/split files}
- This makes it easier for a download tool to verify that a file
found as a potential content of an iso is actually the correct file to
insert.
Is inserting per folder checksum results into the iso image a do-able
thing ? Would a patch to achieve such be considered ?
DaveT.
More information about the fedora-devel-list
mailing list