duplicate packages after up2date failure

[Damian Menscher]

On Sun, 9 Oct 2005, Damian Menscher wrote:

> We were taking advantage of the weekend to update our RHEL4.1 machine to
> RHEL4.2.  It is worth mentioning that it is an x86_64 box.
>
> our RPM database appears to be mangled quite badly, and the standard
> trick of rm /var/lib/rpm/__db.00[1-3] && rpm --rebuilddb doesn't help.
> In particular, the RPM database now thinks that we have *both* the old
> and new versions of several RPMs installed.  For example:
>
> # rpm -q --qf "%{NAME}-%{VERSION}-%{RELEASE} %{ARCH}\n" rpm
> rpm-4.3.3-9_nonptl x86_64
> rpm-4.3.3-11_nonptl x86_64
>
> We have 164 packages which are duplicated in this fashion.  Meanwhile
> up2date wants to update them (since we have the old version) but can't
> (because we have the new version).  So... I'm looking for advice on how
> to handle this.
>
> My guess is that the cleanest thing to do is remove the newer of each
> pair from the database and then run up2date, which should upgrade
> everything (fixing any old files on disk).

Several people have emailed me directly asking about how we resolved 
this, so I figure the best way to get the information out is through a 
followup to my own post:

There is an extra complication since some packages are *supposed* to 
have both i386 and x86_64 packages of the same version number installed. 
So my proposed script errored out since it didn't know what to do. 
Here are is the complete solution I used to resolve the issue:

# try to remove everything, but save that list of problems to /tmp/dupes
for file in `rpm -qa --queryformat="%{NAME} %{ARCH}\n" | sort | uniq -c | grep -v "  1 " | cut -c 9- | cut -d" " -f1`; do rpm -q --last $file | head -1 | cut -d" " -f1; done | grep -v gpg-pubkey | xargs rpm -e --justdb --nodeps 2> /tmp/dupes

# explicitly remove the i386 and x86_64 versions of the problem packages
for rpm in `cut -d\" -f2 /tmp/dupes`; do rpm -e --justdb --nodeps ${rpm}.i386 ${rpm}.x86_64; done

# go back through and fix all the other packages
for file in `rpm -qa --queryformat="%{NAME} %{ARCH}\n" | sort | uniq -c | grep -v "  1 " | cut -c 9- | cut -d" " -f1`; do rpm -q --last $file | head -1 | cut -d" " -f1; done | grep -v gpg-pubkey | xargs rpm -e --justdb --nodeps

# re-sync with Red Hat Network
up2date -p

# try the upgrade again and hope for the best
up2date

The reason I grep out gpg-pubkey is that you might have multiple gpg
keys on your system, and it confuses RPM.  We still ended up with
multiple copies of a few packages (like the kernel) but those are
expected.

Hopefully this is helpful to you, but it comes with no warranty.  All
I can say is it appears to have fixed our problems on a single machine.
You may want to consider some of the other suggestions posted to
fedora-list.

As a side note, when running up2date the second time, I watched its 
progress through frequent use of rpm -q.  Apparently up2date is trying 
to install 200 new packages, then remove the 200 old ones from the 
database.  So if it crashes (as happened to 20% of x86_64-smp users 
attempting the RHEL4.2 upgrade) then it leaves lots of duplicated 
packages (164 in our case).  This is unacceptable behavior: up2date 
should use atomic (at the package-level) operations on the database. Add 
one new package, remove one old one.  Add one new one, remove one old 
one.  This is an incredibly stupid design flaw in the operation of 
up2date, and RedHat should be LARTed for making such a stupid mistake in 
their flagship product (after all, we're paying the big bucks for RHEL 
for the stability of upgrades provided by RHN, and those upgrades are 
distributed through up2date).

Damian Menscher
-- 
-=#| Physics Grad Student & SysAdmin @ U Illinois Urbana-Champaign |#=-
-=#| 488 LLP, 1110 W. Green St, Urbana, IL 61801 Ofc:(217)333-0038 |#=-
-=#| 4602 Beckman, VMIL/MS, Imaging Technology Group:(217)244-3074 |#=-
-=#| <menscher at uiuc.edu> www.uiuc.edu/~menscher/ Fax:(217)333-9819 |#=-
-=#| The above opinions are not necessarily those of my employers. |#=-