FC5t3 - rpm -e hangs up

Jan Andrejkovic jandrejkovic at gmail.com
Sat Mar 4 09:37:13 UTC 2006


Hello Jim and David,

Thank you very much for your advice, the problem is solved!!!!
There is no rpm hangup bug.

After David's note about rpm -Va I have noticed that it takes a 'while' to
finish and after David's question
> How long did you give it before nuking it ?
I have decided that I will be VERY patient. I have left rpm running
overnight and in the morning it was finished.
The problem is that I do not know the exact time. My last record from top
(before I get to bed) is:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
13632 root      25   0  238m 149m 4256 R 98.4 29.8  80:04.16 rpm

It took minimally more than an hour on the Pentium M 1.4Mhz eating all
CPU... I have never seen rpm performing such a long time before.

Thank you very much for your help!!!

Have a nice weekend,

Jan
PS: I never delete files manually. I was in a wrong believe that rpm
database is cleared of xen packages after xen entries for FC4 were deleted
      from the grub.conf after FC4->FC5t3 migration, so I decided co cleanup
my system and I deleted xen kernels manually (but nothing else).

PPS: Here is rpm command I used and complete verbose output:

# rpm -evv --justdb kernel-xen0-devel kernel-xenU-devel --allmatches
D: opening  db environment /var/lib/rpm/Packages create:cdb:mpool
D: opening  db index       /var/lib/rpm/Packages rdonly mode=0x0
D: locked   db index       /var/lib/rpm/Packages
D: opening  db index       /var/lib/rpm/Name rdonly mode=0x0
D:  read h#     468 Header SHA1 digest: OK
(99fd52c1d2a31ea6661ce0d249a26852b471d211)
D: opening  db index       /var/lib/rpm/Pubkeys rdonly mode=0x0
D:  read h#    1040 Header sanity check: OK
D: ========== DSA pubkey id b44269d0 4f2a6fd2 (h#1040)
D:  read h#     732 Header V3 DSA signature: OK, key ID 4f2a6fd2
D:  read h#     117 Header SHA1 digest: OK
(c8d069d827dc6bd4e6293ceca8f1f45568480cfc)
D:  read h#     678 Header V3 DSA signature: OK, key ID 4f2a6fd2
D: ========== --- kernel-xen0-devel-2.6.12-1.1454_FC4 i686/linux 0x0
D: opening  db index       /var/lib/rpm/Requirename rdonly mode=0x0
D: ========== --- kernel-xen0-devel-2.6.13-1.1532_FC4 i686/linux 0x0
D: ========== --- kernel-xenU-devel-2.6.12-1.1454_FC4 i686/linux 0x0
D: ========== --- kernel-xenU-devel-2.6.13-1.1532_FC4 i686/linux 0x0
D: ========== recording tsort relations
D: ========== tsorting packages (order, #predecessors, #succesors, tree,
depth, breadth)
D:     0    0    0    3    1    0   -
kernel-xenU-devel-2.6.13-1.1532_FC4.i686
D: ========== successors only (0 bytes)
D:     1    0    0    0    1    1   -
kernel-xen0-devel-2.6.12-1.1454_FC4.i686
D:     2    0    0    1    1    2   -
kernel-xen0-devel-2.6.13-1.1532_FC4.i686
D:     3    0    0    2    1    3   -
kernel-xenU-devel-2.6.12-1.1454_FC4.i686
D: closed   db index       /var/lib/rpm/Pubkeys
D: closed   db index       /var/lib/rpm/Requirename
D: closed   db index       /var/lib/rpm/Name
D: closed   db index       /var/lib/rpm/Packages
D: closed   db environment /var/lib/rpm/Packages
D: opening  db environment /var/lib/rpm/Packages joinenv
D: opening  db index       /var/lib/rpm/Packages create mode=0x42
D: mounted filesystems:
D:     i        dev    bsize       bavail       iavail mount point
D:     0 0x00000307     1024       286906       167116 /
D:     1 0x00000003     4096            0           -1 /proc
D:     2 0x00000000     4096            0           -1 /sys
D:     3 0x0000000a     4096            0           -1 /dev/pts
D:     4 0x00000011     4096        64439        64438 /dev/shm
D:     5 0x0000030c     1024       121091        98975 /home
D:     6 0x0000030b     1024       397617       131515 /tmp
D:     7 0x00000308     4096       252679      1462737 /usr
D:     8 0x0000030a     4096       120759       285237 /var
D:     9 0x00000013     4096            0           -1
/proc/sys/fs/binfmt_misc
D:    10 0x00000014     4096            0           -1
/var/lib/nfs/rpc_pipefs
D:    11 0x00000015     4096            0           -1 /net
D: sanity checking 4 elements
D: running pre-transaction scripts
D: computing 24608 file fingerprints
D: computing file dispositions
D: opening  db index       /var/lib/rpm/Basenames create mode=0x42
D: ========== --- kernel-xenU-devel-2.6.13-1.1532_FC4 i686-linux 0x0
D:     erase: kernel-xenU-devel-2.6.13-1.1532_FC4 has 5012 files, test = 0
D: opening  db index       /var/lib/rpm/Name create mode=0x42
D:  read h#     678 Header V3 DSA signature: OK, key ID 4f2a6fd2
D:   --- h#     678 kernel-xenU-devel-2.6.13-1.1532_FC4
D: removing "kernel-xenU-devel" from Name index.
D: removing 5012 entries from Basenames index.
D: opening  db index       /var/lib/rpm/Group create mode=0x42
D: removing "System Environment/Kernel" from Group index.
D: opening  db index       /var/lib/rpm/Requirename create mode=0x42
D: removing 6 entries from Requirename index.
D: opening  db index       /var/lib/rpm/Providename create mode=0x42
D: removing 4 entries from Providename index.
D: opening  db index       /var/lib/rpm/Dirnames create mode=0x42
D: removing 1475 entries from Dirnames index.
D: opening  db index       /var/lib/rpm/Requireversion create mode=0x42
D: removing 6 entries from Requireversion index.
D: opening  db index       /var/lib/rpm/Provideversion create mode=0x42
D: removing 4 entries from Provideversion index.
D: opening  db index       /var/lib/rpm/Installtid create mode=0x42
D: removing 1 entries from Installtid index.
D: opening  db index       /var/lib/rpm/Sigmd5 create mode=0x42
D: removing 1 entries from Sigmd5 index.
D: opening  db index       /var/lib/rpm/Sha1header create mode=0x42
D: removing "851a2f954dfbb73ffdca2e99746214e79edb8089" from Sha1header
index.
D: opening  db index       /var/lib/rpm/Filemd5s create mode=0x42
D: removing 5012 entries from Filemd5s index.
D: ========== --- kernel-xen0-devel-2.6.13-1.1532_FC4 i686-linux 0x0
D:     erase: kernel-xen0-devel-2.6.13-1.1532_FC4 has 7414 files, test = 0
D:  read h#     732 Header V3 DSA signature: OK, key ID 4f2a6fd2
D:   --- h#     732 kernel-xen0-devel-2.6.13-1.1532_FC4
D: removing "kernel-xen0-devel" from Name index.
D: removing 7414 entries from Basenames index.
D: removing "System Environment/Kernel" from Group index.
D: removing 6 entries from Requirename index.
D: removing 4 entries from Providename index.
D: removing 2554 entries from Dirnames index.
D: removing 6 entries from Requireversion index.
D: removing 4 entries from Provideversion index.
D: removing 1 entries from Installtid index.
D: removing 1 entries from Sigmd5 index.
D: removing "3b91e9a818dae6f182b83d80d001b1605017dedc" from Sha1header
index.
D: removing 7414 entries from Filemd5s index.
D: ========== --- kernel-xen0-devel-2.6.12-1.1454_FC4 i686-linux 0x0
D:     erase: kernel-xen0-devel-2.6.12-1.1454_FC4 has 7292 files, test = 0
D:  read h#     468 Header SHA1 digest: OK
(99fd52c1d2a31ea6661ce0d249a26852b471d211)
D:   --- h#     468 kernel-xen0-devel-2.6.12-1.1454_FC4
D: removing "kernel-xen0-devel" from Name index.
D: removing 7292 entries from Basenames index.
D: removing "System Environment/Kernel" from Group index.
D: removing 6 entries from Requirename index.
D: removing 4 entries from Providename index.
D: removing 2513 entries from Dirnames index.
D: removing 6 entries from Requireversion index.
D: removing 4 entries from Provideversion index.
D: removing 1 entries from Installtid index.
D: removing 1 entries from Sigmd5 index.
D: removing "99fd52c1d2a31ea6661ce0d249a26852b471d211" from Sha1header
index.
D: removing 7292 entries from Filemd5s index.
D: ========== --- kernel-xenU-devel-2.6.12-1.1454_FC4 i686-linux 0x0
D:     erase: kernel-xenU-devel-2.6.12-1.1454_FC4 has 4890 files, test = 0
D:  read h#     117 Header SHA1 digest: OK
(c8d069d827dc6bd4e6293ceca8f1f45568480cfc)
D:   --- h#     117 kernel-xenU-devel-2.6.12-1.1454_FC4
D: removing "kernel-xenU-devel" from Name index.
D: removing 4890 entries from Basenames index.
D: removing "System Environment/Kernel" from Group index.
D: removing 6 entries from Requirename index.
D: removing 4 entries from Providename index.
D: removing 1434 entries from Dirnames index.
D: removing 6 entries from Requireversion index.
D: removing 4 entries from Provideversion index.
D: removing 1 entries from Installtid index.
D: removing 1 entries from Sigmd5 index.
D: removing "c8d069d827dc6bd4e6293ceca8f1f45568480cfc" from Sha1header
index.
D: removing 4890 entries from Filemd5s index.
D: running post-transaction scripts
D: closed   db index       /var/lib/rpm/Filemd5s
D: closed   db index       /var/lib/rpm/Sha1header
D: closed   db index       /var/lib/rpm/Sigmd5
D: closed   db index       /var/lib/rpm/Installtid
D: closed   db index       /var/lib/rpm/Provideversion
D: closed   db index       /var/lib/rpm/Requireversion
D: closed   db index       /var/lib/rpm/Dirnames
D: closed   db index       /var/lib/rpm/Providename
D: closed   db index       /var/lib/rpm/Requirename
D: closed   db index       /var/lib/rpm/Group
D: closed   db index       /var/lib/rpm/Basenames
D: closed   db index       /var/lib/rpm/Name
D: closed   db index       /var/lib/rpm/Packages
D: closed   db environment /var/lib/rpm/Packages
D: May free Score board((nil))

Have a look on the Time+ value - 80:04.16 and it was not finished at this
point...
On 3/4/06, Jim Cornette <fct-cornette at insight.rr.com> wrote:
>
> Jan Andrejkovic wrote:
> > Hello Jim and everybody who is willing to help,
> >
> > Runlevel 1 did not help, but I have found the source of the problem,
> > unfortunatelly not the solution:
> >
> > The source of the problem is that I have deleted some installed files
> > manually, without RPM.
>
> Not a good practice. This is not wise for any operating system. RPM does
> a great job removing and installing files when the rpm file is properly
> referenced for installation and removal of packages. Not all packages
> are always setup correctly but the bulk of rpms are well thought out by
> those that setup the routines that rpm undergoes. There are many options
> that rpm can handle and many that I am still not familiar with after
> years of usage.
>
> > Anyway I think rpm should be robust and it should not hang if some files
> > are deleted.
>
> It depends upon which files that you deleted. If you deleted a common
> file or a file which is intricate to the program's operation or that of
> the system, it will have no choice but to bomb out. Its primary jobs are
> to allow adding removing of programs in a sane manner, ensuring that the
> program contains all its needed components (binaries, setup files,
> services started, user added, and on) It can only do so if all of the
> components are left intact.
> Since rpm checks for files being in a location, missing or that the file
> is not like the orginal, it does catch the most likely situations. But
> if rpm checked every file for its presence before trying to remove a
> file, it would require a longer time for rpm to complete the task of
> adding or removing the files.
> It sounds like you might have pressed rpm and the package that you were
> trying to remove into a corner case and if you can recall what you
> deleted, it might be helpful to prevent a future incident in the
> package, rpm program or user awareness and familiarity with the program
> concept.
>
> > It should display some warning instead.
>
> Granted, rpm should ideally use its feature for verifying everything is
> present before attempting a package removal. It should at least check
> for the file presence or have a routine to accomplish if the file was
> missing. I believe most rpms are setup to continue or report the problem
> to the terminal output if a problem was encountered.
> There are problems that happen when failures happen with %post and %pre
> scripts that need improvement in design. The major drawbacks are when
> %pre scripts fail and programs are setup to be installed, they may not
> ever install because deps are checked before the installation
> transaction. Things seemed setup for the transaction to do the right
> thing. The problem might be that the package is setup correctly, but
> security programs may not grant permission to rpm to place files, run
> tasks and other unpredictable problems.
> The problem with %post script failures is that the tasks might be
> completed pretty much and there was only a minor failure because the rpm
> was on its cleanup phase. Other %post failures might cause operational
> tasks from completing such as a kernel never getting through to the
> portion where the image needed for the modules and the entry into the
> bootloader never gets completed. I am sure that the packagers keep all
> of these factors in mind and do their best to prevent problems from
> occurring.
>
> >
> > I have used -vv option and here is deailed output from rpm:
> > rm -f /var/lib/rpm/__db*
>
> the deleted portion looks normal to me. I however never packages a
> program for distribution.
>
> > D:     0    0    0    3    1    0   -
> > kernel-xenU-devel-2.6.13-1.1532_FC4.i686
> > D: ========== successors only (0 bytes)
>
> Those that know more about the workings of rpm would know what the six
> variables represent. The display of the rpm output does reveal to me
> something more than I previously knew about rpm internals. I take it
> that the program found no previous version to act upon. I am only going
> on the zero bytes feedback.
>
> > D:     1    0    0    0    1    1
> > -kernel-xen0-devel-2.6.12-1.1454_FC4.i686
> > D:     2    0    0    1    1    2
> > -kernel-xen0-devel-2.6.13-1.1532_FC4.i686
> > D:     3    0    0    2    1    3
> > -kernel-xenU-devel-2.6.12-1.1454_FC4.i686
>
> It looks like the first of six digits is some counter (0,1,2,3)
>
>
> > D: closed   db index       /var/lib/rpm/Pubkeys
> > D: closed   db index       /var/lib/rpm/Requirename
> > D: closed   db index       /var/lib/rpm/Name
> > D: closed   db index       /var/lib/rpm/Packages
> > D: closed   db environment /var/lib/rpm/Packages
> > D: opening  db environment /var/lib/rpm/Packages joinenv
> > D: opening  db index       /var/lib/rpm/Packages create mode=0x42
> > D: mounted filesystems:
> > D:     i        dev    bsize       bavail       iavail mount point
> > D:     0 0x00000307     1024       287812       167116 /
> > D:     1 0x00000003     4096            0           -1 /proc
> > D:     2 0x00000000     4096            0           -1 /sys
> > D:     3 0x0000000a     4096            0           -1 /dev/pts
> > D:     4 0x00000011     4096        64439        64438 /dev/shm
> > D:     5 0x0000030c     1024       135546        99333 /home
> > D:     6 0x0000030b     1024       397619       131519 /tmp
> > D:     7 0x00000308     4096       260065      1462823 /usr
> > D:     8 0x0000030a     4096       157409       285427 /var
> > D:     9 0x00000013     4096            0           -1
> > /proc/sys/fs/binfmt_misc
> > D:    10 0x00000014     4096            0           -1
> > /var/lib/nfs/rpc_pipefs
> > D:    11 0x00000015     4096            0           -1 /net
> > D: sanity checking 4 elements
>
> Rpm and the package being acted upon are trying to be sane.
>
> > D: running pre-transaction scripts
>
> This I take is the %pre factor mentioned earlier. What do I need to do
> before proceeding with the installation of the package?
>
> > D: computing 24608 file fingerprints
> > D: computing file dispositions
> > D: opening  db index       /var/lib/rpm/Basenames create mode=0x42
> > Killed
>
> You might be able to use a program that can actually peek into the
> actual package to see what the following step would be. There is a file
> manager utility/shell program called mc (midnight commander) which is
> able to dive into the rpm content. The script is executable so be sure
> to use the F3 function key to view the file content. Being able to view
> what is actually within a packages rpm might show interest to you and
> enlighten you more than the link you referenced tries to explain.
> I have used mc to view rpms and deb package content and found the
> internals of both formats interesting. It is also possible to retrieve
> files from the rpms, deb or whatever files with mc.
>
> >
> > It hanged after the last step and I had to kill it.
>
> If the similarity is anything like DOS loading stuff, it is usually the
> step following the lockup. It locked and did not reveal anything
> regarding its initialization. Personally, I like feedback before
> performing the function and feedback as to what the results were for the
> tasks.
>
> > I have straced it as well and here is the part of output from strace
> > which is running in a neverending loop:
> > <cut>
> > stat64("/usr/src/kernels/2.6.12- 1.1454_FC4-xenU-i686/arch/i386",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-
> > 1.1454_FC4-xenU-i686/arch/i386/mach-visws", 0xbfbd6784) = -1 ENOENT (No
> > such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-
> 1.1454_FC4-xenU-i686/arch/i386/mach-voyager",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/math-emu
> ",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/mm",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/oprofile
> ",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/pci",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch", 0xbfbd6784)
> > = -1 ENOENT (No such file or directory)
> > stat64("/usr/src/kernels/2.6.12-1.1454_FC4-xenU-i686/arch/i386/power",
> > 0xbfbd6784) = -1 ENOENT (No such file or directory)
> > <cut>
> >
> > I have deleted my FC4 xen kernels in FC5t3 manually because I thought
> > that rpm dabase is already clear. I was wrong.
> > My RPM version is: 4.4.2
> >
> > Do you think I should open bugzilla ticket?
>
> It is a shortcoming of the package, not rpm to be able to handle this
> circumstance. I file bug reports myself for the developer to at least be
> aware of something that could be handled better. Most reports are marked
> as not supported or the like. The awareness for the developer is at
> least presented to the developer. Backburner or not, the rpm routine
> should be able to handle this possible situation.
>
> > Do you have any advice how can I reinstall my xen or what can I do?
>
> rpm has features where you can ignore scripts, delete only the rpmdb
> entry and a host of possible resolutions. Rpm most likely has features
> which will get you out of the problem.
>
> I would run
> rpm -e --justdb --nodeps <continuous-loop-package>
>   and then try to *install* this rpm again if desired. By reviewing the
> output above, running my favorite command above to rid the rpmdb entry
> from the older version package should be enough. Reading your original
> posting, it seems that you already attempted this feature.
>
> >
> > By the way I have found very good rpm guide, but it did not help me
> either:
> > http://www.redhat.com/docs/books/max-rpm
>
> I'll have to take a look at this page later myself. I never read the
> documentation before.
> >
> > Thank you very much,
> >
> > Jan
> >
> > On 3/1/06, *Jim Cornette * <fct-cornette at insight.rr.com
> > <mailto:fct-cornette at insight.rr.com>> wrote:
> >
> >     Jan Andrejkovic wrote:
> >      > Hello,
> >      >
> >      > I have upgraded FC4 to FC5t3. I had some xen problems therefore I
> >     have
> >      > decided to reinstall xen packages.
> >      > But when I try
> >      > rpm -e kernel-xen-hypervisor-devel or rpm -e
> >     kernel-xen-guest-devel rpm
> >      > hangs up and eats almost 100% cpu for long time.
> >      > I need to use kill -9 to stop it.
>
> Were you removing 1454 or 1532. If you have multiple versions installed,
> specifying which version might be needed. RPM should have bombed out
> specifying that multiple versions installed and to specify which version
> that you wanted to remove.
>
> >      >
> >      > I have tried to do
> >      > rm -f /var/lib/rpm/__db*
> >      > and
> >      > rpm --rebuilddb
> >      > but nothing helped after those commands - it hangs again.
> >      >
> >      > I have also tried rpm -e --justdb but it did not help either.
>
> Without specifying the version to remove? Or specifying which verson to
> remove?
>
> >    >
>
> >> Initially it was "yum remove" which hanged first. But as far as I
> > know it uses rpm therefore I report this as rpm problem.
>
>
> I have no idea for certain. I missed a lot of information from your
> original posting as reference to the steps you tried.
>
> >>
> >> Does anybody have the same problem or can anybody tell me some
> >> workaround how I can remove those packages manually and possibly
> > rebuild database without them?
>
> I believe the database does not need rebuilt. The entries within the
> database need corrected by human intervention using the features rpm is
> capable of.
>
> I hope that I am not misleading or jumping off on unrelated to your
> problem. I doubt that is the case though, I probably departed on several
> issues. I was afraid to reply to the posting initially with all the
> information provided in verbose and debugging output.
>
> I'd figure out how best to represent the problem with yum and the
> hypervisor removal and your actions. I believe you attempted using yum
> then rpm and later deleted the files afterward. It sounds like a really
> big bug since the looping and cpu utilization.
>
> >>
> >> Thank you very much,
> >>
> >> Jan
> >
> >
>
> I hope this leads you to a resolution of the problem.
>
> Jim
>
> --
> What I want is all of the power and none of the responsibility.
>
> --
> fedora-test-list mailing list
> fedora-test-list at redhat.com
> To unsubscribe:
> https://www.redhat.com/mailman/listinfo/fedora-test-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/fedora-test-list/attachments/20060304/77f255bc/attachment.htm>


More information about the fedora-test-list mailing list