[sos-devel] [RFC PATCH v2] sosreport: Check for rpm database corruption during initialization

[Aruna Balakrishnaiah]

sosreport runs an rpm query to get the package list. If rpmdb is corrupted
sosreport hangs for ever hence check for rpmdb consistency before running
the rpm query

Signed-off-by: Aruna Balakrishnaiah <aruna at linux.vnet.ibm.com>
---
Changes from v1:
	Addressed issues of the maintainer
	Introduce timeout instead of relying on yum.

When rpmdb is corrupted rpm commands hangs forever,
yum check will result in:

yum check
error: rpmdb: BDB0113 Thread/process 43828/70366497037824 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:

Error: rpmdb open failed 

 sos/policies/__init__.py |   10 ++++++----
 sos/policies/redhat.py   |    8 +++++++-
 2 files changed, 13 insertions(+), 5 deletions(-)

diff --git a/sos/policies/__init__.py b/sos/policies/__init__.py
index 9fcbd55..82c1997 100644
--- a/sos/policies/__init__.py
+++ b/sos/policies/__init__.py
@@ -11,7 +11,8 @@ from os import environ
 from sos.utilities import (ImporterHelper,
                            import_module,
                            get_hash_name,
-                           shell_out)
+                           shell_out,
+                           is_executable)
 from sos.plugins import IndependentPlugin
 from sos import _sos as _
 import hashlib
@@ -85,14 +86,15 @@ class PackageManager(object):
         else:
             return None
 
-    def get_pkg_list(self):
+    def get_pkg_list(self, timeout=300):
         """
         returns a dictionary of packages in the following format:
         {'package_name': {'name': 'package_name', '
                           version': 'major.minor.version'}}
         """
-        if self.query_command:
-            pkg_list = shell_out(self.query_command).splitlines()
+        if self.query_command and timeout and is_executable("timeout"):
+            command = "timeout %ds %s" % (timeout, self.query_command)
+            pkg_list = shell_out(command).splitlines()
             for pkg in pkg_list:
                 if '|' not in pkg:
                     continue
diff --git a/sos/policies/redhat.py b/sos/policies/redhat.py
index 2219246..746c4a5 100644
--- a/sos/policies/redhat.py
+++ b/sos/policies/redhat.py
@@ -46,8 +46,14 @@ class RedHatPolicy(LinuxPolicy):
             'rpm -qa --queryformat "%{NAME}|%{VERSION}\\n"')
         self.valid_subclasses = [RedHatPlugin]
 
+        self.pkgs = self.package_manager.all_pkgs()
+
+        # If rpm query timed out after timeout duration exit
+        if not self.pkgs:
+            sys.exit(1)
+
         # handle PATH for UsrMove
-        if self.package_manager.all_pkgs()['filesystem']['version'][0] == '3':
+        if self.pkgs['filesystem']['version'][0] == '3':
             self.PATH = "/usr/sbin:/usr/bin:/root/bin"
         else:
             self.PATH = "/sbin:/bin:/usr/sbin:/usr/bin:/root/bin"