[Crash-utility] [PATCH 1/1] fix left bit-shift overflow in __exclude_unnecessary_pages()

[Alexander Egorenkov]

Whenever the variables compound_order or private become greater than
31, left bit-shift of 1 overflows, and nr_pages becomes zero. If nr_pages
becomes 0 and pages are being excluded at the end of the PFN loop, the
else branch of the last if statement is entered and pfn is decremented by
1 because nr_pages is 0. Finally, this causes the loop variable pfn to
be assigned the same value as before when the next loop iteration begins
which results in an infinite loop.

This issue appeared on s390 64bit architecture with a dump of 16GB RAM.

This is a simple program to demonstrate the primary issue:

void main(void)
{
        unsigned long long n;
        unsigned long m;

        m = 32;
        n = 1 << m;
        fprintf(stderr, "%llx\n", n);
        n = 1UL << m;
        fprintf(stderr, "%llx\n", n);
}

Signed-off-by: Alexander Egorenkov <egorenar at linux.ibm.com>
---
 makedumpfile.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/makedumpfile.c b/makedumpfile.c
index c063267f15bb..863840b13608 100644
--- a/makedumpfile.c
+++ b/makedumpfile.c
@@ -6210,7 +6210,7 @@ __exclude_unnecessary_pages(unsigned long mem_map,
 		if (OFFSET(page.private) != NOT_FOUND_STRUCTURE)
 			private = ULONG(pcache + OFFSET(page.private));
 
-		nr_pages = 1 << compound_order;
+		nr_pages = 1UL << compound_order;
 		pfn_counter = NULL;
 
 		/*
@@ -6227,7 +6227,7 @@ __exclude_unnecessary_pages(unsigned long mem_map,
 		else if ((info->dump_level & DL_EXCLUDE_FREE)
 		    && info->page_is_buddy
 		    && info->page_is_buddy(flags, _mapcount, private, _count)) {
-			nr_pages = 1 << private;
+			nr_pages = 1UL << private;
 			pfn_counter = &pfn_free;
 		}
 		/*
-- 
2.31.1