[Crash-utility] x86_64 limit of 454 cpu's?

[Takao Indoh]

(4/15/2011 09:04), Dave Anderson wrote:
>
>
> ----- Original Message -----
>> Hi Dave, and company,
>>
>> I get this error trying to open a dump of a large system:
>>
>> crash: compressed kdump: invalid nr_cpus value: 640
>> crash: vmcore: not a supported file format
>>
>> The message is from diskdump.c:
>> if (sizeof(*header) + sizeof(void *) * header->nr_cpus>  block_size ||
>>      header->nr_cpus<= 0) {
>>          error(INFO, "%s: invalid nr_cpus value: %d\n",
>>
>> block_size is the page size of 4096
>> struct disk_dump_header looks like 464 bytes
>> void * is 8
>> So it looks like 454 is the maximum number of cpus.
>> 464 + 454*8 ->  4096
>>
>> Is this intentional?
>> It looks like a restriction that no one ever complained about. But there
>> are systems (Altix UV) with 2048 cpu's.
>>
>> Is there an easy fix?
>>
>> -Cliff
>
> To be honest, I don't know, I didn't design or write that code.

Yes, this is intentional for RHEL4/diskdump. In the RHEL4 kernel,
disk_dump_header is defined as follows.

struct disk_dump_header {
     char                    signature[8];   /* = "DISKDUMP" */
(snip)
     int                     nr_cpus;        /* Number of CPUs */
     struct task_struct      *tasks[NR_CPUS];
};

And maximum logical CPUs of RHEL4 are 32(x86) or 64(x86_64) so
this does not cause any problem.

On the other hands, as you and Dave said, this causes limitation problem
in RHEL5/RHEL6 kernel. But as far as I know, makedumpfile does not use
this "tasks" member, so we can skip here.

if (is_diskdump?)
     if (sizeof(*header) + sizeof(void *) * header->nr_cpus>  block_size ||
         header->nr_cpus<= 0) {
         error(INFO, "%s: invalid nr_cpus value: %d\n",
         goto err;
     }

Something like this. Dave, Ohmichi-san, what do you think?

Thanks,
Takao Indoh

>
> And you're right, although dumpfiles with that many cpus are highly
> unusual, but looking at the code, it certainly does appear that the
> disk_dump_header plus the task pointers for each cpu must fit in a
> "block_size", or page size, and that the sub_header is the first item
> following the contents of the first page:
>
> ---
>   ^     disk_dump_header
>   |     task on cpu 0
> page     ...
>   |     task on cpu x-1
>   V     task on cpu x
> ---
>         sub_header
>         bitmap
>         remaining stuff...
>
> Since your dump is presumably a compressed kdump, I'm wondering
> what the makedumpfile code did in your case?  Did it round up the
> location of the sub_header to a page boundary?
>
> I've cc'd this to Ken'ichi Ohmichi (makedumpfile), and to Takao Indoh
> (original compressed diskdump) for their input.
>
> Thanks,
>    Dave
>
>