<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> <html> <tt>Magnus Damm wrote:</tt> <blockquote TYPE=CITE><tt>Hi Dave,</tt><tt></tt> <tt>Thanks for your comments!</tt><tt></tt> <tt>On Wed, 2006-10-18 at 10:49 -0400, Dave Anderson wrote:</tt> <tt>> Kazuo Moriwaka wrote:</tt> <tt>></tt> <tt>> > Hello,</tt> <tt>> ></tt> <tt>> > From: Dave Anderson <anderson@redhat.com></tt> <tt>> > Subject: Re: [Crash-utility] kdump format may be updated</tt> <tt>> > Date: Tue, 17 Oct 2006 09:01:32 -0400</tt> <tt>> ></tt> <tt>> > > As we discussed before, it would have been preferable in my</tt> <tt>> > > opinion to have the starting-point mfn value for all the domains,</tt> <tt>> > > thereby making the dumpfile usable for all domains instead of</tt> <tt>> > > just dom0. But I will be happy with at least getting this change</tt> <tt>> > > in place so that crash can be used directly on the xen dumpfile</tt> <tt>> > > for dom0 analysis without having to run it through some other</tt> <tt>> > > utility.</tt> <tt>> ></tt> <tt>> > Yes, I remember the discussion and I think it is possible to make headers.</tt> <tt>> > Now, Magnus is cleaning the patch. He and I discussed, but I'm not</tt> <tt>> > enough to convince him that dom0 information is need for crash.</tt> <tt>></tt> <tt>> Just to clarify this discussion. Magnus's patch *does* include the</tt> <tt>> dom0 cr3 information for x86, and I am quite happy with that. With that</tt> <tt>> single, simple, dom0 cr3 value, the crash utility can use the common</tt> <tt>> xen/dom0 vmcore file unmodified.</tt><tt></tt> <tt>Yes, the patch includes that information today. But say that this value</tt> <tt>wouldn't be provided as a crash note - isn't it possible for software to</tt> <tt>locate the value anyhow?</tt> <tt></tt> </blockquote> <tt>Believe me -- if you could tell me how, I would do it, and</tt> <tt>we could avoid this discussion...</tt> <blockquote TYPE=CITE><tt></tt> <tt>Or maybe the CR3 value isn't saved at all? Shouldn't it be saved instead</tt> <tt>when all the other registers are saved?</tt><tt></tt> <tt>And why do we chose to save CR3 and not CR4? I know what CR3 is used</tt> <tt>for, but I kind of dislike the ad-hoc approach of how these things are</tt> <tt>added. Also, maybe more registers are needed under Xen compared to the</tt> <tt>Linux kernel.</tt><tt></tt> <tt>I'm especially thinking about segment registers and descriptor tables.</tt></blockquote> <tt>A bit of history -- and I don't mean to "dumb-down" the discussion...</tt><tt></tt> <tt>If you just give me a dump of memory, I can make the crash utility work</tt> <tt>with it, as long as I can take a physical address and turn it into a</tt> <tt>file offset into the dumpfile.</tt><tt></tt> <tt>The first set of physical memory reads that crash does take the</tt> <tt>unity-mapped virtual addresses of key kernel data, strip off the</tt> <tt>identifier and pass the physical address to the read function of the</tt> <tt>particular memory device, i.e., be it the live memory, netdumps,</tt> <tt>diskdumps, LKCD dumps, mcore dumps, kdumps, xen dumps, and xen</tt> <tt>"xm save" files.</tt><tt></tt> <tt>The issue at hand with the xen kdump vmcore is that its ELF header</tt> <tt>describes physical memory in machine memory terms. To use that</tt> <tt>vmcore with respect to dom0 instead of the xen binary, for each</tt> <tt>required physical address derived from a dom0 kernel virtual address,</tt> <tt>the physical address is a pseudo-physical address with no correlation</tt> <tt>with the physical (machine) memory descriptors in the ELF header.</tt><tt></tt> <tt>So, given that the dom0 pseudo-physical address needs to be</tt> <tt>translated into a machine address, I need to be able to find my way</tt> <tt>to the phys_to_machine_mapping array. From that point on, it's</tt> <tt>becomes a matter of searching the array for the desired pseudo-physical</tt> <tt>address, getting the associated machine address, and then using</tt> <tt>the PT_LOAD segments of the ELF header to find the memory.</tt><tt></tt> <tt>To find the phys_to_machine_mapping array, there are two keys</tt> <tt>to Pandora's box:</tt><tt></tt> <tt>(1) the dom0 cr3 value -- which in a writable page table kernel,</tt> <tt> will contain an mfn value. With that starting point, a page</tt> <tt> table walk can be initiated for the "phys_to_machine_mapping"</tt> <tt> virtual address.</tt><tt></tt> <tt>(2) alternatively, given the dom0 pfn_to_mfn_frame_list_list mfn,</tt> <tt> I also have a starting point in order to reconstruct the</tt> <tt> phys_to_machine_mapping array.</tt><tt></tt> <tt>Either one works. I preferred #2 because it would presumably work</tt> <tt>for both writable and shadow page table kernels. But, I've never</tt> <tt>done any work with shadow page table kernels (Red Hat is going with</tt> <tt>writable...), so I don't know what the ramifications are for those</tt> <tt>kernels.</tt><tt></tt> <tt>In fact, my original suggestion was for an ELF note with an array</tt> <tt>of cr3's or pfn_to_mfn_frame_list_list mfn's, i.e., for dom0 and</tt> <tt>all the other domUs running on the system. That would be an awesome</tt> <tt>capability -- a single vmcore that could be used against the xen</tt> <tt>binary using gdb, or with the crash utility for analyszing dom0</tt> <tt>or any of the other domU sessions.</tt><tt></tt> <tt>But, there apparently was an issue with the idea of having</tt> <tt>an ELF note with an undetermined size.</tt><tt></tt> <tt>Anyway, like I mentioned to Kazuo, I'd be happy with just the</tt> <tt>dom0 "key"...</tt><tt></tt> <tt>Also, in the ELF NT_PRSTATUS note, the cr3 value is not stored</tt> <tt>since it consists of a processor-specific user_regs_struct.</tt> <tt>So if you were to append it there, I could use that instead.</tt> <tt>Your initial patch seems to have put it both there and</tt> <tt>in the new NT_XEN_DOM0_CR3 note:</tt><tt></tt> <tt>+/* The cr3 for dom0 on each of its vcpus</tt> <tt>+ * It is added as ELF_Prstatus prstatus.pr_reg[ELF_NGREG-1)], where</tt> <tt>+ * prstatus is the data of the elf note, and ELF_NGREG was extended</tt> <tt>+ * by one to allow extra space.</tt> <tt>+ * This code runs after all cpus except the crashing one have</tt> <tt>+ * been shutdown so as to avoid having to hold domlist_lock,</tt> <tt>+ * as locking after a crash is playing with fire */</tt> <tt></tt> <tt></tt> <tt>+ buf = append_elf_note(buf, "Xen Domanin-0 CR3",</tt> <tt>+ NT_XEN_DOM0_CR3, &cr3, 4);</tt><tt></tt> <tt>But, again, only for x86.</tt> <tt></tt> <blockquote TYPE=CITE><tt></tt> <tt></tt> <tt>> What I don't understand is whether the same thing is going to be done</tt> <tt>> for x86_64?</tt> <tt>></tt> <tt>> NT_XEN_DOM0_CR3 is #define'd in xen/include/xen/elfcore.h in</tt> <tt>> this patch:</tt> <tt>></tt> <tt>> [Xen-devel] [PATCH 02/04] Kexec / Kdump: Code shared between x86_32 and x86_64</tt> <tt>></tt> <tt>> NT_XEN_DOM0_CR3 is used by the find_dom0_cr3() function in</tt> <tt>> xen/arch/x86/crash.c, in this patch:</tt> <tt>></tt> <tt>> [Xen-devel] [PATCH 03/04] Kexec / Kdump: x86_32 specific code</tt> <tt>></tt> <tt>> But there is no analogous x86_64 usage in this patch:</tt> <tt>></tt> <tt>> [Xen-devel] [PATCH 04/04] Kexec / Kdump: x86_64 specific code</tt> <tt>></tt> <tt>> Is NT_XEN_DOM0_CR3 not being used by x86_64 by mistake,</tt> <tt>> or on purpose? Or perhaps you're saying that it's going to be</tt> <tt>> pulled out completely?</tt><tt></tt> <tt>Yes and maybe. It was a mistake to put it into that patch, but I think</tt> <tt>the patch ends up in the common code anyway. Next version will be</tt> <tt>improved.</tt><tt></tt> <tt>Ripping it out? Well, I'm tempted. But nothing is decided. Feel free to</tt> <tt>argue. =)</tt><tt></tt> <tt>The main reason behind it is that we need to make kexec-tools xen-aware.</tt> <tt>And this awareness may lead to that we don't need any crash notes at all</tt> <tt>in the Xen case. Mostly because kexec-tools only knows about dom0, but</tt> <tt>the crash dump is about the entire system.</tt><tt></tt> <tt>Today we have some code that stores the elf notes in the hypervisor in</tt> <tt>the same format as the kernel, and to pass these notes we need to hook</tt> <tt>in hypercalls in the kernel so that the user-space tool can find the</tt> <tt>address of the crash notes.</tt><tt></tt> <tt>I think it would make sense to let dom0 save it's registers within dom0</tt> <tt>using the good old crash note format, but to use a simpler format for</tt> <tt>registers for the hypervisor. And the let the tools locate the registers</tt> <tt>by looking up global symbols.</tt><tt></tt> <tt>> > I know you don't want to treat xen binary file with crash, but I'm</tt> <tt>> > not clear why. Please discuss with him directly to make up xen kdump</tt> <tt>> > file formats. The patch will be merged into xen-3.0.4.</tt> <tt>> > I hope we can find solution before merge.</tt> <tt>> ></tt> <tt>></tt> <tt>> The crash utility is wholly based upon the internal structure</tt> <tt>> of the Linux kernel.</tt><tt></tt> <tt>So why can't you just require that Xen dumps needs to be cut out with</tt> <tt>dom0 cut?</tt> <tt></tt> </blockquote> <tt>Well, to answer your question with a question:</tt><tt></tt> <tt> Why should it be required if it could be so easily avoided?</tt><tt></tt> <tt>As Henry David Thoreau said, "Simplicity, simplicity, simplicity..."</tt><tt></tt> <tt>My two cents,</tt> <tt> Dave</tt> <tt></tt> <tt></tt> </html>