mtrr funnies

[D. Hugh Redelmeier]

| From: Wolfgang S. Rupprecht <wolfgang.rupprecht+gnus200810 at gmail.com>

| I've got a motherboard with a fairly new chipset (Asus M3A78T
| w. AMD/ATI 790GX) and I'm seeing a funny MTRR setting.  I have 8GB
| memory and am running a 64-bit kernel, but I'm only seeing ~4GB
| mentioned in the MTRR's.  Is this a bug?  Do I need to add the upper
| 4+GB by hand?
| 
|     $ cat /proc/mtrr
|     reg00: base=0x00000000 (   0MB), size=2048MB: write-back, count=1
|     reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
|     reg02: base=0xc0000000 (3072MB), size= 256MB: write-back, count=1
|     reg03: base=0xd0000000 (3328MB), size= 256MB: write-combining, count=1
| 
| It does seem like the kernel sees the whole memory, at least at times,
| but it isn't clear if that high memory is being cached the right way.
| 
|     Oct  7 10:54:30 poblano kernel: Memory: 7678940k/8650752k available (2699k kernel code, 184672k reserved, 1485k data, 980k init)

I agree with your concern.

These days, many BIOSes set up overlapping MTRRs on systems with 4GiB
or more of RAM.  Until the mtrr-cleanup code is in the kernel, this
causes problems.

In your case, you don't seem to have an MTRR covering the top half of
memory.  (In a later post you say that 7.6G of memory is made
available.)

Normally the BIOS would map all of memory linearly EXCEPT to leave a
hole in the top of the 4G range for things on the PCI bus.  The hole
is 1GiB or less, I think.  In your case, it appears that the hole is
3/4 of 1GiB.

The RAM that would otherwise have addresses in the hole is usually
remapped to be above all the other memory.  In your case, you would
have 256M * 3 bytes at and above 8GiB.  Many BIOSes apparently have an
option to not remap -- then that RAM gets thrown away (i.e. it cannot
be addressed so it cannot be use).

One reason to turn off remapping might be that there are not enough
MTRR registers to describe all the regions.  For example, in your
case, you need one more MTRR to cover 4GiB-8GiB and two more to
cover the remapped 256M * 3.  That uses up 7 MTRRs altogether.
Luckilly, there are 8 in the hardware.

Anyway, none of this is the problem you are presenting.

The MTRRs you listed don't cover anything above 4GiB.  The default
memory type (when not covered by an MTRR) is uncachable.  So if Linux
is using your high memory, it must be VERY slow.  Your log message
suggests that high memory is being used.  So your system ought to be
extremely slow.  If it isn't, something else must be going on, and I
don't know what that might be.  Perhaps PAT is saving you.

Are you running in x86_64 mode?  Or with a PAE kernel?  If neither, I
guess that high memory might not be being used.  That would be a waste
of memory.  But I don't think that a non-PAE i386 kernel would report
more than 4GiB of memory on your machine, and you show 7.7 being
reported.

This is surely a BIOS bug.  Or it is a BIOS option that is set in a
way that you don't want.

If you do a dmesg command, you will see kernel messages that report
what the kernel is being told about memory.  I don't understand them
all, but the most obvious are lines that begin with " BIOS-e820: ".
Is PAT mentioned?