[rhelv6-list] no more MTRRs available error on server during driver load

[Tim Mooney]

All-

Running 2.6.32-279.14.1.el6.x86_64 on a RHEL 6.3 server.  The hardware is
an Intel S5520HC motherboard, BIOS is S5500.86B.01.00.0048.022720101215.

We added a Myricom 10 GB card to the server, and now when the system
boots and the myri10ge driver loads, part way through the load we get

mtrr: no more MTRRs available

The card is "working", though write-combining is disabled.

I've done a bunch of reading on memory type range registers, but nearly
all of the info out there relates to X on laptops or desktops.  This is a
server, and we're not running X on it.

Looking in /proc/mtrr , I see

~$cat /proc/mtrr 
reg00: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back
reg01: base=0x080000000 ( 2048MB), size=  256MB, count=1: write-back
reg02: base=0x100000000 ( 4096MB), size= 4096MB, count=1: write-back
reg03: base=0x200000000 ( 8192MB), size= 4096MB, count=1: write-back
reg04: base=0x300000000 (12288MB), size= 1024MB, count=1: write-back
reg05: base=0x340000000 (13312MB), size=  512MB, count=1: write-back
reg06: base=0x360000000 (13824MB), size=  256MB, count=1: write-back
reg07: base=0x0b0000000 ( 2816MB), size=   16MB, count=1: write-combining

First, how do I determine what's actually using up the MTRRs on this
server?  If there's a good guide or tool to finding that, I've definitely
missed it.

Some of the posts I found relating to MTRR and laptops and X indicated
that it was possible to re-write /proc/mtrr to make a write-combining
register available for X, but I haven't found any examples or discussion
of how to essentially do the opposite -- I don't want an MTRR reserved for
graphics, I want it available to other hardware at boot time.

Since the server hardware is relevant in this case, I'll mention that
the S5520HC board has 2 on-board Intel NICs (82575EB), only one of which
is being used, 2 dual-port QLogic FC cards (ISP2432-based), the on-board
LSI/MPT Fusion SAS card, and the Myricom NIC we added.

I have tried both settings for the BIOS option to allow drivers to use
memory above 4 GB, and neither option changes the MTRR status of the
system.

Any tips people could provide would be appreciated.

Tim
-- 
Tim Mooney                                             Tim.Mooney at ndsu.edu
Enterprise Computing & Infrastructure                  701-231-1076 (Voice)
Room 242-J6, IACC Building                             701-231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164