Memory Performance Issue with Fedora Core 2 Kernels
James Foris
james.foris at med.ge.com
Fri Jul 30 21:46:58 UTC 2004
James Wilkinson wrote:
> James Foris wrote:
>
>>I have run into an issue with memory bandwidth using the Fedora Core
>>2 kernels and I need help. I don't know what is wrong, but something
>>killed performance of my custom driver when I ported it from RedHat 7.3
>>to Fedora Core 2. I believe I have narrowed it down to the kernel.
>>
>>My driver requires a large amount of contiguous physical memory for
>>DMA from a PCI device. I use the 'mem=YYY' command line parameter to
>>reserve the top of physical RAM for my driver. Then I allow mapping
>>via mmap() calls to user space. The user space app then uses this
>>pointer to save the data to disk.
>>
>><snip>
>>
>>I have tried this on several platforms and kernels and the results vary,
>>but the common denominator seems to be:
>>
>> Fedora kernel + 32-bit Intel = poor performance (see below)
>
>
> I suggested:
>
>>Wild guess: have you tried turning off the 4G+4G support in the kernel
>>(requires a recompile)?
>
>
> James replied
>
>>I have rebuilt now w/o several patches; 4Kstacks, netdump, and 4g/4g.
>>
>>The problem has gone away. I suspect that 4G/4G is broken.
>
>
> It sounds more "broken as designed", to be honest. It sounds as though
> your setup is doing a *lot* of context switches between user mode and
> kernel mode. The basic trade-off that 4G/4G flushes the TLB each
> context switch, so that kernel and user both get nearly 4 GB, and TLB
> flushes are expensive.
>
> See
> http://lwn.net/Articles/39283/
> and
> http://kerneltrap.org/node/view/2891
>
> May I recommend fedora-devel-list at redhat.com or the LKML? You might not
> have much done about it, but they might be able to recommend
> workarounds.
Actually, we have gotten some of the kernel group at Intel into the game, now.
It seems that the problem can be completely reproduced using the native
"/dev/mem" interface - no custom drivers needed. The intel engineer that
has looked at this believes that either the 4g/4g patch is broken, or that there
are mem driver interactions with 4g/4g that no one has taken into account
(which to me says that potentially both the mem driver and 4g/4g patch need
fixing).
Certainly, removing the 4g/4g patch fixes this problem.
We have been told that Intel will be talking/working with the Red Hat people
directly to resolve this. They seem to think that this is something significant
that should be resolved.
>
> At the very least, it would be good if real-life problems with 4G/4G are
> reported.
>
> James.
>
More information about the fedora-list
mailing list