Bug 800181: NFSv4 on RHEL 6.2 over six times slower than 5.7

[m.roth at 5-cent.us]

For those with threaded mail tools, after it d/l pop-3 at home, I wound up
forwarding this to myself at work, then back to this account, which is
subscribed to the redhat list.

> Subject: Re: Bug 80018: NFSv4 on RHEL 6.2 over six times slower than 5.7
> Date: Tue, 10 Jul 2012 21:47:29 -0600
> From: Corey Kovacs <corey.kovacs at gmail.com>
>
> TAM = Technical Account Manager.
>
> I asked about auditing and you asked about selinux auditing. Since when is
> selinux auditing? I mean the auditd daemon. It can tax the system severely
> if not set up correctly.

/var/log/audit/audit.log
And I *said* I was at home, and couldn't look, but yes, auditd is running.
>
> I asked about your exports file, you give me the format for a generic
> exports file. If you didn't notice, i am an RHCA. I think I know what the
> general format is.

Yeah, and I'm completely unimpressed with your RHCA. You've come on as
*sure* that it's my fault for misconfiguration, not that we might have
found a bug.
>
> I asked about kerberos, you said you didn't know..  how can you NOT know
> if you are using kerberos?
>
> I asked you to give us something to work with. You said "read the damn
> bug". I did, it's so fricking vague it's ridiculous.

Vague? Really?
>
> You seem to have very little information/knowledge of your system which
> isn't too surprising at this point.
<snip>
> On Tue, Jul 10, 2012 at 8:47 PM, mark <m.roth at 5-cent.us> wrote:
<snip>
>>
>> I'll say it one more time: we found the problem on CentOS. We went to
>> our test RHEL system. Updated it. Exported a directory *from* the RHEL box
>> to itself, to /mnt/foo, and ran the test, and got the same results.
>>
>> In fact, I ran it twice today, updating the kernel in between, and with
>> 6.3, it's taking a consistent 7.5 min, instead of the 6.5 we were
>> getting with 6.2
>> <snip>
>>
>>  Now, all that said and done, here are some questions for you which
>> might help us figure what would help.
>>>
>>> 1. What options are present on the mount? (cat /proc/mounts, thinks
>>> like sync can be a problem)

/scratch/foo <same_server>(rw,sync,no_wdelay)

<snip>
>>  2. What does your /etc/exports config look like on your server node
>> (cat /etc/exports)

As if I don't know what/how to tell you that? See above.
<snip>
>>  3. You are using NFSv4, are you using Kerberos with it?

No.
>>
>> I don't believe we have kerborous set with NFS. We do use it for other
>> things.
>>
>>
>>   3.a. If so, what mode are you using for your gss/krb flag? (krb5,
>>> krb5i, krb5p)
>>> 4. What's your network speed? Are you sure? (ethtool ethX to make sure)
>>>
>> Gigabit.
<snip>
>> Do you mean selinux auditing? As I said, doing it on the local drive
>> takes seconds. Doing it from a 5.x NFS server takes about 1.5 min.
Therefore,
>> there's nothing that could affect it on the one server.
>>
>>  7. How many clients are hitting your server and how many nfsd threads
>> are you running on it?
>>
>> No other clients. This is a test system.
>>
>>
>>> This is by no means an exhaustive list of things to look at.
>>>
>>> Anyway, in order to get any real help, you cannot just shout out, "My
>>> stuff is broke, it's Red Hat's fault, no one will listen to me!"
>>>
>>> Give us something to work with.

Maybe you should actually *read* everything I wrote. Screw your I'm an
RHCE, here's a real world test:

>From what I wrote:
1. Is this a) a test system, b) a system in use (prod or dev)?
2. Did it export anything before this test?
3. Is it exporting anything to any other server, or only to the same machine?
4. Did I try unpacking to a local drive?
5. Did I find the problem unpacking to an NFS mounted directory that was
mounted FROM THE SAME MACHINE?

<snip>
          mark