[K12OSN] LTSP Home Server Overload

[k12ltsp]

Hi everyone,

We are having an issue with high loads on our home server. The Home Server
is the central store we use for the home folder, which is shared out by
NFS. It also runs NIS and is the central store for the accounts. We know
it's not an issue with hardware. It's a Quad-Core Xeon 2.4 Ghz, with SAS
Raid drives, and 4 Gigs of RAM. It's running with 2 Gigabit NIC's through
binding.

90% of the  school day, all is well and the load is normal. But
occasionally, the load can spike up to 10, and this causes all of the
linux servers to run slow across the district. The load can suddenly drop
down to reasonable levels at an instant and all is well again.

We did a top and discovered two things:

The WA category is high, meaning the load is caused by the Home server
queuing up I/O processes to the SAS Hard Drives. It also shows that NFS is
the program queuing up the I.O requests.

This means that somewhere, someone in the district is running an
application that creates a significant amount of read-write queries that
is passed to the Home server by NFS. However, we can't tell what
application is causing this load, because from the Home Server's
prospective, it only knows of the queries, but not the remote application
behind it.

Is there a way we can get more information from NFS to identify what these
queries are, or does anyone have any idea what we can try to do to
identify the responsible application and server? I'm wondering if it's
possible to identify which server is generating the bulk of the NFS
queries at the time of the high load.

We are on CentOS 5. Also, we use async for the NFS server.



~~~~~~~~~~~~~~~~~~~~~~
Alan Owen
Assistant to the Director of Information Services
Hermon Information Services/Hermon School Department
"Using Technology to Empower All Students to Succeed in a Changing World."
~~~~~~~~~~~~~~~~~~~~~~~