Problems with SSH and pam_listfile

[John Gorkos]

I am having good success using pam_listfile with my LDAP directory to 
allow/disallow users in specific posixGroups access to servers using SSH.  My 
"auth" section of /etc/pam.d/system-auth on my RHEL 5.2 system looks like 
this:

auth        required      pam_listfile.so onerr=fail item=group sense=allow 
file=/etc/login.group.allowed
auth        required      pam_env.so
auth        sufficient    pam_unix.so nullok try_first_pass
auth        sufficient    pam_ldap.so use_first_pass
auth        requisite     pam_succeed_if.so uid >= 500 quiet
auth        required      pam_deny.so

If a user's UID is in a memberUID field of an objectClass=posixGroup in LDAP 
(ou=Groups,o=XXXX), he can log in via SSH.  If he's not in one of the groups 
enumerated in /etc/login.group.allowed, he's denied... UNLESS he has a public 
key in his ~/.ssh/authorized_keys file.  If that is the case, he's allowed to 
log in with no problems, even if he's not in an allowed group.
Sudo (which is also controlled by LDAP) works correctly, i.e. if a user is not 
in an allowed group, but logs into the system anyway due to an authorized_keys 
entry, he will not be allowed to sudo execute anything.

The problem is that I have users with keys in place already.  We have 
automated processes that use these keys, so I can't be draconian and disallow 
key usage.  On the other hand, I have a fairly fluid set of people moving into 
and out of groups, so I need to be able to control access to these machines 
regardless of whether there is a key in authorized_keys.

Has anyone seen this before, or is there a way that I can re-order my pam 
config to force SSH to respect the group membership requirements?

Thanks.
John Gorkos