<div dir="ltr"><div><div>Thanks, that worked.. users now able to get the password changed with any issues... </div>Will do few more testing on this but at this point looks like that was the issue </div>~Rakesh </div><div class="gmail_extra"> <div class="gmail_quote">On Tue, Jan 13, 2015 at 1:52 PM, Sumit Bose <<a href="mailto:sbose@redhat.com" target="_blank">sbose@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On Tue, Jan 13, 2015 at 12:48:18PM +0530, Rakesh Rajasekharan wrote: > >>>Does it work for the same user from the client if you reset password on > the server, authenticate from the client and then force reset again on the > server? > When I force reset a user, he stil faces the same error "token > manipulation" when tries to login to a client. However, when he tries > getting into the server, he now gets prompted for the password change and > is successfully able to get through. > > So, at this point we have a workaround though something seems not right at > the clients. > >>>Can you add a new client and see whether it works there? > > >>Have you tried re-installing the client? > Yes, I did try reinstalling but that did not help > > > >>>Sorry, I meant the full krb5_child.log ... > > This is how I get the logs in krb5_child. > > when a user tries to authenticate with the random password that I generated, > > WARNING: Your password has expired. > You must change your password now and login again! > Changing password for user hq-testuser. > Current Password: > New password: > Retype new password: > passwd: Authentication token manipulation erro > > And on the krb5_child.log, these are the entries > > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] [unpack_buffer] > (0x0100): ccname: [FILE:/tmp/krb5cc_710600001_XXXXXX] keytab: > [/etc/krb5.keytab] > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_RENEWABLE_LIFETIME] > from environment. > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_LIFETIME] from > environment. > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] > [set_canonicalize_option] (0x0100): SSSD_KRB5_CANONICALIZE is set to [true] > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] [k5c_setup_fast] > (0x0100): SSSD_KRB5_FAST_PRINCIPAL is set to [host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>] > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] [match_principal] > (0x1000): Principal matched to the sample (host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>). > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] [check_fast_ccache] > (0x0200): FAST TGT is still valid. > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] [main] (0x0400): > Will perform password change > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] [changepw_child] > (0x1000): Password change operation > (Tue Jan 13 06:47:39 2015) [[sssd[krb5_child[18004]]]] [changepw_child] > (0x0400): Attempting kinit for realm [<a href="http://TEST.COM" target="_blank">TEST.COM</a>] > > > This does not go beyond this. however, when i attempt another login , the > logs start moving from this point( the time stamp start from 6:54 AM) > > WARNING: Your password has expired. > You must change your password now and login again! > Changing password for user hq-testuser. > Current Password: > New password: > Retype new password: > passwd: Authentication token manipulation erro > > now the krb5_child.log adds following lines > > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [main] (0x0400): > krb5_child started. > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [unpack_buffer] > (0x1000): total buffer size: [134]TEST > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [unpack_buffer] > (0x0100): cmd [241] uid [710600001] gid [710600001] validate [true] > enterprise principal [false] offline [false] UPN [<a href="mailto:hq-testuser@TEST.COM">hq-testuser@TEST.COM</a>] > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [unpack_buffer] > (0x0100): ccname: [FILE:/tmp/krb5cc_710600001_XXXXXX] keytab: > [/etc/krb5.keytab] > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_RENEWABLE_LIFETIME] > from environment. > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_LIFETIME] from > environment. > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] > [set_canonicalize_option] (0x0100): SSSD_KRB5_CANONICALIZE is set to [true] > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [k5c_setup_fast] > (0x0100): SSSD_KRB5_FAST_PRINCIPAL is set to [host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>] > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [match_principal] > (0x1000): Principal matched to the sample (host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>). > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [check_fast_ccache] > (0x0200): FAST TGT is still valid. > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [main] (0x0400): > Will perform online auth > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [tgt_req_child] > (0x1000): Attempting to get a TGT > (Tue Jan 13 06:54:52 2015) [[sssd[krb5_child[23514]]]] [get_and_save_tgt] > (0x0400): Attempting kinit for realm [<a href="http://TEST.COM" target="_blank">TEST.COM</a>] > (Tue Jan 13 06:54:53 2015) [[sssd[krb5_child[23514]]]] [get_and_save_tgt] > (0x0020): 981: [-1765328361][Password has expired] > (Tue Jan 13 06:54:53 2015) [[sssd[krb5_child[23514]]]] [tgt_req_child] > (0x1000): Password was expired > (Tue Jan 13 06:54:56 2015) [[sssd[krb5_child[23514]]]] [k5c_send_data] > (0x0200): Received error code 1432158213 > (Tue Jan 13 06:54:56 2015) [[sssd[krb5_child[23514]]]] [main] (0x0400): > krb5_child completed successfully > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [main] (0x0400): > krb5_child started. > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [unpack_buffer] > (0x1000): total buffer size: [134] > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [unpack_buffer] > (0x0100): cmd [247] uid [710600001] gid [710600001] validate [true] > enterprise principal [false] offline [false] UPN [<a href="mailto:hq-testuser@TEST.COM">hq-testuser@TEST.COM</a>] > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [unpack_buffer] > (0x0100): ccname: [FILE:/tmp/krb5cc_710600001_XXXXXX] keytab: > [/etc/krb5.keytab] > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_RENEWABLE_LIFETIME] > from environment. > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_LIFETIME] from > environment. > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] > [set_canonicalize_option] (0x0100): SSSD_KRB5_CANONICALIZE is set to [true] > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [k5c_setup_fast] > (0x0100): SSSD_KRB5_FAST_PRINCIPAL is set to [host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>] > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [match_principal] > (0x1000): Principal matched to the sample (host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>). > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [check_fast_ccache] > (0x0200): FAST TGT is still valid. > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [main] (0x0400): > Will perform password change checks > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [changepw_child] > (0x1000): Password change operation > (Tue Jan 13 06:55:00 2015) [[sssd[krb5_child[23595]]]] [changepw_child] > (0x0400): Attempting kinit for realm [<a href="http://TEST.COM" target="_blank">TEST.COM</a>] > (Tue Jan 13 06:55:03 2015) [[sssd[krb5_child[23595]]]] [changepw_child] > (0x1000): Initial authentication for change password operation successful. > (Tue Jan 13 06:55:03 2015) [[sssd[krb5_child[23595]]]] [k5c_send_data] > (0x0200): Received error code 0 > (Tue Jan 13 06:55:03 2015) [[sssd[krb5_child[23595]]]] [main] (0x0400): > krb5_child completed successfully > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [main] (0x0400): > krb5_child started. > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [unpack_buffer] > (0x1000): total buffer size: [153] > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [unpack_buffer] > (0x0100): cmd [246] uid [710600001] gid [710600001] validate [true] > enterprise principal [false] offline [false] UPN [<a href="mailto:hq-testuser@TEST.COM">hq-testuser@TEST.COM</a>] > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [unpack_buffer] > (0x0100): ccname: [FILE:/tmp/krb5cc_710600001_XXXXXX] keytab: > [/etc/krb5.keytab] > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_RENEWABLE_LIFETIME] > from environment. > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] > [set_lifetime_options] (0x0100): Cannot read [SSSD_KRB5_LIFETIME] from > environment. > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] > [set_canonicalize_option] (0x0100): SSSD_KRB5_CANONICALIZE is set to [true] > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [k5c_setup_fast] > (0x0100): SSSD_KRB5_FAST_PRINCIPAL is set to [host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>] > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [match_principal] > (0x1000): Principal matched to the sample (host/ > <a href="mailto:qa-dummy-int.test.com@TEST.COM">qa-dummy-int.test.com@TEST.COM</a>). > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [check_fast_ccache] > (0x0200): FAST TGT is still valid. > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [main] (0x0400): > Will perform password change > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [changepw_child] > (0x1000): Password change operation > (Tue Jan 13 06:55:13 2015) [[sssd[krb5_child[24241]]]] [changepw_child] > (0x0400): Attempting kinit for realm [<a href="http://TEST.COM" target="_blank">TEST.COM</a>] > > and again the last line is attempting kinit for realm </div></div>according to some earlier log entries your Kerberos server needs some time to respond. Maybe you are hit by the authentication timeout SSSD uses to not wait indefinitely long for a response. The default is 6s. You can increase it by setting krb5_auth_timeout option in the [domain/...] section in sssd.conf to a higher value. See man sssd-krb5 for more details. HTH bye, Sumit <div class="HOEnZb"><div class="h5"> > > Thanks, > Rakesh > > > On Tue, Jan 13, 2015 at 1:05 AM, Dmitri Pal <<a href="mailto:dpal@redhat.com">dpal@redhat.com</a>> wrote: > > > On 01/12/2015 12:55 PM, Rakesh Rajasekharan wrote: > > > > This is the full log, > > > > Jan 12 17:45:15 10-5-68-5 sshd[29753]: pam_sss(sshd:account): User info > > message: Password expired. Change your password now. > > Jan 12 17:45:15 10-5-68-5 sshd[29753]: Accepted password for hq-testuser > > from 10.5.68.184 port 54048 ssh2 > > Jan 12 17:45:16 10-5-68-5 sshd[29753]: pam_unix(sshd:session): session > > opened for user hq-testuser by (uid=0) > > Jan 12 17:45:16 10-5-68-5 passwd: pam_unix(passwd:chauthtok): user > > "hq-testuser" does not exist in /etc/passwd > > Jan 12 17:45:35 10-5-68-5 passwd: pam_unix(passwd:chauthtok): user > > "hq-testuser" does not exist in /etc/passwd > > Jan 12 17:45:41 10-5-68-5 passwd: pam_sss(passwd:chauthtok): Password > > change failed for user hq-testuser: 22 (Authentication token lock busy) > > Jan 12 17:45:43 10-5-68-5 sshd[30329]: Received disconnect from > > <a href="http://10.5.68.184" target="_blank">10.5.68.184</a>: 11: disconnected by user > > Jan 12 17:45:43 10-5-68-5 sshd[29753]: pam_unix(sshd:session): session > > closed for user hq-testuser > > > > > > >> Does it happen for all users or only users that you migrated? > > Yes it happens for all, I created a new user ( hq-testuser) is a fresh > > one that I created. > > > > I found a workaround for this , users are able to successfully change > > the password by connecting to the IPA master server. > > So, its only the ipa clients that have the issue. > > > > > > Does it work for the same user from the client if you reset password on > > the server, authenticate from the client and then force reset again on the > > server? > > > > Can you add a new client and see whether it works there? > > Have you tried re-installing the client? > > > > > > > > Thanks, > > Rakesh > > > > On Mon, Jan 12, 2015 at 10:57 PM, Jakub Hrozek <<a href="mailto:jhrozek@redhat.com">jhrozek@redhat.com</a>> wrote: > > > >> On Mon, Jan 12, 2015 at 04:01:32PM +0530, Rakesh Rajasekharan wrote: > >> > under /var/log/secure.. have this error > >> > passwd: pam_sss(passwd:chauthtok): Password change failed for user > >> > hq-testuser: 22 (Authentication token lock busy) > >> > >> It looks like the log was trucated, can you post more context? > >> > >> Authentication token lock busy usually means the kadmin servers were > >> offline.. > >> > >> -- > >> Manage your subscription for the Freeipa-users mailing list: > >> <a href="https://www.redhat.com/mailman/listinfo/freeipa-users" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a> > >> Go To <a href="http://freeipa.org" target="_blank">http://freeipa.org</a> for more info on the project > >> > > > > > > > > > > > > -- > > Thank you, > > Dmitri Pal > > > > Sr. Engineering Manager IdM portfolio > > Red Hat, Inc. > > > > > > -- > > Manage your subscription for the Freeipa-users mailing list: > > <a href="https://www.redhat.com/mailman/listinfo/freeipa-users" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a> > > Go To <a href="http://freeipa.org" target="_blank">http://freeipa.org</a> for more info on the project > > > -- > Manage your subscription for the Freeipa-users mailing list: > <a href="https://www.redhat.com/mailman/listinfo/freeipa-users" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a> > Go To <a href="http://freeipa.org" target="_blank">http://freeipa.org</a> for more info on the project </div></div></blockquote></div> </div>