From oliver.graute at gmail.com Mon Oct 24 07:19:34 2016 From: oliver.graute at gmail.com (Oliver Graute) Date: Mon, 24 Oct 2016 09:19:34 +0200 Subject: [Mod_nss-list] Unexpected Session tickets send by Server Message-ID: <20161024071934.GA3981@graute-opti> Hello List, in our setup we use apache 2.4.16 with mod-nss 1.013 and enabled Session Tickets (RFC 5077). The Session Ticket Feature worked with Chrome and Firefox for a while now. The Certificate Database where stored in the filesystem. Now we moved nearly the same Certificates in a slot of a High Security Module. Since then the Firefox Browser is often complaining about unexpected new Session Tickets. The Error is: SSL_ERROR_RX_UNEXPECTED_NEW_SESSION_TICKET I analyzed it with wireshark and I saw that the Server is indeed sending periodical new Session Tickets towards the client what he did not before. Sometimes the Firefox is complaining some times not. The Apache Logs: [Fri Oct 21 14:17:44.083822 2016] [:info] [pid 839] (104)Connection reset by peer: SSL library error -12216 writing data [Fri Oct 21 14:17:44.083954 2016] [:info] [pid 839] SSL Library Error: -12216 Attempt to write encrypted data to underlying socket failed [Fri Oct 21 14:17:44.084430 2016] [:debug] [pid 839] nss_engine_io.c(667): SSL connection destroyed without being closed [Fri Oct 21 14:17:52.972846 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) [Fri Oct 21 14:17:53.878877 2016] [:info] [pid 839] (70014)End of file found: SSL input filter read failed. The Chrome Browser behavior is little bit different. When the Server is sending the (second) Session Ticket its complaining with ERR_SSL_VERSION_OR_CIPHER_MISMATCH. And no further TLS Connection is possible anymore. [Fri Oct 21 14:20:57.300227 2016] [:info] [pid 839] SSL input filter read failed. [Fri Oct 21 14:20:57.300391 2016] [:error] [pid 839] SSL Library Error: -12229 SSL peer was not expecting a handshake message it received [Fri Oct 21 14:20:57.301942 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) [Fri Oct 21 14:20:57.302471 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) [Fri Oct 21 14:20:57.304934 2016] [:info] [pid 839] SSL input filter read failed. [Fri Oct 21 14:20:57.305066 2016] [:error] [pid 839] SSL Library Error: -12279 Client is using unsupported SSL version [Fri Oct 21 14:20:57.305633 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) [Fri Oct 21 14:20:57.307819 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) [Fri Oct 21 14:20:57.310564 2016] [:info] [pid 839] SSL input filter read failed. [Fri Oct 21 14:20:57.310700 2016] [:error] [pid 839] SSL Library Error: -12279 Client is using unsupported SSL version [Fri Oct 21 14:20:57.311263 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) Some ideas how to investigate this issue further? We use TLS 1.2 and Cypher Suite ecdhe_ecdsa_aes_128_gcm_sha_256,+ecdhe_ecdsa_aes_128_sha_256 Best regards, Oliver From rcritten at redhat.com Tue Oct 25 10:36:01 2016 From: rcritten at redhat.com (Rob Crittenden) Date: Tue, 25 Oct 2016 12:36:01 +0200 Subject: [Mod_nss-list] Unexpected Session tickets send by Server In-Reply-To: <20161024071934.GA3981@graute-opti> References: <20161024071934.GA3981@graute-opti> Message-ID: <580F3591.4050308@redhat.com> Oliver Graute wrote: > Hello List, > > in our setup we use apache 2.4.16 with mod-nss 1.013 and enabled Session > Tickets (RFC 5077). The Session Ticket Feature worked with Chrome and > Firefox for a while now. The Certificate Database where stored in the > filesystem. > > Now we moved nearly the same Certificates in a slot of a High Security Module. > > Since then the Firefox Browser is often complaining about unexpected new > Session Tickets. > > The Error is: > > SSL_ERROR_RX_UNEXPECTED_NEW_SESSION_TICKET > > I analyzed it with wireshark and I saw that the Server is indeed sending > periodical new Session Tickets towards the client what he did not before. > > Sometimes the Firefox is complaining some times not. > > The Apache Logs: > > [Fri Oct 21 14:17:44.083822 2016] [:info] [pid 839] (104)Connection reset by peer: SSL library error -12216 writing data > [Fri Oct 21 14:17:44.083954 2016] [:info] [pid 839] SSL Library Error: -12216 Attempt to write encrypted data to underlying socket failed > [Fri Oct 21 14:17:44.084430 2016] [:debug] [pid 839] nss_engine_io.c(667): SSL connection destroyed without being closed > [Fri Oct 21 14:17:52.972846 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > [Fri Oct 21 14:17:53.878877 2016] [:info] [pid 839] (70014)End of file found: SSL input filter read failed. > > > The Chrome Browser behavior is little bit different. When the Server is > sending the (second) Session Ticket its complaining with > ERR_SSL_VERSION_OR_CIPHER_MISMATCH. And no further TLS Connection is > possible anymore. > > [Fri Oct 21 14:20:57.300227 2016] [:info] [pid 839] SSL input filter read failed. > [Fri Oct 21 14:20:57.300391 2016] [:error] [pid 839] SSL Library Error: -12229 SSL peer was not expecting a handshake message it received > [Fri Oct 21 14:20:57.301942 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > [Fri Oct 21 14:20:57.302471 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > [Fri Oct 21 14:20:57.304934 2016] [:info] [pid 839] SSL input filter read failed. > [Fri Oct 21 14:20:57.305066 2016] [:error] [pid 839] SSL Library Error: -12279 Client is using unsupported SSL version > [Fri Oct 21 14:20:57.305633 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > [Fri Oct 21 14:20:57.307819 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > [Fri Oct 21 14:20:57.310564 2016] [:info] [pid 839] SSL input filter read failed. > [Fri Oct 21 14:20:57.310700 2016] [:error] [pid 839] SSL Library Error: -12279 Client is using unsupported SSL version > [Fri Oct 21 14:20:57.311263 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > > > Some ideas how to investigate this issue further? > > We use TLS 1.2 and Cypher Suite ecdhe_ecdsa_aes_128_gcm_sha_256,+ecdhe_ecdsa_aes_128_sha_256 I'm guessing it's some issue in NSS but why in the world it would be behaving differently based on where the token lives is quite strange. What version of NSS do you have installed? Did anything else change other than switching from the soft token to the HSM? I assume that restarting Chrome will cause a fresh handshake so things work again? (obviously not a great work-around, just gathering info). rob From oliver.graute at gmail.com Tue Oct 25 15:05:13 2016 From: oliver.graute at gmail.com (Oliver Graute) Date: Tue, 25 Oct 2016 17:05:13 +0200 Subject: [Mod_nss-list] Unexpected Session tickets send by Server In-Reply-To: <580F3591.4050308@redhat.com> References: <20161024071934.GA3981@graute-opti> <580F3591.4050308@redhat.com> Message-ID: <20161025150513.GB3981@graute-opti> On 25/10/16, Rob Crittenden wrote: > Oliver Graute wrote: > >Hello List, > > > >in our setup we use apache 2.4.16 with mod-nss 1.013 and enabled Session > >Tickets (RFC 5077). The Session Ticket Feature worked with Chrome and > >Firefox for a while now. The Certificate Database where stored in the > >filesystem. > > > >Now we moved nearly the same Certificates in a slot of a High Security Module. > > > >Since then the Firefox Browser is often complaining about unexpected new > >Session Tickets. > > > >The Error is: > > > >SSL_ERROR_RX_UNEXPECTED_NEW_SESSION_TICKET > > > >I analyzed it with wireshark and I saw that the Server is indeed sending > >periodical new Session Tickets towards the client what he did not before. > > > >Sometimes the Firefox is complaining some times not. > > > >The Apache Logs: > > > >[Fri Oct 21 14:17:44.083822 2016] [:info] [pid 839] (104)Connection reset by peer: SSL library error -12216 writing data > >[Fri Oct 21 14:17:44.083954 2016] [:info] [pid 839] SSL Library Error: -12216 Attempt to write encrypted data to underlying socket failed > >[Fri Oct 21 14:17:44.084430 2016] [:debug] [pid 839] nss_engine_io.c(667): SSL connection destroyed without being closed > >[Fri Oct 21 14:17:52.972846 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > >[Fri Oct 21 14:17:53.878877 2016] [:info] [pid 839] (70014)End of file found: SSL input filter read failed. > > > > > >The Chrome Browser behavior is little bit different. When the Server is > >sending the (second) Session Ticket its complaining with > >ERR_SSL_VERSION_OR_CIPHER_MISMATCH. And no further TLS Connection is > >possible anymore. > > > >[Fri Oct 21 14:20:57.300227 2016] [:info] [pid 839] SSL input filter read failed. > >[Fri Oct 21 14:20:57.300391 2016] [:error] [pid 839] SSL Library Error: -12229 SSL peer was not expecting a handshake message it received > >[Fri Oct 21 14:20:57.301942 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > >[Fri Oct 21 14:20:57.302471 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > >[Fri Oct 21 14:20:57.304934 2016] [:info] [pid 839] SSL input filter read failed. > >[Fri Oct 21 14:20:57.305066 2016] [:error] [pid 839] SSL Library Error: -12279 Client is using unsupported SSL version > >[Fri Oct 21 14:20:57.305633 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > >[Fri Oct 21 14:20:57.307819 2016] [:info] [pid 839] Connection to child 0 established (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > >[Fri Oct 21 14:20:57.310564 2016] [:info] [pid 839] SSL input filter read failed. > >[Fri Oct 21 14:20:57.310700 2016] [:error] [pid 839] SSL Library Error: -12279 Client is using unsupported SSL version > >[Fri Oct 21 14:20:57.311263 2016] [:info] [pid 839] Connection to child 0 closed (server xxxx.xxx.xxx.xx:443, client 192.168.1.99) > > > > > >Some ideas how to investigate this issue further? > > > >We use TLS 1.2 and Cypher Suite ecdhe_ecdsa_aes_128_gcm_sha_256,+ecdhe_ecdsa_aes_128_sha_256 > > I'm guessing it's some issue in NSS but why in the world it would be > behaving differently based on where the token lives is quite > strange. > I suppose that its related to NSS to. I investigate it a bit further and found the following Code snippet in NSS 3.21 nss/lib/ssl/ssl3con.c /* Send a NewSessionTicket message if the client sent us * either an empty session ticket, or one that did not verify. * (Note that if either of these conditions was met, then the * server has sent a SessionTicket extension in the * ServerHello message.) */ if (isServer && !ss->ssl3.hs.isResuming && ssl3_ExtensionNegotiated(ss, ssl_session_ticket_xtn) && ssl3_KEAAllowsSessionTicket(ss->ssl3.hs.suite_def->key_exchange_alg)) { /* RFC 5077 Section 3.3: "In the case of a full handshake, the * server MUST verify the client's Finished message before sending * the ticket." Presumably, this also means that the client's * certificate, if any, must be verified beforehand too. */ rv = ssl3_SendNewSessionTicket(ss); if (rv != SECSuccess) { goto xmit_loser; } } After checking the wireshark traces my current assumptions is that nss can't verify the Session Ticket from the Browser and send therefore every time a new one. For the verification of the Session Ticket somehow the HSM is involved. Probably this isn't working. > What version of NSS do you have installed? Did anything else change > other than switching from the soft token to the HSM? NSS 3.21 modnss 1.013 > > I assume that restarting Chrome will cause a fresh handshake so > things work again? (obviously not a great work-around, just > gathering info). yes, restarting apache or the browser fix the problem for a short time until the browser present the Ticket. Best Regards, Oliver