SSL renegotiation with full duplex socket communic

up vote 0 down vote favorite 1

I have a very simple client-server with one blocking socket doing full-duplex communication. I've enabled SSL/TLS to the application. The model is that of a typical producer-consumer. The client produces the data, sends it to the server and the server processes them. The only catch is that, once in a while the server sends data back to the client which the client handles accordingly. Below is a very simple pseudo code of the application: 1 Client: 2 ------- 3 while (true) 4 { 5 if (poll(pollin, timeout=0) || 0 < SSL_pending(ssl)) 6 { 7 SSL_read(); 8 // Handle WANT_READ or WANT_WRITE appropriately. 9 // If no error, handle the received control message. 10 } 11 // produce data. 12 while (!poll(pollout)) 13 ; // Wait until the pipe is ready for a send(). 14 SSL_write(); 15 // Handle WANT_READ or WANT_WRITE appropriately. 16 if (time to renegotiate) 17 SSL_renegotiate(ssl); 18 } 19 20 Server: 21 ------- 22 while (true) 23 { 24 if (poll(pollin, timeout=1s) || 0 < SSL_pending(ssl)) 25 { 26 SSL_read(); 27 // Handle WANT_READ or WANT_WRITE appropriately. 28 // If no error, consume data. 29 } 30 if (control message needs to be sent) 31 { 32 while (!poll(pollout)) 33 ; // Wait until the pipe is ready for a send(). 34 SSL_write(); 35 // Handle WANT_READ or WANT_WRITE appropriately. 36 } 37 } The trouble happens when, for testing purposes, I force SSL renegotiation (lines 16-17). The session starts nice and easy, but after a while, I get the following errors: Client: ------- error:140940F5:SSL routines:SSL3_READ_BYTES:unexpected record Server: ------- error:140943F2:SSL routines:SSL3_READ_BYTES:sslv3 alert unexpected message Turns out, around the same time that the client initiates a renegotiation (line 14), the server ends up sending application data to the client (line 34). The client as part of the renegotiation process receives this application data and bombs with a "unexpected record" error. Similarly, when the server does the subsequent receive (line 26), it ends up receiving a renegotiation data when it was expecting application data. What am I doing wrong? How should I handle/test SSL renegotiations with a full-duplex channel. Note that, there are no threads involved. It's a simple single threaded model with reads/writes happening on either end of the socket. UPDATE : To verify that there is nothing wrong with the application that I have written, I could even reproduce this quite comfortably with OpenSSL's s_client and s_server implementations. I started a s_server and once the s_client got connected to the server, I programmatically send a bunch of application data from the server to the client and a bunch of 'R' (renegotiation requests) from the client to the server. Eventually, they both fail in exactly the same manner as described above. s_client: RENEGOTIATING 4840:error:140940F5:SSL routines:SSL3_READ_BYTES:unexpected record:s3_pkt.c:1258: s_server: Read BLOCK ERROR 4838:error:140943F2:SSL routines:SSL3_READ_BYTES:sslv3 alert unexpected message:s3_pkt.c:1108:SSL alert number 10 4838:error:140940E5:SSL routines:SSL3_READ_BYTES:ssl handshake failure:s3_pkt.c:1185: UPDATE 2: Ok. As suggested by David, I reworked the test application to use non-blocking sockets and always do SSL_read and SSL_write first and do the select based on what they return and I still get the same errors during renegotiations (SSL_write ends up getting application data from the other side in the midst of renegotiation). The question is, at any point in time, if SSL_read returns WANT_READ, can I assume it is because there is nothing in the pipe and go ahead with SSL_write since I have something to write? If not, that's probably why I end up with errors. Either that, or I am doing the renegotiation all wrong. Note, if SSL_read returns WANT_WRITE, I always do a select and call SSL_read again. c  ssl  openssl share improve this question edited Sep 11 '13 at 17:03 asked Sep 10 '13 at 20:38 Karthik 500 2 8

add a comment

2 Answers

active oldest votes

up vote 5 down vote

You're trying to "look through" the SSL black box. This is a huge mistake. if (poll(pollin, timeout=0) || 0 < SSL_pending(ssl)) { SSL_read(); You're making the assumption that in order for SSL_read to make forward progress, it needs to read data from the socket. This is an assumption that can be false. For example, if a renegotiation is in progress, the SSL engine may need to send data next, not read data. while (!poll(pollout)) ; // Wait until the pipe is ready for a send(). SSL_write(); How do you know the SSL engine wants to write data to the pipe? Did it give you a WANT_WRITEindication? If not, maybe it needs to read renegotiation data in order to send. To use SSL in non-blocking mode, just attempt the operation you want to do. If you want to read decrypted data, call SSL_read. If you want to send encrypted data, call SSL_write. Only call poll if the SSL engine tells you to, with a WANT_READ or WANT_WRITE indication. Update:: You have a "half of each" hybrid between blocking and non-blocking approaches. This cannot possibly work. The problem is simple: Until you call SSL_read, you don't know whether or not it needs to read from the socket. If you call poll first, you will block even if SSL_read does not need to read from the socket. If you call SSL_read first, it will block if it does need to read from the socket. SSL_pending won't help you. If SSL_read needs to write to the socket to make forward progress, SSL_pending will return zero, but calling poll will block forever. You have two sane choices: Blocking. Leave the sockets set blocking. Just call SSL_read when you want to read and SSL_write when you want to write. They will block. Blocking sockets can block, that's how they work. Non-blocking. Set the sockets non-blocking. Just call SSL_read when you want to read and SSL_write when you want to write. They will not block. If you get a WANT_READ indication, poll in the read direction. If you get a WANT_WRITE indication, poll in the write direction. Note that it is perfectly normal for SSL_read to return WANT_WRITE, and then you poll in the write direction. Similarly, SSL_write can return WANT_READ, and then you poll in the read direction. Your code would (mostly) work if the implementation of SSL_read was basically, "read some data then decrypt it" and SSL_write was "encrypt some data and send it". The problem is, these functions actually run a sophisticated state machine that reads and writes to the socket as needed and ultimately causes the effect of giving you decrypted data or encrypting your data and sending it. share improve this answer edited Sep 13 '13 at 22:51 answered Sep 10 '13 at 20:53 David Schwartz 98k 6 84 147

As I mentioned in my question, the socket I have is of blocking nature and with blocking sockets, poll() is one way to know you won't end up blocking on recv()/send(). If I don't use poll() and call SSL_read on a blocking socket, I am bound to block until the other side sends something. In that case, how to deal with SSL renegotiations? –  Karthik  Sep 10 '13 at 21:08  1   @Karthik: You have to choose whether to use blocking sockets or non-blocking sockets and stick to that choice. If you want to use non-blocking sockets, set them non-blocking and follow my explanation. If you want to use blocking sockets, don't call poll. A "half of each" hybrid will not work for the reasons I explained -- polling on the socket does not tell you the readiness of the SSL connection. –  David Schwartz Sep 10 '13 at 21:45  There is no "half of each" hybrid. Both sides use blocking sockets and with blocking sockets, poll() is inevitable. Also, see my update. I could get the same behavior with OpenSSL's s_client and s_server both of which use blocking sockets and select()s before doing a SSL_read() or SSL_write. –  Karthik  Sep 10 '13 at 21:49 1   @Karthik Oh, then you have a different bug. You're refusing to call SSL_read even if it can make forward progress just because no data has been received on the socket. What if SSL_read has to write to the socket to make forward progress? –  David Schwartz  Sep 10 '13 at 22:17  1   with such a model, it's impossible to support SSL renegotiations unless non-blocking sockets are used -- Do you want to block or not? It is impossible to use blocking sockets and nevertheless not block. If you do not ever wish to block, you must use non-blocking sockets. –  David Schwartz  Sep 10 '13 at 22:19 show  5 more comments

up vote 1 down vote accepted

After spending time debugging my application with OpenSSL, I figured out the answer to the question I originally posted. I am sharing it here in case it helps others like me. The question I had posted originally had to do with a clear error from OpenSSL indicating that it was receiving application data in the middle of a handshake. What I failed to understand was that OpenSSL gets confused when it receives application data in the middle of a handshake. It's fine to receive handshake data when receiving/sending application data but not the other way around (at least with OpenSSL). That's the thing that I failed to realize. This is also the reason most SSL-enabled applications run fine because most of them are half-duplex in nature (HTTPS for instance) which implicitly guarantees no application data asynchronously arriving at the time of handshake. What this means is that if you are designing a custom client-server full-duplex protocol (which is the case I am in) and want to slap SSL onto it, it's the application's responsibility to initiate a renegotiation when neither end is sending any data. This is clearly documented in Mozilla's NSS API. Not to mention there is an open ticket in OpenSSL's bug repository regarding this issue. The moment I changed my application to initiate a handshake when there is nothing for the client/server to sayto one another, I no longer faced the above errors. Also, I agree with David's comments about blocking sockets and I've read many of his arguments in the OpenSSL mailing list as well. But, the sad thing is that most legacy applications are built around poll and blocking sockets and they "Just Work Fine (TM)". The issue arises when dealing with SSL renegotiation. I still believe at least my application can deal with SSL renegotiation in the presence of blocking sockets since it is a very confined and custom protocol and we (as the application developer) can decide to do the renegotiation when the protocol is quiescent. If that doesn't work, I will go the non-blocking socket route. share improve this answer answered Sep 14 '13 at 16:07 Karthik 500 2 8

This can happen with HTTPS too. You can stream requests (typically GET) and get their responses without blocking between each request/response pair. A re-negotiation in the middle of this would cause similar problems. –  Bruno  Sep 14 '13 at 18:04 Saved me... had data exchange and the handshake... rather wait for the handshake to complete. –  Blacktempel  Feb 27 at 14:08