Getting SSL_ERROR_NO_CYPHER_OVERLAP when attempting a site with a self-signed certificate
I'm developing a web app. Currently, I'm using a self-signed certificate (getting it properly signed comes later).
When I have the web server set so that it only accepts TLS1.1 and TLS1.2, I'm getting a SSL_ERROR_NO_CYPHER_OVERLAP error. And, of course, trying the 'use outdated security' link doesn't work, since the web server won't allow those connections.
If I temporarily allow insecure connections on the web server, Firefox will then allow me to accept the cert. After the cert is accepted, Firefox can then connect over only TLS1.1 and TLS1.2. So, most of the time, Firefox can find a common cypher for TLS1.1/1.2 connections.
(The web server is on an Ubuntu kernel, with OpenSSL1.0.1f.)
Chosen solution
I finally figured out what is going on.
The fix is really in configuring OpenSSL; however, since Firefox is the browser that most readily displays the problem, I'm going to post the answer here.
[NOTE: if you get this error on Firefox, and you're not in control of the web site you visit, you'll need to send an e-mail to the web site administrator, with the error you're getting]
Anyway, at issue is the separation in OpenSSL of the protocols supported vs. the cipher list.
In an app using OpenSSL, if you're using anything older than OpenSSL 1.1.0, you'll need to disable any protocol older than TLSv1. Do this with:
SSL_CTX_set_options(ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3);
(Note that recent versions of OpenSSL before version 1.1.0 have SSLv2 turned off by default, but it doesn't hurt to explicitly disable it with this call. Also note that if you disable TLSv1 [i.e. add ' | SSL_OP_NO_TLSv1' to the above call], you'll break compatibility with some apps that make HTTPS calls; for example Firefox appears to use TLSv1 to do the certificate exchange, before going to stronger protocols for the session).
The key to understanding the SSL_NO_CYPHER_OVERLAP error is that TLSv1 only uses SSLv3 ciphers.
So, I was running into this issue because when I disabled SSLv3, I was also disabling the SSLv3 ciphers. To set the OpenSSL ciphers, use something like:
SSL_CTL_set_cipher_list(ctx, "TLSv1.2:TLSv1:SSLv3:!SSLv2:HIGH:!MEDIUM:!LOW");
If you use instead (as I was originally using):
SSL_CTL_set_cipher_list(ctx, "TLSv1.2:TLSv1:!SSLv3:!SSLv2:HIGH:!MEDIUM:!LOW");
You'll effectively disable TLSv1, since there are no TLSv1-specific ciphers (at least in OpenSSL), and with the SSLv3 ciphers disabled, it isn't possible to establish a TLSv1 connection.
With SSLv3 disabled, but the TLSv1/SSLv3 ciphers enabled, Firefox is able to get the certificates. After this, I see that Firefox then establishes a TLSv1.2 connection.
Most of the above solution is not needed for OpenSSL 1.1.0, since that has no support for SSLv3 at all.
Read this answer in context 👍 4All Replies (1)
Chosen Solution
I finally figured out what is going on.
The fix is really in configuring OpenSSL; however, since Firefox is the browser that most readily displays the problem, I'm going to post the answer here.
[NOTE: if you get this error on Firefox, and you're not in control of the web site you visit, you'll need to send an e-mail to the web site administrator, with the error you're getting]
Anyway, at issue is the separation in OpenSSL of the protocols supported vs. the cipher list.
In an app using OpenSSL, if you're using anything older than OpenSSL 1.1.0, you'll need to disable any protocol older than TLSv1. Do this with:
SSL_CTX_set_options(ctx, SSL_OP_NO_SSLv2 | SSL_OP_NO_SSLv3);
(Note that recent versions of OpenSSL before version 1.1.0 have SSLv2 turned off by default, but it doesn't hurt to explicitly disable it with this call. Also note that if you disable TLSv1 [i.e. add ' | SSL_OP_NO_TLSv1' to the above call], you'll break compatibility with some apps that make HTTPS calls; for example Firefox appears to use TLSv1 to do the certificate exchange, before going to stronger protocols for the session).
The key to understanding the SSL_NO_CYPHER_OVERLAP error is that TLSv1 only uses SSLv3 ciphers.
So, I was running into this issue because when I disabled SSLv3, I was also disabling the SSLv3 ciphers. To set the OpenSSL ciphers, use something like:
SSL_CTL_set_cipher_list(ctx, "TLSv1.2:TLSv1:SSLv3:!SSLv2:HIGH:!MEDIUM:!LOW");
If you use instead (as I was originally using):
SSL_CTL_set_cipher_list(ctx, "TLSv1.2:TLSv1:!SSLv3:!SSLv2:HIGH:!MEDIUM:!LOW");
You'll effectively disable TLSv1, since there are no TLSv1-specific ciphers (at least in OpenSSL), and with the SSLv3 ciphers disabled, it isn't possible to establish a TLSv1 connection.
With SSLv3 disabled, but the TLSv1/SSLv3 ciphers enabled, Firefox is able to get the certificates. After this, I see that Firefox then establishes a TLSv1.2 connection.
Most of the above solution is not needed for OpenSSL 1.1.0, since that has no support for SSLv3 at all.
Modified by gshonle