一次 https 证书引起的 nginx 400 bad request 分析

1. 问题描述

在一台Server上部署有一个https的service(这个service用于为Android Client提供服务)，之前一直正常，从某一天开始突然不能正常访问，有的机型一直在nginx中报 400 bad request， 有的则正常访问.
两个问题：
Question 1: 为什么一部分机型突然不正常了？
Question 2: 为什么是一部分机型正常，一部分机型不正常?

2. 环境描述

本台Server上部署了两套完全不相关的Service，都是使用https， 两套的证书完全不一样，并且不是同一个CA签发的,
上述出错的service是a.com下的service

server {
    listen 443
    server_name a.com
    ssl on
    ssl_certificate a.com.crt;
    ssl_certificate_key a.com.key;
    ...
}

server {
    listen 443
    server_name b.cn
    ssl on
    ssl_certificate b.cn.crt;
    ssl_certificate_key b.cn.key;
    ...
}

3. Q1分析

nginx的access log打出来 400， 可能是建立连接https连接的时候出错了，没有更详细的信息了，所以有以下分析
1. Server上使用tcpdump抓包, wireshark分析结果

tcpdump -w debug.pcap

过程10，Server下发了证书
过程15，Client报错, Alert(Level Fatal, Description: Certificate Unknown)

1. 查看过程10中下发证书的详情
   
   发现请求a.com过程中下发的证书竟然是b.cn的，难以置信，但是至少发现了不可理解的地方，然后google找到nginxg文档中有这样的解释:

Name-based HTTPS servers

A common issue arises when configuring two or more HTTPS servers listening on a single IP address:

server {
    listen          443 ssl;
    server_name     www.example.com;
    ssl_certificate www.example.com.crt;
    ...
}

server {
    listen          443 ssl;
    server_name     www.example.org;
    ssl_certificate www.example.org.crt;
    ...
}
With this configuration a browser receives the default server’s certificate, i.e. www.example.com regardless of the requested server name. This is caused by SSL protocol behaviour. The SSL connection is established before the browser sends an HTTP request and nginx does not know the name of the requested server. Therefore, it may only offer the default server’s certificate.

The oldest and most robust method to resolve the issue is to assign a separate IP address for every HTTPS server:

server {
    listen          192.168.1.1:443 ssl;
    server_name     www.example.com;
    ssl_certificate www.example.com.crt;
    ...
}

server {
    listen          192.168.1.2:443 ssl;
    server_name     www.example.org;
    ssl_certificate www.example.org.crt;
    ...
}

也就是说如果按上面的https配置方式，每次请求的时候，无论请求的是哪个域名，只要是https的请求，nginx
会把默认的证书发下来(至于哪个server_name下的证书是默认的，需要研究), 也就是说访问a.com, 拿到b.cn的证书是可以理解的，因为server_name的信息是在http请求的headers里面，而建立https过程中并没有http的headers信息，所以nginx并不能根据headers发给相应的server_name处理，所以也就出现了400 bad request的问题,这个现象貌似和我们之前配置80的多个server_name不一样，直到看到文档的解释才相信确实如此。暂时的解决方案是只让一个server_name配置443的端口.

4. Q2分析

对于Part 2中的场景，两个机型(Nexus 5, 1+ )请求a.com, N5总是 400 bad request, 而1+，则是正常的, 对于这种现象往下看nginx文档,关于TLS协议的一个扩展字段SNI(Server Name Indication)的解释：

Server Name Indication

A more generic solution for running several HTTPS servers on a single IP address is TLS Server Name Indication extension (SNI, RFC 6066), which allows a browser to pass a requested server name during the SSL handshake and, therefore, the server will know which certificate it should use for the connection. However, SNI has limited browser support. Currently it is supported starting with the following browsers versions:

Opera 8.0;
MSIE 7.0 (but only on Windows Vista or higher);
Firefox 2.0 and other browsers using Mozilla Platform rv:1.8.1;
Safari 3.2.1 (Windows version supports SNI on Vista or higher);
and Chrome (Windows version supports SNI on Vista or higher, too).
Only domain names can be passed in SNI, however some browsers may erroneously pass an IP address of the server as its name if a request includes literal IP address. One should not rely on this.
In order to use SNI in nginx, it must be supported in both the OpenSSL library with which the nginx binary has been built as well as the library to which it is being dynamically linked at run time. OpenSSL supports SNI since 0.9.8f version if it was built with config option “--enable-tlsext”. Since OpenSSL 0.9.8j this option is enabled by default. If nginx was built with SNI support, then nginx will show this when run with the “-V” switch:

$ nginx -V
...
TLS SNI support enabled
...
However, if the SNI-enabled nginx is linked dynamically to an OpenSSL library without SNI support, nginx displays the warning:

nginx was built with SNI support, however, now it is linked
dynamically to an OpenSSL library which has no tlsext support,
therefore SNI is not available

这个字段的存在满足了在建立https链接时就指定要使用的server_name的需求，会在Client Hello中的扩展字段中带上，同时对于不同的浏览器，对TLS协议的SNI不一定支持，所以怀疑N5的Client Hello中没带SNI， 而1+手机则带有SNI，
抓包验证，抓包验证之前还有一个问题，就是在https链接建立过程中的哪一步会带上这个信息呢，这是一个之前不知道的问题，所以在rfc6066中找到了确定解释，在Client Hello中：

3.  Server Name Indication

   TLS does not provide a mechanism for a client to tell a server the
   name of the server it is contacting.  It may be desirable for clients
   to provide this information to facilitate secure connections to
   servers that host multiple 'virtual' servers at a single underlying
   network address.

   In order to provide any of the server names, clients MAY include an
   extension of type "server_name" in the (extended) client hello.  The
   "extension_data" field of this extension SHALL contain
   "ServerNameList" where:

      struct {
          NameType name_type;
          select (name_type) {
              case host_name: HostName;
          } name;
      } ServerName;

      enum {
          host_name(0), (255)
      } NameType;

      opaque HostName<1..2^16-1>;

      struct {
          ServerName server_name_list<1..2^16-1>
      } ServerNameList;

N5 https 的包：

1+ https client hello 的包：

确实符合猜想，1+有Extension: server_name字段， 所以它的请求会正常
这一步说明确实是和机型相关的，Andorid Client请求service 使用的okhttpclient, 并没有明确配置SNI，所以有的机型可能默认支持开启，有的不支持并没有这个字段.

5. 总结

1. 如Part2中的配置不可行，解决方案按Part3中的解释
2. SNI使用需要Client和Server的同时支持
3. 不要让潜在的认知误导自己，验证了事实再看问题

6. Reference

1. Transport Layer Security (TLS) Extensions: Extension Definitions
2. Configuring HTTPS servers
3. Transport Layer Security (TLS) Extensions
4. Server Name Indication
5. Nginx 日志中的400和408