Android 上 Https 双向通信— 深入理解KeyManager 和 TrustManagers
-
- Android 客户端的配置
- 生成SSLContext
- 和Http 客户端关联
- Android 作为Https 的服务端
- AndroidService 支持客户端证书请求
- 使用认证链做认证
- 使用hugo 观察SSL handshark 过程
- 服务端发送证书
- 服务端发送 Certificate Request
- 客户端校验
- 客户端发送证书
- concrypt 库的bug
- Two way ssl uses trustchain, android as a service
- https 握手过程中的KeyManager 和TrustManager 调用
- Android 客户端的配置
在Android 上http 访问采用双向ssl 认证是一种很常见的场景。这种通常是Android作为客户端，访问后台服务器。Android 作为服务端的情况比较少见。 下面就谈谈Android 同时作为服务端和客户端的情况。
Android 客户端的配置
Android 作为客户端https 通信，通常需要一个SSLContext， SSLContext 需要配置一个 TrustManager，如果是双向通信，还需要一个 KeyManager。
- 单行https TrustManager
- 双向https TrustManager KeyManager
- KeyManager 负责提供证书和私钥，证书发给对方peer
- TrustManager 负责验证peer 发来的证书。
生成SSLContext
KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
kmf.init(mKeyStore, mKeyPass.toCharArray());
tmf.init(mTrustStore);
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);和Http 客户端关联
OkHttp 客户端如下：
SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();
X509TrustManager x509TrustManager = Platform.get().trustManager(sslSocketFactory);
OkHttpClient okHttpClient = new OkHttpClient
.Builder()
.addInterceptor(httpLoggingInterceptor)
.sslSocketFactory(sslSocketFactory, x509TrustManager)
.build();
mRetrofit = new Retrofit.Builder()
.baseUrl(mBaseHost)
.client(okHttpClient)
.addConverterFactory(GsonConverterFactory.create())
.build();AndroidAsync 的客户端：
AsyncHttpClient.getDefaultInstance().getSSLSocketMiddleware().setSSLContext(sslContext);
AsyncHttpClient.getDefaultInstance().getSSLSocketMiddleware().setTrustManagers(tmf.getTrustManagers());如果是客户单的话配置还是比较的简单明了的
Android 作为Https 的服务端
在技术选型的时候选择了 AndroidAsync 作为服务端的框架。另外一个是 NanoHTTPD
AndroidAsync 和 NanoHTTPD 的对比
因为要同时实现客户端和服务端，而且AndroidAsync 支持异步，更符合现在的Android 趋势。
AndroidService 支持客户端证书请求
客户端按照上面的配置了一下，服务端也如法炮制sslContext，AndroidAsync 提供了一个SSLTests 的测试用例，采用自签名证书方式。
AsyncHttpServer httpServer = new AsyncHttpServer();
httpServer.listenSecure(8888, sslContext);
httpServer.get("/", new HttpServerRequestCallback() {
@Override
public void onRequest(AsyncHttpServerRequest request, AsyncHttpServerResponse response) {
response.send("hello");
}
});只能实现单向Https ,无法双向。通过抓包对比，发现双向https 需要服务端向客户端发送一个 Certificate Request。但是服务端没有发送。Android 上ssl 握手是通过openssl 实现的。通过查阅一些论文，查看boringssl 源码，是一个变量没有设置导致 handshake 的时候服务端没有发送 Certificate Request。 修改boringssl 不太现实。换个思路这个变量是不是可以通过Java层控制。
public void listenSecure(final int port, final SSLContext sslContext) {
AsyncServer.getDefault().listen(null, port, new ListenCallback() {
@Override
public void onAccepted(AsyncSocket socket) {
AsyncSSLSocketWrapper.handshake(socket, null, port, sslContext.createSSLEngine(), null, null, false,
new AsyncSSLSocketWrapper.HandshakeCallback() {
@Override
public void onHandshakeCompleted(Exception e, AsyncSSLSocket socket) {
if (socket != null)
mListenCallback.onAccepted(socket);
}
});
}
......
});
}通过对 AndroidAsync AsyncHttpServer 的实现分析，SSLContext的方法没有我们要控制的功能。但是ssl 握手的时候创建了一个SSLEngine。SSLEngine 的方法比较多的。
SSLEngine.setNeedClientAuth(true);
这个方法看起来比较靠谱。但是AndroidAsync 框架并没有提供API，想办法把这个类拿出来重写。服务端用SslAsyncHttpServer 替换AsyncHttpServer， Certificate Request 终于发出来了。
class SslAsyncHttpServer extends AsyncHttpServer {
private static final String TAG = "SslAsyncHttpServer";
private SSLEngine mSSLEngine
@Override
public void listenSecure(final int port, final SSLContext sslContext) {
AsyncServer.getDefault().listen(null, port, new ListenCallback() {
@Override
public void onAccepted(AsyncSocket socket) {
mSSLEngine = sslContext.createSSLEngine();
mSSLEngine.setNeedClientAuth(true);
AsyncSSLSocketWrapper.handshake(socket, null, port, mSSLEngine, null, null, false,
new AsyncSSLSocketWrapper.HandshakeCallback() {
@Override
public void onHandshakeCompleted(Exception e, AsyncSSLSocket socket) {
if (socket != null)
getListenCallback().onAccepted(socket);
}
});
}
@Override
public void onListening(AsyncServerSocket socket) {
getListenCallback().onListening(socket);
}
@Override
public void onCompleted(Exception ex) {
getListenCallback().onCompleted(ex);
}
});
}
}使用认证链做认证
在生产环境中 对证书的校验更为严格，通常采用证书链的方式。还是上面的code, 采用证书链的方式以后. handshake 失败。
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: javax.net.ssl.SSLHandshakeException: Handshake failed
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.android.org.conscrypt.OpenSSLEngineImpl.unwrap(OpenSSLEngineImpl.java:441)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:1270)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.koushikdutta.async.AsyncSSLSocketWrapper$5.onDataAvailable(AsyncSSLSocketWrapper.java:194)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.koushikdutta.async.Util.emitAllData(Util.java:23)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.koushikdutta.async.AsyncNetworkSocket.onReadable(AsyncNetworkSocket.java:152)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.koushikdutta.async.AsyncServer.runLoop(AsyncServer.java:821)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.koushikdutta.async.AsyncServer.run(AsyncServer.java:658)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.koushikdutta.async.AsyncServer.access$800(AsyncServer.java:44)
04-27 08:43:33.093 6881-6903/com.louie.certtest W/System.err: at com.koushikdutta.async.AsyncServer$14.run(AsyncServer.java:600)
04-27 08:43:33.094 6881-6903/com.louie.certtest W/System.err: Caused by: javax.net.ssl.SSLProtocolException: SSL handshake terminated: ssl=0xb31d4fc0: Failure in SSL library, usually a protocol error
04-27 08:43:33.094 6881-6903/com.louie.certtest W/System.err: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE (external/boringssl/src/ssl/s3_srvr.c:1945 0xa3b68196:0x00000000)
04-27 08:43:33.094 6881-6903/com.louie.certtest W/System.err: at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake_bio(Native Method)
04-27 08:43:33.094 6881-6903/com.louie.certtest W/System.err: at com.android.org.conscrypt.OpenSSLEngineImpl.unwrap(OpenSSLEngineImpl.java:426)
04-27 08:43:33.094 6881-6903/com.louie.certtest W/System.err: ... 8 morewireshark 抓包后发现，服务端发送了 TCP 的FIN，查看整个握手过程，服务端发送 Certificate Request 后，客户端也发送了 “Certificate”，但是服务端随后就发送了 Fin。又是一个让人头疼的问题。从源码来看，确实是服务端 调用了close.
>4 0.007445 127.0.0.1 127.0.0.1 TLSv1.2 205 Client Hello
>6 0.022138 127.0.0.1 127.0.0.1 TLSv1.2 1652 Server Hello, Certificate, Server Key Exchange, Certificate Request, Server Hello Done
>8 0.029033 127.0.0.1 127.0.0.1 TLSv1.2 204 Certificate, Client Key Exchange, Change Cipher Spec, Hello Request, Hello Request
>9 0.031817 127.0.0.1 127.0.0.1 TCP 66 6666→51878 [FIN, ACK] Seq=1587 Ack=278 Win=131008 Len=0 TSval=1856446 TSecr=1856446查到了 ssl 握手的 RFC 文档
The TLS ProtocolVersion 1.0
7.4.6. Client certificate
When this message will be sent:
This is the first message the client can send after receiving a
server hello done message. This message is only sent if the
server requests a certificate. If no suitable certificate is
available, the client should send a certificate message
containing no certificates. If client authentication is required
by the server for the handshake to continue, it may respond with
a fatal handshake failure alert. Client certificates are sent
using the Certificate structure defined in Section 7.4.2.
然后再看 在wireshark 中看客户端回的 Certificate 字段，长度居然为0.
想看下完整的ssl 握手过程，但是Android上并没有SSL 握手的详细日志。
使用hugo 观察SSL handshark 过程
stackoverflow 上有一篇帖子清奇：
Client Certificate not working from Android - How to debug?
关于hugo 的详细使用参考
hugo
服务端发送证书
04-27 04:05:02.025 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇢ chooseServerAlias(s="EC", principals=null, socket=null) [Thread:"AsyncServer"]
04-27 04:05:02.025 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇠ chooseServerAlias [0ms] = null
04-27 04:05:02.026 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇢ chooseServerAlias(s="RSA", principals=null, socket=null) [Thread:"AsyncServer"]
04-27 04:05:02.026 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇠ chooseServerAlias [0ms] = "1"
04-27 04:05:02.026 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇢ getPrivateKey(s="1") [Thread:"AsyncServer"]
04-27 04:05:02.026 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇠ getPrivateKey [0ms] = RSA Private CRT Key- 从日志上看， 第 1 2 3 4 行是服务端需要发送 Certificate, 在KeyStore 中选择和是的证书Alias
- 5 6 行根据选择的Alias 获取PrivateKey.
04-27 04:05:02.026 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇢ getCertificateChain(s="1") [Thread:"AsyncServer"]
04-27 04:05:02.030 3682-3699/com.louie.certtest V/SslX509KeyManager: ⇠ getCertificateChain [0ms] = [Certificate:
Data:
Version: 3 (0x2)
Serial Number:
21:dd:e7:2c:8c:95:d9:f1
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=XXX_Web_test, O=XXX, C=US
Validity
Not Before: Mar 26 14:38:49 2018 GMT
Not After : Jan 4 20:48:34 2037 GMT
Subject: CN=ae86.XXXXXXX-local.com- 接下来的日志表示找到服务端的证书，服务端的证书会发送给客户端。
服务端发送 Certificate Request
服务端首先根据KeyStore 中的证书链 找出客户端需要发送证书的issure. 从日志上看是一个 Intermediate CA:
04-27 04:05:02.030 3682-3699/com.louie.certtest V/SslX509TrustManager: ⇢ getAcceptedIssuers() [Thread:"AsyncServer"]
04-27 04:05:02.043 3682-3699/com.louie.certtest V/SslX509TrustManager: ⇠ getAcceptedIssuers [0ms] = [Certificate:
Data:
Version: 3 (0x2)
Serial Number:
74:0e:7c:31:e5:5e:2c:9d
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=XXX Root CA, O=XXX, C=US
Validity
Not Before: Jan 4 20:48:34 2017 GMT
Not After : Jan 4 20:48:34 2037 GMT
Subject: CN=XXX Intermediate CA, O=XXX, C=US
客户端校验
客户端校验服务端证书：
04-27 08:35:41.909 6640-6660/com.louie.certtest V/SslX509TrustManager: ⇢ checkServerTrusted(chain=[Certificate:客户端发送证书
客户端根据服务端发送的 Certificate Request 选择合适的证书
从日志上可以看出：
服务端发送的证书 Subject 为:
- C=US, O=XXX, CN=XXX Intermediate CA,
- C=US, O=XXX, CN=XXX Root CA,
- C=US, O=XXX, CN=XXX Root CA]
客户端证书的Issure为:
1. C=US, O=XXX, CN=XXX_Vehicle_test
客户端没有找到合适的证书，所以发送的证书长度为0。
04-27 08:35:41.914 6640-6660/com.louie.certtest V/SslX509KeyManager: ⇢ chooseClientAlias(strings=["EC", "RSA"], principals=[C=US, O=XXX, CN=XXX Intermediate CA, C=US, O=XXX, CN=XXX Root CA, C=US, O=XXX, CN=XXX Root CA], socket=null) [Thread:"AsyncServer"]
04-27 08:35:41.914 6640-6660/com.louie.certtest V/KeyManagerImpl: ⇢ printVar(name="issuersList", object=[C=US, O=XXX, CN=XXX Intermediate CA, C=US, O=XXX, CN=XXX Root CA, C=US, O=XXX, CN=XXX Root CA]) [Thread:"AsyncServer"]
04-27 08:35:41.915 6640-6660/com.louie.certtest V/KeyManagerImpl: ⇠ printVar [0ms]
04-27 08:35:41.915 6640-6660/com.louie.certtest V/KeyManagerImpl: ⇢ printVar(name="issuerFromChain", object=C=US, O=XXX, CN=XXX_Vehicle_test) [Thread:"AsyncServer"]
04-27 08:35:41.915 6640-6660/com.louie.certtest V/KeyManagerImpl: ⇠ printVar [0ms]
04-27 08:35:41.915 6640-6660/com.louie.certtest V/KeyManagerImpl: ⇢ printVar(name="alias", object="1") [Thread:"AsyncServer"]
04-27 08:35:41.915 6640-6660/com.louie.certtest V/KeyManagerImpl: ⇠ printVar [0ms]
04-27 08:35:41.915 6640-6660/com.louie.certtest V/SslX509KeyManager: ⇠ chooseClientAlias [0ms] = nullconcrypt 库的bug
找到原因后，看下服务端为发送的 Certificate Request 为什么不正确。
通过debug ， 服务端调用 在 SSLParametersImpl.java 中的 setCertificateValidation 调用 trustManager.getAcceptedIssuers()。
然后调用 encodeIssuerX509Principals 函数。
void setCertificateValidation(long sslNativePointer) throws IOException {
if(!this.client_mode) {
boolean certRequested;
。。。。。。
if(certRequested) {
X509TrustManager trustManager = this.getX509TrustManager();
X509Certificate[] issuers = trustManager.getAcceptedIssuers();
if(issuers != null && issuers.length != 0) {
byte[][] issuersBytes;
try {
issuersBytes = encodeIssuerX509Principals(issuers);
} catch (CertificateEncodingException var8) {
throw new IOException("Problem encoding principals", var8);
}
NativeCrypto.SSL_set_client_CA_list(sslNativePointer, issuersBytes);
}
}
}
}在encodeIssuerX509Principals 中调用getIssuerX500Principal 获取证书的Issuere.如果我们有三个证书组成认证链:
- [subject=RootCA, issure=RootCA],
- [subject=SecondCA, issure=RootCA]
- [subject=ThirdCA, issure=SecondCA]
getIssuerX500Principal 获取到的是
[RootCA, RootCA, SecondCA]
正确的做法为:getSubjectX500Principal
这样获取到的为:
[RootCA, SecondCA, ThirdCA]
static byte[][] encodeIssuerX509Principals(X509Certificate[] certificates) throws CertificateEncodingException {
byte[][] principalBytes = new byte[certificates.length][];
for(int i = 0; i < certificates.length; ++i) {
principalBytes[i] = certificates[i].getIssuerX500Principal().getEncoded();
}
return principalBytes;
}Two way ssl uses trustchain, android as a service
在Android 8.0 上测试，发现还是有这个问题。Androd作为客户端的场景比较常见，
作为服务端比较少见。向google 提交了一个commit
Two way ssl uses trustchain, android as a service
https 握手过程中的KeyManager 和TrustManager 调用
