TLS/SSL抓包常见方法(一)

概述

目前，随着大家对安全的重视和HTTP2的推进，网站和APP后台基本上都会采用TLS/SSL；那么在TLS/SSL下，有什么方式抓到包的原始内容呢？

由于我习惯于采用WireShark来分析包，因此本文主要介绍如何采用WireShark来抓TLS/SSL包；

Wireshak简介

Wireshark是业界比较出名的一款抓包工具，尤其对包的分析很厉害，可以结合tcpdump使用;

wireshark.png

从Wireshark的配置界面可以看到它支持两种配置方式：

1. 配置私钥：
   通常私钥都放在服务端，如果可以拿到私钥，那肯定是可以解密整个通讯过程；
2. 配置log file:
   Wireshark是开源的，在它的源码里有段注释，介绍了Wireshark支持的log file格式:

/* The format of the file is a series of records with one of the following formats:
 *   - "RSA xxxx yyyy"
 *     Where xxxx are the first 8 bytes of the encrypted pre-master secret (hex-encoded)
 *     Where yyyy is the cleartext pre-master secret (hex-encoded)
 *     (this is the original format introduced with bug 4349)
 *
 *   - "RSA Session-ID:xxxx Master-Key:yyyy"
 *     Where xxxx is the SSL session ID (hex-encoded)
 *     Where yyyy is the cleartext master secret (hex-encoded)
 *     (added to support openssl s_client Master-Key output)
 *     This is somewhat is a misnomer because there's nothing RSA specific
 *     about this.
 *
 *   - "PMS_CLIENT_RANDOM xxxx yyyy"
 *     Where xxxx is the client_random from the ClientHello (hex-encoded)
 *     Where yyyy is the cleartext pre-master secret (hex-encoded)
 *     (This format allows SSL connections to be decrypted, if a user can
 *     capture the PMS but could not recover the MS for a specific session
 *     with a SSL Server.)
 *
 *   - "CLIENT_RANDOM xxxx yyyy"
 *     Where xxxx is the client_random from the ClientHello (hex-encoded)
 *     Where yyyy is the cleartext master secret (hex-encoded)
 *     (This format allows non-RSA SSL connections to be decrypted, i.e.
 *     ECDHE-RSA.)
 */

基于第一种方式，配置私钥的抓包方式比较简单，这边就不做介绍，我们重点看看第二种方式；

生成log文件

前面介绍了，可以通过配置log file,让wireshark对通讯过程进行解密；那么如何生成wireshark支持的文件呢？

浏览器:

Firefox 和 Chrome 都支持NSS Key Log。要想启用NSS LOG，必须要配置系统环境变量中SSLKEYLOGFILE(以MAC为例，其它系统类似)：

• 临时添加环境变量:

mkdir ~/ssl && touch ~/ssl/key.log
export SSLKEYLOGFILE=~/ssl/key.log

• 永久添加环境变量:

mkdir ~/ssl && touch ~/ssl/key.log
#zsh
echo "\nexport SSLKEYLOGFILE=~/ssl/key.log" >> ~/.zshrc && source ~/.zshrc
#bash
echo "\nexport SSLKEYLOGFILE=~/ssl/key.log" >> ~/.bash_profile && . ~/.bash_profile

添加环境变量之后，要重启浏览器，为了确保环境变量生效，建议通过终端启动浏览器，以我的机器为例:

open ~/Applications/Google\ Chrome.app 

之后通过浏览器访问https网站时，会将master key ,client_random等写key.log，根据NSS LOG Format文档，写入的日志有两种格式:

• 格式1:

RSA  <16 bytes of hex encoded encrypted pre master secret>  <96 bytes of hex encoded pre master secret>

• 格式2：

CLIENT_RANDOM  <64 bytes of hex encoded client_random>  <96 bytes of hex encoded master secret>

浏览器根据TLS/SSL握手协商出来的CipherSuites，选择不同的日志格式；如果采用RSA进行秘钥协商，则采用格式1；否则采用格式2；另外格式2要求wireshark版本>=1.8.0；

openssl命令

openssl提供了s_client命令，连接到服务端：

openssl s_client -connect www.baidu.com:443 -debug

result.png

可以看到输出里面包含Session-ID和Master-Key,根据之前的介绍，wireshark支持该种格式;

Java应用

在Java世界，JDK提供了JSSE,可以建立TLS/SSL通讯，但效率一般；由于Openssl使用很广，开源界在此基础上封装了Java版的API；Twitter首先封装了相应的API,后来这部分代码转给了tomcat,也就是Apache Tomcat Native Library;另外google在openssl基础上fork了boringssl,我在之前的工作中用的比较多的是netty从Tomcat Native Library fork出的netty tcnative, 里面包含boringssl、libressl和openssl;

JDK

JDK的SSLSessionImpl类记录了TLS/SSL协商过程中的masterkey:

/*
     * The state of a single session, as described in section 7.1
     * of the SSLv3 spec.
     */
    private final ProtocolVersion       protocolVersion;
    private final SessionId             sessionId;
    private X509Certificate[]   peerCerts;
    private byte                compressionMethod;
    private CipherSuite         cipherSuite;
    private SecretKey           masterSecret;

    void setMasterSecret(SecretKey secret) {
         if (masterSecret == null) {
             masterSecret = secret;
         } else {
             throw new RuntimeException("setMasterSecret() error");
         }
    }

masterSecret就是我们需要的内容;如果是自己动手构建TLS/SSL通讯框架，可以直接在握手成功之后从socket对象获取session,然后通过反射获取masterSecret;如果是采用第三方框架，则可以采用Java的Instrumentation机制实现：

public class Transformer implements ClassFileTransformer {
    private static final Logger log = Logger.getLogger(Transformer.class.getName());

    public byte[] transform(
            ClassLoader loader,
            String classPath,
            Class classBeingRedefined,
            ProtectionDomain protectionDomain,
            byte[] classfileBuffer) throws IllegalClassFormatException {
        String className = classPath.replace("/", ".");
        if (className.equals("sun.security.ssl.SSLSessionImpl") ||
                className.equals("com.sun.net.ssl.internal.ssl.SSLSessionImpl")) {
            try {
                ClassPool pool = ClassPool.getDefault();
                CtClass sslSessionClass = pool.get(className);
                CtMethod method = sslSessionClass.getDeclaredMethod("setMasterSecret");
                method.insertAfter(MasterSecretCallback.class.getName() + ".onMasterSecret(this, $1);");
                return sslSessionClass.toBytecode();
            } catch (Throwable e) {
                log.log(Level.WARNING, "Error instrumenting " + className, e);
                return classfileBuffer;
            }
        } else if (className.equals("sun.security.ssl.Handshaker") ||
                className.equals("com.sun.net.ssl.internal.ssl.Handshaker")) {
            try {
                ClassPool pool = ClassPool.getDefault();
                CtClass sslSessionClass = pool.get(className);
                CtMethod method = sslSessionClass.getDeclaredMethod("calculateConnectionKeys");
                method.insertBefore(MasterSecretCallback.class.getName() + ".onCalculateKeys(session, clnt_random, $1);");
                return sslSessionClass.toBytecode();
            } catch (Throwable e) {
                log.log(Level.WARNING, "Error instrumenting " + className, e);
                return classfileBuffer;
            }
        } else {
            return classfileBuffer;
        }
    }

}

关于该方案，github上已经有人提供了源码extract-ssl-secrets；

Netty

如果采用Netty作为通讯框架，而且采用的是openssl,无法通过Instrument的方式获取master secret;研究netty源码发现，netty是通过JNI调用openssl的c库实现的；openssl1.1.0之后提供了如下API:

#include 

 size_t SSL_get_client_random(const SSL *ssl, unsigned char *out, size_t outlen);
 size_t SSL_get_server_random(const SSL *ssl, unsigned char *out, size_t outlen);
 size_t SSL_SESSION_get_master_key(const SSL_SESSION *session, unsigned char *out, size_t outlen);

我采用的方案是扩展netty tcnative类库，在ssl.c中增加tapSSLKey方法 ：

TCN_IMPLEMENT_CALL(jlong, SSL, tapSSLKey)(TCN_STDARGS, jlong ssl)
{
    SSL *ssl_ = J2P(ssl, SSL *);
    SSL_SESSION *session = NULL;

    if (ssl_ == NULL) {
        tcn_ThrowException(e, "ssl is null");
        return 0;
    }
    session = SSL_get_session(ssl_);
    if (session == NULL) {
        // BoringSSL does not protect against a NULL session. OpenSSL
        // returns 0 if the session is NULL, so do that here.
        return 0;
    }

    UNREFERENCED(o);
   
    unsigned char client_random[SSL3_RANDOM_SIZE];
    unsigned char master_key[SSL_MAX_MASTER_KEY_LENGTH];
    int master_key_length = 0;

    if (session) {
        #if OPENSSL_VERSION_NUMBER >= 0x10100000L
                SSL_get_client_random(ssl_ , client_random, SSL3_RANDOM_SIZE);
                master_key_length = SSL_SESSION_get_master_key(session, master_key,
                        SSL_MAX_MASTER_KEY_LENGTH);
        #else
                if (ssl_ ->s3 && session->master_key_length > 0) {
                    memcpy(client_random, ssl_ ->s3->client_random, SSL3_RANDOM_SIZE);

                    master_key_length = session->master_key_length;
                    memcpy(master_key, session->master_key, master_key_length);
                }
        #endif
    }

    if (master_key_length > 0) {
        init_keylog_file();
        if (keylog_file_fd >= 0) {
            dump_to_fd(keylog_file_fd, client_random, master_key,
                    master_key_length);
        }
    }
    return 1;
}

具体的代码请参见我的github-netty-tcnative

JDK TLS/SSL Debug Utilities

JSSE提供了动态调试工具，启用的方式通过系统属性 javax.net.debug
关于JSSE提供了哪些调试选项，可以通过如下命令查看:

java -Djavax.net.debug=help MyApp

注意: MyApp必须使用了JSSE类；

TrustManager tm = new X509TrustManager() {
    @Override
    public void checkClientTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {

    }

    @Override
    public void checkServerTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {

    }

    @Override
    public X509Certificate[] getAcceptedIssuers() {
        return new X509Certificate[0];
    }
};
//设置SSLContext
SSLContext sslcontext = SSLContext.getInstance("TLS");
sslcontext.init(null, new TrustManager[]{tm}, null);

//打开连接
//要发送的POST请求url?Key=Value&Key2=Value2&Key3=Value3的形式
URL requestUrl = new URL("https://www.baidu.com");
HttpsURLConnection httpsConn = (HttpsURLConnection) requestUrl.openConnection();

//设置套接工厂
httpsConn.setSSLSocketFactory(sslcontext.getSocketFactory());
httpsConn.setRequestMethod("GET");
httpsConn.setDoOutput(true);

BufferedReader in = new BufferedReader(new InputStreamReader(httpsConn.getInputStream()));
int code = httpsConn.getResponseCode();
if (HttpsURLConnection.HTTP_OK == code) {
    String temp = in.readLine();
                     /*连接成一个字符串*/
    while (temp != null) {
        System.out.println(temp);
        temp = in.readLine();
    }
}

usage.png

例子：

• 查看所有调试信息

   java -Djavax.net.debug=all MyApp

• 查看握手信息

java -Djavax.net.debug=ssl:handshake:data MyApp

• 查看每次握手的信息&打印trust managere调试信息

java -Djavax.net.debug=SSL,handshake,data,trustmanager MyApp