Go实战--golang中使用HTTPS以及TSL(.crt、.key、.pem区别以及crypto/tls包介绍)
生命不止，继续go go go!!!
HTTP与HTTPS
在WWDC 2016上，苹果在发布iOS 9的同时也向开发者传递了一个消息，那就是到2017年1月1日时App Store中所有应用都必须启用 App Transport Security应用程序安全传输协议，从而提升应用和系统安全性。
HTTPS是Hyper Text Transfer Protocol Secure的缩写，相比http，多了一个secure，这一个secure是怎么来的呢？这是由TLS（SSL）提供的。
https和http都属于应用层，基于TCP（以及UDP）协议。但是不同的是：
HTTP 缺省工作在TCP协议80端口
HTTPS缺省工作在TCP协议443端口
通俗一句话：相比http，https对于大部分人来说，意味着比较安全。
TLS
安全传输层协议（TLS）用于在两个通信应用程序之间提供保密性和数据完整性。
The TLS/SSL is a public/private key infrastructure (PKI). For most common cases, each client and server must have a private key.
TLS与SSL的差异
SSLv2 and SSLv3 are completely different (and both are now considered insecure). SSLv3 and TLSv1.0 are very similar, but have a few differences.
You could consider TLSv1.0 as SSLv3.1
版本号：TLS记录格式与SSL记录格式相同，但版本号的值不同，TLS的版本1.0使用的版本号为SSLv3.1。
报文鉴别码：SSLv3.0和TLS的MAC算法及MAC计算的范围不同。TLS使用了RFC-2104定义的HMAC算法。SSLv3.0使用了相似的算法，两者差别在于SSLv3.0中，填充字节与密钥之间采用的是连接运算，而HMAC算法采用的是异或运算。但是两者的安全程度是相同的。
伪随机函数：TLS使用了称为PRF的伪随机函数来将密钥扩展成数据块，是更安全的方式。
报警代码：TLS支持几乎所有的SSLv3.0报警代码，而且TLS还补充定义了很多报警代码，如解密失败（decryption_failed）、记录溢出（record_overflow）、未知CA（unknown_ca）、拒绝访问（access_denied）等。
密文族和客户证书：SSLv3.0和TLS存在少量差别，即TLS不支持Fortezza密钥交换、加密算法和客户证书。
certificate_verify和finished消息：SSLv3.0和TLS在用certificate_verify和finished消息计算MD5和SHA-1散列码时，计算的输入有少许差别，但安全性相当。
加密计算：TLS与SSLv3.0在计算主密值（master secret）时采用的方式不同。
填充：用户数据加密之前需要增加的填充字节。在SSL中，填充后的数据长度要达到密文块长度的最小整数倍。而在TLS中，填充后的数据长度可以是密文块长度的任意整数倍（但填充的最大长度为255字节），这种方式可以防止基于对报文长度进行分析的攻击。
openssl
openssl(www.openssl.org) 是sslv2,sslv3,tlsv1的一份完整实现,内部包含了大量加密算法程序.其命令行提供了丰富的加密,验证,证书生成等功能,甚至可以用其建立 一个完整的CA.与其同时,它也提供了一套完整的库函数,可用开发用SSL/TLS的通信程序.
插曲：
2016年10月18日，锤子科技CEO罗永浩在锤子手机发布会上，宣布将200多万元门票收入，以及原计划成立的 Smartisan 公益基金近100万元，全部捐赠给 OpenSSL 基金会和 OpenBSD 基金会。
crt、key以及pem的区别以及生成
crt — Alternate synonymous most common among *nix systems .pem (pubkey).
csr — Certficate Signing Requests (synonymous most common among *nix systems).
cer — Microsoft alternate form of .crt, you can use MS to convert .crt to .cer (DER encoded .cer, or base64[PEM] encoded cer).
pem = The PEM extension is used for different types of X.509v3 files which contain ASCII (Base64) armored data prefixed with a «—– BEGIN …» line. These files may also bear the cer or the crt extension.
der — The DER extension is used for binary DER encoded certificates.
证书(Certificate) .cer .crt
私钥(Private Key).key
证书签名请求(Certificate sign request) .csr
至于pem和der，是编码方式，以上三类均可以使用这两种编码方式，因此.pem和.der(少见)不一定是以上三种(Cert,Key,CSR)中的某一种
PEM - Privacy Enhanced Mail,打开看文本格式,以”—–BEGIN…”开头, “—–END…”结尾,内容是BASE64编码.
查看PEM格式证书的信息:openssl x509 -in certificate.pem -text -noout
Apache和*NIX服务器偏向于使用这种编码格式.
DER - Distinguished Encoding Rules,打开看是二进制格式,不可读.
查看DER格式证书的信息:openssl x509 -in certificate.der -inform der -text -noout
Java和Windows服务器偏向于使用这种编码格式.
x509
X.509是一种非常通用的证书格式。所有的证书都符合ITU-T X.509国际标准，因此(理论上)为一种应用创建的证书可以用于任何其他符合X.509标准的应用。
x509证书一般会用到三类文，key，csr，crt。
Key 是私用密钥openssl格，通常是rsa算法。
Csr 是证书请求文件，用于申请证书。在制作csr文件的时，必须使用自己的私钥来签署申，还可以设定一个密钥。
crt是CA认证后的证书文，（windows下面的，其实是crt），签署人用自己的key给你签署的凭证。
生成.key
rsa算法：
openssl genrsa -out server.key 2048ECDSA算法：
openssl ecparam -genkey -name secp384r1 -out server.key生成.crt
openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650需要输入一些信息：
openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:cn
State or Province Name (full name) [Some-State]:Beijing
Locality Name (eg, city) []:Beijing
Organization Name (eg, company) [Internet Widgits Pty Ltd]:wangshubo
Organizational Unit Name (eg, section) []:wangshubo
Common Name (e.g. server FQDN or YOUR name) []:wangshubo
Email Address []:wangshubo1989@126.com生成pem和key
openssl req -new -nodes -x509 -out server.pem -keyout server.key -days 3650 -subj "/C=DE/ST=NRW/L=Earth/O=Random Company/OU=IT/CN=www.random.com/[email protected]"crypto/tls包介绍
golang中为我们提供了tls包：
Package tls partially implements TLS 1.2, as specified in RFC 5246.
func LoadX509KeyPair
func LoadX509KeyPair(certFile, keyFile string) (Certificate, error)LoadX509KeyPair reads and parses a public/private key pair from a pair of files. The files must contain PEM encoded data. The certificate file may contain intermediate certificates following the leaf certificate to form a certificate chain. On successful return, Certificate.Leaf will be nil because the parsed form of the certificate is not retained.
type Config
A Config structure is used to configure a TLS client or server. After one has been passed to a TLS function it must not be modified. A Config may be reused; the tls package will also not modify it.
type Config struct {
Rand io.Reader
Time func() time.Time
Certificates []Certificate
NameToCertificate map[string]*Certificate
GetCertificate func(*ClientHelloInfo) (*Certificate, error)
GetClientCertificate func(*CertificateRequestInfo) (*Certificate, error)
GetConfigForClient func(*ClientHelloInfo) (*Config, error)
VerifyPeerCertificate func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error
RootCAs *x509.CertPool
NextProtos []string
ServerName string
ClientAuth ClientAuthType
ClientCAs *x509.CertPool
InsecureSkipVerify bool
CipherSuites []uint16
PreferServerCipherSuites bool
SessionTicketsDisabled bool
SessionTicketKey [32]byte
ClientSessionCache ClientSessionCache
MinVersion uint16
MaxVersion uint16
CurvePreferences []CurveID
DynamicRecordSizingDisabled bool
Renegotiation RenegotiationSupport
KeyLogWriter io.Writer
}这里主要关注一下Certificates，是我们要用到的。
func Listen
func Listen(network, laddr string, config *Config) (net.Listener, error)Listen creates a TLS listener accepting connections on the given network address using net.Listen. The configuration config must be non-nil and must include at least one certificate or else set GetCertificate.
func Dial
func Dial(network, addr string, config *Config) (*Conn, error)Dial connects to the given network address using net.Dial and then initiates a TLS handshake, returning the resulting TLS connection. Dial interprets a nil configuration as equivalent to the zero configuration; see the documentation of Config for the defaults.
应用
通过golang生成pem
github上的代码：https://github.com/levigross/go-mutual-tls/blob/master/generate_client_cert.go
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Generate a self-signed X.509 certificate for a TLS server. Outputs to
// 'cert.pem' and 'key.pem' and will overwrite existing files.
package main
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"flag"
"fmt"
"log"
"math/big"
"os"
"time"
)
var (
emailAddress = flag.String("email-address", "", "The email address of the user you wish to create the certificate for")
validFrom = flag.String("start-date", "", "Creation date formatted as Jan 1 15:04:05 2011")
validFor = flag.Duration("duration", 365*24*time.Hour, "Duration that certificate is valid for")
isCA = flag.Bool("ca", false, "whether this cert should be its own Certificate Authority")
rsaBits = flag.Int("rsa-bits", 2048, "Size of RSA key to generate. Ignored if --ecdsa-curve is set")
ecdsaCurve = flag.String("ecdsa-curve", "", "ECDSA curve to use to generate a key. Valid values are P224, P256, P384, P521")
)
func publicKey(priv interface{}) interface{} {
switch k := priv.(type) {
case *rsa.PrivateKey:
return &k.PublicKey
case *ecdsa.PrivateKey:
return &k.PublicKey
default:
return nil
}
}
func pemBlockForKey(priv interface{}) *pem.Block {
switch k := priv.(type) {
case *rsa.PrivateKey:
return &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(k)}
case *ecdsa.PrivateKey:
b, err := x509.MarshalECPrivateKey(k)
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to marshal ECDSA private key: %v", err)
os.Exit(2)
}
return &pem.Block{Type: "EC PRIVATE KEY", Bytes: b}
default:
return nil
}
}
func main() {
flag.Parse()
if len(*emailAddress) == 0 {
log.Fatalf("Missing required --email-address parameter")
}
var priv interface{}
var err error
switch *ecdsaCurve {
case "":
priv, err = rsa.GenerateKey(rand.Reader, *rsaBits)
case "P224":
priv, err = ecdsa.GenerateKey(elliptic.P224(), rand.Reader)
case "P256":
priv, err = ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
case "P384":
priv, err = ecdsa.GenerateKey(elliptic.P384(), rand.Reader)
case "P521":
priv, err = ecdsa.GenerateKey(elliptic.P521(), rand.Reader)
default:
fmt.Fprintf(os.Stderr, "Unrecognized elliptic curve: %q", *ecdsaCurve)
os.Exit(1)
}
if err != nil {
log.Fatalf("failed to generate private key: %s", err)
}
var notBefore time.Time
if len(*validFrom) == 0 {
notBefore = time.Now()
} else {
notBefore, err = time.Parse("Jan 2 15:04:05 2006", *validFrom)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to parse creation date: %s\n", err)
os.Exit(1)
}
}
notAfter := notBefore.Add(*validFor)
serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128)
serialNumber, err := rand.Int(rand.Reader, serialNumberLimit)
if err != nil {
log.Fatalf("failed to generate serial number: %s", err)
}
template := x509.Certificate{
SerialNumber: serialNumber,
Subject: pkix.Name{
Organization: []string{"Acme Co"},
},
NotBefore: notBefore,
NotAfter: notAfter,
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth,
x509.ExtKeyUsageClientAuth},
BasicConstraintsValid: true,
}
template.DNSNames = append(template.DNSNames, "localhost")
template.EmailAddresses = append(template.EmailAddresses, *emailAddress)
if *isCA {
template.IsCA = true
template.KeyUsage |= x509.KeyUsageCertSign
}
derBytes, err := x509.CreateCertificate(rand.Reader, &template, &template, publicKey(priv), priv)
if err != nil {
log.Fatalf("Failed to create certificate: %s", err)
}
certOut, err := os.Create("cert.pem")
if err != nil {
log.Fatalf("failed to open cert.pem for writing: %s", err)
}
pem.Encode(certOut, &pem.Block{Type: "CERTIFICATE", Bytes: derBytes})
certOut.Close()
log.Print("written cert.pem\n")
keyOut, err := os.OpenFile("key.pem", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
log.Print("failed to open key.pem for writing:", err)
return
}
pem.Encode(keyOut, pemBlockForKey(priv))
keyOut.Close()
log.Print("written key.pem\n")
}golang中使用HTTPS
package main
import (
"log"
"net/http"
)
func HelloServer(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "text/plain")
w.Write([]byte("This is an example using https in golang.\n"))
}
func main() {
http.HandleFunc("/hello", HelloServer)
err := http.ListenAndServeTLS(":443", "server.crt", "server.key", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
浏览器访问：
https://localhost/hello
不要疑问，出现了访问12306的效果，很正常，因为这是我们自己做的证书。
关于 为何从12306.cn订票时浏览器总是提醒证书不受信任？请看知乎上的讨论，很精彩：
https://www.zhihu.com/question/25334635
golang中使用tls
server.go
package main
import (
"bufio"
"crypto/tls"
"log"
"net"
)
func main() {
log.SetFlags(log.Lshortfile)
cer, err := tls.LoadX509KeyPair("server.crt", "server.key")
if err != nil {
log.Println(err)
return
}
config := &tls.Config{Certificates: []tls.Certificate{cer}}
ln, err := tls.Listen("tcp", ":443", config)
if err != nil {
log.Println(err)
return
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
log.Println(err)
continue
}
go handleConnection(conn)
}
}
func handleConnection(conn net.Conn) {
defer conn.Close()
r := bufio.NewReader(conn)
for {
msg, err := r.ReadString('\n')
if err != nil {
log.Println(err)
return
}
println(msg)
n, err := conn.Write([]byte("jude\n"))
if err != nil {
log.Println(n, err)
return
}
}
}
client.go
package main
import (
"crypto/tls"
"log"
)
func main() {
log.SetFlags(log.Lshortfile)
conf := &tls.Config{
InsecureSkipVerify: true,
}
conn, err := tls.Dial("tcp", "127.0.0.1:443", conf)
if err != nil {
log.Println(err)
return
}
defer conn.Close()
n, err := conn.Write([]byte("hi\n"))
if err != nil {
log.Println(n, err)
return
}
buf := make([]byte, 100)
n, err = conn.Read(buf)
if err != nil {
log.Println(n, err)
return
}
println(string(buf[:n]))
}
这里编写client代码时候需要注意：InsecureSkipVerify: true
也就是说上面的代码中客户端不对服务端的证书进行验证。
go实现的Client端默认也是要对服务端传过来的数字证书进行校验的，但客户端提示：这个证书是由不知名CA签发的！
