MQTT-mosquitto学习笔记
文章目录
- 1. 编译源码
-
- 安装依赖
- 编译安装
- 2. 基本使用
-
- 2.1 配置
- 2.2 Demo
- 2.3 非匿名
- 2.4 TLS安全通信
-
- 单向验证
- 双向验证
- 3. 指定QoS
-
- QoS 0
- QoS 1
- QoS 2
- 4. 源码分析
-
- mosquitto
- 5. 参考资料
1. 编译源码
官网：https://mosquitto.org/
本次实验编译的版本是v3.1.1。
安装依赖
源码目录README-compileing.md。
我在编译时缺少了以下库：
# cJson
# https://github.com/DaveGamble/cJSON
unzip cJSON-1.7.15.zip
cd cJSON-1.7.15/
mkdir build
cd build
cmake ..
make
make install
# xsltproc
# docbook-xsl
sudo apt-get install xsltproc docbook-xsl
编译安装
make && make install
或者用源码目录下的cmake, 如果要调试的话，需要在CMakeLists.txt里面加上可调式配置：
SET(CMAKE_CXX_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g -ggdb")
SET(CMAKE_CXX_FLAGS_RELEASE "$ENV{CXXFLAGS} -O3 -Wall")
然后编译安装就好了：
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Debug ..
make
make install
经过测试，cmake的方法调试起来比较顺利一些，直接用makefile编译的话调试起来源码行数不太匹配。
主要是获取了以下可执行文件：
$ ll /usr/local/bin/mos*
-rwxr-xr-x 1 root root 198672 4月 11 14:52 /usr/local/bin/mosquitto_ctrl* # 管理工具
-rwxr-xr-x 1 root root 88584 4月 11 14:52 /usr/local/bin/mosquitto_passwd* # 管理账户密码
-rwxr-xr-x 1 root root 158952 4月 11 14:52 /usr/local/bin/mosquitto_pub* # 发布客户端
-rwxr-xr-x 1 root root 192912 4月 11 14:52 /usr/local/bin/mosquitto_rr* # 发送请求、接收响应的客户端
-rwxr-xr-x 1 root root 189368 4月 11 14:52 /usr/local/bin/mosquitto_sub* # 订阅客户端
$ ll /usr/local/sbin/mos*
-rwxr-xr-x 1 root root 1809264 4月 11 14:52 /usr/local/sbin/mosquitto* # broker
安装完后需要执行一下sudo ldconfig，不然搜不到/usr/loca/lib下新安装的so。
2. 基本使用
2.1 配置
$ ll /etc/mosquitto/
total 68
drwxr-xr-x 2 root root 4096 4月 11 14:52 ./
drwxr-xr-x 137 root root 12288 4月 11 14:52 ../
-rw-r--r-- 1 root root 230 4月 11 14:52 aclfile.example # 权限配置，
-rw-r--r-- 1 root root 40142 4月 11 14:52 mosquitto.conf.example
-rw-r--r-- 1 root root 23 4月 11 14:52 pskfile.example
-rw-r--r-- 1 root root 355 4月 11 14:52 pwfile.example
执行mosquitto代理时 -c 指定mosquitto.conf即可，不指定则默认监听1883端口。
2.2 Demo
Broker:
$ mosquitto
1649669075: mosquitto version 2.0.14 starting
1649669075: Using default config.
1649669075: Starting in local only mode. Connections will only be possible from clients running on this machine.
1649669075: Create a configuration file which defines a listener to allow remote access.
1649669075: For more details see https://mosquitto.org/documentation/authentication-methods/
1649669075: Opening ipv4 listen socket on port 1883.
1649669075: Opening ipv6 listen socket on port 1883.
1649669075: mosquitto version 2.0.14 running
订阅（-t指定topic）：
mosquitto_sub -t foobar [-h localhost] -v
发布（-m指定message）：
mosquitto_pub -t foobar [-h localhost] [-p 1883] -m "hello"
然后订阅端就会收到foobar hello消息（格式 Topic Msg），再看看服务端日志：
1649669420: New connection from 127.0.0.1:43066 on port 1883.
1649669420: New client connected from 127.0.0.1:43066 as auto-11ADBEA9-7454-58EF-31EF-03D7916A9C06 (p2, c1, k60).
1649669448: New connection from 127.0.0.1:43068 on port 1883.
1649669448: New client connected from 127.0.0.1:43068 as auto-B93201BC-7CD1-F179-ECCA-4F33181AF365 (p2, c1, k60).
2.3 非匿名
可以参数指定：
-u username
-P password
也可以改配置mosquitto.conf：
# Defaults to false, unless there are no listeners defined in the configuration
# file, in which case it is set to true, but connections are only allowed from
# the local machine.
allow_anonymous false
# See the TLS client require_certificate and use_identity_as_username options
# for alternative authentication options. If a plugin is used as well as
# password_file, the plugin check will be made first.
password_file /etc/mosquitto/pwfile
# If an plugin is used as well as acl_file, the plugin check will be
# made first.
acl_file /etc/mosquitto/aclfile
新增账户：
$ sudo mosquitto_passwd -b /etc/mosquitto/pwfile foo 123
$ sudo mosquitto_passwd -b /etc/mosquitto/pwfile bar 123
$ cat /etc/mosquitto/pwfile
foo:$7$101$cEsUq8j7617mzd+6$jT757TUWXKg3MlNCAhi5wCHATk8I6pNMG7dumaA2GdExL60v9FmwtC8nMuXb0YmM1bx+myGCCN82hD2w1HF3xw==
bar:$7$101$mgWeseFJ6T7wDkpk$K8/S4/Kmr70q4uFVUfu3MgdPjDd8boARSK8uguOyLmEmuvmXq6mRPbhOmPnokNQFarum4YVmMj0z/b+SQNTk6Q==
访问权限aclfile:
# read 订阅权限 、write 发布权限、# 通配符表示所有的
user foo
topic read Topic
user bar
topic write Topic
启动broker：
$ mosquitto -c /etc/mosquitto/mosquitto.conf
使用错误的账户订阅，会失败：
$ mosquitto_sub -t Topic -v -d -u aaa
Client null sending CONNECT
Client null received CONNACK (5)
Connection error: Connection Refused: not authorised.
Client null sending DISCONNECT
必须用acl允许的账户密码（foo/123）才行：
$ mosquitto_sub -t Topic -v -d -u foo -P 123
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
发布也是:
$ mosquitto_pub -t Topic -m "hello" -u bar
Connection error: Connection Refused: not authorised.
Error: The connection was refused.
$ mosquitto_pub -t Topic -m "hello" -u bar -P 123
订阅者成功收到消息：
Client null received PUBLISH (d0, q0, r0, m0, 'Topic', ... (5 bytes))
Topic hello
2.4 TLS安全通信
MQTT安全通信有3种方法：
- 网络层采用VPN专线；
- 传输层使用TLS加密，实现数据加密和身份认证；
- 应用层采用账户密码验证。
本节讲解TLS的方法，需要安装openssl：
sudo apt install openssl libssl-dev
在/etc/mosquitto/执行脚本make_tls.sh：
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of the axTLS project nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# Generate the certificates and keys for testing.
#
PROJECT_NAME="TLS Project"
# Generate the openssl configuration files.
cat > ca_cert.conf << EOF
[ req ]
distinguished_name = req_distinguished_name
prompt = no
[ req_distinguished_name ]
O = $PROJECT_NAME Dodgy Certificate Authority
EOF
cat > server_cert.conf << EOF
[ req ]
distinguished_name = req_distinguished_name
prompt = no
[ req_distinguished_name ]
O = $PROJECT_NAME
CN = 192.168.56.101
EOF
cat > client_cert.conf << EOF
[ req ]
distinguished_name = req_distinguished_name
prompt = no
[ req_distinguished_name ]
O = $PROJECT_NAME Device Certificate
CN = 192.168.56.101
EOF
mkdir ca
mkdir server
mkdir client
mkdir certDER
# private key generation
openssl genrsa -out ca.key 1024
openssl genrsa -out server.key 1024
openssl genrsa -out client.key 1024
# 想从证书颁发机构certificate authority（CA）那里获得 SSL 证书，就必须生成一个证书签署请求certificate signing request（CSR）
openssl req -out ca.csr -key ca.key -new \
-config ./ca_cert.conf
openssl req -out server.csr -key server.key -new \
-config ./server_cert.conf
openssl req -out client.csr -key client.key -new \
-config ./client_cert.conf
# generate the actual certs.
openssl x509 -req -in ca.csr -out ca.crt \
-sha1 -days 5000 -signkey ca.key
openssl x509 -req -in server.csr -out server.crt \
-sha1 -CAcreateserial -days 5000 \
-CA ca.crt -CAkey ca.key
openssl x509 -req -in client.csr -out client.crt \
-sha1 -CAcreateserial -days 5000 \
-CA ca.crt -CAkey ca.key
# 将 PEM 编码的证书.crt转换为 DER 编码的证书.der
# DER 格式通常与 Java 一起使用。
openssl x509 -in ca.crt -outform DER -out ca.der
openssl x509 -in server.crt -outform DER -out server.der
openssl x509 -in client.crt -outform DER -out client.der
mv ca.crt ca.key ca/
mv server.crt server.key server/
mv client.crt client.key client/
mv ca.der server.der client.der certDER/
chmod 644 ca/ca.key
chmod 644 server/server.key
chmod 644 client/client.key
rm *cert.conf
rm *.csr
rm *.srl
生成的文件有证书（.crt），私钥（.key）。
测试一下：
$ openssl verify -CAfile ca/ca.crt server/server.crt
server/server.crt: OK
$ openssl verify -CAfile ca/ca.crt client/client.crt
client/client.crt: OK
单向验证
单向的意思是，仅客户端验证服务端返回的ca证书。
再次配置/etc/mosquitto/mosquitto.conf:
# Note that the recommended port for MQTT over TLS is 8883,
# but this must be set manually.
port 8883
cafile /etc/mosquitto/ca/ca.crt
# Path to the PEM encoded server certificate.
certfile /etc/mosquitto/server/server.crt
# Path to the PEM encoded keyfile.
keyfile /etc/mosquitto/server/server.key
use_identity_as_username false
require_certificate false
启动broker：
$ sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v
订阅：
$ mosquitto_sub -t Topic -d -u foo -P 123
Error: Connection refused
$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0
发布：
mosquitto_pub -t Topic -m "hello" -u bar -P 123 --cafile /etc/mosquitto/ca/ca.crt
双向验证
Client验证服务器端的证书，Server验证客户端证书。
改mosquitto.conf的两个字段：
use_identity_as_username true
require_certificate true
启动Broker:
$ sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v
订阅：
$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt
Client null sending CONNECT
Error: Protocol error
$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt --cert /etc/mosquitto/client/client.crt --key /etc/mosquitto/client/client.key
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0
发布：
$ mosquitto_pub -t Topic -m "hello" -u bar -P 123 --cafile /etc/mosquitto/ca/ca.crt --cert /etc/mosquitto/client/client.crt --key /etc/mosquitto/client/client.key
3. 指定QoS
接下来会通过抓包来分析mqtt，所以要把tls去掉，只保留用户信息即可，mosquitto.conf如下：
keyfile /etc/mosquitto/server/server.key
password_file /etc/mosquitto/pwfile
acl_file /etc/mosquitto/aclfile
启动Broker：
sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v
这部分内容，需要结合文档第3章来分析。
QoS 0
mosquitto_pub -q参数可以指定QoS（默认0）。
订阅：
$ mosquitto_sub -t Topic -d -u foo -P 123 -v -q 0
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0
第一个由订阅端发出的CONNECT包是最长的，注意Msg Len这22个字节是指 【可变头(截图两个框中间的部分)+Payload】。Payload主要就是客户端ID，明文账户名和口令。
简单说下Connect Flags中的will flag，可以翻译成遗嘱消息，如果这个bit设为1，那么在网络连接非正常关闭的情况下，服务端就会发布遗嘱消息。
订阅的主题和QoS，则是在第3个包，SUBSCRIBE 里：
MQ Telemetry Transport Protocol, Subscribe Request
Header Flags: 0x82, Message Type: Subscribe Request
1000 .... = Message Type: Subscribe Request (8)
.... 0010 = Reserved: 2
Msg Len: 10
Message Identifier: 1
Topic Length: 5
Topic: Topic
Requested QoS: At most once delivery (Fire and Forget) (0)
发布：
$ mosquitto_pub -t foobar -m "hello" -d
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q0, r0, m1, 'foobar', ... (5 bytes))
Client null sending DISCONNECT
增加了7个包，最后两个心跳包是碰巧抓到的，排除掉。
- 前两个是发布端与Broker的connect交互，和订阅时差不多；
- 然后是两个publish发布包，第一个是发布端给Broker的（因为三方都在一个虚拟机里，所以都是127.0.0.1），第二个是Broker发给订阅端的；
- 最后是发布端与Broker断开连接的包，断开的理由官网提供了很多，但我这里只有一个Fixed Header，没有Variable Header，以后碰到再说吧。
QoS 1
订阅：
mosquitto_sub -t Topic -d -u foo -P 123 -v -q 1
仍然是4个数据包，只不过SUBSCRIBE包的QoS字节变成了1。
发布：
$ mosquitto_pub -t Topic -m "hello" -u bar -P 123 -d -q 1
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q1, r0, m1, 'Topic', ... (5 bytes))
Client null received PUBACK (Mid: 1, RC:0)
Client null sending DISCONNECT
从调试信息可以看出，多出了一个PUBACK响应。
从抓到的数据包来看，也是多了两个PUBACK响应包。
从broker日志可以看出，
1649749139: New connection from 127.0.0.1:52304 on port 1883.
1649749139: New client connected from 127.0.0.1:52304 as auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (p2, c1, k60, u'bar').
1649749139: No will message specified.
1649749139: Sending CONNACK to auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (0, 0)
1649749139: Received PUBLISH from auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (d0, q1, r0, m1, 'Topic', ... (5 bytes)) # 收到发布端消息
1649749139: Sending PUBLISH to auto-60541609-17C4-B477-7348-50276571D3C1 (d0, q1, r0, m1, 'Topic', ... (5 bytes)) # 发给订阅端
1649749139: Sending PUBACK to auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (m1, rc0)
# 发给发布端PUBACK
1649749139: Received PUBACK from auto-60541609-17C4-B477-7348-50276571D3C1 (Mid: 1, RC:0)
# 收到订阅端的PUBACK
1649749139: Received DISCONNECT from auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278
1649749139: Client auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 disconnected.
# 断开与发布端的连接
QoS 2
QoS 2因为往来的数据包太多，为了方便分析，订阅端使用QoS 0。
发布：
mosquitto_pub -t Topic -m "hello" -u bar -P 123 -d -q 2
先看下broker日志：
1649749959: New connection from 127.0.0.1:53004 on port 1883.
1649749959: New client connected from 127.0.0.1:53004 as auto-FF7F8A46-058D-A421-0856-B629C0247399 (p2, c1, k60, u'foo').
1649749959: No will message specified.
1649749959: Sending CONNACK to auto-FF7F8A46-058D-A421-0856-B629C0247399 (0, 0)
1649749959: Received SUBSCRIBE from auto-FF7F8A46-058D-A421-0856-B629C0247399
1649749959: Topic (QoS 0)
1649749959: auto-FF7F8A46-058D-A421-0856-B629C0247399 0 Topic
1649749959: Sending SUBACK to auto-FF7F8A46-058D-A421-0856-B629C0247399
# 以上是订阅端
1649749969: New connection from 127.0.0.1:53006 on port 1883.
1649749969: New client connected from 127.0.0.1:53006 as auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (p2, c1, k60, u'bar').
1649749969: No will message specified.
1649749969: Sending CONNACK to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (0, 0)
1649749969: Received PUBLISH from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (d0, q2, r0, m1, 'Topic', ... (5 bytes))
# 收到发布端消息
1649749969: Sending PUBREC to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (m1, rc0)
1649749969: Received PUBREL from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (Mid: 1)
# PUBREL 是发布端对 PUBREC 的响应
1649749969: Sending PUBLISH to auto-FF7F8A46-058D-A421-0856-B629C0247399 (d0, q0, r0, m0, 'Topic', ... (5 bytes))
# 发给订阅端
1649749969: Sending PUBCOMP to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (m1)
# PUBCOMP 是Broker对 PUBREL 的响应
1649749969: Received DISCONNECT from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD
1649749969: Client auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD disconnected.
# 与发布端断开连接
再看下数据包，与日志对的上：
这里有个细节，两个publish message的长度不一样（82和80），这两个字节是可变头（Variable header）的Packet Identifier标识符字段，在QoS 1和2两个服务等级里才存在，用来匹配类似PUBLISH和PUBACK这样的请求/响应包，具体可参考文档2.3.1.
整理一下QoS2的流程图：
publisher broker PUBLISH Received(PUBREC) Released(PUBREL) Completed(PUBCOMP) publisher broker
4. 源码分析
mosquitto
主函数里的主循环及响应处理：
rc = mosquitto_main_loop(listensock, listensock_count);
int mosquitto_main_loop(struct mosquitto__listener_sock *listensock, int listensock_count)
{
//...
rc = mux__handle(listensock, listensock_count);
//...
}
调试过程中肯定会超时，比如我用订阅端发送订阅请求，QoS为0，那么超时后订阅端会多次发送CONNECT请求。
Broker响应CONNACK的函数栈：
net__write(struct mosquitto * mosq, const void * buf, size_t count) (\home\starr\Documents\CProject\mosquitto\lib\net_mosq.c:1045)
packet__write(struct mosquitto * mosq) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:252)
packet__queue(struct mosquitto * mosq, struct mosquitto__packet * packet) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:172)
send__connack(struct mosquitto * context, uint8_t ack, uint8_t reason_code, const mosquitto_property * properties) (\home\starr\Documents\CProject\mosquitto\src\send_connack.c:108)
connect__on_authorised(struct mosquitto * context, void * auth_data_out, uint16_t auth_data_out_len) (\home\starr\Documents\CProject\mosquitto\src\handle_connect.c:314)
handle__connect(struct mosquitto * context) (\home\starr\Documents\CProject\mosquitto\src\handle_connect.c:929)
handle__packet(struct mosquitto * context) (\home\starr\Documents\CProject\mosquitto\src\read_handle.c:64)
packet__read(struct mosquitto * mosq) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:553)
loop_handle_reads_writes(struct mosquitto * context, uint32_t events) (\home\starr\Documents\CProject\mosquitto\src\mux_epoll.c:295)
mux_epoll__handle() (\home\starr\Documents\CProject\mosquitto\src\mux_epoll.c:207)
mux__handle(struct mosquitto__listener_sock * listensock, int listensock_count) (\home\starr\Documents\CProject\mosquitto\src\mux.c:76)
mosquitto_main_loop(struct mosquitto__listener_sock * listensock, int listensock_count) (\home\starr\Documents\CProject\mosquitto\src\loop.c:205)
main(int argc, char ** argv) (\home\starr\Documents\CProject\mosquitto\src\mosquitto.c:562)
libc.so.6!__libc_start_main(int (*)(int, char **, char **) main, int argc, char ** argv, int (*)(int, char **, char **) init, void (*)(void) fini, void (*)(void) rtld_fini, void * stack_end) (\build\glibc-uZu3wS\glibc-2.27\csu\libc-start.c:310)
_start (Unknown Source:0)