MQTT-mosquitto学习笔记

文章目录

  • 1. 编译源码
    • 安装依赖
    • 编译安装
  • 2. 基本使用
    • 2.1 配置
    • 2.2 Demo
    • 2.3 非匿名
    • 2.4 TLS安全通信
      • 单向验证
      • 双向验证
  • 3. 指定QoS
    • QoS 0
    • QoS 1
    • QoS 2
  • 4. 源码分析
    • mosquitto
  • 5. 参考资料

1. 编译源码

官网:https://mosquitto.org/

本次实验编译的版本是v3.1.1。

安装依赖

源码目录README-compileing.md。

我在编译时缺少了以下库:

# cJson
# https://github.com/DaveGamble/cJSON
unzip cJSON-1.7.15.zip
cd cJSON-1.7.15/
mkdir build
cd build
cmake ..
make
make install

# xsltproc 
# docbook-xsl
sudo apt-get install xsltproc docbook-xsl

编译安装

make && make install

或者用源码目录下的cmake, 如果要调试的话,需要在CMakeLists.txt里面加上可调式配置:

SET(CMAKE_CXX_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g -ggdb")
SET(CMAKE_CXX_FLAGS_RELEASE "$ENV{CXXFLAGS} -O3 -Wall")

然后编译安装就好了:

mkdir build && cd build 
cmake -DCMAKE_BUILD_TYPE=Debug ..
make
make install

经过测试,cmake的方法调试起来比较顺利一些,直接用makefile编译的话调试起来源码行数不太匹配。

主要是获取了以下可执行文件:

$ ll /usr/local/bin/mos*
-rwxr-xr-x 1 root root 198672 411 14:52 /usr/local/bin/mosquitto_ctrl*	# 管理工具
-rwxr-xr-x 1 root root  88584 411 14:52 /usr/local/bin/mosquitto_passwd*	# 管理账户密码
-rwxr-xr-x 1 root root 158952 411 14:52 /usr/local/bin/mosquitto_pub*	# 发布客户端
-rwxr-xr-x 1 root root 192912 411 14:52 /usr/local/bin/mosquitto_rr*		# 发送请求、接收响应的客户端
-rwxr-xr-x 1 root root 189368 411 14:52 /usr/local/bin/mosquitto_sub*	# 订阅客户端
$ ll /usr/local/sbin/mos*
-rwxr-xr-x 1 root root 1809264 411 14:52 /usr/local/sbin/mosquitto*		# broker

安装完后需要执行一下sudo ldconfig,不然搜不到/usr/loca/lib下新安装的so。

2. 基本使用

2.1 配置

$ ll /etc/mosquitto/
total 68
drwxr-xr-x   2 root root  4096 411 14:52 ./
drwxr-xr-x 137 root root 12288 411 14:52 ../
-rw-r--r--   1 root root   230 411 14:52 aclfile.example	# 权限配置,
-rw-r--r--   1 root root 40142 411 14:52 mosquitto.conf.example
-rw-r--r--   1 root root    23 411 14:52 pskfile.example
-rw-r--r--   1 root root   355 411 14:52 pwfile.example

执行mosquitto代理时 -c 指定mosquitto.conf即可,不指定则默认监听1883端口。

2.2 Demo

Broker:

$ mosquitto
1649669075: mosquitto version 2.0.14 starting
1649669075: Using default config.
1649669075: Starting in local only mode. Connections will only be possible from clients running on this machine.
1649669075: Create a configuration file which defines a listener to allow remote access.
1649669075: For more details see https://mosquitto.org/documentation/authentication-methods/
1649669075: Opening ipv4 listen socket on port 1883.
1649669075: Opening ipv6 listen socket on port 1883.
1649669075: mosquitto version 2.0.14 running

订阅(-t指定topic):

mosquitto_sub -t foobar [-h localhost] -v

发布(-m指定message):

mosquitto_pub -t foobar [-h localhost] [-p 1883] -m "hello"

然后订阅端就会收到foobar hello消息(格式 Topic Msg),再看看服务端日志:

1649669420: New connection from 127.0.0.1:43066 on port 1883.
1649669420: New client connected from 127.0.0.1:43066 as auto-11ADBEA9-7454-58EF-31EF-03D7916A9C06 (p2, c1, k60).
1649669448: New connection from 127.0.0.1:43068 on port 1883.
1649669448: New client connected from 127.0.0.1:43068 as auto-B93201BC-7CD1-F179-ECCA-4F33181AF365 (p2, c1, k60).

2.3 非匿名

可以参数指定:

-u username
-P password

也可以改配置mosquitto.conf:

# Defaults to false, unless there are no listeners defined in the configuration
# file, in which case it is set to true, but connections are only allowed from
# the local machine.
allow_anonymous false

# See the TLS client require_certificate and use_identity_as_username options
# for alternative authentication options. If a plugin is used as well as
# password_file, the plugin check will be made first.
password_file /etc/mosquitto/pwfile

# If an plugin is used as well as acl_file, the plugin check will be
# made first.
acl_file /etc/mosquitto/aclfile

新增账户:

$ sudo mosquitto_passwd -b /etc/mosquitto/pwfile foo 123
$ sudo mosquitto_passwd -b /etc/mosquitto/pwfile bar 123
$ cat /etc/mosquitto/pwfile
foo:$7$101$cEsUq8j7617mzd+6$jT757TUWXKg3MlNCAhi5wCHATk8I6pNMG7dumaA2GdExL60v9FmwtC8nMuXb0YmM1bx+myGCCN82hD2w1HF3xw==
bar:$7$101$mgWeseFJ6T7wDkpk$K8/S4/Kmr70q4uFVUfu3MgdPjDd8boARSK8uguOyLmEmuvmXq6mRPbhOmPnokNQFarum4YVmMj0z/b+SQNTk6Q==

访问权限aclfile:

# read 订阅权限 、write 发布权限、# 通配符表示所有的
user foo
topic read Topic

user bar
topic write Topic

启动broker:

$ mosquitto -c /etc/mosquitto/mosquitto.conf

使用错误的账户订阅,会失败:

$ mosquitto_sub -t Topic -v -d -u aaa
Client null sending CONNECT
Client null received CONNACK (5)
Connection error: Connection Refused: not authorised.
Client null sending DISCONNECT

必须用acl允许的账户密码(foo/123)才行:

$ mosquitto_sub -t Topic -v -d -u foo -P 123
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK

发布也是:

$ mosquitto_pub -t Topic -m "hello" -u bar
Connection error: Connection Refused: not authorised.
Error: The connection was refused.
$ mosquitto_pub -t Topic -m "hello" -u bar -P 123

订阅者成功收到消息:

Client null received PUBLISH (d0, q0, r0, m0, 'Topic', ... (5 bytes))
Topic hello

2.4 TLS安全通信

MQTT安全通信有3种方法:

  • 网络层采用VPN专线;
  • 传输层使用TLS加密,实现数据加密和身份认证;
  • 应用层采用账户密码验证。

本节讲解TLS的方法,需要安装openssl:

sudo apt install openssl libssl-dev

/etc/mosquitto/执行脚本make_tls.sh:

# * Redistributions in binary form must reproduce the above copyright
#   notice, this list of conditions and the following disclaimer in the
#   documentation and/or other materials provided with the distribution.
# * Neither the name of the axTLS project nor the names of its
#   contributors may be used to endorse or promote products derived
#   from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#
# Generate the certificates and keys for testing.
#

PROJECT_NAME="TLS Project"
  
# Generate the openssl configuration files.
cat > ca_cert.conf << EOF 
[ req ]
distinguished_name     = req_distinguished_name
prompt                 = no
[ req_distinguished_name ]
 O                      = $PROJECT_NAME Dodgy Certificate Authority
EOF

  
cat > server_cert.conf << EOF 
[ req ]
distinguished_name     = req_distinguished_name
prompt                 = no
[ req_distinguished_name ]
 O                      = $PROJECT_NAME
 CN                     = 192.168.56.101
EOF

cat > client_cert.conf << EOF 
[ req ]
distinguished_name     = req_distinguished_name
prompt                 = no
[ req_distinguished_name ]
 O                      = $PROJECT_NAME Device Certificate
 CN                     = 192.168.56.101
EOF


mkdir ca
mkdir server
mkdir client
mkdir certDER


# private key generation
openssl genrsa -out ca.key 1024
openssl genrsa -out server.key 1024
openssl genrsa -out client.key 1024


# 想从证书颁发机构certificate authority(CA)那里获得 SSL 证书,就必须生成一个证书签署请求certificate signing request(CSR)
openssl req -out ca.csr -key ca.key -new \
            -config ./ca_cert.conf
openssl req -out server.csr -key server.key -new \
            -config ./server_cert.conf
openssl req -out client.csr -key client.key -new \
            -config ./client_cert.conf
  
# generate the actual certs.
openssl x509 -req -in ca.csr -out ca.crt \
            -sha1 -days 5000 -signkey ca.key
openssl x509 -req -in server.csr -out server.crt \
            -sha1 -CAcreateserial -days 5000 \
            -CA ca.crt -CAkey ca.key
openssl x509 -req -in client.csr -out client.crt \
            -sha1 -CAcreateserial -days 5000 \
            -CA ca.crt -CAkey ca.key

# 将 PEM 编码的证书.crt转换为 DER 编码的证书.der
# DER 格式通常与 Java 一起使用。
openssl x509 -in ca.crt -outform DER -out ca.der
openssl x509 -in server.crt -outform DER -out server.der
openssl x509 -in client.crt -outform DER -out client.der


mv ca.crt ca.key ca/
mv server.crt server.key server/
mv client.crt client.key client/
mv ca.der server.der client.der certDER/

chmod 644 ca/ca.key
chmod 644 server/server.key
chmod 644 client/client.key

rm *cert.conf
rm *.csr
rm *.srl

生成的文件有证书(.crt),私钥(.key)。

测试一下:

$ openssl verify -CAfile ca/ca.crt server/server.crt
server/server.crt: OK
$ openssl verify -CAfile ca/ca.crt client/client.crt
client/client.crt: OK

单向验证

单向的意思是,仅客户端验证服务端返回的ca证书。

再次配置/etc/mosquitto/mosquitto.conf:

# Note that the recommended port for MQTT over TLS is 8883,
# but this must be set manually.
port 8883

cafile /etc/mosquitto/ca/ca.crt

# Path to the PEM encoded server certificate.
certfile /etc/mosquitto/server/server.crt

# Path to the PEM encoded keyfile.
keyfile /etc/mosquitto/server/server.key

use_identity_as_username false
require_certificate false

启动broker:

$ sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v

订阅:

$ mosquitto_sub -t Topic -d -u foo -P 123
Error: Connection refused
$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

发布:

mosquitto_pub -t Topic -m "hello" -u bar -P 123 --cafile /etc/mosquitto/ca/ca.crt

双向验证

Client验证服务器端的证书,Server验证客户端证书。

改mosquitto.conf的两个字段:

use_identity_as_username true
require_certificate true

启动Broker:

$ sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v

订阅:

$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt
Client null sending CONNECT
Error: Protocol error
$ mosquitto_sub -t Topic -d -u foo -P 123 --cafile /etc/mosquitto/ca/ca.crt --cert /etc/mosquitto/client/client.crt --key /etc/mosquitto/client/client.key
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

发布:

$ mosquitto_pub -t Topic -m "hello" -u bar -P 123 --cafile /etc/mosquitto/ca/ca.crt --cert /etc/mosquitto/client/client.crt --key /etc/mosquitto/client/client.key

3. 指定QoS

接下来会通过抓包来分析mqtt,所以要把tls去掉,只保留用户信息即可,mosquitto.conf如下:

keyfile /etc/mosquitto/server/server.key
password_file /etc/mosquitto/pwfile
acl_file /etc/mosquitto/aclfile

启动Broker:

sudo mosquitto -c /etc/mosquitto/mosquitto.conf -v

这部分内容,需要结合文档第3章来分析。

QoS 0

mosquitto_pub -q参数可以指定QoS(默认0)。

订阅:

$ mosquitto_sub -t Topic -d -u foo -P 123 -v -q 0
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: Topic, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

MQTT-mosquitto学习笔记_第1张图片

第一个由订阅端发出的CONNECT包是最长的,注意Msg Len这22个字节是指 【可变头(截图两个框中间的部分)+Payload】。Payload主要就是客户端ID,明文账户名和口令。

简单说下Connect Flags中的will flag,可以翻译成遗嘱消息,如果这个bit设为1,那么在网络连接非正常关闭的情况下,服务端就会发布遗嘱消息。

订阅的主题和QoS,则是在第3个包,SUBSCRIBE 里:

MQ Telemetry Transport Protocol, Subscribe Request
    Header Flags: 0x82, Message Type: Subscribe Request
        1000 .... = Message Type: Subscribe Request (8)
        .... 0010 = Reserved: 2
    Msg Len: 10
    Message Identifier: 1
    Topic Length: 5
    Topic: Topic
    Requested QoS: At most once delivery (Fire and Forget) (0)

发布:

$ mosquitto_pub -t foobar -m "hello" -d
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q0, r0, m1, 'foobar', ... (5 bytes))
Client null sending DISCONNECT

MQTT-mosquitto学习笔记_第2张图片

增加了7个包,最后两个心跳包是碰巧抓到的,排除掉。

  • 前两个是发布端与Broker的connect交互,和订阅时差不多;
  • 然后是两个publish发布包,第一个是发布端给Broker的(因为三方都在一个虚拟机里,所以都是127.0.0.1),第二个是Broker发给订阅端的;
  • 最后是发布端与Broker断开连接的包,断开的理由官网提供了很多,但我这里只有一个Fixed Header,没有Variable Header,以后碰到再说吧。

QoS 1

订阅:

mosquitto_sub -t Topic -d -u foo -P 123 -v -q 1

仍然是4个数据包,只不过SUBSCRIBE包的QoS字节变成了1。

发布:

$ mosquitto_pub -t Topic -m "hello" -u bar -P 123 -d -q 1
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q1, r0, m1, 'Topic', ... (5 bytes))
Client null received PUBACK (Mid: 1, RC:0)
Client null sending DISCONNECT

从调试信息可以看出,多出了一个PUBACK响应。

MQTT-mosquitto学习笔记_第3张图片

从抓到的数据包来看,也是多了两个PUBACK响应包。

从broker日志可以看出,

1649749139: New connection from 127.0.0.1:52304 on port 1883.

1649749139: New client connected from 127.0.0.1:52304 as auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (p2, c1, k60, u'bar').
1649749139: No will message specified.
1649749139: Sending CONNACK to auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (0, 0)
1649749139: Received PUBLISH from auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (d0, q1, r0, m1, 'Topic', ... (5 bytes))	# 收到发布端消息

1649749139: Sending PUBLISH to auto-60541609-17C4-B477-7348-50276571D3C1 (d0, q1, r0, m1, 'Topic', ... (5 bytes))	# 发给订阅端

1649749139: Sending PUBACK to auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 (m1, rc0)
# 发给发布端PUBACK

1649749139: Received PUBACK from auto-60541609-17C4-B477-7348-50276571D3C1 (Mid: 1, RC:0)
# 收到订阅端的PUBACK

1649749139: Received DISCONNECT from auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278
1649749139: Client auto-B4E753DD-ABD1-A473-6C42-7D00E7FD2278 disconnected.
# 断开与发布端的连接

QoS 2

QoS 2因为往来的数据包太多,为了方便分析,订阅端使用QoS 0。

发布:

mosquitto_pub -t Topic -m "hello" -u bar -P 123 -d -q 2

先看下broker日志:

1649749959: New connection from 127.0.0.1:53004 on port 1883.
1649749959: New client connected from 127.0.0.1:53004 as auto-FF7F8A46-058D-A421-0856-B629C0247399 (p2, c1, k60, u'foo').
1649749959: No will message specified.
1649749959: Sending CONNACK to auto-FF7F8A46-058D-A421-0856-B629C0247399 (0, 0)
1649749959: Received SUBSCRIBE from auto-FF7F8A46-058D-A421-0856-B629C0247399
1649749959:     Topic (QoS 0)
1649749959: auto-FF7F8A46-058D-A421-0856-B629C0247399 0 Topic
1649749959: Sending SUBACK to auto-FF7F8A46-058D-A421-0856-B629C0247399
# 以上是订阅端


1649749969: New connection from 127.0.0.1:53006 on port 1883.
1649749969: New client connected from 127.0.0.1:53006 as auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (p2, c1, k60, u'bar').
1649749969: No will message specified.
1649749969: Sending CONNACK to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (0, 0)
1649749969: Received PUBLISH from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (d0, q2, r0, m1, 'Topic', ... (5 bytes))
# 收到发布端消息

1649749969: Sending PUBREC to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (m1, rc0)
1649749969: Received PUBREL from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (Mid: 1)
# PUBREL 是发布端对 PUBREC 的响应

1649749969: Sending PUBLISH to auto-FF7F8A46-058D-A421-0856-B629C0247399 (d0, q0, r0, m0, 'Topic', ... (5 bytes))
# 发给订阅端

1649749969: Sending PUBCOMP to auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD (m1)
# PUBCOMP 是Broker对 PUBREL 的响应

1649749969: Received DISCONNECT from auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD
1649749969: Client auto-80721B03-A077-1EE8-BD1B-D9A43CA3C3CD disconnected.
# 与发布端断开连接

再看下数据包,与日志对的上:

MQTT-mosquitto学习笔记_第4张图片

这里有个细节,两个publish message的长度不一样(82和80),这两个字节是可变头(Variable header)的Packet Identifier标识符字段,在QoS 1和2两个服务等级里才存在,用来匹配类似PUBLISH和PUBACK这样的请求/响应包,具体可参考文档2.3.1.

整理一下QoS2的流程图:

publisher broker PUBLISH Received(PUBREC) Released(PUBREL) Completed(PUBCOMP) publisher broker

4. 源码分析

mosquitto

主函数里的主循环及响应处理:

rc = mosquitto_main_loop(listensock, listensock_count);

int mosquitto_main_loop(struct mosquitto__listener_sock *listensock, int listensock_count)
{
    //...
	rc = mux__handle(listensock, listensock_count);
    //...
}

调试过程中肯定会超时,比如我用订阅端发送订阅请求,QoS为0,那么超时后订阅端会多次发送CONNECT请求。

Broker响应CONNACK的函数栈:

net__write(struct mosquitto * mosq, const void * buf, size_t count) (\home\starr\Documents\CProject\mosquitto\lib\net_mosq.c:1045)
packet__write(struct mosquitto * mosq) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:252)
packet__queue(struct mosquitto * mosq, struct mosquitto__packet * packet) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:172)
send__connack(struct mosquitto * context, uint8_t ack, uint8_t reason_code, const mosquitto_property * properties) (\home\starr\Documents\CProject\mosquitto\src\send_connack.c:108)
connect__on_authorised(struct mosquitto * context, void * auth_data_out, uint16_t auth_data_out_len) (\home\starr\Documents\CProject\mosquitto\src\handle_connect.c:314)
handle__connect(struct mosquitto * context) (\home\starr\Documents\CProject\mosquitto\src\handle_connect.c:929)
handle__packet(struct mosquitto * context) (\home\starr\Documents\CProject\mosquitto\src\read_handle.c:64)
packet__read(struct mosquitto * mosq) (\home\starr\Documents\CProject\mosquitto\lib\packet_mosq.c:553)
loop_handle_reads_writes(struct mosquitto * context, uint32_t events) (\home\starr\Documents\CProject\mosquitto\src\mux_epoll.c:295)
mux_epoll__handle() (\home\starr\Documents\CProject\mosquitto\src\mux_epoll.c:207)
mux__handle(struct mosquitto__listener_sock * listensock, int listensock_count) (\home\starr\Documents\CProject\mosquitto\src\mux.c:76)
mosquitto_main_loop(struct mosquitto__listener_sock * listensock, int listensock_count) (\home\starr\Documents\CProject\mosquitto\src\loop.c:205)
main(int argc, char ** argv) (\home\starr\Documents\CProject\mosquitto\src\mosquitto.c:562)
libc.so.6!__libc_start_main(int (*)(int, char **, char **) main, int argc, char ** argv, int (*)(int, char **, char **) init, void (*)(void) fini, void (*)(void) rtld_fini, void * stack_end) (\build\glibc-uZu3wS\glibc-2.27\csu\libc-start.c:310)
_start (Unknown Source:0)

5. 参考资料

你可能感兴趣的:(IOT,iot)