TDengine学习笔记-安装
TDengine学习笔记-安装
-
-
- 前期规划：
- 1.准备安装环境
- 2.下载安装包并安装
- 3.编辑配置文件
- 4.启动数据库
- 5.登录数据库并进行测试
- 测试1
- 测试2
- 6.相关目录与文件
- 写在最后
-
TDengine简介官方描述
TDengine是涛思数据面对高速增长的物联网大数据市场和技术挑战推出的创新性的大数据处理产品，它不依赖任何第三方软件，也不是优化或包装了一个开源的数据库或流式计算产品，而是在吸取众多传统关系型数据库、NoSQL数据库、流式计算引擎、消息队列等软件的优点之后自主开发的产品，在时序空间大数据处理上，有着自己独到的优势。
TDengine的模块之一是时序数据库。但除此之外，为减少研发的复杂度、系统维护的难度，TDengine还提供缓存、消息队列、订阅、流式计算等功能，为物联网、工业互联网大数据的处理提供全栈的技术方案，是一个高效易用的物联网大数据平台。与Hadoop等典型的大数据平台相比，它具有如下鲜明的特点：
- 10倍以上的性能提升
- 强大的分析功能
- 与第三方工具无缝连接
- 零运维成本、零学习成本
[以上描述来自官网]
进入正题
前期规划：
操作系统: CentOS Linux release 7.6 【安装开发包避免缺失依赖包】
关闭SELinux、关闭防火墙
IP：192.168.1.76 ms06.dyq.com ms06
安装包下载
官方安装介绍
官方文档
下载安装包需要提供邮件
随后会收到下载的邮件
虽然官方提供了rpm包，但由于个人习惯，还是选择了二进制安装包。
1.准备安装环境
将IP地址和域名写入/etc/hosts
[root@ms06 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.71 ms01.dyq.com ms01
192.168.1.72 ms02.dyq.com ms02
192.168.1.73 ms03.dyq.com ms03
192.168.1.74 ms04.dyq.com ms04
192.168.1.75 ms05.dyq.com ms05
192.168.1.76 ms06.dyq.com ms06
2.下载安装包并安装
上传安装包
[root@ms06 soft]# ll
total 9180
-rw------- 1 root root 9398005 Mar 6 08:56 TDengine-server-2.0.16.0-Linux-x64.tar.gz
解压缩安装包
[root@ms06 soft]# tar xvf TDengine-server-2.0.16.0-Linux-x64.tar.gz
TDengine-server-2.0.16.0/
TDengine-server-2.0.16.0/connector/
TDengine-server-2.0.16.0/connector/grafanaplugin/
TDengine-server-2.0.16.0/connector/grafanaplugin/package-lock.json
................
TDengine-server-2.0.16.0/examples/python/taosdemo/taosdemo.py
TDengine-server-2.0.16.0/examples/python/taosdemo/README.md
TDengine-server-2.0.16.0/examples/python/taosdemo/requirements.txt
TDengine-server-2.0.16.0/taos.tar.gz
[root@ms06 soft]# ll
total 9180
drwxrwxr-x 5 1000 1000 90 Feb 10 09:27 TDengine-server-2.0.16.0
-rw------- 1 root root 9398005 Mar 6 08:56 TDengine-server-2.0.16.0-Linux-x64.tar.gz
[root@ms06 soft]# cd TDengine-server-2.0.16.0
[root@ms06 TDengine-server-2.0.16.0]# ll
total 2292
drwxrwxr-x 6 1000 1000 104 Feb 10 09:27 connector
drwxrwxr-x 2 1000 1000 33 Feb 10 09:27 driver
drwxrwxr-x 10 1000 1000 98 Feb 10 09:27 examples
-rwxrwxr-x 1 1000 1000 34689 Feb 10 09:27 install.sh
-rw-rw-r-- 1 1000 1000 2306969 Feb 10 09:27 taos.tar.gz
执行安装脚本，一路确定就可以，不需要输入任何参数，相关参数的配置可以在安装后进行设置。
[root@ms06 TDengine-server-2.0.16.0]# ./install.sh
Start to install TDengine...
Created symlink from /etc/systemd/system/multi-user.target.wants/taosd.service to /etc/systemd/system/taosd.service.
System hostname is: ms06.dyq.com
Enter FQDN:port (like h1.taosdata.com:6030) of an existing TDengine cluster node to join
OR leave it blank to build one:
Enter your email address for priority support or enter empty to skip:
To configure TDengine : edit /etc/taos/taos.cfg
To start TDengine : sudo systemctl start taosd
To access TDengine : taos -h ms06.dyq.com to login into TDengine server
TDengine is installed successfully!
3.编辑配置文件
firstEp: 用于连接集群首节点。
fqdn：数据节点的FQDN，可以设置为节点域名或ip地址。
serverPort：服务的端口号，默认值为6030
dataDir：数据文件目录，默认值：/var/lib/taos。
logDir：日志文件目录，默认值：/var/log/taos。
tempDir：临时目录，默认值：/tmp/
[root@ms06 ~]# grep -v '^#' /etc/taos/taos.cfg |sed '/^$/d'
firstEp ms06.dyq.com:6030
fqdn ms06.dyq.com
serverPort 6030
logDir /var/log/taos
dataDir /var/lib/taos
tempDir /tmp/
配置好配置文件后，就可以启动数据库了。
4.启动数据库
安装程序已经自动在系统中添加了启动文件，可以直接使用systemctl启停服务。
[root@ms06 TDengine-server-2.0.16.0]# cat /etc/systemd/system/taosd.service
[Unit]
Description=TDengine server service
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/taosd
ExecStartPre=/usr/local/taos/bin/startPre.sh
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TimeoutStartSec=0
StandardOutput=null
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
[Install]
WantedBy=multi-user.target
[root@ms06 ~]# systemctl start taosd
[root@ms06 ~]# systemctl status taosd
● taosd.service - TDengine server service
Loaded: loaded (/etc/systemd/system/taosd.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2021-03-06 09:02:32 CST; 3s ago
Process: 6701 ExecStartPre=/usr/local/taos/bin/startPre.sh (code=exited, status=0/SUCCESS)
Main PID: 6707 (taosd)
CGroup: /system.slice/taosd.service
└─6707 /usr/bin/taosd
Mar 06 09:02:32 ms06.dyq.com systemd[1]: Starting TDengine server service...
Mar 06 09:02:32 ms06.dyq.com systemd[1]: Started TDengine server service.
Mar 06 09:02:32 ms06.dyq.com TDengine:[6707]: Starting TDengine service...
Mar 06 09:02:32 ms06.dyq.com TDengine:[6707]: Started TDengine service successfully.
5.登录数据库并进行测试
TDengine的客户端程序是taos 可以用–help查看相关参数
[root@ms06 ~]# taos --help
Usage: taos [OPTION...]
-A, --user=Auth The user auth to use when connecting to the
server.
-c, --config-dir=CONFIG_DIR Configuration directory.
-C, --dump-config Dump configuration.
-d, --database=DATABASE Database to use when connecting to the server.
-D, --directory=DIRECTORY Use multi-thread to import all SQL files in the
directory separately.
-f, --file=FILE Script to run without enter the shell.
-h, --host=HOST TDengine server FQDN to connect. The default host
is localhost.
-l, --pktlen=PKTLEN Packet length used for net test, default is 1000
bytes.
-n, --netrole=NETROLE Net role when network connectivity test, default
is startup, options:
client|server|rpc|startup|sync.
-p, --password[=PASSWORD] The password to use when connecting to the server.
-P, --port=PORT The TCP/IP port number to use for the connection.
-r, --raw-time Output time as uint64_t.
-s, --commands=COMMANDS Commands to run without enter the shell.
-t, --timezone=TIMEZONE Time zone of the shell, default is local.
-T, --thread=THREADNUM Number of threads when using multi-thread to
import data.
-u, --user=USER The user name to use when connecting to the
server.
-?, --help Give this help list
--usage Give a short usage message
-V, --version Print program version
Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.
Report bugs to <[email protected]>.
[root@ms06 ~]# taos -h ms06.dyq.com
Welcome to the TDengine shell from Linux, Client Version:2.0.16.0
Copyright (c) 2020 by TAOS Data, Inc. All rights reserved.
taos> show databases\G;
*************************** 1.row ***************************
name: log
created_time: 2021-03-06 09:02:33.371
ntables: 4
vgroups: 1
replica: 1
quorum: 1
days: 10
keep0,keep1,keep(D): 30,30,30
cache(MB): 1
blocks: 3
minrows: 100
maxrows: 4096
wallevel: 1
fsync: 3000
comp: 2
cachelast: 0
precision: us
update: 0
status: ready
Query OK, 1 row(s) in set (0.001267s)
PS：\G参数的使用与MySQL不同，在MySQL中\G后面不需要加[;]，否则会报错；而在TDenging中，\G后面必须加[;]才能执行。
taos> show databases\G
->
->
->
->
->
创建一个测试库test
taos> create database test1;
Query OK, 0 row(s) affected (0.002404s)
taos> show databases\G;
*************************** 1.row ***************************
name: log
created_time: 2021-03-06 09:02:33.371
ntables: 4
vgroups: 1
replica: 1
quorum: 1
days: 10
keep0,keep1,keep(D): 30,30,30
cache(MB): 1
blocks: 3
minrows: 100
maxrows: 4096
wallevel: 1
fsync: 3000
comp: 2
cachelast: 0
precision: us
update: 0
status: ready
*************************** 2.row ***************************
name: test1
created_time: 2021-03-06 09:03:27.079
ntables: 0
vgroups: 0
replica: 1
quorum: 1
days: 1
keep0,keep1,keep(D): 3650,3650,3650
cache(MB): 16
blocks: 6
minrows: 100
maxrows: 4096
wallevel: 1
fsync: 3000
comp: 2
cachelast: 0
precision: ms
update: 0
status: ready
Query OK, 2 row(s) in set (0.001666s)
创建测试表，并插入数据。因为是时序数据库，因此第一个字段类型必须是TIMESTAMP。
taos> use test1;
Database changed.
taos> create table tb_test1 (tt TIMESTAMP,name1 BINARY(100));
Query OK, 0 row(s) affected (0.008029s)
taos> show tables;
table_name | created_time | columns | stable_name | uid | tid | vgId |
==========================================================================================================================================================
tb_test1 | 2021-03-06 09:04:27.918 | 2 | | 844424946914194 | 1 | 3 |
Query OK, 1 row(s) in set (0.002966s)
taos> show tables\G;
*************************** 1.row ***************************
table_name: tb_test1
created_time: 2021-03-06 09:04:27.918
columns: 2
stable_name:
uid: 844424946914194
tid: 1
vgId: 3
Query OK, 1 row(s) in set (0.002657s)
taos> INSERT INTO tb_test1 VALUES ('2021-03-06 08:44:01.990', 'test1') ;
Query OK, 1 row(s) affected (0.001934s)
taos> select * from tb_test1;
tt | name1 |
===========================================================
2021-03-06 08:44:01.990 | test1 |
Query OK, 1 row(s) in set (0.005092s)
测试1
在之前的文章中，看到如果时间序列一致，则插入的新数据会覆盖旧数据。对此进行测试。
taos> INSERT INTO tb_test1 VALUES ('2021-03-06 08:44:01.990', 'test2') ;
Query OK, 1 row(s) affected (0.000555s)
taos> select * from tb_test1;
tt | name1 |
===========================================================
2021-03-06 08:44:01.990 | test1 |
Query OK, 1 row(s) in set (0.004198s)
结果显示，数据没有被覆盖。以后再详细测一下。
测试2
之前知乎上有篇文章介绍TDengine，说无法在已有数据的时间戳前插入数据，测试如下：
taos> INSERT INTO tb_test1 VALUES ('2021-03-06 08:44:01.980', 'test2') ;
Query OK, 1 row(s) affected (0.000480s)
taos> select * from tb_test1;
tt | name1 |
===========================================================
2021-03-06 08:44:01.980 | test2 |
2021-03-06 08:44:01.990 | test1 |
Query OK, 2 row(s) in set (0.003131s)
貌似也是可以的。
6.相关目录与文件
刚才在配置文件中看到，数据文件的存储目录是/var/lib/taos/
[root@ms06 ~]# cd /var/lib/taos/
[root@ms06 taos]# ls
dnode mnode vnode vnode_bak
[root@ms06 taos]# ll
total 0
drwxr-xr-x 2 root root 87 Mar 6 09:02 dnode
drwxr-xr-x 3 root root 17 Mar 6 09:02 mnode
drwxr-xr-x 4 root root 34 Mar 6 09:04 vnode
drwxr-xr-x 2 root root 6 Mar 6 09:02 vnode_bak
官方介绍，数据是存储在vnode下，查看如下：
[root@ms06 vnode]# ls
vnode2 vnode3
[root@ms06 vnode]# ll vnode3/
total 4
-rw-rw-rw- 1 root root 500 Mar 6 09:04 config.json
drwxr-xr-x 3 root root 33 Mar 6 09:04 tsdb
drwxr-xr-x 2 root root 18 Mar 6 09:04 wal
config.json 文件中存储了刚才创建的数据库test的相关信息
[root@ms06 vnode3]# cat config.json
{
"db": "0.test1",
"cfgVersion": 0,
"vgCfgVersion": 0,
"cacheBlockSize": 16,
"totalBlocks": 6,
"daysPerFile": 1,
"daysToKeep": 3650,
"daysToKeep1": 3650,
"daysToKeep2": 3650,
"minRowsPerFileBlock": 100,
"maxRowsPerFileBlock": 4096,
"precision": 0,
"compression": 2,
"walLevel": 1,
"fsync": 3000,
"replica": 1,
"dbReplica": 1,
"wals": 3,
"quorum": 1,
"update": 0,
"cacheLastRow": 0,
"nodeInfos": [{
"nodeId": 1,
"nodeEp": "ms06.dyq.com:6030"
}]
}
查看数据文件，发现没有文件，可能数据还在wal日志。
[root@ms06 vnode3]# cd tsdb/
[root@ms06 tsdb]# ll
total 4
-rwxr-xr-x 1 root root 512 Mar 6 09:04 current
drwxr-xr-x 2 root root 6 Mar 6 09:04 data
[root@ms06 tsdb]# ll data/
total 0
查看wal日志，因为wal日志不是文本文件，因此用strings查看了一下，发现了之前插入的数据。
[root@ms06 vnode3]# cd wal/
[root@ms06 wal]# ll
total 4
-rwxrwxrwx 1 root root 1067 Mar 6 09:11 wal1
[root@ms06 wal]# ll wal1
-rwxrwxrwx 1 root root 1067 Mar 6 09:11 wal1
[root@ms06 wal]# file wal1
wal1: AmigaOS bitmap font
[root@ms06 wal]# strings wal1
tb_test1
name1
test1
test2
test2
写在最后
因为我的环境之前安装过MySQL和PostgreSQL，因此相关依赖包应该都很全，在安装TDengine时没有遇到依赖关系的问题。下次准备在一个全新环境安装测试一下。
总体来说，TDengine安装还是超超超简单的，尤其是与Oracle相比。