doris的单节点安装部署(详细)
doris的单节点安装部署
环境准备：
1.jdk8
进入官网:https://www.oracle.com/java/technologies/downloads/
在linux服务上创建jdk安装的目录
mkdir /itsoftware/jdk #创建jdk安装路径
cd /itsoftware/jdk #进入jdk安装目录
解压缩包
cd /itsoftware/jdk #进入jdk安装目录
tar -zxvf jdk-8u212-linux-x64.tar.gz #对压缩包解压
配置环境变量
vim /etc/profile #编辑配置文件
在文件末尾添加
export JAVA_HOME=/itsoftware/jdk/jdk1.8.0_212
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
export PATH=$PATH:${JAVA_PATH}
重新加载配置文件
source /etc/profile
验证jdk是否安装成功
javac -version
![[图片]](http://img.e-com-net.com/image/info8/22e10da290aa464fb2d29e594ce39205.jpg)
修改limits文件
vim /etc/security/limits.conf
在文件的末尾处添加或者修改
* soft nofile 204800
* hard nofile 204800
* soft nproc 204800
* hard nproc 204800
vim /etc/sysctl.conf
#添加
fs.file-max = 6553560
vm.max_map_count = 2000000
#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
执行生效
sysctl -p
安装doris
官网网址：https://doris.apache.org/zh-CN/docs/dev/get-starting/quick-start
解压 tar.gz 文件
tar -zxvf apache-doris-2.0.2-bin-x64.tar.gz
mv apache-doris-2.0.2-bin-x64 /opt/doris
配置环境：
[root@10-63-0-181 conf]# vim /etc/profile.d/doris.sh
export DORIS_HOME=/opt/doris
export PATH=$PATH:$DORIS_HOME/fe/bin:$DORIS_HOME/be/bin
配置生效：
source /etc/profile.d/doris.sh
FE配置
进入doris下的fe目录，修改其中conf目录下的fe.conf文件
注意：
这个参数我们在安装的时候是必须要配置的，特别是当一台机器拥有多个IP地址的时候，我们要为 FE 指定唯一的IP地址。
这里假设你的节点 IP 是 172.23.16.32，那么我们可以通过掩码的方式配置为 172.23.16.0/24。
[root@10-63-0-181 conf]# pwd
/opt/doris/fe/conf
[root@10-63-0-181 conf]# vim fe.conf
更改配置：
meta_dir = /data/doris-meta
priority_networks = 10.63.0.0/24
启动 FE
在 FE 安装目录下执行下面的命令，来完成 FE 的启动。
./bin/start_fe.sh --daemon
查看 FE 运行状态
你可以通过下面的命令来检查 Doris 是否启动成功
[root@10-63-0-181 fe]# curl http://10.63.0.181:8030/api/bootstrap
{"msg":"success","code":0,"data":{"replayedJournalId":0,"queryPort":0,"rpcPort":0,"version":""},"count":0}[root@10-63-0-181 fe]#
[root@10-63-0-181 fe]#
注意：
- 这里我们使用 Doris 内置的默认用户 root 进行登录，密码是空
- 这是一个 Doris 的管理界面，只能拥有管理权限的用户才能登录，普通用户不能登录。
连接 FE
我们下面通过 MySQL 客户端来连接 Doris FE，下载免安装的 MySQL 客户端
解压刚才下载的 MySQL 客户端，在 bin/ 目录下可以找到 mysql 命令行工具。然后执行下面的命令连接 Doris。
[root@10-63-0-181 bin]# ./mysql -uroot -P9030 -h10.63.0.181
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 0
Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show frontends\G;
*************************** 1. row ***************************
Name: fe_2b76e132_4efe_4425_8919_f95876206eda
Host: 10.63.0.181
EditLogPort: 9010
HttpPort: 8030
QueryPort: 9030
RpcPort: 9020
Role: FOLLOWER
IsMaster: true
ClusterId: 1092995366
Join: true
Alive: true
ReplayedJournalId: 319
LastHeartbeat: 2023-10-23 16:01:08
IsHelper: true
ErrMsg:
Version: doris-2.0.2-rc05-ae923f7
CurrentConnected: Yes
1 row in set (0.06 sec)
停止 FE 节点
Doris FE 的停止可以通过下面的命令完成
./bin/stop_fe.sh
配置 BE
进入doris下的be目录，修改其中conf目录下的be.conf文件
[root@10-63-0-181 conf]# pwd
/opt/doris/be/conf
[root@10-63-0-181 conf]# vim be.conf
更改配置：
priority_networks = 10.63.0.0/24
storage_root_path = /data/doris-data
启动 BE
在 BE 安装目录下执行下面的命令，来完成 BE 的启动。
./bin/start_be.sh --daemon
添加 BE 节点到集群
通过MySQL 客户端连接到 FE 之后执行下面的 SQL，将 BE 添加到集群中
ALTER SYSTEM ADD BACKEND "10.60.0.181:9050";
[root@10-63-0-181 bin]# ./mysql -uroot -P 9030 -h 10.63.0.181
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> SHOW BACKENDS\G;
*************************** 1. row ***************************
BackendId: 10065
Host: 10.63.0.181
HeartbeatPort: 9050
BePort: 9060
HttpPort: 8040
BrpcPort: 8060
LastStartTime: 2023-10-24 10:40:27
LastHeartbeat: 2023-10-25 15:44:35
Alive: true
SystemDecommissioned: false
TabletNum: 22
DataUsedCapacity: 1.417 KB
TrashUsedCapcacity: 0.000
AvailCapacity: 199.782 GB
TotalCapacity: 199.902 GB
UsedPct: 0.06 %
MaxDiskUsedPct: 0.06 %
RemoteUsedCapacity: 0.000
Tag: {"location" : "default"}
ErrMsg:
Version: doris-2.0.2-rc05-ae923f7
Status: {"lastSuccessReportTabletsTime":"2023-10-25 15:44:34","lastStreamLoadTime":-1,"isQueryDisabled":false,"isLoadDisabled":false}
- Alive : true表示节点运行正常
验证：
创建数据表
创建一个数据库
create database demo;
创建数据表
use demo;
CREATE TABLE IF NOT EXISTS demo.example_tbl
(
`user_id` LARGEINT NOT NULL COMMENT "用户id",
`date` DATE NOT NULL COMMENT "数据灌入日期时间",
`city` VARCHAR(20) COMMENT "用户所在城市",
`age` SMALLINT COMMENT "用户年龄",
`sex` TINYINT COMMENT "用户性别",
`last_visit_date` DATETIME REPLACE DEFAULT "1970-01-01 00:00:00" COMMENT "用户最后一次访问时间",
`cost` BIGINT SUM DEFAULT "0" COMMENT "用户总消费",
`max_dwell_time` INT MAX DEFAULT "0" COMMENT "用户最大停留时间",
`min_dwell_time` INT MIN DEFAULT "99999" COMMENT "用户最小停留时间"
)
AGGREGATE KEY(`user_id`, `date`, `city`, `age`, `sex`)
DISTRIBUTED BY HASH(`user_id`) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1"
);
示例数据
10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
将上面的数据保存在test.csv文件中。
1. 导入数据
这里我们通过Stream load 方式将上面保存到文件中的数据导入到我们刚才创建的表里。
[root@10-63-0-181 ~]# vi test.csv
[root@10-63-0-181 ~]# cat test.csv
10000,2017-10-01,北京,20,0,2017-10-01 06:00:00,20,10,10
10000,2017-10-01,北京,20,0,2017-10-01 07:00:00,15,2,2
10001,2017-10-01,北京,30,1,2017-10-01 17:05:45,2,22,22
10002,2017-10-02,上海,20,1,2017-10-02 12:59:12,200,5,5
10003,2017-10-02,广州,32,0,2017-10-02 11:20:00,30,11,11
10004,2017-10-01,深圳,35,0,2017-10-01 10:00:15,100,3,3
10004,2017-10-03,深圳,35,0,2017-10-03 10:20:22,11,6,6
[root@10-63-0-181 ~]# curl --location-trusted -u root: -T test.csv -H "column_separator:," http://10.63.0.181:8030/api/demo/example_tbl/_stream_load
{
"TxnId": 2,
"Label": "3d320737-a9db-4546-a145-5543e56d44a5",
"Comment": "",
"TwoPhaseCommit": "false",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 7,
"NumberLoadedRows": 7,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 399,
"LoadTimeMs": 284,
"BeginTxnTimeMs": 20,
"StreamLoadPutTimeMs": 199,
"ReadDataTimeMs": 0,
"WriteDataTimeMs": 11,
"CommitAndPublishTimeMs": 51
}
[root@10-63-0-181 bin]# ./mysql -uroot -P 9030 -h 10.63.0.181
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.7.99 Doris version doris-2.0.2-rc05-ae923f7
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use demo;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from example_tbl;
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| user_id | date | city | age | sex | last_visit_date | cost | max_dwell_time | min_dwell_time |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| 10000 | 2017-10-01 | 北京 | 20 | 0 | 2017-10-01 07:00:00 | 35 | 10 | 2 |
| 10001 | 2017-10-01 | 北京 | 30 | 1 | 2017-10-01 17:05:45 | 2 | 22 | 22 |
| 10002 | 2017-10-02 | 上海 | 20 | 1 | 2017-10-02 12:59:12 | 200 | 5 | 5 |
| 10003 | 2017-10-02 | 广州 | 32 | 0 | 2017-10-02 11:20:00 | 30 | 11 | 11 |
| 10004 | 2017-10-01 | 深圳 | 35 | 0 | 2017-10-01 10:00:15 | 100 | 3 | 3 |
| 10004 | 2017-10-03 | 深圳 | 35 | 0 | 2017-10-03 10:20:22 | 11 | 6 | 6 |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
6 rows in set (0.07 sec)
mysql> select * from example_tbl where city='上海';
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| user_id | date | city | age | sex | last_visit_date | cost | max_dwell_time | min_dwell_time |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
| 10002 | 2017-10-02 | 上海 | 20 | 1 | 2017-10-02 12:59:12 | 200 | 5 | 5 |
+---------+------------+--------+------+------+---------------------+------+----------------+----------------+
1 row in set (0.09 sec)
mysql> select city, sum(cost) as total_cost from example_tbl group by city;
+--------+------------+
| city | total_cost |
+--------+------------+
| 上海 | 200 |
| 深圳 | 111 |
| 北京 | 37 |
| 广州 | 30 |
+--------+------------+
4 rows in set (0.13 sec)
mysql>