Doris部署

Doris部署

1 环境准备

1.1 系统环境准备

jdk,防火墙等关闭

1.2 Doris安装包获取

1.2.1 自行编译

参考: Doris编译 - Apache Doris

以1.1.3版本在Docker环境下编译为例

编译机器上需要预先安装docker,在此不进行赘述

镜像版本	commit id	doris 版本
apache/incubator-doris:build-env	before ff0dd0d	0.8.x, 0.9.x
apache/incubator-doris:build-env-1.1	ff0dd0d	0.10.x, 0.11.x
apache/incubator-doris:build-env-1.2	4ef5a8c	0.12.x - 0.14.0
apache/incubator-doris:build-env-1.3.1	ad67dd3	0.14.x
apache/doris:build-env-for-0.15.0	a81f4da or later	0.15.0
apache/incubator-doris:build-env-latest	before 0efef1b
apache/doris:build-env-for-1.0.0		1.0.0
apache/doris:build-env-for-1.1.0		1.1.0
apache/doris:build-env-ldb-toolchain-latest		1.1.x, trunk
apache/doris:build-env-ldb-toolchain-no-avx2-latest		1.1.x, trunk

注意：

1. 名称中带有 no-avx2 字样的镜像中的第三方库，可以运行在不支持 avx2 指令的 CPU 上。可以配合 USE_AVX2=0 选项，编译 Doris。

1. 编译镜像 ChangeLog。

1. doris 0.14.0 版本仍然使用apache/incubator-doris:build-env-1.2 编译，0.14.x 版本的代码将使用apache/incubator-doris:build-env-1.3.1。

1. 从 build-env-1.3.1 的docker镜像起，同时包含了 OpenJDK 8 和 OpenJDK 11，请通过 java -version 确认默认 JDK 版本。也可以通过以下方式切换版本（建议默认使用 JDK8）

   切换到 JDK 8：

   alternatives --set java java-1.8.0-openjdk.x86_64
   alternatives --set javac java-1.8.0-openjdk.x86_64
   export JAVA_HOME=/usr/lib/jvm/java-1.8.0
   

   切换到 JDK 11：

   alternatives --set java java-11-openjdk.x86_64
   alternatives --set javac java-11-openjdk.x86_64
   export JAVA_HOME=/usr/lib/jvm/java-11
   

下载Doris源码

$ wget https://dist.apache.org/repos/dist/dev/doris/xxx.tar.gz
or
$ git clone https://github.com/apache/doris.git

查看编译机器是否支持avx2指令集

$ cat /proc/cpuinfo | grep avx2
# 不支持则使用以下命令进行编译
$ USE_AVX2=0  sh build.sh
# 支持的
$ sh build.sh

编译FE,BE

# 在源码主目录下
sh build.sh

编译broker

# 进入fs_brokers/apache_hdfs_broker执行build.sh
sh build.sh

编译完成后，产出文件在 output/ 目录中。

1.2.2 官方下载

下载 - Apache Doris

2 安装

2.1 节点规划

节点名称	ip	服务	系统版本
doris01	192.168.31.210	FE	CentOS 7.8
doris02	192.168.31.211	BE	CentOS 7.8
doris03	192.168.31.212	BE	CentOS 7.8
doris04	192.168.31.213	BE	CentOS 7.8

2.2 修改文件读写大小(所有节点)

sudo vim /etc/security/limits.conf 

#添加如下值
* soft nofile 65535 
* hard nofile 65535 
* soft nproc 65535 
* hard nproc 65535 

# 需重启,保存之后,临时命令 ulimit -n 65535 临时生效
ulimit -n 65535

2.3 部署FE(doris01)

mkdir /data/doris/doris-meta
cd /opt/module/apache-doris-fe-1.1.3-bin
vim conf/fe.conf

# store metadata, must be created before start FE.
# Default value is ${DORIS_HOME}/doris-meta
meta_dir = /data/doris/doris-meta

# Choose one if there are more than one ip except loopback address. 
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24
# Default value is empty.
priority_networks = 192.168.31.210/24

bin/start_fe.sh --daemon

client登录FE后,添加BE

ALTER SYSTEM ADD BACKEND "192.168.31.211:9050"; 
ALTER SYSTEM ADD BACKEND "192.168.31.212:9050"; 
ALTER SYSTEM ADD BACKEND "192.168.31.213:9050"; 

2.4 部署BE(doris02-04)

mkdir /data/doris/doris-storage
cd /opt/module/apache-doris-be-1.1.3-bin-x86_64
vim conf/be.conf

# Choose one if there are more than one ip except loopback address. 
# Note that there should at most one ip match this list.
# If no ip match this rule, will choose one randomly.
# use CIDR format, e.g. 10.10.10.0/24
# Default value is empty.
priority_networks = 192.168.31.211/24

# data root path, separate by ';'
# you can specify the storage medium of each root path, HDD or SSD
# you can add capacity limit at the end of each root path, seperate by ','
# eg:
# storage_root_path = /home/disk1/doris.HDD,50;/home/disk2/doris.SSD,1;/home/disk2/doris
# /home/disk1/doris.HDD, capacity limit is 50GB, HDD;
# /home/disk2/doris.SSD, capacity limit is 1GB, SSD;
# /home/disk2/doris, capacity limit is disk capacity, HDD(default)
# 
# you also can specify the properties by setting ':', seperate by ','
# property 'medium' has a higher priority than the extension of path
#
# Default value is ${DORIS_HOME}/storage, you should create it by hand.
storage_root_path = /data/doris/doris-storage

bin/start_be.sh --daemon