ApacheDoris入门操作和BE启动失败问题解决

Doris详细介绍

https://blog.csdn.net/qihoo_tech/article/details/96472699

一.配置

FE:		    10G(内存) 	100G
BE1:		20G(内存)	200G
BE2:	    20G(内存)	200G
BE3:		20G(内存)	200G

Ⅰ).BE

vi be.conf

# INFO,WARNING,ERROR,FATAL
sys_log_level=INFO

# ports for admin,web,heartbeat service
be_port=9060
be_rpc_port=9070
webserver_port=8040
heartbeat_service_port=9050
brpc_port=8060

# Choose one if there are more than one ip except loopback address
# 24为网关
priority_networks=hostname/24

# storage_root_path=/home/disk1/palo.HDD,50;/home/disk2/palo.SSD,1;/home/disk2/palo
# 目录需手动创建
storage_root_path=/data/opt/be/palo

Ⅱ).FE

vi fe.conf

# INFO,WARNING,ERROR,FATAL
sys_log_level=INFO

# store metadata,create it if it is not exist
meta_dir=${DORIS_HOME}/palo-meta
http_port=8030
rpc_port=9020
query_port=9030
edit_log_port=9010

# Choose one if there are more than one ip except loopback address
# 24为网关
priority_networks=hostname/24

Ⅲ).网络通信端口

二.服务命令

Ⅰ).启停

a).启动服务FE和BE

sh  /data/opt/fe/bin/start_fe.sh --daemo
sh  /data/opt/be/bin/start_be.sh --daemon

b).停止服务

sh  /data/opt/fe/bin/stop_fe.sh
sh 	/data/opt/be/bin/stop_be.sh

Ⅱ).验证

URL: http://hostname:8030/system
查看FE节点: [http://hostname:8030/system?path=//frontends]

三.业务命令

Ⅰ).连接FE

mysql -h 192.1.1.1 -P9030 -uroot

urser

Doris    1234
root 	  root

Ⅱ).创建数据库

create database doris;

Ⅲ).创建用户

create user 'doris' identified by '1234';

Ⅳ).赋权

grant all on doris to doris;

Ⅴ).添加BE

ALTER SYSTEM ADD BACKEND "hostname:9050";

Ⅵ).查看BE状态

SHOW PROC '/backends';

使用 mysql-client 连接到 FE，并执行

SHOW PROC '/backends';

查看 BE 运行情况。如一切正常，isAlive 列应为 true

Ⅶ).删除BE

ALTER SYSTEM DROPP BACKEND "hostname:9050";

Ⅷ).添加FE

ALTER SYSTEM ADD FOLLOWER "hostname:9050";

Ⅸ).查看FE状态

SHOW PROC '/frontends';

Ⅹ).删除FE

ALTER SYSTEM DROPP FOLLOWER "hostname:9050";

Ⅺ).创建表

CREATE TABLE doris.table2
(
  event_day DATE,
  siteid INT DEFAULT '10',
  citycode SMALLINT,
  username VARCHAR(32) DEFAULT '',
  pv BIGINT SUM DEFAULT '0'
)
AGGREGATE KEY(event_day,citycode,username)
PARTITION BY RANGE(event_day)
(
  PARTITION p1 VALUES LESS THAN ('2019-06-30'),
  PARTITION p1 VALUES LESS THAN ('2019-07-31'),
  PARTITION p1 VALUES LESS THAN ('2019-08-31')
)
DISTRIBUTED BY HASH(siteid) BUCKETS 10
PROPERTIES("replication_num"="1")

Ⅻ).导入数据

1.database: doris
2.tablename: table2
3.label: 每个批次不可重复
4.column_separator: 数据文件中字段之间特殊符号分割(|-%7c, ,-%2c, t-%09, x01-%01)

分割符	对应值
竖线	%7c
,	%2c
\t	%09
\x01	%01

curl --location-trusted -u db_user -T path/filename.txt http://hostname/8030/api/doris/table2/_load?label=100\&column_separator=%7c

curl  --location-trusted  -u root:root  -T	 table2_data(文件名)   http://192.1.1.1:8030/api/example_db(库名)/table2(表名)/_load?label=table2_20170707 -V

XIII)).创建MySQL引擎表

CREATE TABLE IF NOT EXISTES doris.table_name
(
  'agent_id' INT NOT NULL,
  'accept_time' DATETIME NOT NULL,
  'signature' VARCHAR(770) NOT NULL,
  'bad_app_count' BIGINT,
  'total_count' BIGINT,
  'version' INT DEFAULT "0",
  'method_type' VARCHAR(20)
)
ENGINE=mysql
PROPERTIES
(
  "host"="db_hostname",
  "port"="3306",
  "user"="db_user",
  "password"="password",
  "database"="database",
  "table"="table_name"
)

启动失败

四.启动失败问题

Ⅰ).BE启动失败

修改最大文件句柄数

echo "* soft nofile 204800"  >> /etc/security/limits.conf
echo "* hard nofile 204800"  >> /etc/security/limits.conf

echo "* soft nproc 204800"  >> /etc/security/limits.conf
echo "* hard nproc 204800 "  >> /etc/security/limits.conf

修改 /etc/sysctl.conf, 加入
echo   fs.file-max = 6553560  >> /etc/sysctl.conf

查看是否成功

cat /etc/security/limits.conf
cat /etc/sysctl.conf

重启生效

reboot  -h  now

Ⅱ).BE启动成功,但是FE添加失败

java.net.NoRouteToHostException: No route to host (Host unreachable) |

解决:(centos7环境)

直接关闭防火墙

systemctl stop firewalld.service        #停止firewall
systemctl disable firewalld.service    #禁止firewall开机启动

reboot -h now