doris安装文档翻译-标准部署(Standard deployment)

文档地址: 对应1.2版本

Standard deployment - Apache Doris

This topic is about the hardware and software environment needed to deploy Doris, the recommended deployment mode, cluster scaling, and common problems occur in creating and running clusters.

Before continue reading, you might want to compile Doris following the instructions in the Compile topic.

翻译:

本主题是关于部署Doris所需的硬件和软件环境、建议的部署模式、集群扩展以及创建和运行集群时出现的常见问题。

在继续阅读之前，您可能需要按照编译主题中的说明编译Doris。

Software and Hardware Requirements​

Overview​

Doris, as an open source OLAP database with an MPP architecture, can run on most mainstream commercial servers. For you to take full advantage of the high concurrency and high availability of Doris, we recommend that your computer meet the following requirements:

翻译:

软件和硬件要求​

概述​

Doris作为一个具有MPP体系结构的开源OLAP数据库，可以在大多数主流商业服务器上运行。为了充分利用Doris的高并发性和高可用性，我们建议您的计算机满足以下要求：

Linux Operating System Version Requirements​

Linux System	Version
Centos	7.1 and above
Ubuntu	16.04 and above

Software Requirements​

Soft	Version
Java	1.8 and above
GCC	4.8.2 and above

OS Installation Requirements​

Set the maximum number of open file descriptors in the system

设置系统中打开的文件描述符的最大数量

vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536

Clock synchronization​

The metadata in Doris requires a time precision of less than 5000ms, so all machines in all clusters need to synchronize their clocks to avoid service exceptions caused by inconsistencies in metadata caused by clock problems.

翻译:

时钟同步​

Doris中的元数据需要小于5000ms的时间精度，因此所有集群中的所有机器都需要同步它们的时钟，以避免由于时钟问题导致的元数据不一致而导致的服务异常。

Close the swap partition​

The Linux swap partition can cause serious performance problems for Doris, so you need to disable the swap partition before installation.

翻译:

关闭交换分区​

Linux交换分区可能会给Doris带来严重的性能问题，因此在安装之前需要禁用交换分区。

Linux file system​

Both ext4 and xfs file systems are supported.

翻译:

Linux文件系统​

支持ext4和xfs文件系统。

Development Test Environment (部署测试环境)​

Module	CPU	Memory	Disk	Network	Number of Instances
Frontend	8 core +	8GB +	SSD or SATA, 10GB + *	Gigabit Network Card	1
Backend	8 core +	16GB +	SSD or SATA, 50GB + *	Gigabit Network Card	1-3*

Production Environment (部署生产环境)​

Module	CPU	Memory	Disk	Network	Number of Instances (Minimum Requirements)
Frontend	16 core +	64GB +	SSD or RAID card, 100GB + *	10,000 Mbp network card	1-3*
Backend	16 core +	64GB +	SSD or SATA, 100G + *	10-100 Mbp network card	3 *

Note 1:

1. The disk space of FE is mainly used to store metadata, including logs and images. It usually ranges from several hundred MB to several GB.
2. The disk space of BE is mainly used to store user data. The total disk space taken up is 3 times the total user data (3 copies). Then an additional 40% of the space is reserved for background compaction and intermediate data storage.
3. On one single machine, you can deploy multiple BE instances but only one FE instance. If you need 3 copies of the data, you need to deploy 3 BE instances on 3 machines (1 instance per machine) instead of 3 BE instances on one machine). Clocks of the FE servers must be consistent (allowing a maximum clock skew of 5 seconds).
4. The test environment can also be tested with only 1 BE instance. In the actual production environment, the number of BE instances directly determines the overall query latency.
5. Disable swap for all deployment nodes.

 翻译:

注1：

FE的磁盘空间主要用于存储元数据，包括日志和图像。它的大小通常在几百MB到几GB之间。

BE的磁盘空间主要用于存储用户数据。占用的总磁盘空间是总用户数据（3个副本）的3倍。然后，额外保留40%的空间用于背景压缩和中间数据存储。

在一台机器上，可以部署多个BE实例，但只能部署一个FE实例。如果您需要3个数据副本，则需要在3台计算机上部署3个BE实例（每台计算机1个实例），而不是在一台计算机上配置3个BE示例。FE服务器的时钟必须一致（允许最大5秒的时钟偏移）。

测试环境也可以只使用1个be实例进行测试。在实际的生产环境中，BE实例的数量直接决定了整个查询延迟。

禁用所有部署节点的交换。

Note 2: Number of FE nodes

1. FE nodes are divided into Followers and Observers based on their roles. (Leader is an elected role in the Follower group, hereinafter referred to as Follower, too.)
2. The number of FE nodes should be at least 1 (1 Follower). If you deploy 1 Follower and 1 Observer, you can achieve high read availability; if you deploy 3 Followers, you can achieve high read-write availability (HA).
3. The number of Followers must be odd, and there is no limit on the number of Observers.
4. According to past experience, for business that requires high cluster availability (e.g. online service providers), we recommend that you deploy 3 Followers and 1-3 Observers; for offline business, we recommend that you deploy 1 Follower and 1-3 Observers.

翻译: 

注2：FE节点数量

FE节点根据其角色分为跟随者和观察者。（领导者是追随者群体中的民选角色，以下也称为追随者。）

FE节点的数量应至少为1（1个跟随器）。如果部署1个关注者和1个观察者，可以实现高读取可用性；如果部署3个Followers，则可以实现高读写可用性（HA）。

关注者的数量必须是奇数，并且观察者的数量没有限制。

根据以往的经验，对于需要高集群可用性的业务（如在线服务提供商），我们建议您部署3名追随者和1-3名观察员；对于离线业务，我们建议您部署1个追随者和1-3个观察员。

• Usually we recommend 10 to 100 machines to give full play to Doris' performance (deploy FE on 3 of them (HA) and BE on the rest).
• The performance of Doris is positively correlated with the number of nodes and their configuration. With a minimum of four machines (one FE, three BEs; hybrid deployment of one BE and one Observer FE to provide metadata backup) and relatively low configuration, Doris can still run smoothly.
• In hybrid deployment of FE and BE, you might need to be watchful for resource competition and ensure that the metadata catalogue and data catalogue belong to different disks.

翻译: 

通常我们推荐10到100台机器，以充分发挥Doris的性能（其中3台部署FE（HA），其余部署BE）。

Doris的性能与节点数量及其配置呈正相关。Doris至少有四台机器（一个FE、三个BE；一个BE和一个Observer FE的混合部署以提供元数据备份），并且配置相对较低，因此仍然可以平稳运行。

在FE和BE的混合部署中，您可能需要警惕资源竞争，并确保元数据目录和数据目录属于不同的磁盘。

Broker Deployment​

Broker is a process for accessing external data sources, such as hdfs. Usually, deploying one broker instance on each machine should be enough.

翻译: 

Broker部署​
Broker是一个用于访问外部数据源（如hdfs）的进程。通常，在每台机器上部署一个代理实例就足够了。
​

Network Requirements​

Doris instances communicate directly over the network. The following table shows all required ports.

翻译:

网络要求​

Doris实例直接通过网络进行通信。下表显示了所有必需的端口。

Instance Name	Port Name	Default Port	Communication Direction	Description
BE	be_port	9060	FE --> BE	Thrift server port on BE for receiving requests from FE (BE上用于接收FE请求的Thrift服务器端口)
BE	webserver_port	8040	BE <--> BE	HTTP server port on BE
BE	heartbeat_service_port	9050	FE --> BE	Heart beat service port (thrift) on BE, used to receive heartbeat from FE (BE上的心跳服务端口（节俭），用于接收来自FE的心跳)
BE	brpc_port	8060	FE <--> BE, BE <--> BE	BRPC port on BE for communication between BEs (BE上用于BE之间通信的BRPC端口)
FE	http_port	8030	FE <--> FE, user <--> FE	HTTP server port on FE
FE	rpc_port	9020	BE --> FE, FE <--> FE	Thrift server port on FE; The configurations of each FE should be consistent. (FE上的Thrift服务器端口；每个FE的配置应一致。)
FE	query_port	9030	user <--> FE	MySQL server port on FE
FE	edit_log_port	9010	FE <--> FE	Port on FE for BDBJE communication
Broker	broker ipc_port	8000	FE --> Broker, BE --> Broker	Thrift server port on Broker for receiving requests

Note:

1. When deploying multiple FE instances, make sure that the http_port configuration of each FE is consistent.
2. Make sure that each port has access in its proper direction before deployment.

注：

1. 部署多个FE实例时，请确保每个FE的http_port配置一致。

2. 在部署之前，请确保每个端口都能以正确的方向访问。

IP Binding​

Because of the existence of multiple network cards, or the existence of virtual network cards caused by the installation of docker and other environments, the same host may have multiple different IPs. Currently Doris does not automatically identify available IPs. So when you encounter multiple IPs on the deployment host, you must specify the correct IP via the priority_networks configuration item.

翻译:

由于存在多个网卡，或者安装docker等环境导致的虚拟网卡的存在，同一台主机可能具有多个不同的IP。目前，Doris无法自动识别可用的IP。因此，当您在部署主机上遇到多个IP时，必须通过priority_networks配置项指定正确的IP。

priority_networks is a configuration item that both FE and BE have. It needs to be written in fe.conf and be.conf. It is used to tell the process which IP should be bound when FE or BE starts. Examples are as follows:

priority_networks=10.1.3.0/24

This is a representation of CIDR. FE or BE will find the matching IP based on this configuration item as their own local IP.

翻译:

priority_networks是FE和BE都具有的配置项。它需要写在fe.conf和be.conf中。它用于告诉进程在fe或be启动时应该绑定哪个IP。示例如下：

priority_networks=10.1.3.0/24

这是CIDR的表示。FE或BE将基于该配置项找到匹配的IP作为它们自己的本地IP。

Note: Configuring priority_networks and starting FE or BE only ensure the correct IP binding of FE or BE. You also need to specify the same IP in ADD BACKEND or ADD FRONTEND statements, otherwise the cluster cannot be created. For example:

BE is configured as priority_networks = 10.1.3.0/24'..

If you use the following IP in the ADD BACKEND statement: ALTER SYSTEM ADD BACKEND "192.168.0.1:9050";

Then FE and BE will not be able to communicate properly.

At this point, you must DROP the wrong BE configuration and use the correct IP to perform ADD BACKEND.

The same works for FE.

Broker currently does not have the priority_networks configuration item, nor does it need. Broker's services are bound to 0.0.0.0 by default. You can simply execute the correct accessible BROKER IP when using ADD BROKER.

翻译:

注意：配置priority_networks并启动FE或BE只能确保FE或BE的IP绑定正确。您还需要在ADD BACKEND或ADD FRONTEND语句中指定相同的IP，否则无法创建群集。例如：

BE配置为priority_networks=10.1.3.0/24’...

如果在ADD BACKEND语句中使用以下IP：ALTER SYSTEM ADD BACKEND“192.168.0.1:9050”；

那么FE和BE将无法正常通信。

此时，必须删除错误的BE配置，并使用正确的IP执行ADD BACKEND。

FE也是如此。

Broker当前没有priority_networks配置项，也不需要。默认情况下，Broker的服务绑定到0.0.0.0。使用ADD BROKER时，您只需执行正确的可访问BROKER IP即可。

Table Name Case Sensitivity​

By default, table names in Doris are case-sensitive. If you need to change that, you may do it before cluster initialization. The table name case sensitivity cannot be changed after cluster initialization is completed.

See the lower_case_table_names section in Variables for details.

翻译:

表名称区分大小写​

默认情况下，Doris中的表名区分大小写。如果需要更改，可以在集群初始化之前进行更改。在完成群集初始化后，无法更改表名称区分大小写。

有关详细信息，请参阅Variables中的lower_case_table_names部分。

Cluster Deployment​

Manual Deployment​

Deploy FE​

• Copy the FE deployment file into the specified node

  Find the Fe folder under the output generated by source code compilation, copy it into the specified deployment path of FE nodes and put it the corresponding directory.

   翻译:

        将FE部署文件复制到指定的节点中

        在源代码编译生成的输出下找到Fe文件夹，将其复制到指定的Fe节点部署路径中，并放在相应的目录中。

• Configure FE

  1. The configuration file is conf/fe.conf. Note: meta_dir indicates the metadata storage location. The default value is ${DORIS_HOME}/doris-meta. The directory needs to be created manually.

     Note: For production environments, it is better not to put the directory under the Doris installation directory but in a separate disk (SSD would be the best); for test and development environments, you may use the default configuration.

  2. The default maximum Java heap memory of JAVA_OPTS in fe.conf is 4GB. For production environments, we recommend that it be adjusted to more than 8G.

     翻译:

        配置FE

        配置文件为conf/fe.conf。注意：meta_dir表示元数据的存储位置。默认值为${DORIS_HOME}/DORIS meta。该目录需要手动创建。

        注意：对于生产环境，最好不要将目录放在Doris安装目录下，而是放在一个单独的磁盘中（SSD最好）；对于测试和开发环境，可以使用默认配置。

fe.conf中Java_OPTS的默认最大Java堆内存为4GB。对于生产环境，我们建议将其调整到8G以上。

• Start FE

  bin/start_fe.sh --daemon

  The FE process starts and enters the background for execution. Logs are stored in the log/ directory by default. If startup fails, you can view error messages by checking out log/fe.log or log/fe.out.

• For details about deployment of multiple FEs, see the FE scaling section.

翻译:

        启动FE

        bin/start_fe.sh --daemon

        FE过程开始并进入后台执行。默认情况下，日志存储在log/目录中。如果启动失败，您可以通过检查log/fe.log或log/fe.out来查看错误消息。

        有关部署多个FE的详细信息，请参阅FE缩放部分。

Deploy BE​

• Copy the BE deployment file into all nodes that are to deploy BE on

  Find the BE folder under the output generated by source code compilation, copy it into to the specified deployment paths of the BE nodes.

• Modify all BE configurations

  Modify be/conf/be.conf, which mainly involves configuring storage_root_path: data storage directory. By default, under be/storage, the directory needs to be created manually. Use ; to separate multiple paths (do not add ; after the last directory).

翻译:

       部署BE

         将BE部署文件复制到要在其上部署BE的所有节点中

         在源代码编译生成的输出下找到BE文件夹，将其复制到BE节点的指定部署路径中。

      修改所有BE配置

        修改be/conf/be.conf，主要是配置storage_root_path：数据存储目录。默认情况下，在be/storage下，需要手动创建目录。使用；以分隔多个路径（不要添加；在最后一个目录之后）。

You may specify the directory storage medium in the path: HDD or SSD. You may also add capacility limit to the end of every path and use , for separation. Unless you use a mix of SSD and HDD disks, you do not need to follow the configuration methods in Example 1 and Example 2 below, but only need to specify the storage directory; you do not need to modify the default storage medium configuration of FE, either.

翻译:

您可以在路径中指定目录存储介质：HDD或SSD。您也可以在每条路径的末尾添加容量限制，并使用，以进行分离。除非混合使用SSD和HDD磁盘，否则不需要按照下面示例1和示例2中的配置方法进行配置，只需要指定存储目录即可；您也不需要修改FE的默认存储介质配置。

Example 1:

Note: For SSD disks, add .SSD to the end of the directory; for HDD disks, add .HDD.

翻译:

示例1：

注意：对于SSD磁盘，请添加。SSD到目录的末尾；对于HDD磁盘，请添加。HDD。

`storage_root_path=/home/disk1/doris.HDD;/home/disk2/doris.SSD;/home/disk2/doris`

Description

* 1./home/disk1/doris.HDD: The storage medium is HDD;
* 2./home/disk2/doris.SSD: The storage medium is SSD;
* 3./home/disk2/doris: The storage medium is HDD (default).

Example 2:

Note: You do not need to add the .SSD or .HDD suffix, but to specify the medium in the storage_root_path parameter

翻译:

示例2：

注意：您不需要添加.SSD或.HDD后缀，但要在storage_root_path参数中指定介质

`storage_root_path=/home/disk1/doris,medium:HDD;/home/disk2/doris,medium:SSD`
  

Description

* 1./home/disk1/doris,medium:HDD  :  The storage medium is HDD;
* 2./home/disk2/doris,medium:SSD  :  The storage medium is SSD.

• BE webserver_port configuration

  If the BE component is installed in hadoop cluster, you need to change configuration webserver_port=8040 to avoid port used.

        BE Web服务器端口配置

        如果BE组件安装在hadoop集群中，则需要更改配置webserver_port=8040以避免使用端口。

• Set JAVA_HOME environment variable

  Since Version 1.2.0 Java UDF is supported since version 1.2, so BEs are dependent on the Java environment. It is necessary to set the `JAVA_HOME` environment variable before starting. You can also do this by adding `export JAVA_HOME=your_java_home_path` to the first line of the `start_be.sh` startup script.

   

        设置JAVA_HOME环境变量

        由于1.2版本支持1.2.0版本的Java UDF，因此BE依赖于Java环境。启动前必须设置“JAVA_HOME”环境变量。您还可以通过将“export JAVA_HOME=your_JAVA_HOME_path”添加到“start_be.sh”启动脚本的第一行来完成此操作。

• Install Java UDF

  Since Version 1.2.0 Because Java UDF is supported since version 1.2, you need to download the JAR package of Java UDF from the official website and put them under the lib directory of BE, otherwise it may fail to start.

        安装Java UDF

        1.2.0版本起由于1.2版本起支持Java UDF，需要从官网下载Java UDF的JAR包，放在BE的lib目录下，否则可能无法启动。

• Add all BE nodes to FE

  BE nodes need to be added in FE before they can join the cluster. You can use mysql-client (Download MySQL 5.7) to connect to FE:

  ./mysql-client -h fe_host -P query_port -uroot

  fe_host is the node IP where FE is located; query_port is in fe/conf/fe.conf; the root account is used by default and no password is required in login.

  After login, execute the following command to add all the BE host and heartbeat service port:

  ALTER SYSTEM ADD BACKEND "be_host:heartbeat_service_port";

  be_host is the node IP where BE is located; heartbeat_service_port is in be/conf/be.conf.

        将所有BE节点添加到FE

        BE节点需要在FE中添加，然后才能加入集群。您可以使用mysql客户端（下载mysql 5.7）连接到FE：

        ./mysql-client -h fe_host -P query_port -uroot

        fe.host是fe所在的节点IP；query_port位于fe/conf/fe.conf中；默认情况下使用根帐户，登录时不需要密码。

        登录后，执行以下命令添加所有BE主机和心跳服务端口：

        ALTER SYSTEM ADD BACKEND“be_host:heartbeat_service_port”；

        be_host是be所在的节点IP；heartbeat_service_port位于be/conf/be.conf中。

        

• Start BE

  bin/start_be.sh --daemon

  The BE process will start and go into the background for execution. Logs are stored in be/log/directory by default. If startup fails, you can view error messages by checking out be/log/be.log or be/log/be.out.

        翻译:

        BE进程将启动并进入后台执行。默认情况下，日志存储在be/log/目录中。如果启动失败，您可以通过签出be/log/be.log或be/log/be.out来查看错误消息。

• View BE status

  Connect to FE using mysql-client and execute SHOW PROC '/backends'; to view BE operation status. If everything goes well, the isAlive column should be true.

        翻译:

        使用mysql客户端连接到FE并执行SHOW PROC '/backends'；以查看BE操作状态。如果一切顺利，isAlive列应该为true。

(Optional) FS_Broker Deployment​

Broker is deployed as a plug-in, which is independent of Doris. If you need to import data from a third-party storage system, you need to deploy the corresponding Broker. By default, Doris provides fs_broker for HDFS reading and object storage (supporting S3 protocol). fs_broker is stateless and we recommend that you deploy a Broker for each FE and BE node.

翻译:

Broker被部署为一个插件，它独立于Doris。如果需要从第三方存储系统导入数据，则需要部署相应的Broker。默认情况下，Doris为HDFS读取和对象存储提供fs_broker（支持S3协议）。fs_broker是无状态的，我们建议您为每个FE和BE节点部署一个broker。

• Copy the corresponding Broker directory in the output directory of the source fs_broker to all the nodes that need to be deployed. It is recommended to keep the Broker directory on the same level as the BE or FE directories.

        翻译:

        将源fs_Broker的输出目录中对应的Broker目录复制到所有需要部署的节点。建议将Broker目录与BE或FE目录保持在同一级别。

• Modify the corresponding Broker configuration

  You can modify the configuration in the corresponding broker/conf/directory configuration file.

        翻译:

        修改相应的Broker配置

        您可以在相应的broker/conf/目录配置文件中修改配置。

• Start Broker

  bin/start_broker.sh --daemon

• Add Broker

  To let Doris FE and BE know which nodes Broker is on, add a list of Broker nodes by SQL command.

  Use mysql-client to connect the FE started, and execute the following commands:

  ALTER SYSTEM ADD BROKER broker_name "broker_host1:broker_ipc_port1","broker_host2:broker_ipc_port2",...;

  broker\_host is the Broker node ip; broker_ipc_port is in conf/apache_hdfs_broker.conf in the Broker configuration file.

• View Broker status

  Connect any started FE using mysql-client and execute the following command to view Broker status: SHOW PROC '/brokers';

        翻译:

        添加Broker

        要让Doris FE和BE知道Broker在哪个节点上，请通过SQL命令添加Broker节点列表。

        使用mysql客户端连接启动的FE，并执行以下命令：

        ALTER SYSTEM添加BROKER BROKER_name“BROKER_host1:BROKER_ipc_port1”，“BROKER_host2:BROKER_ipc _port2”，...；

        broker\_host是broker节点ip；broker_ipc_port位于broker配置文件的conf/apache_hdfs_broker.conf中。

        查看Broker状态

                使用mysql客户端连接任何已启动的FE，并执行以下命令来查看Broker状态：SHOW PROC'/brokers'；

Note: In production environments, daemons should be used to start all instances to ensure that processes are automatically pulled up after they exit, such as Supervisor. For daemon startup, in Doris 0.9.0 and previous versions, you need to remove the last & symbol in the start_xx.sh scripts. In Doris 0.10.0 and the subsequent versions, you may just call sh start_xx.sh directly to start.

翻译:

注意：在生产环境中，应该使用守护程序来启动所有实例，以确保进程在退出后自动启动，例如Supervisor。对于守护进程启动，在Doris 0.9.0和以前的版本中，您需要删除start_xx.sh脚本中的最后一个&符号。在Doris 0.10.0及后续版本中，您可以直接调用sh start_xx.sh来启动。

FAQ​

Process-Related Questions​

1. How can we know whether the FE process startup succeeds?

   After the FE process starts, metadata is loaded first. Based on the role of FE, you can see transfer from UNKNOWN to MASTER/FOLLOWER/OBSERVER in the log. Eventually, you will see the thrift server started log and can connect to FE through MySQL client, which indicates that FE started successfully.

   You can also check whether the startup was successful by connecting as follows:

   http://fe_host:fe_http_port/api/bootstrap

   If it returns:

   {"status":"OK","msg":"Success"}

   The startup is successful; otherwise, there may be problems.

   Note: If you can't see the information of boot failure in fe. log, you may check in fe. out.

翻译:

我们如何知道FE流程启动是否成功？

FE过程开始后，首先加载元数据。根据FE的角色，您可以在日志中看到从UNKNOWN到MASTER/FOLOWER/OBSERVER的转移。最终，您将看到节俭服务器启动的日志，并可以通过MySQL客户端连接到FE，这表明FE成功启动。

您还可以通过以下连接来检查启动是否成功：

http://fe_host:fe_http_port/api/bootstrap

如果返回：

｛“status”：“OK”，“msg”：“Success”｝

启动成功；否则，可能会出现问题。

注意：如果在fe中看不到引导失败的信息。日志，您可以在fe中查看.out

2.  How can we know whether the BE process startup succeeds?

After the BE process starts, if there have been data there before, it might need several minutes for data index loading.

If BE is started for the first time or the BE has not joined any cluster, the BE log will periodically scroll the words waiting to receive first heartbeat from frontend, meaning that BE has not received the Master's address through FE's heartbeat and is waiting passively. Such error log will disappear after sending the heartbeat by ADD BACKEND in FE. If the word master client', get client from cache failed. host:, port: 0, code: 7 appears after receiving the heartbeat, it indicates that FE has successfully connected BE, but BE cannot actively connect FE. You may need to check the connectivity of rpc_port from BE to FE.

If BE has been added to the cluster, the heartbeat log from FE will be scrolled every five seconds: get heartbeat, host:xx. xx.xx.xx, port:9020, cluster id:xxxxxxx, indicating that the heartbeat is normal.

Secondly, if the word finish report task success. return code: 0 is scrolled every 10 seconds in the log, that indicates that the BE-to-FE communication is normal.

Meanwhile, if there is a data query, you will see the rolling logs and the execute time is xxx logs, indicating that BE is started successfully, and the query is normal.

You can also check whether the startup was successful by connecting as follows:

http://be_host:be_http_port/api/health

If it returns:

{"status": "OK","msg": "To Be Added"}

That means the startup is successful; otherwise, there may be problems.

Note: If you can't see the information of boot failure in be.INFO, you may see it in be.out.

翻译:

2. 我们如何知道BE进程启动是否成功？

BE进程启动后，如果以前有数据，则可能需要几分钟的时间来加载数据索引。

如果BE是第一次启动，或者BE没有加入任何集群，BE日志将定期滚动等待从前端接收第一个心跳的单词，这意味着BE没有通过FE的心跳接收到Master的地址，而是被动等待。这样的错误日志将在FE中的ADD BACKEND发送心跳后消失。如果单词master client，则从缓存获取客户端失败。主机：，端口：0，代码：7接收到心跳后出现，表示FE已成功连接BE，但BE无法主动连接FE。您可能需要检查rpc_port从BE到FE的连接。

如果BE已添加到集群中，则FE的心跳日志将每五秒钟滚动一次：getheartbeat，host:xx。xx.xx.xx，端口：9020，集群id:xxxxxxx，表示心跳正常。

其次，如果单词完成报告任务成功。返回代码：日志中每10秒滚动一次0，表示BE到FE通信正常。

同时，如果有数据查询，会看到滚动日志，执行时间为xxx日志，表示BE启动成功，查询正常。

您还可以通过以下连接来检查启动是否成功：

http://be_host:be_http_port/api/health

如果返回：

｛“status”：“OK”，“msg”：“待添加”｝

这意味着启动是成功的；否则，可能会出现问题。

注意：如果你在be.INFO中看不到引导失败的信息，你可以在be.out中看到。

3. How can we confirm that the connectivity of FE and BE is normal after building the system?

Firstly, you need to confirm that FE and BE processes have been started separately and worked normally. Then, you need to confirm that all nodes have been added through ADD BACKEND or ADD FOLLOWER/OBSERVER statements.

If the heartbeat is normal, BE logs will show get heartbeat, host:xx.xx.xx.xx, port:9020, cluster id:xxxxx; if the heartbeat fails, you will see backend [10001] got Exception: org.apache.thrift.transport.TTransportException in FE's log, or other thrift communication abnormal log, indicating that the heartbeat from FE to 10001 BE fails. Here you need to check the connectivity of the FE to BE host heartbeat port.

If the BE-to-FE communication is normal, the BE log will display the words finish report task success. return code: 0. Otherwise, the words master client, get client from cache failed will appear. In this case, you need to check the connectivity of BE to the rpc_port of FE.

1. What is the Doris node authentication mechanism?

   In addition to Master FE, the other role nodes (Follower FE, Observer FE, Backend) need to register to the cluster through the ALTER SYSTEM ADD statement before joining the cluster.

   When Master FE is started for the first time, a cluster_id is generated in the doris-meta/image/VERSION file.

   When FE joins the cluster for the first time, it first retrieves the file from Master FE. Each subsequent reconnection between FEs (FE reboot) checks whether its cluster ID is the same as that of other existing FEs. If it is not the same, the FE will exit automatically.

   When BE first receives the heartbeat of Master FE, it gets the cluster ID from the heartbeat and records it in the cluster_id file of the data directory. Each heartbeat after that compares that to the cluster ID sent by FE. If the cluster IDs are not matched, BE will refuse to respond to FE's heartbeat.

   The heartbeat also contains Master FE's IP. If the Master FE changes, the new Master FE will send the heartbeat to BE together with its own IP, and BE will update the Master FE IP it saved.

         翻译:

        什么是Doris节点身份验证机制？

        除了Master FE之外，其他角色节点（Follower FE、Observer FE、Backend）在加入集群之前还需要通过ALTER SYSTEM ADD语句注册到集群。

        当第一次启动Master FE时，会在doris meta/image/VERSION文件中生成cluster_id。

        当FE第一次加入集群时，它首先从主FE检索文件。FE之间的每次后续重新连接（FE重新启动）都会检查其集群ID是否与其他现有FE的集群ID相同。如果不相同，FE将自动退出。

        BE第一次接收到Master FE的心跳时，会从心跳中获取集群ID，并将其记录在数据目录的cluster_ID文件中。之后的每个心跳将其与FE发送的集群ID进行比较。如果集群ID不匹配，BE将拒绝响应FE的心跳。

        心跳还包含主FE的IP。如果主FE发生变化，新的主FE将把心跳和它自己的IP一起发送给BE，BE将更新它保存的主FE IP。

1. priority_network

   priority_network is a configuration item that both FE and BE have. It is used to help FE or BE identify their own IP addresses in the cases of multi-network cards. priority_network uses CIDR notation: RFC 4632

   If the connectivity of FE and BE is confirmed to be normal, but timeout still occurs in creating tables, and the FE log shows the error message backend does not find. host:xxxx.xxx.XXXX, this means that there is a problem with the IP address automatically identified by Doris and that the priority_network parameter needs to be set manually.

   The explanation to this error is as follows. When the user adds BE through the ADD BACKEND statement, FE recognizes whether the statement specifies hostname or IP. If it is a hostname, FE automatically converts it to an IP address and stores it in the metadata. When BE reports on the completion of the task, it carries its own IP address. If FE finds that the IP address reported by BE is different from that in the metadata, the above error message will show up.

   Solutions to this error: 1) Set priority_network in FE and BE separately. Usually FE and BE are in one network segment, so their priority_network can be set to be the same. 2) Put the correct IP address instead of the hostname in the ADD BACKEND statement to prevent FE from getting the wrong IP address

翻译:

优先级_网络

priority_network是FE和BE都具有的配置项。在多网卡的情况下，它用于帮助FE或BE识别自己的IP地址。priority_network使用CIDR表示法：RFC 4632

如果FE和BE的连接被确认为正常，但在创建表时仍然发生超时，并且FE日志显示后端找不到错误消息。host:xxxx.xxx.xxx，这意味着Doris自动识别的IP地址有问题，需要手动设置priority_network参数。

对此错误的解释如下。当用户通过ADD BACKEND语句添加BE时，FE会识别该语句指定的是主机名还是IP。如果是主机名，FE会自动将其转换为IP地址，并将其存储在元数据中。当BE报告任务完成时，它会携带自己的IP地址。如果FE发现BE报告的IP地址与元数据中的不同，则会显示上述错误消息。

此错误的解决方案：1）分别在FE和BE中设置priority_network。通常FE和BE在一个网段中，因此它们的优先级_network可以设置为相同。2） 在ADD BACKEND语句中输入正确的IP地址而不是主机名，以防止FE获取错误的IP地址。

​

5. What is the number of file descriptors of a BE process?

The number of file descriptor of a BE process is determined by two parameters: min_file_descriptor_number/max_file_descriptor_number.

If it is not in the range [min_file_descriptor_number, max_file_descriptor_number], error will occurs when starting a BE process. You may use the ulimit command to reset the parameters.

The default value of min_file_descriptor_number is 65536.

The default value of max_file_descriptor_number is 131072.

For example, the command ulimit -n 65536; means to set the number of file descriptors to 65536.

After starting a BE process, you can use cat /proc/$pid/limits to check the actual number of file descriptors of the process.

If you have used supervisord and encountered a file descriptor error, you can fix it by modifying minfds in supervisord.conf.

翻译:

BE进程的文件描述符数量是多少？

BE进程的文件描述符数量由两个参数决定：最小文件描述符数量/最大文件描述符数量。

如果它不在范围[min_file_descriptor_number，max_file_desscriptor_number]内，则启动BE进程时将发生错误。您可以使用ulimit命令重置参数。

min_file_descriptor_number的默认值为65536。

max_file_descriptor_number的默认值为131072。

例如，命令ulimit-n 65536；意味着将文件描述符的数量设置为65536。

启动BE进程后，可以使用cat/proc/$pid/limits检查进程的文件描述符的实际数量。

如果您使用过supervisord并遇到文件描述符错误，可以通过在supervisord.conf中修改minfds来修复它。