apache druid(时序数据库)集群搭建

准备环境

mysql、zookeeper、hadoop(HDFS)环境
jdk1.8(每台druid服务器都需要安装)

druid的服务器列表

服务器地址	服务器角色	备注
10.32.10.240	druid-query	Broker、Router
10.32.10.241	druid-query	Broker、Router
10.32.10.242	druid-master	coordinator、overlord
10.32.10.243	druid-master	coordinator、overlord
10.32.10.244	druid-data	Historical、MiddleManager
10.32.10.245	druid-data	Historical、MiddleManager

开始搭建druid集群

下载apache-druid-0.18.0-bin.tar.gz并上传到服务器(/mnt/druid)

所有服务器都需要操作

进入/mnt/druid，解压tar包

  tar -zxvf apache-druid-0.18.0-bin.tar.gz

进行druid的集群配置共公目录/mnt/druid/apache-druid-0.18.0/conf/druid/cluster/_common

  cd apache-druid-0.18.0/conf/druid/cluster/_common/

配置common.runtime.properties

  vim common.runtime.properties

修改common.runtime.properties配置文件

druid.extensions.loadList=["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "mysql->metadata-storage", "druid-hdfs-storage", "druid-kafka-indexing-service"]
druid.host=10.32.10.240  #本机服务器地址
### zookeeper配置
druid.zk.service.host=192.168.11.111:2181,192.168.11.112:2181,192.168.11.113:2181
druid.zk.paths.base=/druid018
# Metadata storage
# For MySQL (make sure to include the MySQL JDBC driver on the classpath):
druid.metadata.storage.type=mysql
druid.metadata.storage.connector.connectURI=jdbc:mysql://127.0.0.1:3306/druid018?characterEncoding=UTF8
druid.metadata.storage.connector.user=root
druid.metadata.storage.connector.password=root
# For HDFS:
druid.storage.type=hdfs
druid.storage.storageDirectory=hdfs://10.32.10.16:8020/druid018/segments
#
# Indexing service logs
#
# For HDFS:
druid.indexer.logs.type=hdfs
druid.indexer.logs.directory=hdfs://10.32.10.16:8020/druid018/indexing-logs

修改完成后，创建druid日志目录

  mkdir -p /mnt/druid/apache-druid-0.18.0/var/sv/

引入以下mysql包到apache-druid-0.18.0/extensions/mysql-metadata-storage目录

mysql-connector-java-5.1.48.jar、password-connector-java-5.1.48.jar

配置master coordinator-overlord（242 243 节点同步操作)

进行配置文件目录
  cd /mnt/druid/apache-druid-0.18.0/conf/druid/cluster/master/coordinator-overlord
jvm.config配置

-server
-Xms15g
-Xmx15g
-XX:+ExitOnOutOfMemoryError
-XX:+UseG1GC
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
-Dderby.stream.error.file=var/druid/derby.log

runtime.properties配置

druid.service=druid/coordinator
druid.plaintextPort=8081
druid.coordinator.startDelay=PT10S
druid.coordinator.period=PT5S
# Run the overlord service in the coordinator process
druid.coordinator.asOverlord.enabled=true
druid.coordinator.asOverlord.overlordService=druid/overlord
druid.indexer.queue.startDelay=PT5S
druid.indexer.runner.type=remote
druid.indexer.storage.type=metadata

启动druid的master节点
/apache-druid-0.18.0/bin/start-cluster-master-no-zk-server &

配置 data节点 MiddleManager Historical （244 245节点同时操作）

MiddleManager配置：
进行配置文件目录
  cd /mnt/druid/apache-druid-0.18.0/conf/druid/cluster/data/middleManager
jvm.config配置

-server
-Xms1g
-Xmx1g
-XX:+ExitOnOutOfMemoryError
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager

runtime.properties配置

druid.service=druid/middleManager
druid.plaintextPort=8091
# Number of tasks per middleManager
druid.worker.capacity=4
# Task launch parameters
druid.indexer.runner.javaOpts=-server -Xms1g -Xmx1g -XX:MaxDirectMemorySize=1g ->Duser.timezone=UTC -Dfile.encoding=UTF-8 -XX:+ExitOnOutOfMemoryError ->Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
druid.indexer.task.baseTaskDir=var/druid/task
# HTTP server threads
druid.server.http.numThreads=60
# Processing threads and buffers on Peons
druid.indexer.fork.property.druid.processing.numMergeBuffers=2
druid.indexer.fork.property.druid.processing.buffer.sizeBytes=100000000
druid.indexer.fork.property.druid.processing.numThreads=1
# Hadoop indexing
druid.indexer.task.hadoopWorkingPath=var/druid/hadoop-tmp

Historical配置：
进行配置文件目录
  cd /mnt/druid/apache-druid-0.18.0/conf/druid/cluster/data/historical
jvm.config配置

-server
-Xms8g
-Xmx8g
-XX:MaxDirectMemorySize=13g
-XX:+ExitOnOutOfMemoryError
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
runtime.properties配置
druid.service=druid/historical
druid.plaintextPort=8083
# HTTP server threads
druid.server.http.numThreads=60
# Processing threads and buffers
druid.processing.buffer.sizeBytes=500000000
druid.processing.numMergeBuffers=4     #1:4 cpu核数
druid.processing.numThreads=15      # cpu核数-1
druid.processing.tmpDir=var/druid/processing
# Segment storage
druid.segmentCache.locations=[{"path":"var/druid/segment-cache","maxSize":300000000000}]
druid.server.maxSize=300000000000
# Query cache
druid.historical.cache.useCache=true
druid.historical.cache.populateCache=true
druid.cache.type=caffeine
druid.cache.sizeInBytes=256000000

启动命令：
/apache-druid-0.18.0/bin/start-cluster-data-server &

配置 query节点 broker & router(240 241节点同时操作）

borker配置：
进行配置文件目录
  cd /mnt/druid/apache-druid-0.18.0/conf/druid/cluster/query/broker
jvm.config配置

-server
-Xms12g
-Xmx12g
-XX:MaxDirectMemorySize=6g
-XX:+ExitOnOutOfMemoryError
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
runtime.properties配置
druid.service=druid/broker
druid.plaintextPort=8082
# HTTP server settings
druid.server.http.numThreads=60
# HTTP client settings
druid.broker.http.numConnections=50
druid.broker.http.maxQueuedBytes=10000000
# Processing threads and buffers
druid.processing.buffer.sizeBytes=500000000
druid.processing.numMergeBuffers=6
druid.processing.numThreads=1
druid.processing.tmpDir=var/druid/processing
# Query cache disabled -- push down caching and merging instead
druid.broker.cache.useCache=false
druid.broker.cache.populateCache=false

borker配置：
进行配置文件目录
  cd /mnt/druid/apache-druid-0.18.0/conf/druid/cluster/query/router
jvm.config配置

-server
-Xms1g
-Xmx1g
-XX:+UseG1GC
-XX:MaxDirectMemorySize=128m
-XX:+ExitOnOutOfMemoryError
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
runtime.properties配置
druid.service=druid/router
druid.plaintextPort=8888
# HTTP proxy
druid.router.http.numConnections=50
druid.router.http.readTimeout=PT5M
druid.router.http.numMaxThreads=100
druid.server.http.numThreads=100
# Service discovery
druid.router.defaultBrokerServiceName=druid/broker
druid.router.coordinatorServiceName=druid/coordinator
# Management proxy to coordinator / overlord: required for unified web console.
druid.router.managementProxy.enabled=true

启动命令：
/apache-druid-0.18.0/bin/start-cluster-query-server &

配置完成后，由druid的query节点的8888端口提供可视化页面，可以通过nginx来反向代理10.32.10.240:8888和10.32.10.241:8888

注意：启动完成后要通过exit退出服务器连接，不能直接关闭窗口或断开连接，不然druid的后台进程也会被关闭