Presto 配置

Presto架构图

图片 12.png

下载presto-server-317.tar.gz
解压到/opt/app/presto-server-317,
建立软连接：ln -s /opt/app/presto-server-317 /opt/app/presto-server

/etc/profile

export PRESTO_HOME=/opt/app/presto-server

export HADOOP_CLASSPATH=`hadoop classpath`
export HADOOP_CONF_DIR=/etc/hadoop/conf
export SPARK_MAJOR_VERSION=2

export FLUME_HOME=/opt/app/apache-flume-1.9.0-bin
export PATH=$FLUME_HOME/bin:$PRESTO_HOME/bin:$FLINK_HOME/bin:$JAVA_HOME/bin:.:$PATH

整体目录

[umecron@umetrip-hdp26-147xxx presto-server]$ ll
total 208
drwxr-xr-x  3 umecron hadoop    144 Mar 29 19:17 bin
drwxr-xr-x  3 umecron hadoop    133 Jul 30 21:08 etc
drwxr-xr-x  2 umecron hadoop   8192 Feb 22 10:46 lib
-rw-r--r--  1 umecron hadoop 191539 Feb 22 10:46 NOTICE
drwxr-xr-x 30 umecron hadoop   4096 Feb 22 10:46 plugin
-rw-r--r--  1 umecron hadoop    126 Feb 22 10:46 README.txt

etc 目录

[umecron@umetrip-hdp26-147xxx presto-server]$ ll etc/
total 16
drwxr-xr-x 2 umecron hadoop  37 Jul 31 16:24 catalog
-rw-r--r-- 1 umecron hadoop 368 Apr 28 14:38 config.properties
-rw-r--r-- 1 umecron hadoop 244 Jul 30 21:08 jvm.config
-rw-r--r-- 1 umecron hadoop  25 Mar 25 15:04 log.properties
-rw-r--r-- 1 umecron hadoop  87 Apr 28 14:33 node.properties

1.config.properties

coordinator配置文件

coordinator=true
node-scheduler.include-coordinator=true

http-server.http.port=8084
http-server.threads.max=1500

query.max-memory=30GB
query.max-memory-per-node=10GB
query.max-total-memory-per-node=15GB
query.max-length=100000000

discovery-server.enabled=true
discovery.uri=http://10.5.xxx.xxx:8084

task.max-worker-threads=32

failure-detector.threshold=0.10

exchange.min-error-duration=30m
exchange.max-error-duration=150m
exchange.http-client.request-timeout=30m

woker配置文件

[umecron@umetrip-hdp26-147xxx etc]$  cat config.properties 
coordinator=false

http-server.http.port=8084
http-server.threads.max=1500

query.max-memory=30GB
query.max-memory-per-node=10GB
query.max-total-memory-per-node=15GB
query.max-length=100000000

discovery.uri=http://10.5.xxx.xxx:8084 # 与coordinator相同

task.max-worker-threads=32

exchange.min-error-duration=30m
exchange.max-error-duration=150m
exchange.http-client.request-timeout=30m

2.jvm.config

[umecron@umetrip-hdp26-147xxx etc]$ cat jvm.config 
-server
-Xmx64G
-XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent
-XX:+CMSClassUnloadingEnabled
-XX:+AggressiveOpts
-XX:+HeapDumpOnOutOfMemoryError
-XX:OnOutOfMemoryError=kill -9 %p
-XX:ReservedCodeCacheSize=150M
#-DHADOOP_USER_NAME=hdfs

3.node.properties

[umecron@umetrip-hdp26-147xxx etc]$ cat node.properties 
node.environment=uat
node.id=10-5-xxx-xxx
node.data-dir=/opt/appdata/disk01/log/presto

4. catalog/hive.properties

[umecron@umetrip-hdp26-147xxx etc]$ cat  catalog/hive.properties 
connector.name=hive-hadoop2
hive.metastore.uri=thrift://10.5.147.xxx:9083
hive.config.resources=/etc/hadoop/conf/core-site.xml,/etc/hadoop/conf/hdfs-site.xml

hive.allow-drop-table=true
#hive.storage-format=PARQUET
hive.metastore-cache-ttl=0s
hive.metastore-refresh-interval=1s
hive.max-partitions-per-writers=1000
#hive.metastore.username=hdfs

#hive.hdfs.authentication.type=NONE
#hive.hdfs.impersonation.enabled=true

Server命令：

./launcher.py status
./launcher.py start
./launcher.py stop
./launcher.py restart

Cli 命令

./presto-cli --server 10.5.xxx.xxx:8084 --catalog hive --schema default
--server 指定coordinator的IP
--catalog 指定哪种类型数据源
--schema 数据库名

presto 中schema即database.