IP |
Hostname |
角色 |
软件 |
备注 |
192.168.56.11 |
Hbase1 |
Coordinator Presto-Sql-ui |
presto-server yanagishima |
|
192.168.56.12 |
Hbase2 |
Worker |
presto-server |
|
192.168.56.13 |
Hbase3 |
Worker |
presto-server |
|
范围:hbase1
命令:
tar -xvf jdk-8u181-linux-x64.tar.gz
tar -xvf presto-server-0.231.1.tar.gz
范围:hbase1/hbase2/hbase3
1)命令:(添加红色内容)
vi /etc/profile
export JAVA_HOME=/opt/jdk1.8.0_181
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
2)命令:
source /etc/profile
3)命令:
java -version
范围:hbase1
命令:
mkdir -p /opt/presto-server-0.231.1/etc
mkdir -p /opt/presto-server-0.231.1/data
mkdir -p /opt/presto-server-0.231.1/etc/catalog/
【说明】
etc文件夹存放配置信息
data文件夹内存放查询的缓存信息
etc/catalog文件夹下存放连接信息
范围:hbase1
1)命令:
vi /opt/presto-server-0.231.1/etc/node.properties
node.environment=production
node.id=ffffffff-ffff-ffff-ffff-fffffffffff1
node.data-dir=/opt/presto-server-0.231.1/data
2)命令:
vi /opt/presto-server-0.231.1/etc/config.properties
coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8080
query.max-memory=2GB
query.max-memory-per-node=1GB
query.max-total-memory-per-node=1GB
discovery-server.enabled=true
discovery.uri=http://192.168.56.11:8080
3)命令:
vi /opt/presto-server-0.231.1/etc/jvm.config
-server
-Xmx2G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError
范围:hbase1
命令:
scp -r /opt/presto-server-0.231.1 [email protected]:/opt/presto-server-0.231.1
scp -r /opt/presto-server-0.231.1 [email protected]:/opt/presto-server-0.231.1
修改配置
范围:hbase2/hbase3
1)命令:
vi /opt/presto-server-0.231.1/etc/node.properties
2)命令:
vi /opt/presto-server-0.231.1/etc/config.properties
范围:hbase1/hbase2/hbase3
命令:
/opt/presto-server-0.231.1/bin/launcher start
也可以用
/opt/presto-server-0.231.1/bin/launcher run
这两个命令的作用相同,run方式可立即观察日志
http://192.168.56.11:8080
presto是一个开源的分布式sql查询引擎,数据量支持GB到PB字节,主要用于处理秒级查询的场合。yanagishima是presto的可视化客户端。
范围:hbase1
命令:
mkdir -p /opt/yanagishima-19.0/data
范围:hbase1
命令:
vi /opt/yanagishima-19.0/conf/yanagishima.properties
jetty.port=7080
presto.query.max-run-time-seconds=1800
presto.max-result-file-byte-size=1073741824
presto.datasources=zjs-presto
auth.zjs-presto=false
presto.coordinator.server.zjs-presto=http://192.168.56.11:8080
presto.redirect.server.zjs-presto=http://192.168.56.11:8080
catalog.zjs-presto=mysql
schema.zjs-presto=crm
select.limit=500
audit.http.header.name=some.auth.header
use.audit.http.header.name=false
to.values.query.limit=500
check.datasource=false
sql.query.engines=presto,hive,mysql,mongo,sqlserver
cors.enabled=false
范围:hbase1
命令:
/opt/yanagishima-19.0/bin/yanagishima-start.sh
在etc/catalog下创建不同数据库的连接配置信息
范围:hbase1
命令:
vi /opt/presto-server-0.231.1/etc/catalog/mysql001.properties
connector.name=mysql
connection-url=jdbc:mysql://192.168.56.11:3306
connection-user=root
connection-password=hive
范围:hbase1
命令:
vi /opt/presto-server-0.231.1/etc/catalog/mongodb001.properties
connector.name=mongodb
mongodb.seeds=192.168.56.12:27017
mongodb.schema-collection=admin
范围:hbase1
命令:
vi /opt/presto-server-0.231.1/etc/catalog/sqlserver001.properties
connector.name=sqlserver
connection-url=jdbc:sqlserver://192.168.56.13:1433
connection-user=sa
connection-password=R*****
范围:hbase1
命令:
vi /opt/presto-server-0.231.1/etc/catalog/redis001.properties
connector.name=redis
redis.table-names=schema1.table1,schema1.table2
redis.nodes=host:port
范围:hbase1
命令:
scp -r /opt/presto-server-0.231.1/etc/catalog/*.properties [email protected]:/opt/presto-server-0.231.1/etc/catalog/
scp -r /opt/presto-server-0.231.1/etc/catalog/*.properties [email protected]:/opt/presto-server-0.231.1/etc/catalog/
范围:hbase1/hbase2/hbase3
1)命令:
/opt/presto-server-0.231.1/bin/launcher stop
2)命令:
/opt/presto-server-0.231.1/bin/launcher start
范围:hbase1
1)命令:
/opt/yanagishima-19.0/bin/yanagishima-shutdown.sh
2)命令:
/opt/yanagishima-19.0/bin/yanagishima-start.sh
登录http://192.168.56.11:7080
执行以下语句,这里只选取了sqlserver、mysql、mongodb三种数据库类型
SELECT * FROM sqlserver001.dbo.MSreplication_options as a
left join mysql001.crm.user as b on b.name=a.optname
left join mongodb001.admin.test as c on c.name=b.name
执行成功的结果如下: