HPLSQL(Hadoop存储过程实现)的安装和简单应用

HPLSQL搭建所需环境：

 

• Java 1.6 or higher
• Hadoop 1.x. and 2.x

 

1.下载安装包并授权

Chmod +x /hplsql

2.环境变量设置

vi hplsql  配置

Hadoop环境变量

和hadoopjar包的环境变量

export "HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH/jars/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/lib/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_CONF_DIR/"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/mapreduce/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/mapreduce/lib/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/hdfs/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/hdfs/lib/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/yarn/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/yarn/lib/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/common/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HADOOP_HOME/share/hadoop/common/lib/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_HOME/lib/*"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_HOME/conf"

export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"

3.HPLSQL安装

/hplsql --version

HPL/SQL x.x.x

4.添加HPLSQL环境变量配置

export PATH=$PATH:

4.1配置队列

如果hivesever2没有启动，需要配置并启动

4.2vi hplsql-site.xml

配置：

--设置默认连接

hplsql.conn.default

hive2conn

The default connection profile

--连接hive2

hplsql.conn.hive2conn

org.apache.hive.jdbc.HiveDriver;jdbc:hive2://127.16.21.17:10000

HiveServer2 JDBC connection

--可以不添加

hplsql.conn.hive.conn

org.apache.hive.jdbc.HiveDriver;jdbc:hive://

Hive embedded JDBC (not requiring Hiveserver)

--设置队列、资源获取、

hplsql.conn.init.hive2conn

         set mapred.job.queue.name=default;

         set hive.execution.engine=mr;

         use _default;

Statements for execute after connection to the database

 

hplsql.conn.covert.hiveconn

true

Convert SQL statements before execution

 

5.运行hplsql与应用

hplsql -e "CURRENT_DATE+1"

 

hplsql -e "SELECT * FROM src LIMIT 1"

or

 

②hplsql -f script.sql

Use HPL/SQL in Shell Scripts

Get a value from HPL/SQL script:

 

MDATE=$(hplsql -e "NVL(MIN_PARTITION_DATE(sales, local_dt, code='A'), '1970-01-01')")

START=$(hplsql -e 'CURRENT_DATE - 1')

 

可以通过 –main 参数指定要执行的部分

Hplsql –f script.sql –main funcation_name