作业HUE01： 大数据可视化工具Hue安装使用

下载依赖包

yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi gcc gcc-c++ krb5-devel libtidy libxml2-devel libxslt-devel openldap-devel python-devel sqlite-devel openssl-devel gmp-devel

获取源代码

$ git clone https://github.com/cloudera/hue.git
hue-3.7.0-cdh5.3.6.tar.gz

解压

进入路径

$ cd hue

编译

$ make apps

hadoop配置文件修改

hdfs-site.xml

<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.secondary.http-address</name>
<value>hadoop0:50090</value>
</property>

<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>
<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>
</configuration>

core-site.xml

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop0:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/jit/cdh356/hadoop-2.5.0-cdh5.3.6/data</value>
</property>
<property>
  <name>hadoop.http.staticuser.user</name>
  <value>jit</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.hosts</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.hue.groups</name>
  <value>*</value>
</property>
</configuration>

修改hive-site.xml配置文件

配置metastore

<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hadoop0:3306/hive?createDatabaseIfNotExist=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
</property>
<property>
  <name>hive.metastore.uris</name>
  <value>thrift://hadoop0:9083</value>
</property>
</configuration>

启动metastore 服务

[jit@hadoop0 hive-0.13.1-cdh5.3.6]$ nohup bin/hive --service metastore

启动hiveserver2

[jit@hadoop0 hive-0.13.1-cdh5.3.6]$ bin/hiveserver2

如果出现访问权的错误则执行

hdfs dfs -chmod -R o+x /tmp

修改配置文件desktop/conf/hue.ini

[desktop]

  # Set this to a random string, the longer the better.
  # This is used for secure hashing in the session store.
  secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o

  # Webserver listens on this address and port
  http_host=hadoop0
  http_port=8888

  # Time zone name
  time_zone=Asia/Shanghai

  # Enable or disable Django debug mode.
  django_debug_mode=false

  # Enable or disable backtrace for server error
  http_500_debug_mode=false
#---------------------------------------
[[[mysql]]]
      # Name to show in the UI.
    nice_name="My SQL DB"

      # For MySQL and PostgreSQL, name is the name of the database.
      # For Oracle, Name is instance of the Oracle server. For express edition
      # this is 'xe' by default.
      ## name=mysqldb

      # Database backend to use. This can be:
      # 1. mysql
      # 2. postgresql
      # 3. oracle
     engine=mysql

      # IP or hostname of the database to connect to.
     host=hadoop0

      # Port the database server is listening to. Defaults are:
      # 1. MySQL: 3306
      # 2. PostgreSQL: 5432
      # 3. Oracle Express Edition: 1521
      port=3306

      # Username to authenticate with when connecting to the database.
       user=hive

      # Password matching the username to authenticate with when
      # connecting to the database.
      password=hive
#---------------------------------------
[hadoop]

  # Configuration for HDFS NameNode
  # ------------------------------------------------------------------------
  [[hdfs_clusters]]
    # HA support by using HttpFs

    [[[default]]]
      # Enter the filesystem uri
      fs_defaultfs=hdfs://hadoop0:8020

      # NameNode logical name.
      ## logical_name=

      # Use WebHdfs/HttpFs as the communication mechanism.
      # Domain should be the NameNode or HttpFs host.
      # Default port is 14000 for HttpFs.
      webhdfs_url=http://hadoop0:50070/webhdfs/v1

      # Change this if your HDFS cluster is Kerberos-secured
      ## security_enabled=false

      # Default umask for file and directory creation, specified in an octal value.
      ## umask=022
       hadoop_bin=/home/jit/cdh356/hadoop-2.5.0-cdh5.3.6/bin
    hadoop_hdfs_home=/home/jit/cdh356/hadoop-2.5.0-cdh5.3.6/
      # Directory of the Hadoop configuration
    hadoop_conf_dir=/home/jit/cdh356/hadoop-2.5.0-cdh5.3.6/etc/hadoop

  # Configuration for YARN (MR2)
  # ------------------------------------------------------------------------
  [[yarn_clusters]]

    [[[default]]]
      # Enter the host on which you are running the ResourceManager
     resourcemanager_host=hadoop0

      # The port where the ResourceManager IPC listens on
      resourcemanager_port=8032

      # Whether to submit jobs to this cluster
      submit_to=True

      # Resource Manager logical name (required for HA)
      ## logical_name=

      # Change this if your YARN cluster is Kerberos-secured
      ## security_enabled=false

      # URL of the ResourceManager API
     resourcemanager_api_url=http://hadoop0:8088

      # URL of the ProxyServer API
     proxy_api_url=http://hadoop0:8088

      # URL of the HistoryServer API
     history_server_api_url=http://hadoop0:19888
#---------------------------------------
[beeswax]

  # Host where HiveServer2 is running.
  # If Kerberos security is enabled, use fully-qualified domain name (FQDN).
 hive_server_host=hadoop0

  # Port where HiveServer2 Thrift server runs on.
hive_server_port=10000

  # Hive configuration directory, where hive-site.xml is located
 hive_conf_dir=/opt/cdh/hive-0.13.1-cdh5.3.6/conf

  # Timeout in seconds for thrift calls to Hive service
  server_conn_timeout=120

启动
sbin/mr-jobhistory-daemon.sh start historyserver
启动
build/env/bin/supervisor

hdfs和yarn的集成效果。如果上传有权限错误，这修改相应权限

如果jobbrower没有信息，可以尝试删除用户名admin

查询编辑器hive

查询编辑器 DB查询 mysql