编译hadoop

$ $HADOOP_HOME/bin/hadoop fs -mkdir /tmp

$ $HADOOP_HOME/bin/hadoop fs -mkdir /user/hive/warehouse

$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /tmp

$ $HADOOP_HOME/bin/hadoop fs -chmod g+w /user/hive/warehouse


出现报错:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

原因是hadoop从网站下载时,是32bit,而系统是64bit,需要下载源码本地编译,然后将编译好的库放入到环境中


源码安装hadoop

mvn package -Pdist,native -DskipTests -Dtar


HIVE debug模式

./hive -hiveconf hive.root.logger=DEBUG,console


Hadoop debug模式

export HADOOP_ROOT_LOGGER=DEBUG,console


执行hive出错:


14/05/29 11:03:57 [main]: ERROR metastore.MetaStoreDirectSql: Self-test query [select "DB_ID" from "DBS"] failed; direct SQL is disabled
javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS"".
        at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451)


死活找不到问题,换了HIVE版本,好了

从 apache-hive-0.13.0-bin该文apache-hive-0.12.0-bin


启动 hive --service hwi 

报错

FATAL hwi.HWIServer: HWI WAR file not found at /usr/local/hadoop/hive/usr/local/hadoop/hive/lib/hive-hwi-0.12.0.war

在hive-site.xml中写入:

        <property>
                <name>hive.hwi.war.file</name>
                <value>lib/hive-hwi-0.12.0.war</value>         
                <description>This sets the path to the HWI war file, relative to ${HIVE_HOME}. </descr
iption>
        </property> 


浏览器连接:

http://10.58.49.255:9999/hwi/


启动php hive

hive --service hiveserver >/dev/null 2>/dev/null &


spawn-fcgi -a 127.0.0.1 -p 3334 -C 10 -u www-data -f /usr/bin/php-cgi



jps 显示进程


hdfs  getconf -namenodes

显示namenodes


编译hadoop java源码,替换

share/hadoop/common/hadoop-common-2.4.0.jar


问题:

Problem accessing /hwi/. Reason:

    No Java compiler available

cp lib/ant.jar hive/lib/

cp lib/ant-launcher.jar hive/lib/


cp /usr/lib/jvm/java-6-oracle/lib/tools.jar /usr/lib/jvm/java-6-oracle/jre/lib/


    Unable to find a javac compiler;
com.sun.tools.javac.Main is not on the classpath.
Perhaps JAVA_HOME does not point to the JDK.
It is currently set to "/usr/lib/jvm/java-6-oracle/jre"

cp /usr/lib/jvm/java-6-oracle/lib/tools.jar lib/


<name>dfs.http.address</name>
<value>bspspo:50070</value>
</property>
<property>
<name>dfs.secondary.http.address</name>
<value>bspspo:50090</value>
</property>
<property>
<name>dfs.namenode.backup.address</name>
<value>bspspo:9002</value>
</property>
<property>
<name>dfs.namenode.backup.http-address</name>
<value>bspspo:9003</value>
</property>


Hive 快速入门

https://cwiki.apache.org/confluence/display/Hive/GettingStarted

Hive语言手册

https://cwiki.apache.org/confluence/display/Hive/LanguageManual

Hive指南

https://cwiki.apache.org/confluence/display/Hive/Tutorial


你可能感兴趣的:(编译hadoop)