hadoop学习笔记

 

1 hadoop linux环境搭建参考

 a 安装jdk 

http://blog.csdn.net/yang_hui1986527/article/details/6677450

 b 搭建hadoop

http://hadoop.apache.org/common/docs/r1.0.3/single_node_setup.html

ssh启动伪分布式hadoop需要用root去操作，要先su root一下。 

root密码设定

http://tech.ddvip.com/2007-09/119040780435337.html

 

2 常见问题集

http://pages.cs.brandeis.edu/~cs147a/lab/hadoop-troubleshooting/ 

 

3 配置不高的话，使用伪分布式容易死机。

本机实践可以考虑用单机版即可

 

4 ssh作用是配置免登陆，方便文件在namenode和datanode的复制，以及master去操作slave的进程

http://www.hadoopor.com/thread-837-1-1.html

 

5 window

http://blog.csdn.net/savechina/article/details/5656937

http://hayesdavis.net/2008/06/14/running-hadoop-on-windows/

 

 java home配置，应对路径空格的方案

export JAVA_HOME=/cygdrive/c/Progra~1/Java/jdk1.7.0_01

 

windows文件权限问题

Failed to set permissions of path: \tmp\hadoop-

http://hi.baidu.com/fedora_12/item/43a8c4baafbfbbf963388eeb

 

eclipse

http://blog.csdn.net/yanical/article/details/4474830

http://wiki.apache.org/hadoop/EclipsePlugIn?rename=EclipseMain.png&action=AttachFile&ticket=0050449d3e.0369edc75bb8d724861ad27d193a8c6dd91e1252&drawing=EclipseMain

 

window 调试环境，伪分布式依然存在问题

蛋疼的could only be replicated to 0 nodes, instead of 1