Ubuntu下安装及配置单点hadoop

  环境： Ubuntu10.10， Hadoop 0.21.0首先准备工作：
1，我是在deskTop版本的Ubuntu下 安装的所以，需要先安装 ssh server。这个很好找，直接到到新立得里 搜索ssh，第一个就是。
2，安装sun jdk6，切忌一定要是java6及其以上版本。先到更新管理器里把canonical的源加进去。
    之后sudo apt-get update.
3, sudo apt-get sun-java6-jdk
4, sudo update-java-alternatives -s java-6-sun
5,增加一个用户组用户，用于hadoop 运行及访问。

1. sudo addgroup hadoop
   
2. sudo adduser --ingroup hadoop hadoop

复制代码

6,生成SSH证书， 配置SSH加密key

1. user@ubuntu:~$ su - hadoop
   
2. hadoop@ubuntu:~$ ssh-keygen -t rsa -P ""
   
3. Generating public/private rsa key pair.
   
4. Enter file in which to save the key (/home/hadoop/.ssh/id_rsa):
   
5. Created directory '/home/hadoop/.ssh'.
   
6. Your identification has been saved in /home/hadoop/.ssh/id_rsa.
   
7. Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
   
8. The key fingerprint is:
   
9. 9b:82:ea:58:b4:e0:35:d7:ff:19:66:a6:ef:ae:0e:d2 hadoop@ubuntu
   
10. The key's randomart image is:
    
11. [...snipp...]
    
12. hadoop@ubuntu:~$

复制代码

1. hadoop@ubuntu:~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

复制代码

配置完成，测试一下：

1. hadoop@ubuntu:~$ ssh localhost
   
2. The authenticity of host 'localhost (::1)' can't be established.
   
3. RSA key fingerprint is d7:87:25:47:ae:02:00:eb:1d:75:4f:bb:44:f9:36:26.
   
4. Are you sure you want to continue connecting (yes/no)? yes
   
5. Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
   
6. Linux ubuntu 2.6.32-22-generic #33-Ubuntu SMP Wed Apr 28 13:27:30 UTC 2010 i686 GNU/Linux
   
7. Ubuntu 10.04 LTS
   
8. [...snipp...]
   
9. hadoop@ubuntu:~$

复制代码

8，禁用ipV6配置：
打开/etc/sysctl.conf， 此 文件需要root权限。
再次文件中，追加如下：

1. net.ipv6.conf.all.disable_ipv6 = 1
   
2. net.ipv6.conf.default.disable_ipv6 = 1
   
3. net.ipv6.conf.lo.disable_ipv6 = 1

复制代码

重启，测试是否配置成功：

1. $ cat /proc/sys/net/ipv6/conf/all/disable_ipv6

复制代码

如果是1就ok了。
下面就是安装Hadoop了。
首先是：下载，解压缩，分配权限。
下载就不说了。
下载后运行如下：

1. $ cd /usr/local
   
2. $ sudo tar xzf hadoop-0.20.2.tar.gz
   
3. $ sudo mv hadoop-0.20.2 hadoop
   
4. $ sudo chown -R hadoop:hadoop hadoop

复制代码

ook，现在安装完毕。
下面说说如何配置和 启动：
基本思路是，配置JDK地址，配置core-site.xml,配置 mapred-site.xml, hdfs-site.xml.
首先建立一个用来存放 数据的目录：mkdir /usr/local/hadoop- datastore

打开conf/core-site.xml,配置如下

1. <configuration>
   
2.   <property>
   
3.     <name>hadoop.tmp.dir</name>
   
4.     <value>/usr/local/hadoop-datastore/</value>
   
5.     <description>A base for other temporary directories.</description>
   
6.   </property>
   
7. 
   
8.   <property>
   
9.     <name>fs.default.name</name>
   
10.     <value>hdfs://localhost:54310</value>
    
11.     <description>The name of the default file system.  A URI whose
    
12.   scheme and authority determine the FileSystem implementation.  The
    
13.   uri's scheme determines the config property (fs.SCHEME.impl) naming
    
14.   the FileSystem implementation class.  The uri's authority is used to
    
15.   determine the host, port, etc. for a filesystem.</description>
    
16.   </property>
    
17. </configuration>

复制代码

mapred-site.xml如下：

1. <configuration>
   
2. <property>
   
3.   <name>mapred.job.tracker</name>
   
4.   <value>localhost:54311</value>
   
5.   <description>The host and port that the MapReduce job tracker runs
   
6.   at.  If "local", then jobs are run in-process as a single map
   
7.   and reduce task.
   
8.   </description>
   
9. </property>
   
10. </configuration>

复制代码

hdfs-site.xml如下：

1. <configuration>
   
2. <property>
   
3.   <name>dfs.replication</name>
   
4.   <value>1</value>
   
5.   <description>Default block replication.
   
6.   The actual number of replications can be specified when the file is created.
   
7.   The default is used if replication is not specified in create time.
   
8.   </description>
   
9. </property>
   
10. </configuration>

复制代码

ok，配置完毕
格式化HDFS：

1. hadoop@ubuntu:~$ /hadoop/bin/hadoop namenode -format

复制代码

启动HDFS和MapReduce

1. hadoop@ubuntu:~$ /bin/start-all.sh

复制代码

停止 服务的脚本是：

1. hadoop@ubuntu:~$ /bin/stop-all.sh

http://www.hadoopor.com/thread-2674-1-1.html