安装ubuntu操作系统
物理主机就可以连接到虚拟机,需要保证虚拟机安装了sshd并且服务开启,防火墙已关闭。
sudo apt-get install openssh-server
sudo apt-get install openssh-client
sudo /etc/init.d/ssh start
ps -ef | grep ssh
安装后添加hadoop用户组和用户并添加root用户权限
sudo addgroup hadoop
sudo adduser -ingroup hadoop hadoop
sudo vi /etc/sudoers
设置hostname为hadoop,注意不能保留127.0.0.1 localhost这个
vi /etc/hosts
192.168.233.3 hadoop
配置ssh无密码登陆
ssh-keygen -t rsa -P ""
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh hadoop
1.解压
tar zxvf jdk-7u67-linux-x64.tar.gz
2.配置环境变量
vi ~/.profile
export JAVA_HOME=/home/hadoop/jdk1.7.0_67
export PATH=$JAVA_HOME/bin:$PATH
1.解压,注意文件属组
tar zxvf hadoop-2.5.2.tar.gz
2.配置环境变量
#HADOOP VARIABLES START
export HADOOP_HOME=/home/hadoop/hadoop-2.5.2
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
#HADOOP VARIABLES END
3.修改配置文件
/home/hadoop/hadoop-2.5.2/etc/hadoop/core-site.xml增加
<property>
<name>fs.default.namename>
<value>hdfs://hadoop:9000value>
property>
<property>
<name>hadoop.tmp.dirname>
<value>/home/hadoop/hadoop-2.5.2/hadoop-${user.name}value>
property>
/home/hadoop/hadoop-2.5.2/etc/hadoop/hdfs-site.xml增加
<property>
<name>dfs.namenode.name.dirname>
<value>file:/home/hadoop/dfs/namevalue>
property>
<property>
<name>dfs.namenode.data.dirname>
<value>file:/home/hadoop/dfs/datavalue>
property>
<property>
<name>dfs.replicationname>
<value>1value>
property>
<property>
<name>dfs.permissionsname>
<value>falsevalue>
property>
<property>
<name>dfs.web.uginame>
<value>jack,supergroupvalue>
property>
从模板copy一个/home/hadoop/hadoop-2.5.2/etc/hadoop/mapred-site.xml增加
<property>
<name>mapreduce.framework.namename>
<value>yarnvalue>
<final>truefinal>
property>
/home/hadoop/hadoop-2.5.2/etc/hadoop/yarn-site.xml增加
<property>
<name>yarn.nodemanager.aux-servicesname>
<value>mapreduce_shufflevalue>
property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce_shuffle.classname>
<value>org.apache.hadoop.mapred.ShuffleHandlervalue>
property>
<property>
<name>yarn.resourcemanager.hostnamename>
<value>hadoopvalue>
property>
4.启动hadoop
cd /home/hadoop/hadoop-2.5.2
./bin/hdfs namenode -format
./sbin/start-all.sh
1.dfs上创建input目录
bin/hadoop fs -mkdir -p input
2.把hadoop目录下的README.txt拷贝到dfs新建的input里
bin/hadoop fs -copyFromLocal README.txt input
3.运行WordCount
bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.5.2-sources.jar org.apache.hadoop.examples.WordCount input output
4.运行完毕后,查看单词统计结果
bin/hadoop fs -cat output/*
假如程序的输出路径为output,如果该文件夹已经存在,先删除
bin/hadoop dfs -rm -r output
参考链接 http://www.cnblogs.com/huligong1234/p/4136331.html