配置ubuntu环境:
1. 防火墙
》查看防火墙状态:sudo ufw status
》关闭本机防火墙:控制面板——window防火墙——打开或关闭windows防火墙——关闭防火墙
2. jdk和scp
》下载32位jdk:
http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
安装winscp,配置虚拟机的ip和访问用户/密码:
-rw-rw-r-- 1 hadoop hadoop 188157665 Sep 10 05:21 jdk-8u144-linux-i586.tar.gz
hadoop@ubuntu:~/profile/download$ pwd
/home/hadoop/profile/download
》安装jdk:
root@ubuntu:/home/hadoop/profile/download# tar zxvf jdk-8u144-linux-i586.tar.gz -C /root/directory/java
》查看安装情况:
root@ubuntu:~/directory/java# pwd
/root/directory/java
root@ubuntu:~/directory/java# ls
jdk1.8.0_144
》配置环境变量:
export JAVA_HOME=/root/directory/java/jdk1.8.0_144
export PATH=${JAVA_HOME}/bin:$PATH
》加载及测试:
root@ubuntu:/etc# source profile
root@ubuntu:/etc# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) Client VM (build 25.144-b01, mixed mode)
3. 搭建伪分布模式
》设置主机名称
sed -i 's/ubuntu/dj/g' /etc/hostname
sudo gedit /etc/hostname
sed -i 's/ubuntu/dj/g' /etc/hosts
sudo gedit /etc/hosts
》下载包:http://www.apache.org/dyn/closer.cgi/hadoop/common/
选择下载:http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.7.4/hadoop-2.7.4.tar.gz
通过scp上传,解压:
tar -zxvf /home/hadoop/profile/download/hadoop-2.7.4.tar.gz-C /root/directory/
》设置环境变量:
vi ~/.bash_profile
export HADOOP_HOME=/root/directory/hadoop-2.7.4
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
》加载环境变量:
source ~/.bash_profile
》配置 hadoop-env.sh
export JAVA_HOME=/root/directory/java/jdk1.8.0_144
配置hdfs-site.xml
配置core-site.xml:
配置mapred-site.xml:
cp mapred-site.xml.template mapred-site.xml
vi mapred-site.xml
配置yarn-site.xml
对hdfs进行格式化:hdfs namenode -format
启动hdfs和yarn:start-all.sh
查看进程:
root@dj:~/directory/hadoop-2.7.4/sbin# jps
3890 ResourceManager
2837 NameNode
3142 SecondaryNameNode
2957 DataNode
4013 NodeManager
4045 Jps
查看页面:
后台日志报错:
doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager.
修改配置文件;
yarn-site.xml
配置host:vi /etc/hosts
添加:192.168.1.119 dj
修改配置:
core-site.xml
yarn-site.xml
配置免秘钥登陆:
ssh-keygen -t rsa
ssh-copy-id -i .ssh/id_rsa.pub root@dj
检测免秘钥是否成功:start-all.sh
root@dj:~# jps
2980 DataNode
3527 Jps
2857 NameNode
3353 ResourceManager
3162 SecondaryNameNode
3483 NodeManager
root@dj:~#
无需输入密码,登陆成功