Hadoop伪分布式搭建

Hadoop伪分布式搭建

一、CentOS 安装

1、下载并安装 VMware® Workstation 15 Pro

2、安装 CentOS-7-x86_64-Minimal-1908.iso

3、IP：192.168.xxx.xxx 、host：hadoop

二、Hadoop 环境搭建

下载Hadoop2.9.2

安装 Hadoop

1. 上传 hadoop-2.9.2.tar.gz 到 home 目录
2. 解压 tar -zxvf /home/hadoop-2.9.2.tar.gz -C /opt/
3. 配置环境变量

[root@hadoop ~]# vi /etc/profile
# set jdk environment
export JAVA_HOME=/opt/jdk1.8.0_112
export CLASSPATH=$JAVA_HOME/lib/
export PATH=$PATH:$JAVA_HOME/bin
[root@hadoop ~]# source /etc/profile
[root@hadoop ~]# java -version

 关闭防火墙

[root@hadoop ~]# systemctl stop firewalld.service
[root@hadoop ~]# systemctl disable firewalld.service

 

[root@hadoop ~]# vi /etc/hosts
# 192.168.xxx.xxx hadoop

 

[root@hadoop ~]# vi /etc/selinux/config
SELINUX=disabled

root@hadoop ~]# vi /opt/hadoop-2.9.2/etc/hadoop/hadoop-env.sh
export JAVA_HOME=/opt/jdk1.8.0_112

[root@hadoop ~]# vi /opt/hadoop-2.9.2/etc/hadoop/core-site.xml

fs.defaultFS
hdfs://hadoop:9000


hadoop.tmp.dir
/home/hadoop/temp

 修改 hdfs-site.xml

[root@hadoop ~]# vi /opt/hadoop-2.9.2/etc/hadoop/hdfs-site.xml

dfs.replication
1


dfs.name.dir
/home/hadoop/hdfs/name


dfs.data.dir
/home/hadoop/hdfs/data

 [root@hadoop ~]# cd /opt/hadoop-2.9.2/etc/hadoop/

 [root@hadoop ~]# cp mapred-site.xml.template mapred-site.xml

mapreduce.framework.name
yarn

 

[root@hadoop ~]# vi /opt/hadoop-2.9.2/etc/hadoop/yarn-site.xml

yarn.resourcemanager.hostname
hadoop

[root@hadoop ~]# cd /opt/hadoop-2.9.2/etc/hadoop

[root@hadoop ~]# hadoop namenode -format 

启动测试

[root@hadoop ~]# cd /opt/hadoop-2.9.2/etc/hadoop

[root@hadoop ~]# start-all.sh