一、问题描述
hdfs开机就进入安全模式,执行强制离开安全模式命令以后仍自动进入安全模式。
二、解决方法
通过hadoop日志查看强制进入安全模式的原因
三、解决步骤
1、查询hdfs的状态
[hadoop@gbase8a1bin]$hdfsdfsadmin-safemodeget
SafemodeisON
2、强制离开安全模式
[hadoop@gbase8a1bin]$hdfsdfsadmin-safemodeleave
[hadoop@gbase8a1bin]$hdfsdfsadmin-safemodeget
SafemodeisON
强制离开安全模式以后,仍然快速切换回安全模式,说明有实质的报错导致hdfs强制安全模式
3、查看namenode日志,确定强制安全模式的原因
2020-07-0912:47:37,607WARNorg.apache.hadoop.hdfs.server.namenode.FSNamesystem:NameNodelowonavailablediskspace.Alreadyinsafemode.
2020-07-0912:47:37,607INFOorg.apache.hadoop.hdfs.StateChange:STATE*SafemodeisON.
ResourcesarelowonNN.Pleaseaddorfreeupmoreresourcesthenturnoffsafemodemanually.NOTE:Ifyouturnoffsafemodebeforeaddingresources,theNNwillimmediatelyreturntosafemode.Use"hdfs dfsadmin -safemode leave"toturnsafemodeoff.
2020-07-0912:47:42,608WARNorg.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker:Spaceavailableonvolume'/dev/mapper/rhel-root'is103092224,whichisbelowtheconfiguredreservedamount104857600
报错信息显示:磁盘空间不足,hadoop自动进入安全模式 ,/dev/mapper/rhel-root对应的文件系统即根目录磁盘空间不足。
4、扩容lv: /dev/mapper/rhel-root
[root@gbase8a1~]# lvextend -L +6G /dev/rhel/root
查看根目录的文件系统类型
[root@gbase8a1~]# df -T
FilesystemType1K-blocksUsedAvailableUse%Mountedon
/dev/mapper/rhel-rootxfs144568326598756785807646%/
devtmpfsdevtmpfs1008704010087040%/dev
tmpfstmpfs10243329210242401%/dev/shm
tmpfstmpfs1024332906810152641%/run
tmpfstmpfs1024332010243320%/sys/fs/cgroup
/dev/sda1xfs103833615787288046416%/boot
/dev/mapper/vg_gbase8a-lv_gbase8axfs83783684698184368018457%/opt
/dev/mapper/vg_hbase-lv_hbasexfs1669120490572117854830%/hbase
/dev/mapper/vg_hive-lv_hivexfs60723204737336133498479%/hive
tmpfstmpfs204868162048521%/run/user/42
tmpfstmpfs20486802048680%/run/user/0
tmpfstmpfs20486802048680%/run/user/1002
[root@gbase8a1~]#
根目录的文件系统类型是: xfs
5、扩容文件系统:根目录
[root@gbase8a1~]# xfs_growfs /dev/rhel/root
[root@gbase8a1~]# df -h
FilesystemSizeUsedAvailUse%Mountedon
/dev/mapper/rhel-root14G6.3G7.5G46%/
devtmpfs986M0986M0%/dev
tmpfs1001M92K1001M1%/dev/shm
tmpfs1001M8.9M992M1%/run
tmpfs1001M01001M0%/sys/fs/cgroup
/dev/sda11014M155M860M16%/boot
/dev/mapper/vg_gbase8a-lv_gbase8a8.0G4.5G3.6G57%/opt
/dev/mapper/vg_hbase-lv_hbase1.6G480M1.2G30%/hbase
/dev/mapper/vg_hive-lv_hive5.8G4.6G1.3G79%/hive
tmpfs201M16K201M1%/run/user/42
tmpfs201M0201M0%/run/user/0
tmpfs201M0201M0%/run/user/1002
[root@gbase8a1~]#
根目录扩容成功。
6、再次强制hdfs离开安全模式
[hadoop@gbase8a1bin]$hdfsdfsadmin-safemodeleave
[hadoop@gbase8a1bin]$hdfsdfsadmin-safemodeget
SafemodeisOFF
观察一段时间后,确认hdfs安全模式一直处于关闭状态。
至此,hdfs由于空间不足导致的强制安全模式状态问题解决。