HDFS常用shell命令

HDFS 文件操作命令
注，其实常用命令不用网上搜，和linux下的命令很类似，触类旁通，直接在linux 上 hadoop fs 看一下就行了，不需要刻意去记

我把 linux 上的 help 列举下，方便直接看吧，hdfs dfs 与 hadoop fs 效果一样

常用的就是

hdfs dfs -copyFromLocal /local/data /hdfs/data：将本地文件上传到 hdfs 上（原路径只能是一个文件）

hdfs dfs -put /tmp/ /hdfs/ ：和 copyFromLocal 区别是，put 原路径可以是文件夹等

hadoop fs -ls / ：查看根目录文件

hadoop fs -ls /tmp/data：查看/tmp/data目录

hadoop fs -cat /tmp/a.txt ：查看 a.txt，与 -text 一样

hadoop fs -mkdir dir：创建目录dir

hadoop fs -rmr dir：删除目录dir

刚开始学hdfs的时候，不知道怎么cd到hdfs的下级目录，我以为同linux命令一样是cd进去的，然而。。恩，不知道有新司机也一样想

[orco@node1 sbin]$ hadoop fs
Usage: hadoop fs [generic options]
    [-appendToFile  ... ]
    [-cat [-ignoreCrc]  ...]
    [-checksum  ...]
    [-chgrp [-R] GROUP PATH...]
    [-chmod [-R] ... | OCTALMODE> PATH...]
    [-chown [-R] [OWNER][:[GROUP]] PATH...]
    [-copyFromLocal [-f] [-p] [-l]  ... ]
    [-copyToLocal [-p] [-ignoreCrc] [-crc]  ... ]
    [-count [-q] [-h]  ...]
    [-cp [-f] [-p | -p[topax]]  ... ]
    [-createSnapshot  []]
    [-deleteSnapshot  ]
    [-df [-h] [ ...]]
    [-du [-s] [-h]  ...]
    [-expunge]
    [-find  ...  ...]
    [-get [-p] [-ignoreCrc] [-crc]  ... ]
    [-getfacl [-R] ]
    [-getfattr [-R] {-n name | -d} [-e en] ]
    [-getmerge [-nl]  ]
    [-help [cmd ...]]
    [-ls [-d] [-h] [-R] [ ...]]
    [-mkdir [-p]  ...]
    [-moveFromLocal  ... ]
    [-moveToLocal  ]
    [-mv  ... ]
    [-put [-f] [-p] [-l]  ... ]
    [-renameSnapshot   ]
    [-rm [-f] [-r|-R] [-skipTrash]  ...]
    [-rmdir [--ignore-fail-on-non-empty]  ...]
    [-setfacl [-R] [{-b|-k} {-m|-x } ]|[--set  ]]
    [-setfattr {-n name [-v value] | -x name} ]
    [-setrep [-R] [-w]   ...]
    [-stat [format]  ...]
    [-tail [-f] ]
    [-test -[defsz] ]
    [-text [-ignoreCrc]  ...]
    [-touchz  ...]
    [-truncate [-w]   ...]
    [-usage [cmd ...]]

HDFS的管理命令
一般管理员才会用下面的命令，举例：

hdfs dfsadmin -report：显示所有dataNode

hdfs dfsadmin -safemode leave：离开安全模式

[orco@node1 hadoop-2.7.3]$ bin/hdfs dfsadmin
Usage: hdfs dfsadmin
Note: Administrative commands can only be run as the HDFS superuser.
    [-report [-live] [-dead] [-decommissioning]]
    [-safemode ]
    [-saveNamespace]
    [-rollEdits]
    [-restoreFailedStorage true|false|check]
    [-refreshNodes]
    [-setQuota  ...]
    [-clrQuota ...]
    [-setSpaceQuota  [-storageType ] ...]
    [-clrSpaceQuota [-storageType ] ...]
    [-finalizeUpgrade]
    [-rollingUpgrade []]
    [-refreshServiceAcl]
    [-refreshUserToGroupsMappings]
    [-refreshSuperUserGroupsConfiguration]
    [-refreshCallQueue]
    [-refresh   [arg1..argn]
    [-reconfig ...>  ]
    [-printTopology]
    [-refreshNamenodes datanode_host:ipc_port]
    [-deleteBlockPool datanode_host:ipc_port blockpoolId [force]]
    [-setBalancerBandwidth in bytes per second>]
    [-fetchImage ]
    [-allowSnapshot ]
    [-disallowSnapshot ]
    [-shutdownDatanode  [upgrade]]
    [-getDatanodeInfo ]
    [-metasave filename]
    [-triggerBlockReport [-incremental] ]
    [-help [cmd]]

单独启动某个服务

hadoop-deamon.sh start namenode

启动一个 DN

hadoop-daemon.sh start datanode

hadoop-daemon.sh start nodemanager