1、基本语法
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs 具体命令
或者
[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs 具体命令
2、HDFS参数大全
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs
Usage: hadoop fs [generic options]
[-appendToFile ... ]
[-cat [-ignoreCrc] ...]
[-checksum ...]
[-chgrp [-R] GROUP PATH...]
[-chmod [-R] PATH...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-copyFromLocal [-f] [-p] [-l] [-d] ... ]
[-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] ... ]
[-count [-q] [-h] [-v] [-t []] [-u] [-x] ...]
[-cp [-f] [-p | -p[topax]] [-d] ... ]
[-createSnapshot []]
[-deleteSnapshot ]
[-df [-h] [ ...]]
[-du [-s] [-h] [-x] ...]
[-expunge]
[-find ... ...]
[-get [-f] [-p] [-ignoreCrc] [-crc] ... ]
[-getfacl [-R] ]
[-getfattr [-R] {-n name | -d} [-e en] ]
[-getmerge [-nl] [-skip-empty-file] ]
[-help [cmd ...]]
[-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [ ...]]
[-mkdir [-p] ...]
[-moveFromLocal ... ]
[-moveToLocal ]
[-mv ... ]
[-put [-f] [-p] [-l] [-d] ... ]
[-renameSnapshot ]
[-rm [-f] [-r|-R] [-skipTrash] [-safely] ...]
[-rmdir [--ignore-fail-on-non-empty] ...]
[-setfacl [-R] [{-b|-k} {-m|-x } ]|[--set ]]
[-setfattr {-n name [-v value] | -x name} ]
[-setrep [-R] [-w] ...]
[-stat [format] ...]
[-tail [-f] ]
[-test -[defsz] ]
[-text [-ignoreCrc] ...]
[-touchz ...]
[-truncate [-w] ...]
[-usage [cmd ...]]
Generic options supported are
-conf specify an application configuration file
-D use value for given property
-fs specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
-jt specify a ResourceManager
-files specify comma separated files to be copied to the map reduce cluster
-libjars specify comma separated jar files to include in the classpath.
-archives specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
command [genericOptions] [commandOptions]
3、HDFS常用命令实操
(1) -help：输出这个命令参数
[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs -help rm
(2) -ls: 显示目录信息
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -ls /
(3) -mkdir：在hdfs上创建目录
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -mkdir -p /aaa/bbb/cc/dd
(4) -moveFromLocal从本地剪切粘贴到hdfs
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -moveFromLocal /home/hadoop/a.txt /aaa/bbb/cc/dd
(5) -moveToLocal：从hdfs剪切粘贴到本地（没有实现）
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -help moveToLocal
-moveToLocal :
Not implemented yet
(6) --appendToFile ：追加一个文件到已经存在的文件末尾
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -appendToFile ./hello.txt /hello.txt
(7) -cat:显示文件内容
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -cat /hello.txt
(8) –tail:显示一个文件的末尾
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -tail /weblog/access_log.1
(9) -chgrp 、-chmod、-chown：linux文件系统中的用法一样，修改文件所属权限
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -chmod 666 /hello.txt
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -chown someuser:somegrp /hello.txt
(10) -copyFromLocal：从本地文件系统中拷贝文件到hdfs路径去
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -copyFromLocal ./jdk.tar.gz /aaa/
(11) -copyToLocal：从hdfs拷贝到本地
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -copyToLocal /user/hello.txt ./hello.txt
(12) -cp ：从hdfs的一个路径拷贝到hdfs的另一个路径
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -cp /aaa/jdk.tar.gz /bbb/jdk.tar.gz.2
(13) -mv：在hdfs目录中移动文件
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -mv /aaa/jdk.tar.gz-入门 /
(14) -get：等同于copyToLocal，就是从hdfs下载文件到本地
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -get /user/hello.txt ./
(15) -getmerge:合并下载多个文件，比如hdfs的目录 /aaa/下有多个文件:log.1, log.2,log.3,...
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -getmerge /aaa/log.* ./log.sum
(16) -put:等同于copyFromLocal
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -put /aaa/jdk.tar.gz /
(17) -rm:删除文件或文件夹
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -rm -r /aaa/bbb/
(18) -rmdir:删除空目录
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -rmdir /aaa/bbb/ccc
(19) -df:统计文件系统的可用空间信息
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -df -h /
(20) -du统计文件夹的大小信息
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -du -s -h /user
(21) -count:统计一个指定目录下的文件节点数量
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -count /user
嵌套文件层级:包含文件的总数
(22) -setrep:设置hdfs中文件的副本数量
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -setrep 3 /aaa/jdk.tar.gz
尖叫提示：这里设置的副本数只是记录在namenode的元数据中，是否真的会有这么多副本，还得看datanode的数量。
假设设置副本数为3,datanode节点数为2,最多2个副本，当datanode节点数为10，最多副本数为3
(23) 修改hdfs文件备份系数
hadoop dfs -setrep [-R]
[victor@node1 hadoop-2.8.2]$ bin/hadoop dfs -setrep -w 1 -R /user/hadoop/dir1
就是把目录下所有文件备份系数设置为1.这个需要一些时间，感觉会扫描所有指定目录下的文件
(24) hdfs离开安全模式
[victor@node1 hadoop-2.8.2]$ bin/hadoop dfsadmin -safemode leave
(25) hadoop kill掉 yarn任务
[victor@node1 hadoop-2.8.2]$ bin/hadoop job -kill job_1526884826758_0015