0 写法类似于linux命令
[root@hadoop0 hadoop]# hadoop Warning: $HADOOP_HOME is deprecated. Usage: hadoop [--config confdir] COMMAND where COMMAND is one of: namenode -format format the DFS filesystem secondarynamenode run the DFS secondary namenode namenode run the DFS namenode datanode run a DFS datanode dfsadmin run a DFS admin client mradmin run a Map-Reduce admin client fsck run a DFS filesystem checking utility fs run a generic filesystem user client balancer run a cluster balancing utility fetchdt fetch a delegation token from the NameNode jobtracker run the MapReduce job Tracker node pipes run a Pipes job tasktracker run a MapReduce task Tracker node historyserver run job history servers as a standalone daemon job manipulate MapReduce jobs queue get information regarding JobQueues version print the version jar <jar> run a jar file distcp <srcurl> <desturl> copy file or directories recursively archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive classpath prints the class path needed to get the Hadoop jar and the required libraries daemonlog get/set the log level for each daemon or CLASSNAME run the class named CLASSNAME Most commands print help when invoked w/o parameters.
[root@hadoop0 hadoop]# hadoop fs Warning: $HADOOP_HOME is deprecated. Usage: java FsShell [-ls <path>] [-lsr <path>] [-du <path>] [-dus <path>] [-count[-q] <path>] [-mv <src> <dst>] [-cp <src> <dst>] [-rm [-skipTrash] <path>] [-rmr [-skipTrash] <path>] [-expunge] [-put <localsrc> ... <dst>] [-copyFromLocal <localsrc> ... <dst>] [-moveFromLocal <localsrc> ... <dst>] [-get [-ignoreCrc] [-crc] <src> <localdst>] [-getmerge <src> <localdst> [addnl]] [-cat <src>] [-text <src>] [-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>] [-moveToLocal [-crc] <src> <localdst>] [-mkdir <path>] [-setrep [-R] [-w] <rep> <path/file>] [-touchz <path>] [-test -[ezd] <path>] [-stat [format] <path>] [-tail [-f] <file>] [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...] [-chown [-R] [OWNER][:[GROUP]] PATH...] [-chgrp [-R] GROUP PATH...] [-help [cmd]]
1. 常用命令简述
对hdfs操作的命令格式是hadoop fs
1.1 -ls <path> 表示对hdfs下一级目录的查看
1.2 -lsr <path> 表示对hdfs目录的递归查看
1.3 -mkdir <path> 创建目录
1.4 -put <src> <des> 从linux上传文件到hdfs
1.5 -get <src> <des> 从hdfs下载文件到linux
1.6 -text <path> 查看文件内容
1.7 -rm <path> 表示删除文件
1.7 -rmr <path> 表示递归删除文件
常用命令详述:
1.0 上述命令 并不一定非要在 hadoop目录下执行,可以在别的linux目录下执行
1.1 -ls <path> 表示对hdfs下一级目录的查看
[root@master hadoop]# hadoop fs -ls hdfs://master:9000/ Warning: $HADOOP_HOME is deprecated. Found 1 items drwxr-xr-x - root supergroup 0 2014-07-19 21:07 /usr [root@master hadoop]# 对上面信息的解释如下: 1 hdfs://master:9000是hadoop core-site.xml内配置的信息,也就是hdfs的根路径 2 hdfs://master:9000 写法可以类比于 http://192.168.1.113:8080 (协议类型://主机:端口) 3 请进入到hadoop目录内执行上面的命令,上述命令是展示hdfs根目录下的文件 4 drwxr-xr-x - root supergroup 0 2014-07-19 21:07 /usr 第一位为d表示目录, -表文件 第十一位 - 表示副本个数, 副本个数只有文件才有, 如果类型是文件夹的话(文件夹的作用仅仅是组织文件,不占用物理空间)则副本个数用-表示 第十四位 0 表示文件大小 第十五位 表示文件最近修改时间 第十六位 .usr 表示文件名 综述: 查看hdfs根目录下的内容,得到 /usr这个文件 同样用上述方式,可以进入 hdfs的 /usr目录看下面的文件 [root@master hadoop]# hadoop fs -ls hdfs://master:9000/usr
1.2 -lsr <path> 表示对hdfs目录的递归查看
[root@master hadoop]# hadoop fs -lsr hdfs://master:9000/ Warning: $HADOOP_HOME is deprecated. drwxr-xr-x - root supergroup 0 2014-07-19 21:07 /usr drwxr-xr-x - root supergroup 0 2014-07-19 21:07 /usr/local drwxr-xr-x - root supergroup 0 2014-07-19 21:08 /usr/local/hadoop drwxr-xr-x - root supergroup 0 2014-07-19 21:08 /usr/local/hadoop/tmp drwxr-xr-x - root supergroup 0 2014-07-30 06:42 /usr/local/hadoop/tmp/mapred drwx------ - root supergroup 0 2014-07-30 06:42 /usr/local/hadoop/tmp/mapred/system -rw------- 1 root supergroup 4 2014-07-30 06:42 /usr/local/hadoop/tmp/mapred/system/jobtracker.info
执行 [root@master hadoop]# hadoop fs -lsr / Warning: $HADOOP_HOME is deprecated. drwxr-xr-x - root supergroup 0 2014-07-19 21:07 /usr drwxr-xr-x - root supergroup 0 2014-07-19 21:07 /usr/local drwxr-xr-x - root supergroup 0 2014-07-19 21:08 /usr/local/hadoop drwxr-xr-x - root supergroup 0 2014-07-19 21:08 /usr/local/hadoop/tmp drwxr-xr-x - root supergroup 0 2014-07-30 06:42 /usr/local/hadoop/tmp/mapred drwx------ - root supergroup 0 2014-07-30 06:42 /usr/local/hadoop/tmp/mapred/system -rw------- 1 root supergroup 4 2014-07-30 06:42 /usr/local/hadoop/tmp/mapred/system/jobtracker.info 其中在执行 # hadoop fs -lsr / 时, 因为没有发现hdfs协议的路径,因此会自动去 hadoop/conf/core-site.xml 中找hdfs默认路径, 如下所示,此文件中配置的fs.default.name默认路径为 hdfs://master:9000 因此/就隐士的代表了配置文件中配置的hdfs根路径 <configuration> <property> <name>fs.default.name</name> <value>hdfs://master:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/usr/local/hadoop/tmp</value> </property> </configuration>
1.3 -mkdir <path> 创建目录
[root@master conf]# hadoop fs -mkdir /d1 Warning: $HADOOP_HOME is deprecated. [root@master conf]# hadoop fs -ls / Warning: $HADOOP_HOME is deprecated. Found 2 items drwxr-xr-x - root supergroup 0 2014-07-31 07:28 /d1 drwxr-xr-x - root supergroup 0 2014-07-19 21:07 /usr
1.4 -put <src> <des> 从linux上传文件到hdfs
[root@master conf]# hadoop fs -put ./core-site.xml hdfs://master:9000/d1 Warning: $HADOOP_HOME is deprecated. [root@master conf]# hadoop fs -ls /d1 Warning: $HADOOP_HOME is deprecated. Found 1 items -rw-r--r-- 1 root supergroup 398 2014-07-31 07:35 /d1/core-site.xml [root@master conf]# ./core-site.xml 表示当前目录(.)下的文件 core-site.xml 上述命令是将当前目录conf下的文件 core-site.xml 上传到 hdfs根目录/d1目录内
1.5 -get <src> <des> 从hdfs下载文件到linux
[root@master sbin]# cd /root/Desktop/ [root@master Desktop]# hadoop fs -get /d1/core-site.xml . Warning: $HADOOP_HOME is deprecated. [root@master Desktop]# ls core-site.xml 进入桌面,然后将 hdfs/d1/core-site.xml 下载到桌面 hadoop fs -get /d1/core-site.xml . 中的.表示linux目录中的当前目录
1.6 -text <path> 查看文件内容
[root@master ~]# hadoop fs -text /d1/core-site.xml Warning: $HADOOP_HOME is deprecated. <?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.default.name</name> <value>hdfs://master:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/usr/local/hadoop/tmp</value> </property> </configuration> [root@master ~]#
1.7 -rm <path> 表示删除文件
[root@master ~]# hadoop fs -rm /d1/core-site.xml Warning: $HADOOP_HOME is deprecated. Deleted hdfs://master:9000/d1/core-site.xml [root@master ~]# hadoop fs -ls /d1 Warning: $HADOOP_HOME is deprecated. Found 1 items -rw-r--r-- 1 root supergroup 71528 2014-07-31 07:48 /d1/ifconfig [root@master ~]#