hadoop fs -help
[xxx@zhujiip shell]$ hadoop fs -help
Usage: hadoop fs [generic options]
[-appendToFile … ]
[-cat [-ignoreCrc] …]
[-checksum …]
[-chgrp [-R] GROUP PATH…]
[-chmod [-R]
[-chown [-R] [OWNER][:[GROUP]] PATH…]
[-copyFromLocal [-f] [-p] [-l] … ]
[-copyToLocal [-p] [-ignoreCrc] [-crc] … ]
[-count [-q] [-h] [-v] [-x] …]
[-cp [-f] [-p | -p[topax]] … ]
[-createSnapshot []]
[-deleteSnapshot ]
[-df [-h] [ …]]
[-du [-s] [-h] [-x] …]
[-expunge]
[-find … …]
[-get [-p] [-ignoreCrc] [-crc] … ]
[-getfacl [-R] ]
[-getfattr [-R] {-n name | -d} [-e en] ]
[-getmerge [-nl] ]
[-help [cmd …]]
[-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [ …]]
[-mkdir [-p] …]
[-moveFromLocal … ]
[-moveToLocal ]
[-mv … ]
[-put [-f] [-p] [-l] … ]
[-renameSnapshot ]
[-rm [-f] [-r|-R] [-skipTrash] …]
[-rmdir [–ignore-fail-on-non-empty]
[-setfacl [-R] [{-b|-k} {-m|-x
[-setfattr {-n name [-v value] | -x name} ]
[-setrep [-R] [-w] …]
[-stat [format] …]
[-tail [-f] ]
[-test -[defsz] ]
[-text [-ignoreCrc] …]
[-touchz …]
[-usage [cmd …]]
-appendToFile … :
Appends the contents of all the given local files to the given dst file. The dst
file will be created if it does not exist. If is -, then the input is
read from stdin.
-cat [-ignoreCrc] … :
Fetch all files that match the file pattern and display their content on
stdout.
-checksum … :
Dump checksum information for files that match the file pattern to stdout.
Note that this requires a round-trip to a datanode storing each block of the
file, and thus is not efficient to run on a large number of files. The checksum
of a file depends on its content, block size and the checksum algorithm and
parameters used for creating the file.
-chgrp [-R] GROUP PATH… :
This is equivalent to -chown … :GROUP …
-chmod [-R]
Changes permissions of a file. This works similar to the shell’s chmod command
with a few exceptions.
-R modifies the files recursively. This is the only option currently
supported.
Mode is the same as mode used for the shell’s command. The only
letters recognized are ‘rwxXt’, e.g. +t,a+r,g-w,+rwx,o=r.
Mode specifed in 3 or 4 digits. If 4 digits, the first may be 1 or
0 to turn the sticky bit on or off, respectively. Unlike the
shell command, it is not possible to specify only part of the
mode, e.g. 754 is same as u=rwx,g=rx,o=r.
If none of ‘augo’ is specified, ‘a’ is assumed and unlike the shell command, no
umask is applied.
-chown [-R] [OWNER][:[GROUP]] PATH… :
Changes owner and group of a file. This is similar to the shell’s chown command
with a few exceptions.
-R modifies the files recursively. This is the only option currently
supported.
If only the owner or group is specified, then only the owner or group is
modified. The owner and group names may only consist of digits, alphabet, and
any of [-_./@a-zA-Z0-9]. The names are case sensitive.
WARNING: Avoid using ‘.’ to separate user name and group though Linux allows it.
If user names have dots in them and you are using local file system, you might
see surprising results since the shell command ‘chown’ is used for local files.
-copyFromLocal [-f] [-p] [-l] … :
Identical to the -put command.
-copyToLocal [-p] [-ignoreCrc] [-crc] … :
Identical to the -get command.
-count [-q] [-h] [-v] [-x] … :
Count the number of directories, files and bytes under the paths
that match the specified file pattern. The output columns are:
DIR_COUNT FILE_COUNT CONTENT_SIZE PATHNAME
or, with the -q option:
QUOTA REM_QUOTA SPACE_QUOTA REM_SPACE_QUOTA
DIR_COUNT FILE_COUNT CONTENT_SIZE PATHNAME
The -h option shows file sizes in human readable format.
The -v option displays a header line.
The -x option excludes snapshots from being calculated.
-cp [-f] [-p | -p[topax]] … :
Copy files that match the file pattern to a destination. When copying
multiple files, the destination must be a directory. Passing -p preserves status
[topax] (timestamps, ownership, permission, ACLs, XAttr). If -p is specified
with no , then preserves timestamps, ownership, permission. If -pa is
specified, then preserves permission also because ACL is a super-set of
permission. Passing -f overwrites the destination if it already exists. raw
namespace extended attributes are preserved if (1) they are supported (HDFS
only) and, (2) all of the source and target pathnames are in the /.reserved/raw
hierarchy. raw namespace xattr preservation is determined solely by the presence
(or absence) of the /.reserved/raw prefix and not by the -p option.
-createSnapshot [] :
Create a snapshot on a directory
-deleteSnapshot :
Delete a snapshot from a directory
-df [-h] [ …] :
Shows the capacity, free and used space of the filesystem. If the filesystem has
multiple partitions, and no path to a particular partition is specified, then
the status of the root partitions will be shown.
-h Formats the sizes of files in a human-readable fashion rather than a number
of bytes.
-du [-s] [-h] [-x] … :
Show the amount of space, in bytes, used by the files that match the specified
file pattern. The following flags are optional:
-s Rather than showing the size of each individual file that matches the
pattern, shows the total (summary) size.
-h Formats the sizes of files in a human-readable fashion rather than a number
of bytes.
-x Excludes snapshots from being counted.
Note that, even without the -s option, this only shows size summaries one level
deep into a directory.
The output is in the form
size disk space consumed name(full path)
-expunge :
Empty the Trash
-find … … :
Finds all files that match the specified expression and
applies selected actions to them. If no is specified
then defaults to the current working directory. If no
expression is specified then defaults to -print.
The following primary expressions are recognised:
-name pattern
-iname pattern
Evaluates as true if the basename of the file matches the
pattern using standard file system globbing.
If -iname is used then the match is case insensitive.
-print
-print0
Always evaluates to true. Causes the current pathname to be
written to standard output followed by a newline. If the -print0
expression is used then an ASCII NULL character is appended rather
than a newline.
The following operators are recognised:
expression -a expression
expression -and expression
expression expression
Logical AND operator for joining two expressions. Returns
true if both child expressions return true. Implied by the
juxtaposition of two expressions and so does not need to be
explicitly specified. The second expression will not be
applied if the first fails.
-get [-p] [-ignoreCrc] [-crc] … :
Copy files that match the file pattern to the local name. is kept.
When copying multiple files, the destination must be a directory. Passing -p
preserves access and modification times, ownership and the mode.
-getfacl [-R] :
Displays the Access Control Lists (ACLs) of files and directories. If a
directory has a default ACL, then getfacl also displays the default ACL.
-R List the ACLs of all files and directories recursively.
File or directory to list.
-getfattr [-R] {-n name | -d} [-e en] :
Displays the extended attribute names and values (if any) for a file or
directory.
-R Recursively list the attributes for all files and directories.
-n name Dump the named extended attribute value.
-d Dump all extended attribute values associated with pathname.
-e Encode values after retrieving them.Valid encodings are “text”,
“hex”, and “base64”. Values encoded as text strings are enclosed
in double quotes ("), and values encoded as hexadecimal and
base64 are prefixed with 0x and 0s, respectively.
The file or directory.
-getmerge [-nl] :
Get all the files in the directories that match the source file pattern and
merge and sort them to only one file on local fs. is kept.
-nl Add a newline character at the end of each file.
-help [cmd …] :
Displays help for given command or all commands if none is specified.
-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [ …] :
List the contents that match the specified file pattern. If path is not
specified, the contents of /user/ will be listed. For a directory a
list of its direct children is returned (unless -d option is specified).
Directory entries are of the form:
permissions - userId groupId sizeOfDirectory(in bytes)
modificationDate(yyyy-MM-dd HH:mm) directoryName
and file entries are of the form:
permissions numberOfReplicas userId groupId sizeOfFile(in bytes)
modificationDate(yyyy-MM-dd HH:mm) fileName
-C Display the paths of files and directories only.
-d Directories are listed as plain files.
-h Formats the sizes of files in a human-readable fashion
rather than a number of bytes.
-q Print ? instead of non-printable characters.
-R Recursively list the contents of directories.
-t Sort files by modification time (most recent first).
-S Sort files by size.
-r Reverse the order of the sort.
-u Use time of last access instead of modification for
display and sorting.
-mkdir [-p] … :
Create a directory in specified location.
-p Do not fail if the directory already exists
-moveFromLocal … :
Same as -put, except that the source is deleted after it’s copied.
-moveToLocal :
Not implemented yet
-mv … :
Move files that match the specified file pattern to a destination .
When moving multiple files, the destination must be a directory.
-put [-f] [-p] [-l] … :
Copy files from the local file system into fs. Copying fails if the file already
exists, unless the -f flag is given.
Flags:
-p Preserves access and modification times, ownership and the mode.
-f Overwrites the destination if it already exists.
-l Allow DataNode to lazily persist the file to disk. Forces
replication factor of 1. This flag will result in reduced
durability. Use with care.
-renameSnapshot :
Rename a snapshot from oldName to newName
-rm [-f] [-r|-R] [-skipTrash] … :
Delete all files that match the specified file pattern. Equivalent to the Unix
command "rm "
-skipTrash option bypasses trash, if enabled, and immediately deletes
-f If the file does not exist, do not display a diagnostic message or
modify the exit status to reflect an error.
-[rR] Recursively deletes directories
-rmdir [–ignore-fail-on-non-empty]
… :Removes the directory entry specified by each directory argument, provided it is
empty.
-setfacl [-R] [{-b|-k} {-m|-x
Sets Access Control Lists (ACLs) of files and directories.
Options:
-b Remove all but the base ACL entries. The entries for user, group
and others are retained for compatibility with permission bits.
-k Remove the default ACL.
-R Apply operations to all files and directories recursively.
-m Modify ACL. New entries are added to the ACL, and existing entries
are retained.
-x Remove specified ACL entries. Other ACL entries are retained.
–set Fully replace the ACL, discarding all existing entries. The
compatibility with permission bits.
File or directory to modify.
-setfattr {-n name [-v value] | -x name} :
Sets an extended attribute name and value for a file or directory.
-n name The extended attribute name.
-v value The extended attribute value. There are three different encoding
methods for the value. If the argument is enclosed in double quotes,
then the value is the string inside the quotes. If the argument is
prefixed with 0x or 0X, then it is taken as a hexadecimal number. If
the argument begins with 0s or 0S, then it is taken as a base64
encoding.
-x name Remove the extended attribute.
The file or directory.
-setrep [-R] [-w] … :
Set the replication level of a file. If is a directory then the command
recursively changes the replication factor of all files under the directory tree
rooted at .
-w It requests that the command waits for the replication to complete. This
can potentially take a very long time.
-R It is accepted for backwards compatibility. It has no effect.
-stat [format] … :
Print statistics about the file/directory at in the specified format.
Format accepts filesize in blocks (%b), group name of owner(%g), filename (%n),
block size (%o), replication (%r), user name of owner(%u), modification date
(%y, %Y)
-tail [-f] :
Show the last 1KB of the file.
-f Shows appended data as the file grows.
-test -[defsz] :
Answer various questions about , with result via exit status.
-d return 0 if is a directory.
-e return 0 if exists.
-f return 0 if is a file.
-s return 0 if file is greater than zero bytes in size.
-z return 0 if file is zero bytes in size, else return 1.
-text [-ignoreCrc] … :
Takes a source file and outputs the file in text format.
The allowed formats are zip and TextRecordInputStream and Avro.
-touchz … :
Creates a file of zero length at with current time as the timestamp of
that . An error is returned if the file exists with non-zero length
-usage [cmd …] :
Displays the usage for given command or all commands if none is specified.
Generic options supported are
-conf specify an application configuration file
-D
-fs
-jt
-files specify comma separated files to be copied to the map reduce cluster
-libjars specify comma separated jar files to include in the classpath.
-archives specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]
[itzxyy@hd-dn-01 shell]$ hadoop fs -help
使用方法:hadoop fs[通用选项]
[-appendToFile < localsrc >…< dst >)
[-cat [-ignoreCrc] …]
(校验和< src >…]
[-chgrp [-R]组路径…]
(chmod - r <模式(模式)…| OCTALMODE >路径…]
[-chown [-R] [OWNER][:[GROUP]] PATH…]
[-copyFromLocal] [-f] [-p] [-l] …< dst >)
[-copyToLocal] [-p] [-ignoreCrc] [-crc] …< localdst >)
[-count [-q] [-h] [-v] [-x] …]
[-cp [-f] [-p | -p[topax]] …< dst >)
[-createSnapshot < snapshotDir > [< snapshotName >]]
[-deleteSnapshot < snapshotDir > < snapshotName >)
[-df [-h] […]]
[-du [-s] [-h] [-x] …]
(删除)
(找到<路径>…<表达式>…]
[-get [-p] [-ignoreCrc] [-crc] …< localdst >)
[-getfacl (- r) <路径>)
[-getfattr [-R] {-n name | -d} [-e en] ]
[-getmerge [-nl] ]
[- [cmd…]]
[- ls [c] [- d] [h] [q] [- r] [t] [s] (- r) (- u)[<路径>…]]
[-mkdir [-p] …]
[-moveFromLocal < localsrc >…< dst >)
[-moveToLocal < src > < localdst >)
(mv < src >…< dst >)
[-put [-f] [-p] [-l] …< dst >)
[-renameSnapshot ]
[-rm [-f] [-r| -r] [-skipTrash] …]
[-rmdir[——ignore-fail-on-non-empty]
…]
[-setfacl [- r] [{- b | - k} {- m | - x < acl_spec >} <路径>]|[——设置< acl_spec > <路径>]]
[-setfattr {-n name [-v value] | -x name} ]
[-setrep [-R] [-w] …]
[-stat [format] …]
(多边形[f] <文件>)
[以及- [defsz] <路径>]
[-text [-ignoreCrc] …]
[-touchz <路径>…]
[使用[cmd…]]
-appendToFile < localsrc >…< dst >:
将所有给定本地文件的内容附加到给定dst文件。dst的
如果文件不存在，将创建该文件。如果为-，则输入为
从stdin读取。
-cat [-ignoreCrc] …:
获取所有与文件模式匹配的文件，并显示它们的内容
stdout。
校验和< src >…:
将匹配文件模式的文件的校验和信息转储到stdout。
注意，这需要到存储
文件，因此在大量文件上运行效率不高。校验和
文件的大小取决于它的内容、块大小和校验和算法以及
用于创建文件的参数。
-chgrp [-R]组路径…:
这相当于-chown…:集团…
chmod (- r) <模式(模式)…| OCTALMODE >路径…:
更改文件的权限。这类似于shell的chmod命令
除了少数例外。
-R递归地修改文件。这是目前唯一的选择
支持。
模式与shell命令使用的模式相同。唯一的
识别的字母是’rwxXt’，例如+t,a+r,g-w，+rwx,o=r。
<以3位或4位数字指定的八进制>模式。如果4位数字，第一个可能是1或
0分别打开或关闭粘性位。不像
命令，则不可能只指定
例如，754等于u=rwx,g=rx,o=r。
如果没有指定’augo’，则假定’a’，与shell命令不同，没有
umask。
-chown [-R] [OWNER][:[GROUP]] PATH…:
更改文件的所有者和组。这类似于shell的chown命令
除了少数例外。
-R递归地修改文件。这是目前唯一的选择
支持。
如果只指定了所有者或组，则只指定了所有者或组
修改。所有者名和组名只能由数字、字母和
[-_. / @a-zA-Z0-9]。这些名称区分大小写。
警告:避免使用’。虽然Linux允许用户名和组分开。
如果用户名中有圆点，并且您正在使用本地文件系统，则可能需要
查看令人惊讶的结果，因为shell命令’chown’用于本地文件。
-copyFromLocal [-f] [-p] [-l] …< dst >:
与-put命令相同。
-copyToLocal [-p] [-ignoreCrc] [-crc] …< localdst >:
与-get命令相同。
-count [-q] [-h] [-v] [-x] …:
计算路径下的目录、文件和字节数
匹配指定的文件模式。输出列为:
DIR_COUNT FILE_COUNT CONTENT_SIZE路径名
或者，使用-q选项:
配额REM_QUOTA SPACE_QUOTA REM_SPACE_QUOTA
DIR_COUNT