1、基本語法
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs 具體命令
或者
[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs 具體命令
2、HDFS參數大全
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs
Usage: hadoop fs [generic options]
[-appendToFile <localsrc> ... <dst>]
[-cat [-ignoreCrc] <src> ...]
[-checksum <src> ...]
[-chgrp [-R] GROUP PATH...]
[-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
[-chown [-R] [OWNER][:[GROUP]] PATH...]
[-copyFromLocal [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
[-copyToLocal [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
[-count [-q] [-h] [-v] [-t [<storage type>]] [-u] [-x] <path> ...]
[-cp [-f] [-p | -p[topax]] [-d] <src> ... <dst>]
[-createSnapshot <snapshotDir> [<snapshotName>]]
[-deleteSnapshot <snapshotDir> <snapshotName>]
[-df [-h] [<path> ...]]
[-du [-s] [-h] [-x] <path> ...]
[-expunge]
[-find <path> ... <expression> ...]
[-get [-f] [-p] [-ignoreCrc] [-crc] <src> ... <localdst>]
[-getfacl [-R] <path>]
[-getfattr [-R] {-n name | -d} [-e en] <path>]
[-getmerge [-nl] [-skip-empty-file] <src> <localdst>]
[-help [cmd ...]]
[-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [<path> ...]]
[-mkdir [-p] <path> ...]
[-moveFromLocal <localsrc> ... <dst>]
[-moveToLocal <src> <localdst>]
[-mv <src> ... <dst>]
[-put [-f] [-p] [-l] [-d] <localsrc> ... <dst>]
[-renameSnapshot <snapshotDir> <oldName> <newName>]
[-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ...]
[-rmdir [--ignore-fail-on-non-empty] <dir> ...]
[-setfacl [-R] [{-b|-k} {-m|-x <acl_spec>} <path>]|[--set <acl_spec> <path>]]
[-setfattr {-n name [-v value] | -x name} <path>]
[-setrep [-R] [-w] <rep> <path> ...]
[-stat [format] <path> ...]
[-tail [-f] <file>]
[-test -[defsz] <path>]
[-text [-ignoreCrc] <src> ...]
[-touchz <path> ...]
[-truncate [-w] <length> <path> ...]
[-usage [cmd ...]]
Generic options supported are
-conf <configuration file> specify an application configuration file
-D <property=value> use value for given property
-fs <file:///|hdfs://namenode:port> specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
-jt <local|resourcemanager:port> specify a ResourceManager
-files <comma separated list of files> specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars> specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives> specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
command [genericOptions] [commandOptions]
3、HDFS常用命令實操
(1) -help：輸出這個命令參數
[victor@node1 hadoop-2.8.2]$ bin/hdfs dfs -help rm
(2) -ls: 顯示目錄信息
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -ls /
(3) -mkdir：在hdfs上創(chuàng)建目錄
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -mkdir -p /aaa/bbb/cc/dd
(4) -moveFromLocal從本地剪切粘貼到hdfs
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -moveFromLocal /home/hadoop/a.txt /aaa/bbb/cc/dd
(5) -moveToLocal：從hdfs剪切粘貼到本地（沒有實現）
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -help moveToLocal
-moveToLocal <src> <localdst> :
Not implemented yet
(6) --appendToFile ：追加一個文件到已經存在的文件末尾
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -appendToFile ./hello.txt /hello.txt
(7) -cat:顯示文件內容
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -cat /hello.txt
(8) –tail:顯示一個文件的末尾
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -tail /weblog/access_log.1
(9) -chgrp 、-chmod、-chown：linux文件系統(tǒng)中的用法一樣，修改文件所屬權限
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -chmod 666 /hello.txt
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -chown someuser:somegrp /hello.txt
(10) -copyFromLocal：從本地文件系統(tǒng)中拷貝文件到hdfs路徑去
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -copyFromLocal ./jdk.tar.gz /aaa/
(11) -copyToLocal：從hdfs拷貝到本地
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -copyToLocal /user/hello.txt ./hello.txt
(12) -cp ：從hdfs的一個路徑拷貝到hdfs的另一個路徑
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -cp /aaa/jdk.tar.gz /bbb/jdk.tar.gz.2
(13) -mv：在hdfs目錄中移動文件
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -mv /aaa/jdk.tar.gz-入門 /
(14) -get：等同于copyToLocal，就是從hdfs下載文件到本地
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -get /user/hello.txt ./
(15) -getmerge:合并下載多個文件，比如hdfs的目錄 /aaa/下有多個文件:log.1, log.2,log.3,...
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -getmerge /aaa/log.* ./log.sum
(16) -put:等同于copyFromLocal
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -put /aaa/jdk.tar.gz /
(17) -rm:刪除文件或文件夾
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -rm -r /aaa/bbb/
(18) -rmdir:刪除空目錄
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -rmdir /aaa/bbb/ccc
(19) -df:統(tǒng)計文件系統(tǒng)的可用空間信息
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -df -h /
(20) -du統(tǒng)計文件夾的大小信息
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -du -s -h /user
(21) -count:統(tǒng)計一個指定目錄下的文件節(jié)點數量
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -count /user
嵌套文件層級:包含文件的總數
(22) -setrep:設置hdfs中文件的副本數量
[victor@node1 hadoop-2.8.2]$ bin/hadoop fs -setrep 3 /aaa/jdk.tar.gz
尖叫提示：這里設置的副本數只是記錄在namenode的元數據中，是否真的會有這么多副本，還得看datanode的數量。
假設設置副本數為3,datanode節(jié)點數為2,最多2個副本，當datanode節(jié)點數為10，最多副本數為3
(23) 修改hdfs文件備份系數
hadoop dfs -setrep [-R] <path> 如果有-R將修改子目錄文件的性質
[victor@node1 hadoop-2.8.2]$ bin/hadoop dfs -setrep -w 1 -R /user/hadoop/dir1
就是把目錄下所有文件備份系數設置為1.這個需要一些時間，感覺會掃描所有指定目錄下的文件
(24) hdfs離開安全模式
[victor@node1 hadoop-2.8.2]$ bin/hadoop dfsadmin -safemode leave
(25) hadoop kill掉 yarn任務
[victor@node1 hadoop-2.8.2]$ bin/hadoop job -kill job_1526884826758_0015
