wget介紹

wget是由c語(yǔ)言寫(xiě)的網(wǎng)絡(luò)下載工具，支持HTTP和FTP協(xié)議，支持代理服務(wù)器和斷點(diǎn)續(xù)傳功能，能夠自動(dòng)遞歸遠(yuǎn)程主機(jī)的目錄?？珊笈_(tái)運(yùn)行。

wget的使用

下載

用法：wget [OPTION]... [URL]...? ?（wget 軟件包下載網(wǎng)址? ? 如：wget?http://example.com/packages/）如需添加選項(xiàng)操作，可參考參數(shù)說(shuō)明。

參數(shù)

較為常用的參數(shù)是 -t：設(shè)定重試次數(shù)（0表示無(wú)限次，即不間斷下載），-b：后臺(tái)模式（將下載行為放到后臺(tái)），-c：繼續(xù)下載（斷網(wǎng)導(dǎo)致的中斷下載。加上該參數(shù)，即可斷點(diǎn)續(xù)傳） ，-P/directory ：將文件下載到指定目錄，-i filelist.txt：批量下載文件中的內(nèi)容。

1、下載192.168.1.168首頁(yè)并且顯示下載信息

wget -d http://192.168.1.168

2、下載192.168.1.168首頁(yè)并且不顯示任何信息

wget -q http://192.168.1.168

3、批量下載的情形，把所有需要下載文件的地址放到 filename.txt 中，然后 wget 就會(huì)自動(dòng)為你下載所有文件了。

wget -i filelist.txt

4、下載到指定目錄

wget -P/tmp ftp://user:passwd@url/file

把文件file下載到/tmp目錄下。

Startup:

? -V,? --version? ? ? ? ? ? ? ? ? display the version of Wget and exit

? -h,? --help? ? ? ? ? ? ? ? ? ? ? print this help

? -b,? --background? ? ? ? ? ? ? ? go to background after startup

? -e,? --execute=COMMAND? ? ? ? ? execute a `.wgetrc'-style command

Logging and input file:

? -o,? --output-file=FILE? ? ? ? ? log messages to FILE

? -a,? --append-output=FILE? ? ? ? append messages to FILE

? -d,? --debug? ? ? ? ? ? ? ? ? ? print lots of debugging information

? -q,? --quiet? ? ? ? ? ? ? ? ? ? quiet (no output)

? -v,? --verbose? ? ? ? ? ? ? ? ? be verbose (this is the default)

? -nv, --no-verbose? ? ? ? ? ? ? ? turn off verboseness, without being quiet

? ? ? --report-speed=TYPE? ? ? ? output bandwidth as TYPE.? TYPE can be bits

? -i,? --input-file=FILE? ? ? ? ? download URLs found in local or external FILE

? -F,? --force-html? ? ? ? ? ? ? ? treat input file as HTML

? -B,? --base=URL? ? ? ? ? ? ? ? ? resolves HTML input-file links (-i -F)

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? relative to URL

? ? ? --config=FILE? ? ? ? ? ? ? specify config file to use

? ? ? --no-config? ? ? ? ? ? ? ? do not read any config file

? ? ? --rejected-log=FILE? ? ? ? log reasons for URL rejection to FILE

Download:

? -t,? --tries=NUMBER? ? ? ? ? ? ? set number of retries to NUMBER (0 unlimits)

? ? ? --retry-connrefused? ? ? ? retry even if connection is refused

? ? ? --retry-on-http-error=ERRORS? ? comma-separated list of HTTP errors to retry

? -O,? --output-document=FILE? ? ? write documents to FILE

? -nc, --no-clobber? ? ? ? ? ? ? ? skip downloads that would download to

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? existing files (overwriting them)

? ? ? --no-netrc? ? ? ? ? ? ? ? ? don't try to obtain credentials from .netrc

? -c,? --continue? ? ? ? ? ? ? ? ? resume getting a partially-downloaded file

? ? ? --start-pos=OFFSET? ? ? ? ? start downloading from zero-based position OFFSET

? ? ? --progress=TYPE? ? ? ? ? ? select progress gauge type

? ? ? --show-progress? ? ? ? ? ? display the progress bar in any verbosity mode

? -N,? --timestamping? ? ? ? ? ? ? don't re-retrieve files unless newer than

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? local

? ? ? --no-if-modified-since? ? ? don't use conditional if-modified-since get

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? requests in timestamping mode

? ? ? --no-use-server-timestamps? don't set the local file's timestamp by

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? the one on the server

? -S,? --server-response? ? ? ? ? print server response

? ? ? --spider? ? ? ? ? ? ? ? ? ? don't download anything

? -T,? --timeout=SECONDS? ? ? ? ? set all timeout values to SECONDS

? ? ? --dns-timeout=SECS? ? ? ? ? set the DNS lookup timeout to SECS

? ? ? --connect-timeout=SECS? ? ? set the connect timeout to SECS

? ? ? --read-timeout=SECS? ? ? ? set the read timeout to SECS

? -w,? --wait=SECONDS? ? ? ? ? ? ? wait SECONDS between retrievals

? ? ? --waitretry=SECONDS? ? ? ? wait 1..SECONDS between retries of a retrieval

? ? ? --random-wait? ? ? ? ? ? ? wait from 0.5*WAIT...1.5*WAIT secs between retrievals

? ? ? --no-proxy? ? ? ? ? ? ? ? ? explicitly turn off proxy

? -Q,? --quota=NUMBER? ? ? ? ? ? ? set retrieval quota to NUMBER

? ? ? --bind-address=ADDRESS? ? ? bind to ADDRESS (hostname or IP) on local host

? ? ? --limit-rate=RATE? ? ? ? ? limit download rate to RATE

? ? ? --no-dns-cache? ? ? ? ? ? ? disable caching DNS lookups

? ? ? --restrict-file-names=OS? ? restrict chars in file names to ones OS allows

? ? ? --ignore-case? ? ? ? ? ? ? ignore case when matching files/directories

? -4,? --inet4-only? ? ? ? ? ? ? ? connect only to IPv4 addresses

? -6,? --inet6-only? ? ? ? ? ? ? ? connect only to IPv6 addresses

? ? ? --prefer-family=FAMILY? ? ? connect first to addresses of specified family,

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? one of IPv6, IPv4, or none

? ? ? --user=USER? ? ? ? ? ? ? ? set both ftp and http user to USER

? ? ? --password=PASS? ? ? ? ? ? set both ftp and http password to PASS

? ? ? --ask-password? ? ? ? ? ? ? prompt for passwords

? ? ? --use-askpass=COMMAND? ? ? specify credential handler for requesting

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? username and password.? If no COMMAND is

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? specified the WGET_ASKPASS or the SSH_ASKPASS

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? environment variable is used.

? ? ? --no-iri? ? ? ? ? ? ? ? ? ? turn off IRI support

? ? ? --local-encoding=ENC? ? ? ? use ENC as the local encoding for IRIs

? ? ? --remote-encoding=ENC? ? ? use ENC as the default remote encoding

? ? ? --unlink? ? ? ? ? ? ? ? ? ? remove file before clobber

? ? ? --xattr? ? ? ? ? ? ? ? ? ? turn on storage of metadata in extended file attributes

Directories:

? -nd, --no-directories? ? ? ? ? ? don't create directories

? -x,? --force-directories? ? ? ? force creation of directories

? -nH, --no-host-directories? ? ? don't create host directories

? ? ? --protocol-directories? ? ? use protocol name in directories

? -P,? --directory-prefix=PREFIX? save files to PREFIX/..

? ? ? --cut-dirs=NUMBER? ? ? ? ? ignore NUMBER remote directory components

HTTP options:

? ? ? --http-user=USER? ? ? ? ? ? set http user to USER

? ? ? --http-password=PASS? ? ? ? set http password to PASS

? ? ? --no-cache? ? ? ? ? ? ? ? ? disallow server-cached data

? ? ? --default-page=NAME? ? ? ? change the default page name (normally

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? this is 'index.html'.)

? -E,? --adjust-extension? ? ? ? ? save HTML/CSS documents with proper extensions

? ? ? --ignore-length? ? ? ? ? ? ignore 'Content-Length' header field

? ? ? --header=STRING? ? ? ? ? ? insert STRING among the headers

? ? ? --compression=TYPE? ? ? ? ? choose compression, one of auto, gzip and none. (default: none)

? ? ? --max-redirect? ? ? ? ? ? ? maximum redirections allowed per page

? ? ? --proxy-user=USER? ? ? ? ? set USER as proxy username

? ? ? --proxy-password=PASS? ? ? set PASS as proxy password

? ? ? --referer=URL? ? ? ? ? ? ? include 'Referer: URL' header in HTTP request

? ? ? --save-headers? ? ? ? ? ? ? save the HTTP headers to file

? -U,? --user-agent=AGENT? ? ? ? ? identify as AGENT instead of Wget/VERSION

? ? ? --no-http-keep-alive? ? ? ? disable HTTP keep-alive (persistent connections)

? ? ? --no-cookies? ? ? ? ? ? ? ? don't use cookies

? ? ? --load-cookies=FILE? ? ? ? load cookies from FILE before session

? ? ? --save-cookies=FILE? ? ? ? save cookies to FILE after session

? ? ? --keep-session-cookies? ? ? load and save session (non-permanent) cookies

? ? ? --post-data=STRING? ? ? ? ? use the POST method; send STRING as the data

? ? ? --post-file=FILE? ? ? ? ? ? use the POST method; send contents of FILE

? ? ? --method=HTTPMethod? ? ? ? use method "HTTPMethod" in the request

? ? ? --body-data=STRING? ? ? ? ? send STRING as data. --method MUST be set

? ? ? --body-file=FILE? ? ? ? ? ? send contents of FILE. --method MUST be set

? ? ? --content-disposition? ? ? honor the Content-Disposition header when

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? choosing local file names (EXPERIMENTAL)

? ? ? --content-on-error? ? ? ? ? output the received content on server errors

? ? ? --auth-no-challenge? ? ? ? send Basic HTTP authentication information

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? without first waiting for the server's

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? challenge

HTTPS (SSL/TLS) options:

? ? ? --secure-protocol=PR? ? ? ? choose secure protocol, one of auto, SSLv2,

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? SSLv3, TLSv1, TLSv1_1, TLSv1_2 and PFS

? ? ? --https-only? ? ? ? ? ? ? ? only follow secure HTTPS links

? ? ? --no-check-certificate? ? ? don't validate the server's certificate

? ? ? --certificate=FILE? ? ? ? ? client certificate file

? ? ? --certificate-type=TYPE? ? client certificate type, PEM or DER

? ? ? --private-key=FILE? ? ? ? ? private key file

? ? ? --private-key-type=TYPE? ? private key type, PEM or DER

? ? ? --ca-certificate=FILE? ? ? file with the bundle of CAs

? ? ? --ca-directory=DIR? ? ? ? ? directory where hash list of CAs is stored

? ? ? --crl-file=FILE? ? ? ? ? ? file with bundle of CRLs

? ? ? --pinnedpubkey=FILE/HASHES? Public key (PEM/DER) file, or any number

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? of base64 encoded sha256 hashes preceded by

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 'sha256//' and separated by ';', to verify

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? peer against

? ? ? --random-file=FILE? ? ? ? ? file with random data for seeding the SSL PRNG

? ? ? --ciphers=STR? ? ? ? ? Set the priority string (GnuTLS) or cipher list string (OpenSSL) directly.

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Use with care. This option overrides --secure-protocol.

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? The format and syntax of this string depend on the specific SSL/TLS engine.

HSTS options:

? ? ? --no-hsts? ? ? ? ? ? ? ? ? disable HSTS

? ? ? --hsts-file? ? ? ? ? ? ? ? path of HSTS database (will override default)

FTP options:

? ? ? --ftp-user=USER? ? ? ? ? ? set ftp user to USER

? ? ? --ftp-password=PASS? ? ? ? set ftp password to PASS

? ? ? --no-remove-listing? ? ? ? don't remove '.listing' files

? ? ? --no-glob? ? ? ? ? ? ? ? ? turn off FTP file name globbing

? ? ? --no-passive-ftp? ? ? ? ? ? disable the "passive" transfer mode

? ? ? --preserve-permissions? ? ? preserve remote file permissions

? ? ? --retr-symlinks? ? ? ? ? ? when recursing, get linked-to files (not dir)

FTPS options:

? ? ? --ftps-implicit? ? ? ? ? ? ? ? use implicit FTPS (default port is 990)

? ? ? --ftps-resume-ssl? ? ? ? ? ? ? resume the SSL/TLS session started in the control connection when

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? opening a data connection

? ? ? --ftps-clear-data-connection? ? cipher the control channel only; all the data will be in plaintext

? ? ? --ftps-fallback-to-ftp? ? ? ? ? fall back to FTP if FTPS is not supported in the target server

WARC options:

? ? ? --warc-file=FILENAME? ? ? ? save request/response data to a .warc.gz file

? ? ? --warc-header=STRING? ? ? ? insert STRING into the warcinfo record

? ? ? --warc-max-size=NUMBER? ? ? set maximum size of WARC files to NUMBER

? ? ? --warc-cdx? ? ? ? ? ? ? ? ? write CDX index files

? ? ? --warc-dedup=FILENAME? ? ? do not store records listed in this CDX file

? ? ? --no-warc-compression? ? ? do not compress WARC files with GZIP

? ? ? --no-warc-digests? ? ? ? ? do not calculate SHA1 digests

? ? ? --no-warc-keep-log? ? ? ? ? do not store the log file in a WARC record

? ? ? --warc-tempdir=DIRECTORY? ? location for temporary files created by the

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? WARC writer

Recursive download:

? -r,? --recursive? ? ? ? ? ? ? ? specify recursive download

? -l,? --level=NUMBER? ? ? ? ? ? ? maximum recursion depth (inf or 0 for infinite)

? ? ? --delete-after? ? ? ? ? ? ? delete files locally after downloading them

? -k,? --convert-links? ? ? ? ? ? make links in downloaded HTML or CSS point to

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? local files

? ? ? --convert-file-only? ? ? ? convert the file part of the URLs only (usually known as the basename)

? ? ? --backups=N? ? ? ? ? ? ? ? before writing file X, rotate up to N backup files

? -K,? --backup-converted? ? ? ? ? before converting file X, back up as X.orig

? -m,? --mirror? ? ? ? ? ? ? ? ? ? shortcut for -N -r -l inf --no-remove-listing

? -p,? --page-requisites? ? ? ? ? get all images, etc. needed to display HTML page

? ? ? --strict-comments? ? ? ? ? turn on strict (SGML) handling of HTML comments

Recursive accept/reject:

? -A,? --accept=LIST? ? ? ? ? ? ? comma-separated list of accepted extensions

? -R,? --reject=LIST? ? ? ? ? ? ? comma-separated list of rejected extensions

? ? ? --accept-regex=REGEX? ? ? ? regex matching accepted URLs

? ? ? --reject-regex=REGEX? ? ? ? regex matching rejected URLs

? ? ? --regex-type=TYPE? ? ? ? ? regex type (posix|pcre)

? -D,? --domains=LIST? ? ? ? ? ? ? comma-separated list of accepted domains

? ? ? --exclude-domains=LIST? ? ? comma-separated list of rejected domains

? ? ? --follow-ftp? ? ? ? ? ? ? ? follow FTP links from HTML documents

? ? ? --follow-tags=LIST? ? ? ? ? comma-separated list of followed HTML tags

? ? ? --ignore-tags=LIST? ? ? ? ? comma-separated list of ignored HTML tags

? -H,? --span-hosts? ? ? ? ? ? ? ? go to foreign hosts when recursive

? -L,? --relative? ? ? ? ? ? ? ? ? follow relative links only

? -I,? --include-directories=LIST? list of allowed directories

? ? ? --trust-server-names? ? ? ? use the name specified by the redirection

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? URL's last component

? -X,? --exclude-directories=LIST? list of excluded directories

? -np, --no-parent? ? ? ? ? ? ? ? don't ascend to the parent directory

拓展

當(dāng)我們?cè)谑褂脀get命令下載時(shí)，添加了參數(shù)-b 后，運(yùn)行命令時(shí)，會(huì)返回下圖：

pid代表了后臺(tái)任務(wù)的進(jìn)程，可以通過(guò)kill掉這個(gè)id來(lái)終止這次下載。wget-log.2文件記錄了下載的輸出信息（持續(xù)寫(xiě)入，直至完全下載完畢）。

wget查看后臺(tái)任務(wù)進(jìn)度

1. 通過(guò)斷點(diǎn)續(xù)傳查看： 運(yùn)行 wget -c 正在執(zhí)行的下載地址。

2. 通過(guò)產(chǎn)看wget-log文件產(chǎn)看： 運(yùn)行tail -f wget-log 或cat?wget-log 即可，終止查看時(shí)按ctrl + c。

http://www.itdecent.cn/p/2e2ba8ecc22a

https://blog.csdn.net/wanglc7/article/details/85136418