看了不少書和資料,自認(rèn)為對于 python 中的線程、進程、協(xié)程等略知一二了。
想實現(xiàn)一個多線程池的模型,但是也不想用 queue 甚至是 celery 這些,查了很多資料,國內(nèi)的原創(chuàng)的不多,并且基本都是停留在 python 2.7 的時代,而且國內(nèi)的文章即便用 google 搜索,大部分文章也是互相轉(zhuǎn)載。國外的資料比較好的還是在 stackoverflow,國內(nèi)的簡書上好文章不少。
搜了半天,在 python 的官方文檔上赫然有著一個例子:
The concurrent.futures module provides a high-level interface for asynchronously executing callables.
The asynchronous execution can be performed with threads, using ThreadPoolExecutor, or separate processes, using ProcessPoolExecutor. Both implement the same interface, which is defined by the abstract Executor class.
用 concurrent.futures 即可,然后
ThreadPoolExecutor is an Executor subclass that uses a pool of threads to execute calls asynchronously.
找了好久差點想自己實現(xiàn)(擔(dān)心自己寫的很爛)的線程池就在這里,例子如下:
import urllib.request
URLS = ['http://www.foxnews.com/',
'http://www.cnn.com/',
'http://europe.wsj.com/',
'http://www.bbc.co.uk/',
'http://some-made-up-domain.com/']
# Retrieve a single page and report the URL and contents
def load_url(url, timeout):
with urllib.request.urlopen(url, timeout=timeout) as conn:
return conn.read()
# We can use a with statement to ensure threads are cleaned up promptly
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
# Start the load operations and mark each future with its URL
future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
for future in concurrent.futures.as_completed(future_to_url):
url = future_to_url[future]
try:
data = future.result()
except Exception as exc:
print('%r generated an exception: %s' % (url, exc))
else:
print('%r page is %d bytes' % (url, len(data)))
這個例子寫得很清楚,可以直接運行,不過建議把那些網(wǎng)址換掉,因為 g*w 的關(guān)系。
沒想到 python 的官方文檔做的這么好,我準(zhǔn)備從頭到底先通讀一遍。