當(dāng)標(biāo)準(zhǔn)scrapyd 下載 pipeline 無(wú)法滿足需求時(shí)，可以自定義pipeline。
僅舉例文件下載和圖片下載pipeline。
擴(kuò)展文件(圖片) FilesPipeline (ImagesPipeline)僅需重寫(xiě)以下兩個(gè)方法：

get_media_request(self, item, info)   # 返回一個(gè)Request對(duì)象
 # 當(dāng)上面的Requsts下載完成后回調(diào)這個(gè)方法，然后填充files或images字段
item_completed(self, results, item, info) 

舉例：

pipelines.py

import scrapyd
from scrapy.pipelines.images import ImagesPipeline
from scrapy.exceptions import DropItem

class MyImagePipeline(ImagesPipeline):

    def get_media_requests(self, item, info):
        for image_url in item['image_urls']:
            yield scrapy.Request(image_url)

    def item_completed(self, request, item, info):
        image_paths = [x['path'] for ok, x in request if ok]
        if not image_paths:
            raise DropItem("item contains no images")
        item['image_paths'] = image_paths
        return item