【1】 Aspera下載NCBI數(shù)據(jù)

單個文件下載命令：

ascp -i "C:\Users\Administrator\AppData\Local\Programs\Aspera\Aspera Connect\etc\asperaweb_id_dsa.openssh" -k 1 -T -l 200m anonftp@ftp-trace.ncbi.nlm.nih.gov:/sra/sra-instant/reads/ByRun/sra/SRR/SRR121/SRR1218151/SRR1218151.sra E:\aspera

多個文件下載命令：

ascp -QT -k1 -l 200m -i "C:\Users\Administrator\AppData\Local\Programs\Aspera\Aspera Connect\etc\asperaweb_id_dsa.putty" --mode=recv --host=ftp-trace.ncbi.nlm.nih.gov --user=anonftp --file-list=E:\aspera\filelist.txt H:\NCBI

• 下載文件列表存放在文件E:\aspera\filelist.txt
• windows CMD路徑中如果有空格Aspera Connect，全路徑用雙引號包起來

無標(biāo)題.png

【2】批量校驗(yàn)

100多個大文件，每個都有獨(dú)立的md5校驗(yàn)碼，手動校驗(yàn)太費(fèi)勁。也沒找到其他批量校驗(yàn)工具，就自己寫了個python腳本（新手上路，代碼不是很python）。代碼如下：

#!/usr/bin/env python           
#coding : utf-8 

import hashlib
import linecache
#import os #本來想通過OS調(diào)用外部md5程序，路徑搞不定，放棄了
import subprocess #subprocess調(diào)用測試成功，下面注釋掉的第二部分

count = 0
wrong_c = 0
fail_list = [] 
for i in range(0,113):
    m = hashlib.md5()
    if i<10:
        filename1 = "I:\\NCBI\\nr.0"+str(i)+".tar.gz"
        filename2 = "I:\\NCBI\\nr.0"+str(i)+".tar.gz.md5"
    else:
        filename1 = "I:\\NCBI\\nr."+str(i)+".tar.gz"
        filename2 = "I:\\NCBI\\nr."+str(i)+".tar.gz.md5"
    with open(filename1,'rb') as f:
        for line in f:
            m.update(line)
    real_md5 = m.hexdigest() #print(m.hexdigest()) #47a6b079cc33a4f312786b46e61e0305
    print(filename1 + " is verifying.......")
    print(real_md5)
    """
    md5_cmd = "D:\\md5\\md5.exe -n" + filename1
    rs = os.popen(md5_cmd).read()
    os_md5 = rs.strip().lower()
    print(os_md5)
    """
    '''
    obj = subprocess.Popen(['D:\\md5\\md5.exe','-n',filename1], shell = True, stdin=subprocess.PIPE, stdout=subprocess.PIPE ,stderr=subprocess.PIPE)
    raw_md5 = obj.stdout.read()
    str_md5 = raw_md5.strip().decode("utf-8").lower()
    print(str_md5)
    '''
    the_line = linecache.getline(filename2, 1)
    ori_md5 = the_line.split(" ")[0].strip()
    print(ori_md5)
    if real_md5 == ori_md5:
        count += 1
        print("The file nr."+str(i)+".tar.gz is OK!")
    else:
        wrong_c += 1
        fail_list.append("nr."+str(i)+".tar.gz")
        print("The file nr."+str(i)+".tar.gz has been damaged!")
    sum_files = count + wrong_c
    print("Validation of %d files in total, %d passed, %d failed!" %         (sum_files, count, wrong_c))
if not fail_list:
    print("Go on analyzing with all the files!")
else:
    print("Failed files include:")
    print(",".join(str(i) for i in fail_list))