實(shí)例:磁力鏈接轉(zhuǎn)種子
網(wǎng)址:https://itorrents.org/
需求:通過該網(wǎng)站,根據(jù)磁力鏈接下種子文件
一般的下載格式:
請(qǐng)求地址:http://itorrents.org/torrent/INFO_HASH_IN_HEX.torrent
請(qǐng)求實(shí)例:http://itorrents.org/torrent/B415C913643E5FF49FE37D304BBB5E6E11AD5101.torrent
更新
【20200406】itorrents網(wǎng)站的Cloudflare升級(jí)之后,腳本出現(xiàn)
+Function("return escape")()(("")["italics"]())[2]+"o"+(undefined+"")[2]+(t
等等,可讀性降低,在簡(jiǎn)化JS腳本時(shí)需要更多措施
必要Cookie
- __cfduid:這個(gè)cookie在返回503時(shí)候從請(qǐng)求頭得到,用于之后的認(rèn)證過程
- cf_clearance: 訪問網(wǎng)站真正使用的cookie,請(qǐng)求頭加入它,可以認(rèn)證通過并正常訪問到網(wǎng)站,該cookie的存活時(shí)間為2小時(shí)
代碼獲取cookie
使用的平臺(tái)和工具:
- 平臺(tái):JRE 8 或 Android 6.0
- 軟件:Postman,模擬http請(qǐng)求
第三方j(luò)ar包:
- jsoup: 爬取網(wǎng)頁源代碼,包括html和Js
- rhino: 讓安卓能夠執(zhí)行js腳本
- okhttp: 用于發(fā)網(wǎng)絡(luò)請(qǐng)求,獲取cookie,下種子文件等操作
具體過程:
? 首先代碼發(fā)送請(qǐng)求,讓服務(wù)器返回503。
//創(chuàng)建OkHttpClient對(duì)象
OkHttpClient client = new OkHttpClient.Builder().build();
//創(chuàng)建Request對(duì)象,設(shè)置一個(gè)url地址, 設(shè)置請(qǐng)求方式。
Request request = new Request.Builder()
.addHeader("User-Agent", USER_AGENT)
.url("https://itorrents.org").method("GET", null)
.build();
//創(chuàng)建一個(gè)call對(duì)象,參數(shù)就是Request請(qǐng)求對(duì)象
Call call = client.newCall(request);
//同步調(diào)用,返回Response,會(huì)拋出IO異常
Response response = call.execute();
? 出現(xiàn)503錯(cuò)誤,首先從返回頭部中獲取第一個(gè)cookie( __cfduid)的值
Headers headers = response.headers();
String setCookieStr = headers.get("Set-Cookie");
String __cfduid = setCookieStr.split(";")[0].trim();
? 使用Jsoup爬取返回的網(wǎng)頁數(shù)據(jù)
// 調(diào)用該句獲取返回輸入流
InputStream is = response.body().byteStream();
private String getErrorHtml(InputStream is){
String resultHtml = "";
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(is, "utf-8"));
String line = null;
while ((line = reader.readLine()) != null) {
resultHtml += line;
}
} catch (IOException e){
EvLog.e(TAG, "IOException: " + e.getMessage());
}
return resultHtml;
}
//獲取表單所有參數(shù)
private Map<String, String> getNewHttpParams(String resultHtml) throws Exception {
Document doc = Jsoup.parse(resultHtml);
//取得表單
Element loginForm = doc.getElementById("challenge-form");
//取得script下面的JS變量
Element js = doc.getElementsByTag("script").get(0);
//過濾JS內(nèi)容,只保留有效內(nèi)容
String jsData = js.data();
jsData = jsData.substring(jsData.indexOf("setTimeout"), jsData.indexOf("'; 121'"));
jsData = "var " + jsData.substring(jsData.indexOf("f,") + 2).trim();
String tiStr = jsData.substring(jsData.indexOf("t = document.createElement('div');"), jsData.indexOf("challenge-form") + 18);
jsData = jsData.replace(tiStr, "").replace("a.value", "var a").replace("t.length", "13");
//...執(zhí)行JS代碼
return params;
}
? 觀察返回結(jié)果:只有第四個(gè)參數(shù)的值沒有體現(xiàn)
<form id="challenge-form" action="/cdn-cgi/l/chk_jschl" method="get">
<input type="hidden" name="s" value="79e2e9582b532dc8c1803b475404232b947dd65e-1552293281-1800-AaPJ3gbam22HO+RRAxJmBJLQP/wyX5Gxit+iGz77w0hI8Ph74sfEKxz5xGm3RWVJzpTVdlM93S4QTzpDHf6HgUeyLd3E1kiC0B0d7rUOEKmd"></input>
<input type="hidden" name="jschl_vc" value="17505afb059e796938098575225488f4"/>
<input type="hidden" name="pass" value="1552293285.342-v/1sNwvefM"/>
<input type="hidden" id="jschl-answer" name="jschl_answer"/>
</form>
? 第四個(gè)參數(shù)需要分析返回網(wǎng)頁的JS,代碼如下
//
<![CDATA[
(function(){
var a = function() {try{return !!window.addEventListener} catch(e) {return !1} },
b = function(b, c) {a() ? document.addEventListener("DOMContentLoaded", b, c) : document.attachEvent("onreadystatechange", b)};
b(function(){
var a = document.getElementById('cf-content');a.style.display = 'block';
setTimeout(function(){
var s,t,o,p,b,r,e,a,k,i,n,g,f, XcSJxuC={"sKGOcaDtX":+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![])+(+[])+(!+[]+!![]+!![]+!![])+(!+[]+!![])+(!+[]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]))/+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(+!![])+(!+[]+!![]+!![]+!![]+!![])+(+[])+(+[]))};
t = document.createElement('div');
t.innerHTML="<a href='/'>x</a>";
t = t.firstChild.href;r = t.match(/https?:\/\//)[0];
t = t.substr(r.length); t = t.substr(0,t.length-1);
a = document.getElementById('jschl-answer');
f = document.getElementById('challenge-form');
;XcSJxuC.sKGOcaDtX-=+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(!+[]+!![]+!![]+!![])+(+!![])+(+[])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(+!![]))/+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![])+(+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]));a.value = +XcSJxuC.sKGOcaDtX.toFixed(10) + t.length; '; 121'
f.action += location.hash;
f.submit();
}, 4000);
}, false);
})();
//]]>
? 分析可以發(fā)現(xiàn),變量a存儲(chǔ)第四個(gè)參數(shù)值,f是提交的表單。因此,如何得到a變量的值是關(guān)鍵
a.value = +XcSJxuC.sKGOcaDtX.toFixed(10) + t.length;
? 由兩個(gè)部分組成,一個(gè)是XcSJxuC.sKGOcaDtX,另一個(gè)是t.length,首先解決t.length,t的值經(jīng)過下面幾個(gè)操作
t = document.createElement('div'); //創(chuàng)建div標(biāo)簽
t.innerHTML="<a href='/'>x</a>"; //添加子元素
t = t.firstChild.href; //獲得a標(biāo)簽的href屬性,這里是網(wǎng)站根路徑https://itorrents.org/
r = t.match(/https?:\/\//)[0]; //獲取t的https://前綴
t = t.substr(r.length); //獲取子串 itorrents.org/
t = t.substr(0,t.length-1); //去掉子串最后一個(gè)字符'/',結(jié)果 itorrents.org
? 因此t.length值為13。每次返回的這一結(jié)果固定,因此直接替換成13即可
? 解決XcSJxuC.sKGOcaDtX的問題,將js簡(jiǎn)化,跟其有關(guān)的只有如下操作
XcSJxuC={"sKGOcaDtX":+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![])+(+[])+(!+[]+!![]+!![]+!![])+(!+[]+!![])+(!+[]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]))/+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(+!![])+(!+[]+!![]+!![]+!![]+!![])+(+[])+(+[]))};
XcSJxuC.sKGOcaDtX-=+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(!+[]+!![]+!![]+!![])+(+!![])+(+[])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(+!![]))/+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![])+(+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]));
/*
[],數(shù)組符號(hào),默認(rèn)值為0,但不能單獨(dú)使用
!, 取反符號(hào),本身不具備數(shù)值意義,不能單獨(dú)使用,需要和[]配合使用
!+[],取反加數(shù)組,值為1
(+[]), 加數(shù)組,值為0
+![],取反數(shù)組,值為0
+!![],二次取反數(shù)組,值為1
var b = +((!+[]+!![]+![])+(![]+![]+!![]+!+[])); 值為4
*/
? 因此,在理論上是可以通過執(zhí)行JS腳本把值算出來的,使用Jsoup獲取全部腳本后,采取字符串操作將js過濾成如下代碼:
var XcSJxuC={"sKGOcaDtX":+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![])+(+[])+(!+[]+!![]+!![]+!![])+(!+[]+!![])+(!+[]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]))/+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(+!![])+(!+[]+!![]+!![]+!![]+!![])+(+[])+(+[]))};
XcSJxuC.sKGOcaDtX-=+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(!+[]+!![]+!![]+!![])+(+!![])+(+[])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(+!![]))/+((!+[]+!![]+!![]+!![]+!![]+!![]+!![]+[])+(!+[]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![])+(+[])+(+!![])+(!+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![])+(!+[]+!![]+!![]));
var a = +XcSJxuC.sKGOcaDtX.toFixed(10) + 13;
? 最后返回的 a 就是第四個(gè)參數(shù)的值,因此,使用rhino庫 ,寫一個(gè)方法,讓android能執(zhí)行js代碼,并獲取結(jié)果
// 執(zhí)行JS腳本,獲得所需參數(shù)的值
private String getJsAnswer(String jsString) {
//Enter a Context
org.mozilla.javascript.Context context = org.mozilla.javascript.Context.enter();
context.setOptimizationLevel(-1);
try{
//Initializing standard objects
Scriptable scope = context.initStandardObjects();
ScriptableObject.putProperty(scope, "javaContext",
org.mozilla.javascript.Context.javaToJS(org.mozilla.javascript.Context.getCurrentContext(), scope));
ScriptableObject.putProperty(scope, "javaLoader",
org.mozilla.javascript.Context.javaToJS(getClass().getClassLoader(), scope));
//Evaluating a script
context.evaluateString(scope, jsString, HOST, 1, null);
Object a = scope.get("a", scope);
if (a == Scriptable.NOT_FOUND) {
EvLog.e(TAG, "answer not found.");
} else {
return org.mozilla.javascript.Context.toString(a);
}
} catch (Exception e){
EvLog.e(TAG, "Exception: " + e.getMessage());
} finally {
org.mozilla.javascript.Context.exit();
}
return null;
}
? 如果是純javaSE或者javaEE項(xiàng)目,支持javax,其自帶js引擎可以幫助你執(zhí)行js代碼,更加方便,如下:
// 執(zhí)行JS腳本,獲得所需參數(shù)的值
private String getJsAnswer(String jsString) {
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("JavaScript");
try {
engine.eval(jsString);
return engine.get("a") + "";
} catch (ScriptException e) {
e.printStackTrace();
}
return null;
}
? 最后算出來結(jié)果,比如最后算出來jschl_answer為22.6738726886,結(jié)合前三個(gè)參數(shù),使用okhttp包,GET方式提交表單到指定地址,這里需要注意延遲4秒再發(fā)送請(qǐng)求。
? 請(qǐng)求頭要帶上第一個(gè)獲取的cookie __cfduid,代碼如下
//模擬cloudflare延時(shí)4秒,以便產(chǎn)生新的cookie,也標(biāo)志著此為耗時(shí)操作,只能在子線程中完成
Thread.sleep(4 * 1000);
OkHttpClient client = new OkHttpClient.Builder().cookieJar(new CookieJar() {
@Override
public void saveFromResponse(HttpUrl url, List<Cookie> cookies) {
List<Cookie> list = cookieStore.get(HOST);
if(list.addAll(cookies)){
EvLog.i(TAG, "saveFromResponse -- 添加進(jìn)cookiestore");
cookieStore.put(HOST, list);
} else {
EvLog.i(TAG, "saveFromResponse -- 重置cookiestore");
cookieStore.put(HOST, cookies);
}
}
@Override
public List<Cookie> loadForRequest(HttpUrl url) {
List<Cookie> cookies = cookieStore.get(HOST);
return cookies != null ? cookies : new ArrayList<>();
}
}).build();
//創(chuàng)建Request對(duì)象,設(shè)置一個(gè)url地址, 設(shè)置請(qǐng)求方式。
Request request = new Request.Builder()
.addHeader("User-Agent", USER_AGENT)
.addHeader("Cookie", __cfduid)
.addHeader("Referer", "https://itorrent.org")
.addHeader("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8")
.addHeader("Connection", "keep-alive")
.addHeader("Host", "itorrent.org")
.url(jsChl).method("GET", null)
.build();
//創(chuàng)建一個(gè)call對(duì)象,參數(shù)就是Request請(qǐng)求對(duì)象
Call call = client.newCall(request);
//同步調(diào)用,返回Response,會(huì)拋出IO異常
Response response = call.execute();
if(response.code() == HttpURLConnection.HTTP_OK) {
// 獲得新的 Cookie
List<Cookie> cookies = cookieStore.get(HOST);
for(Cookie c : cookies) {
if(c.name().equals("cf_clearance")) {
return c;
}
}
} else {
EvLog.e(TAG, "請(qǐng)求失敗,錯(cuò)誤碼: " + response.code());
}
? 如果返回200,則能拿到真正需要的cf_clearance的值,使用偏好設(shè)置將cookie持久化
? 拿到第二個(gè)cookie之后,加入到請(qǐng)求頭當(dāng)中,就可以成功訪問并下載種子文件
String downloadUrl = "http://itorrents.org/torrent/B415C913643E5FF49FE37D304BBB5E6E11AD5101.torrent";
//創(chuàng)建OkHttpClient對(duì)象
OkHttpClient client = new OkHttpClient.Builder().build();
//創(chuàng)建Request對(duì)象,設(shè)置一個(gè)url地址, 設(shè)置請(qǐng)求方式。
Request request = new Request.Builder()
.addHeader("User-Agent", USER_AGENT)
.addHeader("Cookie", cookie)
.url(downloadUrl).method("GET", null)
.build();
//創(chuàng)建一個(gè)call對(duì)象,參數(shù)就是Request請(qǐng)求對(duì)象
Call call = client.newCall(request);
//同步調(diào)用,返回Response,會(huì)拋出IO異常
Response response = call.execute();
//獲取返回?cái)?shù)據(jù)
if(response.code() == HttpURLConnection.HTTP_OK) {
FileUtils.copyInputStreamToFile(response.body().byteStream(), saveTo);
EvLog.i(TAG, "Download torrent success!!!");
}