一、前言

從現(xiàn)在開始會(huì)偶爾寫一些付費(fèi)的內(nèi)容，之所以要寫付費(fèi)內(nèi)容，主要基于以下幾點(diǎn)，占比權(quán)重依次降低。
1、我喜歡錢。
2、建立有用知識(shí)的門檻，把有用的東西給予真正需要的人。
3、通過付費(fèi)倒逼我寫出更好的文章，如果文章不好你可以盡情吐槽，前提是你花了錢。

二、抖音介紹

抖音現(xiàn)在真的是越來越火，我們公司的“一面APP”也想做對(duì)抖音APP的數(shù)據(jù)抓取。基于此，我下載抖音，開始研究它的HTTP請(qǐng)求內(nèi)容，后來我發(fā)現(xiàn)它有毒，越刷越上癮，總感覺下一個(gè)視頻一定會(huì)更精彩，根本停不下來，似乎早已忘記當(dāng)初為什么下載抖音...
當(dāng)然，最后我還是控制著了。

三、獲取抖音中某個(gè)“挑戰(zhàn)”下熱門視頻及單個(gè)視頻的URL

關(guān)于對(duì)APP進(jìn)行抓包的方法，在這我不做講解，這個(gè)方法在網(wǎng)上可以找到很多相關(guān)資料（收費(fèi)價(jià)值不大）。

如果我們要獲取抖音中某個(gè)“挑戰(zhàn)”下熱門視頻，大概的步驟如下：
1）配置好Fiddler或Charles（我這邊是使用Charles），通過Charles可以對(duì)手機(jī)APP進(jìn)行抓包，如下圖1所示；

圖1

2）在抖音APP上進(jìn)行挑戰(zhàn)的內(nèi)容搜索，例如搜索“絕地求生”，如下圖2所示；

圖2

3）點(diǎn)擊“絕地求生”挑戰(zhàn)，查看Charles中的HTTP請(qǐng)求數(shù)據(jù)，找出獲取挑戰(zhàn)下視頻的API，如下圖3所示；

圖3

我們發(fā)現(xiàn)在我們點(diǎn)擊“絕地求生”進(jìn)入這個(gè)挑戰(zhàn)的詳情頁面時(shí)，抖音APP向服務(wù)端申請(qǐng)了兩個(gè)接口請(qǐng)求。
一個(gè)是detail，它里面包含的是這個(gè)挑戰(zhàn)的一些具體信息，因?yàn)槲倚枰氖且曨l信息，所以這個(gè)我們先不管。
另一個(gè)是aweme，它的接口內(nèi)容如下：

https://aweme.snssdk.com/aweme/v1/challenge/aweme/?iid=30373511894&device_id=35781128184&os_api=18&app_name=aweme&channel=App%20Store&idfa=811A8841-030F-4AEA-B934-C2A56489C32D&device_platform=iphone&build_number=17805&vid=A4BB3AF4-7981-4805-995B-78419881DC11&openudid=99bf2b608173e21cefd562496d2cf21fa8eba580&device_type=iPhone7,2&app_version=1.7.8&version_code=1.7.8&os_version=11.3&screen_width=750&aid=1128&ac=WIFI&ch_id=1574030716416014&count=21&cursor=0&pull_type=2&query_type=0&type=5&mas=001469ed4a7a3f61de046e21c8c7ae8bcb64a983471da18ec0c8d1&as=a1c5206d9e676a38ee4960&ts=1524500606

同時(shí)通過查看這個(gè)接口返回的JSON數(shù)據(jù)，也確定這個(gè)是獲取視頻的API，如下是返回的JSON視頻數(shù)據(jù)（只保留aweme_list中的第一條數(shù)據(jù)，用于分析）：

{
  "has_more" : 1,
  "cursor" : 21,
  "rid" : "201804240023290100100511301210FF",
  "status_code" : 0,
  "extra" : {
    "logid" : "201804240023290100100511301210FF",
    "now" : 1524500609305,
    "fatal_item_ids" : []
  },
  "aweme_list" : [
    {
      "text_extra" : [],
      "status" : {
        "is_delete" : false,
        "allow_share" : true,
        "is_private" : false,
        "allow_comment" : true,
        "private_status" : 0,
        "with_goods" : false
      },
      "author_user_id" : 60978405851,
      "is_hash_tag" : 0,
      "create_time" : 1501112821,
      "aweme_type" : 0,
      "cha_list" : [
        {
          "schema" : "aweme:\/\/aweme\/challenge\/detail?cid=1574030716416014",
          "user_count" : 0,
          "author" : {},
          "cha_name" : "絕地求生",
          "type" : 0,
          "desc" : "吃雞吃雞！今晚吃雞！",
          "cid" : "1574030716416014"
        }
      ],
      "region" : "CN",
      "user_digged" : 0,
      "share_info" : {
        "share_weibo_desc" : "#在抖音，記錄美好生活#空氣泳",
        "share_title" : "空氣泳",
        "share_url" : "https:\/\/www.iesdouyin.com\/share\/video\/6447230455191178509\/?region=CN&mid=6443685673772944141&titleType=title",
        "share_desc" : "在抖音，記錄美好生活"
      },
      "video" : {
        "dynamic_cover" : {
          "url_list" : [
            "https:\/\/p3.pstatp.com\/obj\/30b20017abc8da2a1379",
            "https:\/\/pb9.pstatp.com\/obj\/30b20017abc8da2a1379",
            "https:\/\/pb3.pstatp.com\/obj\/30b20017abc8da2a1379"
          ],
          "uri" : "30b20017abc8da2a1379"
        },
        "play_addr_lowbr" : {
          "url_list" : [
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0"
          ],
          "uri" : "fe29c6e0c74a40f6baf22400552d6d2f"
        },
        "width" : 540,
        "ratio" : "720p",
        "play_addr" : {
          "url_list" : [
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0"
          ],
          "uri" : "fe29c6e0c74a40f6baf22400552d6d2f"
        },
        "cover" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/300x400\/30b600031f0eca5498aa.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/300x400\/30b600031f0eca5498aa.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/300x400\/30b600031f0eca5498aa.jpeg"
          ],
          "uri" : "300x400\/30b600031f0eca5498aa"
        },
        "height" : 960,
        "bit_rate" : [
          {
            "bit_rate" : 4325045,
            "gear_name" : "normal",
            "quality_type" : 0
          }
        ],
        "origin_cover" : {
          "url_list" : [
            "http:\/\/p1.pstatp.com\/large\/30b600031f0eca5498aa.jpeg",
            "http:\/\/pb3.pstatp.com\/large\/30b600031f0eca5498aa.jpeg",
            "http:\/\/pb3.pstatp.com\/large\/30b600031f0eca5498aa.jpeg"
          ],
          "uri" : "large\/30b600031f0eca5498aa"
        },
        "duration" : 3820,
        "download_addr" : {
          "url_list" : [
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=0&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/aweme.snssdk.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0",
            "https:\/\/api.amemv.com\/aweme\/v1\/play\/?video_id=fe29c6e0c74a40f6baf22400552d6d2f&line=1&ratio=720p&watermark=1&media_type=4&vr_type=0&test_cdn=None&improve_bitrate=0"
          ],
          "uri" : "fe29c6e0c74a40f6baf22400552d6d2f"
        },
        "has_watermark" : true
      },
      "is_vr" : false,
      "vr_type" : 0,
      "share_url" : "https:\/\/www.douyin.com\/share\/video\/6447230455191178509\/?mid=6443685673772944141",
      "sort_label" : "",
      "music" : {
        "cover_hd" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/1080x1080\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "cover_large" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/720x720\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "status" : 1,
        "extra" : "{\"has_edited\": 0}",
        "user_count" : 0,
        "title" : "Iceland",
        "duration" : 32,
        "play_url" : {
          "url_list" : [
            "http:\/\/p1.pstatp.com\/obj\/29c70000654ea97a6169",
            "http:\/\/p9.pstatp.com\/obj\/29c70000654ea97a6169",
            "http:\/\/p3.pstatp.com\/obj\/29c70000654ea97a6169"
          ],
          "uri" : "29c70000654ea97a6169"
        },
        "mid" : "6443685673772944141",
        "offline_desc" : "",
        "schema_url" : "",
        "cover_medium" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/200x200\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "is_original" : false,
        "id" : 6443685673772944141,
        "cover_thumb" : {
          "url_list" : [
            "http:\/\/p3.pstatp.com\/live\/100x100\/2bd9005342019b56cf7e.jpeg"
          ],
          "uri" : "2bd9005342019b56cf7e"
        },
        "audio_track" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/obj\/46bd000dbd65c3dfd8ba",
            "https:\/\/pb3.pstatp.com\/obj\/46bd000dbd65c3dfd8ba",
            "https:\/\/pb3.pstatp.com\/obj\/46bd000dbd65c3dfd8ba"
          ],
          "uri" : "46bd000dbd65c3dfd8ba"
        },
        "effects_data" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/obj\/2efe00235ebf732bb56f",
            "https:\/\/pb3.pstatp.com\/obj\/2efe00235ebf732bb56f",
            "https:\/\/pb3.pstatp.com\/obj\/2efe00235ebf732bb56f"
          ],
          "uri" : "2efe00235ebf732bb56f"
        },
        "source_platform" : 22,
        "author" : "FlyBoy,Gavrielle",
        "id_str" : "6443685673772944141"
      },
      "is_top" : 0,
      "aweme_id" : "6447230455191178509",
      "cmt_swt" : false,
      "risk_infos" : {
        "warn" : false,
        "content" : "",
        "risk_sink" : false,
        "type" : 0
      },
      "label_origin_author" : {
        "url_list" : [
          "https:\/\/p3.pstatp.com\/obj\/216a001ff805affc1dab",
          "https:\/\/pb9.pstatp.com\/obj\/216a001ff805affc1dab",
          "https:\/\/pb3.pstatp.com\/obj\/216a001ff805affc1dab"
        ],
        "uri" : "216a001ff805affc1dab"
      },
      "label_origin_author_text" : "發(fā)起者",
      "label_top" : {
        "url_list" : [
          "https:\/\/p3.pstatp.com\/obj\/c150000f34767e2cb56",
          "https:\/\/pb9.pstatp.com\/obj\/c150000f34767e2cb56",
          "https:\/\/pb3.pstatp.com\/obj\/c150000f34767e2cb56"
        ],
        "uri" : "c150000f34767e2cb56"
      },
      "is_ads" : false,
      "statistics" : {
        "digg_count" : 19578,
        "aweme_id" : "6447230455191178509",
        "share_count" : 283,
        "play_count" : 1296122,
        "comment_count" : 100
      },
      "video_labels" : [],
      "rate" : 12,
      "bodydance_score" : 0,
      "desc" : "空氣泳",
      "author" : {
        "special_lock" : 0,
        "is_binded_weibo" : false,
        "shield_follow_notice" : 0,
        "nickname_lock" : 0,
        "avatar_larger" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/1080x1080\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/1080x1080\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/1080x1080\/38c90028f622dd195aaf.jpeg"
          ],
          "uri" : "38c90028f622dd195aaf"
        },
        "follow_status" : 0,
        "with_commerce_entry" : false,
        "original_music_qrcode" : null,
        "authority_status" : 0,
        "is_ad_fake" : false,
        "prevent_download" : false,
        "enterprise_verify" : false,
        "verification_type" : 1,
        "app_id" : 1128,
        "weibo_url" : "",
        "community_discipline_status" : 0,
        "risk_flag" : 0,
        "need_recommend" : 1,
        "live_rec_level" : 0,
        "unique_id" : "MysticzZ",
        "hide_location" : false,
        "enterprise_verify_reason" : "",
        "short_id" : "17612748",
        "account_region" : "",
        "star_use_new_download" : false,
        "signature" : "頹",
        "avatar_medium" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/720x720\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/720x720\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/720x720\/38c90028f622dd195aaf.jpeg"
          ],
          "uri" : "38c90028f622dd195aaf"
        },
        "verify_info" : "",
        "create_time" : 1495648525,
        "story_open" : false,
        "region" : "CN",
        "hide_search" : false,
        "avatar_thumb" : {
          "url_list" : [
            "https:\/\/p1.pstatp.com\/aweme\/100x100\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/100x100\/38c90028f622dd195aaf.jpeg",
            "https:\/\/pb3.pstatp.com\/aweme\/100x100\/38c90028f622dd195aaf.jpeg"
          ],
          "uri" : "38c90028f622dd195aaf"
        },
        "school_poi_id" : "",
        "update_before" : 0,
        "shield_comment_notice" : 0,
        "original_music_cover" : null,
        "shield_digg_notice" : 0,
        "bind_phone" : "",
        "live_verify" : 0,
        "birthday" : "1999-03-28",
        "school_type" : 0,
        "uid" : "60978405851",
        "avatar_uri" : "38c90028f622dd195aaf",
        "weibo_schema" : "",
        "is_verified" : true,
        "custom_verify" : "",
        "commerce_user_level" : 0,
        "gender" : 1,
        "has_register_notice" : 0,
        "reflow_page_gid" : 0,
        "reflow_page_uid" : 0,
        "nickname" : "Atlantis.",
        "weibo_verify" : "",
        "language" : "zh-Hans",
        "share_qrcode_uri" : "216a0035fc39c9e9376f",
        "room_id" : 0,
        "constellation" : 1,
        "school_name" : "",
        "weibo_name" : ""
      },
      "is_relieve" : false
    },
    {
        ...第二條數(shù)據(jù)
    }
    ...其他更多的數(shù)據(jù)
  ]
}

4）對(duì)這個(gè)API進(jìn)行分析（破解或者繞過API的加密），找出可以直接獲取JSON數(shù)據(jù)的純種API；
API的內(nèi)容為：
https://aweme.snssdk.com/aweme/v1/challenge/aweme/?iid=30373511894&device_id=35781128184&os_api=18&app_name=aweme&channel=App%20Store&idfa=811A8841-030F-4AEA-B934-C2A56489C32D&device_platform=iphone&build_number=17805&vid=A4BB3AF4-7981-4805-995B-78419881DC11&openudid=99bf2b608173e21cefd562496d2cf21fa8eba580&device_type=iPhone7,2&app_version=1.7.8&version_code=1.7.8&os_version=11.3&screen_width=750&aid=1128&ac=WIFI&ch_id=1574030716416014&count=21&cursor=0&pull_type=2&query_type=0&type=5&mas=001469ed4a7a3f61de046e21c8c7ae8bcb64a983471da18ec0c8d1&as=a1c5206d9e676a38ee4960&ts=1524500606。
為了便于分析，我們先把一些無關(guān)的手機(jī)設(shè)備和版本信息除掉： https://aweme.snssdk.com/aweme/v1/challenge/aweme/?ch_id=1574030716416014&count=21&cursor=0&pull_type=2&query_type=0&type=5&as=a1c5206d9e676a38ee4960&ts=1524500606
它現(xiàn)在只剩下ch_id、count、cursor、pull_type、query_type、type、as以及ts幾個(gè)參數(shù)，其中pull_type、query_type和type三個(gè)是固定的，不用管。cursor和count是獲取的數(shù)據(jù)的開始和數(shù)量，ch_id應(yīng)該是挑戰(zhàn)的唯一標(biāo)示。接下來就是as和ts兩個(gè)參數(shù)，通過多次實(shí)驗(yàn)，我們發(fā)現(xiàn)這個(gè)API使用了as和cp參數(shù)進(jìn)行了加密處理，也就是說我們無法使用這個(gè)API直接獲取到API的數(shù)據(jù)，需要對(duì)as、ts的加密算法進(jìn)行破解。當(dāng)然，上面我也提過，面對(duì)這個(gè)情況，要么破解算法要么繞過加密，原則上是能夠繞過的就不要選擇破解，因?yàn)槠平獾碾y度很大，需要逆向，這個(gè)以后再講（我也沒做過）。
APP或PC上破解不了的接口，可以嘗試使用m站。這個(gè)是我們的一貫原則。
我們發(fā)現(xiàn)抖音中的挑戰(zhàn)有一個(gè)分享按鈕，我們?cè)囍窒沓鲞@個(gè)H5頁面來進(jìn)行分析，分享之后的URL為：
https://www.iesdouyin.com/share/challenge/1574030716416014?utm_campaign=client_share&app=aweme&utm_medium=ios&iid=30373511894&utm_source=weixin&tt_from=weixin&utm_source=weixin&utm_medium=aweme_ios&utm_campaign=client_share&uid=58875748483&did=30373511894
在瀏覽器上打開這個(gè)URL，通過分析它的請(qǐng)求，只看XHR，見下圖4：