selenium、chromedirver

driver = webdriver.Chrome()
driver.get("https://www.zhipin.com/web/geek/job?query=%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90&city=100010000")

獲取當前頁面信息,一定要等待網(wǎng)頁加載完成,針對反爬措施,要點手動與自動相結(jié)合,手動登錄(自動難以實現(xiàn)),翻頁可以自動,一定要注意到driver始終獲取的是當前頁面最新信息

elements = driver.find_elements(By.CSS_SELECTOR, ".job-card-wrapper")
jobinfo_csv(elements)
def jobTaglist(taglist):
    strTaglist = ""
    for el in taglist:
        strTaglist += el.text+"*"
    return strTaglist
def jobinfo_csv(elements):
    jobInfo = {}
    for element in elements:
        jobInfo["job-name"] = element.find_element(By.CSS_SELECTOR, ".job-name").text
        jobInfo['job-area'] = element.find_element(By.CSS_SELECTOR, ".job-area").text
        jobInfo['salary'] = element.find_element(By.CSS_SELECTOR, ".salary").text
        t = element.find_element(By.CSS_SELECTOR, ".job-card-footer .tag-list").find_elements(By.XPATH, "./li")
        jobInfo['technologylist']= jobTaglist(t)
        taglist = element.find_element(By.CSS_SELECTOR, ".tag-list").find_elements(By.XPATH, "./li")
        jobInfo['taglist']= jobTaglist(taglist)
        companyTaglist = element.find_element(By.CSS_SELECTOR, ".company-tag-list").find_elements(By.XPATH, "./li")
        jobInfo['companyTaglist']= jobTaglist(companyTaglist)
        jobInfo['company-name'] = element.find_element(By.CSS_SELECTOR, ".company-name").text
        jobInfo['info-desc'] = element.find_element(By.CSS_SELECTOR, ".info-desc").text
        pddata = pd.DataFrame([jobInfo])
        pddata.to_csv('static.csv',mode='a',encoding='utf-8',header=False,index=False)
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務。

相關閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容