Python實(shí)戰(zhàn)計(jì)劃學(xué)習(xí)筆記-爬取商品信息

結(jié)果圖###

result.png

代碼###

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from bs4 import BeautifulSoup

star_alls = []
star_all = []
infos = []
num = 0
path = 'D:/Users/Jessy zhou/Plan-for-combating-master/1_week/1.2 web-element/1.2homework/webpage/index.html'


def getText(item):
    return '' if item == 0 else ''.join(item.get_text())


def getSrc(item):
    return '' if item == 0 else ''.join(item.get('src'))


with open(path, 'r') as web_data:
    soup = BeautifulSoup(web_data.read(), 'html.parser')

    imgs = soup.select('div.thumbnail > img')
    titles = soup.select('div.caption > h4 > a')
    descs = soup.select('div.caption > p')
    prices = soup.select('div.caption > h4.pull-right')
    reviews = soup.select('div.ratings > p.pull-right')
    # stars =soup.select('div.ratings > p:nth-of-type(2)')
    stars = soup.find_all('span', 'glyphicon')

    for star in stars:
        span = str(star)
        if 'empty' in span:
            span_star = span.replace('<span class="glyphicon glyphicon-star-empty"></span>', u'☆')
        else:
            span_star = span.replace('<span class="glyphicon glyphicon-star"></span>', u'★')
        star_alls.append(span_star)
    star_alls = star_alls[2:]
    limit = len(star_alls) / 5
    while num < limit:
        star_all.append(star_alls[num * 5:(num + 1) * 5])
        num += 1

    for img, title, desc, price, review, star_c in zip(imgs, titles, descs, prices, reviews, star_all):
        data = {
            'img': getSrc(img),
            'title': getText(title),
            'desc': getText(desc),
            'price': getText(price),
            'review': getText(review),
            'star': star_c
        }
        infos.append(data)

總結(jié)###

1.為了顯示★,需要加u以顯示
2.find_all的方法也不是非常的簡(jiǎn)單,而且replace的字符串過(guò)長(zhǎng),應(yīng)該可以得到簡(jiǎn)化

最后編輯于
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀(guān)點(diǎn),簡(jiǎn)書(shū)系信息發(fā)布平臺(tái),僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容