理論內(nèi)容

貝葉斯定理

貝葉斯定理是描述條件概率關(guān)系的定律
$$P(A|B) = \cfrac{P(B|A) * P(A)}{P(B)}$$

樸素貝葉斯分類器

樸素貝葉斯分類器是一種基于概率的分類器，我們做以下定義：

• B：具有特征向量B
• A：屬于類別A

有了這個(gè)定義，我們解釋貝葉斯公式

• P(A|B)：具有特征向量B樣本屬于A類別的概率（計(jì)算目標(biāo)）
• P(B|A)：在A類別中B向量出現(xiàn)的概率（訓(xùn)練樣本中的數(shù)據(jù)）
• P(A)：A類出現(xiàn)的概率（訓(xùn)練樣本中的頻率）
• P(B)：B特征向量出現(xiàn)的概率（訓(xùn)練樣本中的頻率）

對于樸素貝葉斯分類器，進(jìn)一步假設(shè)特征向量之間無關(guān)，那么樸素貝葉斯分類器公式可以如下表示$$P(A|B) = \cfrac{P(A)\prod P(B_{i} |A)}{P(B)}$$

以上公式右側(cè)的值都可以在訓(xùn)練樣本中算得。進(jìn)行預(yù)測時(shí)，分別計(jì)算每個(gè)類別的概率，取概率最高的一個(gè)類別。

特征向量為連續(xù)值的樸素貝葉斯分類器

對于連續(xù)值，有以下兩種處理方式

• 將連續(xù)值按區(qū)間離散化
• 假設(shè)特征向量服從正態(tài)分布或其他分布（很強(qiáng)的先驗(yàn)假設(shè)），由樣本中估計(jì)出參數(shù)，計(jì)算貝葉斯公式時(shí)帶入概率密度

代碼實(shí)現(xiàn)

導(dǎo)入數(shù)據(jù)——文本新聞數(shù)據(jù)

# from sklearn.datasets import fetch_20newsgroups
# news = fetch_20newsgroups(subset='all')
# print(len(news.data))
# print(news.data[0])

from sklearn import datasets
train = datasets.load_files("./20newsbydate/20news-bydate-train")
test = datasets.load_files("./20newsbydate/20news-bydate-test")

print(train.DESCR)
print(len(train.data))
print(train.data[0])

None
11314
b"From: cubbie@garnet.berkeley.edu (                               )\nSubject: Re: Cubs behind Marlins? How?\nArticle-I.D.: agate.1pt592$f9a\nOrganization: University of California, Berkeley\nLines: 12\nNNTP-Posting-Host: garnet.berkeley.edu\n\n\ngajarsky@pilot.njin.net writes:\n\nmorgan and guzman will have era's 1 run higher than last year, and\n the cubs will be idiots and not pitch harkey as much as hibbard.\n castillo won't be good (i think he's a stud pitcher)\n\n       This season so far, Morgan and Guzman helped to lead the Cubs\n       at top in ERA, even better than THE rotation at Atlanta.\n       Cubs ERA at 0.056 while Braves at 0.059. We know it is early\n       in the season, we Cubs fans have learned how to enjoy the\n       short triumph while it is still there.\n"

處理數(shù)據(jù)——特征抽?。ㄎ淖窒蛄炕?/h1>

from sklearn.feature_extraction.text import CountVectorizer
vec = CountVectorizer(stop_words="english",decode_error='ignore')
train_vec = vec.fit_transform(train.data)
test_vec = vec.transform(test.data)
print(train_vec.shape)

(11314, 129782)

模型訓(xùn)練

from sklearn.naive_bayes import MultinomialNB
bays = MultinomialNB()
bays.fit(train_vec,train.target)

MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)

模型評(píng)估

使用自帶評(píng)估器

bays.score(test_vec,test.target)

0.80244291024960168

調(diào)用評(píng)估器

from sklearn.metrics import classification_report
y = bays.predict(test_vec)
print(classification_report(test.target,y,target_names=test.target_names))

                          precision    recall  f1-score   support

             alt.atheism       0.80      0.81      0.80       319
           comp.graphics       0.65      0.80      0.72       389
 comp.os.ms-windows.misc       0.80      0.04      0.08       394
comp.sys.ibm.pc.hardware       0.55      0.80      0.65       392
   comp.sys.mac.hardware       0.85      0.79      0.82       385
          comp.windows.x       0.69      0.84      0.76       395
            misc.forsale       0.89      0.74      0.81       390
               rec.autos       0.89      0.92      0.91       396
         rec.motorcycles       0.95      0.94      0.95       398
      rec.sport.baseball       0.95      0.92      0.93       397
        rec.sport.hockey       0.92      0.97      0.94       399
               sci.crypt       0.80      0.96      0.87       396
         sci.electronics       0.79      0.70      0.74       393
                 sci.med       0.88      0.87      0.87       396
               sci.space       0.84      0.92      0.88       394
  soc.religion.christian       0.81      0.95      0.87       398
      talk.politics.guns       0.72      0.93      0.81       364
   talk.politics.mideast       0.93      0.94      0.94       376
      talk.politics.misc       0.68      0.62      0.65       310
      talk.religion.misc       0.88      0.44      0.59       251

             avg / total       0.81      0.80      0.78      7532