1.Getting and preparing the data
每行包括：label，句子
>> head cooking.stackexchange.txt
__label__sauce __label__cheese How much does potato starch affect a cheese sauce recipe?__label__food-safety __label__acidity Dangerous pathogens capable of growing in acidic environments __label__cast-iron __label__stove How do I cover up the white spots on my cast iron stove?__label__restaurant Michelin Three Star Restaurant; but if the chef is not there__label__knife-skills__label__dicing Without knife skills, how can I quickly and accurately dice vegetables?
在訓(xùn)練之前將數(shù)據(jù)分為訓(xùn)練集和交叉驗(yàn)證集 4：1
>> wc cooking.stackexchange.txt
? 15404? 169582 1401900 cooking.stackexchange.txt
>> head -n 12404 cooking.stackexchange.txt > cooking.train
>> tail -n 3000 cooking.stackexchange.txt > cooking.valid
2.Our first classifier
訓(xùn)練分類器
>> ./fasttext supervised -input cooking.train -output model_cooking
Read 0M words
Number of words:? 14543
Number of labels: 735
Progress: 100.0% words/sec/thread:? 90012 lr:? 0.000000 loss: 10.222594 ETA:? 0h 0m
?-input 指示訓(xùn)練集?
-output定義在哪里存放model文件
訓(xùn)練的最后會(huì)生成 model.bin文件存放分類器
交互式測(cè)試分類器：
>> ./fasttext predict model_cooking.bin -
>> Why not put knives in the dishwasher?
__label__baking
>> Why not put knives in the dishwasher?
__label__food-safety
執(zhí)行交叉驗(yàn)證：
>> ./fasttext test model_cooking.bin cooking.valid
N 3000
P@1 0.138
R@1 0.0595
Number of examples: 3000
precision:P@1
recall:R@1
每5次迭代計(jì)算一次precision和recall
>> ./fasttext test model_cooking.bin cooking.valid 5
N 3000
P@5 0.0677
R@5 0.146
Number of examples: 3000
3.Advanced readers: precision and recall
The precision is the number of correct labels among the labels predicted by fastText.
The recall is the number of labels that successfully were predicted, among all the real labels.
4.Making the model better
4.1 preprocessing the data
>>cat cooking.stackexchange.txt | sed -e "s/\([.\!?,'/()]\)/ \1 /g" | tr "[:upper:]" "[:lower:]" > cooking.preprocessed.txt
head -n 12404 cooking.preprocessed.txt > cooking.train
?tail -n 3000 cooking.preprocessed.txt > cooking.valid
使用預(yù)訓(xùn)練過(guò)的數(shù)據(jù)訓(xùn)練模型：
>> ./fasttext supervised -input cooking.train -output model_cooking
Read 0M words
Number of words: 8952
Number of labels: 735
Progress: 100.0% words/sec/thread: 101142?? lr: 0.000000?? loss: 11.018550?? ETA: 0h 0m
>> ./fasttext test model_cooking.bin cooking.valid
N 3000
P@1 0.172
R@1 0.0744
Number of examples: 3000
4.2 more epochs and larger learning rate
默認(rèn)情況下fastText 只進(jìn)行5次迭代
使用?-epoch?自定義迭代次數(shù)
>> ./fasttext supervised -input cooking.train -output model_cooking -epoch 25
Read 0M words
Number of words: 8952
Number of labels: 735
Progress: 100.0% words/sec/thread: 92990 ? lr: 0.000000?? loss: 7.257324 ? ETA: 0h 0m
>> ./fasttext test model_cooking.bin cooking.valid
N 3000
P@1 0.514
R@1 0.222
Number of examples: 3000
加快學(xué)習(xí)速度--改變learning rate
>> ./fasttext supervised -input cooking.train -output model_cooking -lr 1.0
Read 0M words
Number of words: 8952
Number of labels: 735
Progress: 100.0% words/sec/thread: 91682???? lr: 0.000000 ? ? loss: 6.346271 ? ? ETA: 0h 0m
>> ./fasttext test model_cooking.bin cooking.valid
N 3000
P@1 0.579
R@1 0.25
Number of examples: 3000
4.3 word n-grams
使用bigrams訓(xùn)練模型
>> ./fasttext supervised -input cooking.train -output model_cooking -lr 1.0 -epoch 25 -wordNgrams 2
Read 0M words
Number of words: 8952
Number of labels: 735
Progress: 100.0% words/sec/thread: 93126 ?? lr: 0.000000 ? ? loss: 3.139972???? ETA: 0h 0m
>> ./fasttexttestmodel_cooking.bin cooking.valid
N 3000
P@1 0.61
R@1 0.264
Number of examples: 3000
提升模型準(zhǔn)確率的幾種方法：
preprocessing the data ;
changing the number of epochs (using the option-epoch, standard range[5 - 50]) ;
changing the learning rate (using the option-lr, standard range[0.1 - 1.0]) ;
using word n-grams (using the option-wordNgrams, standard range[1 - 5]).
5.Scaling things up
使用 hierarchical softmax可以讓模型訓(xùn)練的更快 This can be done with the option -loss hs:
>> ./fasttext supervised -input cooking.train -output model_cooking -lr 1.0 -epoch 25 -wordNgrams 2 -bucket 200000 -dim 50 -loss hs
Read 0M words
Number of words:? 8952
Number of labels: 735
Progress: 100.0% words/sec/thread: 2139399 lr:? 0.000000 loss:? 2.142308 ETA:? 0h 0m
