mnist實(shí)例--用簡(jiǎn)單的神經(jīng)網(wǎng)絡(luò)來訓(xùn)練和測(cè)試
mnist實(shí)例--卷積神經(jīng)網(wǎng)絡(luò)CNN
簡(jiǎn)單神經(jīng)網(wǎng)絡(luò)沒有卷積功能，只有簡(jiǎn)單的三層：輸入層，隱藏層和輸出層。
數(shù)據(jù)從輸入層輸入，在隱藏層進(jìn)行加權(quán)變換，最后在輸出層進(jìn)行輸出。輸出的時(shí)候，我們可以使用softmax回歸，輸出屬于每個(gè)類別的概率值。表示如下：
其中，x1,x2,x3為輸入數(shù)據(jù)，經(jīng)過運(yùn)算后，得到三個(gè)數(shù)據(jù)屬于某個(gè)類別的概率值y1,y2,y3. 用簡(jiǎn)單的公式表示如下：
在訓(xùn)練過程中，我們將真實(shí)的結(jié)果和預(yù)測(cè)的結(jié)果相比（交叉熵比較法），會(huì)得到一個(gè)殘差。公式如下：
y 是我們預(yù)測(cè)的概率值, y' 是實(shí)際的值。這個(gè)殘差越小越好，我們可以使用梯度下降法，不停地改變W和b的值，使得殘差逐漸變小，最后收斂到最小值。這樣訓(xùn)練就完成了，我們就得到了一個(gè)模型（W和b的最優(yōu)化值）。
簡(jiǎn)單神經(jīng)網(wǎng)絡(luò)的訓(xùn)練例子（非卷積）：
# -*- coding: utf-8 -*-
"""
@author: gongjia copy
"""
import tensorflow as tf
import tensorflow.examples.tutorials.mnist.input_data as input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
x = tf.placeholder(tf.float32, [None, 784])
y_actual = tf.placeholder(tf.float32, shape=[None, 10])
W = tf.Variable(tf.zeros([784,10])) #初始化權(quán)值W
b = tf.Variable(tf.zeros([10])) #初始化偏置項(xiàng)b
y_predict = tf.nn.softmax(tf.matmul(x,W) + b) #加權(quán)變換并進(jìn)行softmax回歸，得到預(yù)測(cè)概率
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_actual*tf.log(y_predict),reduction_indies=1)) #求交叉熵
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy) #用梯度下降法使得殘差最小
correct_prediction = tf.equal(tf.argmax(y_predict,1), tf.argmax(y_actual,1)) #在測(cè)試階段，測(cè)試準(zhǔn)確度計(jì)算
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) #多個(gè)批次的準(zhǔn)確度均值
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
for i in range(1000): #訓(xùn)練階段，迭代1000次
batch_xs, batch_ys = mnist.train.next_batch(100) #按批次訓(xùn)練，每批100行數(shù)據(jù)
sess.run(train_step, feed_dict={x: batch_xs, y_actual: batch_ys}) #執(zhí)行訓(xùn)練
if(i%100==0): #每訓(xùn)練100次，測(cè)試一次
print "accuracy:",sess.run(accuracy, feed_dict={x: mnist.test.images, y_actual: mnist.test.labels})
每訓(xùn)練100次，測(cè)試一次，隨著訓(xùn)練次數(shù)的增加，測(cè)試精度也在增加。訓(xùn)練結(jié)束后，1W行數(shù)據(jù)測(cè)試的平均精度為91%左右，不是太高，肯定沒有CNN高。
# -*- coding: utf-8 -*-
"""
@author: gongjia copy
"""
import tensorflow as tf
import tensorflow.examples.tutorials.mnist.input_data as input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True) #下載并加載mnist數(shù)據(jù)
x = tf.placeholder(tf.float32, [None, 784]) #輸入的數(shù)據(jù)占位符
y_actual = tf.placeholder(tf.float32, shape=[None, 10]) #輸入的標(biāo)簽占位符
#定義一個(gè)函數(shù)，用于初始化所有的權(quán)值 W
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
#定義一個(gè)函數(shù)，用于初始化所有的偏置項(xiàng) b
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
#定義一個(gè)函數(shù)，用于構(gòu)建卷積層
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
#定義一個(gè)函數(shù)，用于構(gòu)建池化層
def max_pool(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],strides=[1, 2, 2, 1], padding='SAME')
#構(gòu)建網(wǎng)絡(luò)
x_image = tf.reshape(x, [-1,28,28,1]) #轉(zhuǎn)換輸入數(shù)據(jù)shape,以便于用于網(wǎng)絡(luò)中
W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1) #第一個(gè)卷積層
h_pool1 = max_pool(h_conv1) #第一個(gè)池化層
W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2) #第二個(gè)卷積層
h_pool2 = max_pool(h_conv2) #第二個(gè)池化層
W_fc1 = weight_variable([7 * 7 * 64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64]) #reshape成向量
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1) #第一個(gè)全連接層
keep_prob = tf.placeholder("float")
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob) #dropout層
W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])
y_predict=tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2) #softmax層
cross_entropy = -tf.reduce_sum(y_actual*tf.log(y_predict)) #交叉熵
train_step = tf.train.GradientDescentOptimizer(1e-3).minimize(cross_entropy) #梯度下降法
correct_prediction = tf.equal(tf.argmax(y_predict,1), tf.argmax(y_actual,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) #精確度計(jì)算
sess=tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
for i in range(20000):
batch = mnist.train.next_batch(50)
if i%100 == 0: #訓(xùn)練100次，驗(yàn)證一次
train_acc = accuracy.eval(feed_dict={x:batch[0], y_actual: batch[1], keep_prob: 1.0})
print('step',i,'training accuracy',train_acc)
train_step.run(feed_dict={x: batch[0], y_actual: batch[1], keep_prob: 0.5})
test_acc=accuracy.eval(feed_dict={x: mnist.test.images, y_actual: mnist.test.labels, keep_prob: 1.0})
print("test accuracy",test_acc)
('step', 18800, 'training accuracy', 0.079999998)
('step', 18900, 'training accuracy', 0.059999999)
('step', 19000, 'training accuracy', 0.1)
('step', 19100, 'training accuracy', 0.059999999)
('step', 19200, 'training accuracy', 0.12)
('step', 19300, 'training accuracy', 0.14)
('step', 19400, 'training accuracy', 0.079999998)
('step', 19500, 'training accuracy', 0.039999999)
('step', 19600, 'training accuracy', 0.16)
('step', 19700, 'training accuracy', 0.1)
('step', 19800, 'training accuracy', 0.079999998)
('step', 19900, 'training accuracy', 0.1)
('test accuracy', 0.097999997)
