TF2 基礎(chǔ) ：from_tensor_slices,shuffle,batch

tf

主要總結(jié)整理TF2的使用過程總的基礎(chǔ)的的知識點(diǎn)

圖片數(shù)據(jù)的切片，混合，打包

測試數(shù)據(jù)集為：

import tensorflow as tf
import numpy as np
 

np.random.seed(10)#固定每次的隨機(jī)數(shù)
features, labels = (np.random.sample((4, 2, 2)),  # 模擬6組數(shù)據(jù)，每組數(shù)據(jù)3個特征
                    np.random.sample((4, 1)))  # 模擬6組數(shù)據(jù)，每組數(shù)據(jù)對應(yīng)一個標(biāo)簽，注意兩者的維數(shù)必須匹配

print((features))  #  打印feature數(shù)據(jù)
print((labels))  #  打印標(biāo)簽數(shù)據(jù)

得到測試用的數(shù)據(jù)集為：

feture:
[[[0.77132064 0.02075195]
  [0.63364823 0.74880388]]

 [[0.49850701 0.22479665]
  [0.19806286 0.76053071]]

 [[0.16911084 0.08833981]
  [0.68535982 0.95339335]]

 [[0.00394827 0.51219226]
  [0.81262096 0.61252607]]]

lables:
[[0.72175532]
 [0.29187607]
 [0.91777412]
 [0.71457578]]

我們用features表示4張大小為[2,2]的圖片數(shù)據(jù)，labels表示4張圖片的標(biāo)簽數(shù)據(jù)，例如屬于哪一類圖片

from_tensor_slices

執(zhí)行以下操作對數(shù)據(jù)集進(jìn)行切片操作

data = tf.data.Dataset.from_tensor_slices((features, labels))

輸出的結(jié)果是包含每一張圖片的數(shù)據(jù)信息和特征信息的節(jié)點(diǎn)，4張圖片對應(yīng)的是4個Tensor節(jié)點(diǎn)

----------from_tensor_slices--------------

(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=array([[0.77132064, 0.02075195],
       [0.63364823, 0.74880388]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.72175532])>)
       
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.49850701, 0.22479665],
       [0.19806286, 0.76053071]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.29187607])>)
       
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.16911084, 0.08833981],
       [0.68535982, 0.95339335]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.91777412])>)
       
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.00394827, 0.51219226],
       [0.81262096, 0.61252607]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.71457578])>)

shuffle

對from_tensor_slices處理的數(shù)據(jù)，進(jìn)行混合，混合就是打亂原數(shù)組之間的順序，數(shù)組的數(shù)據(jù)大小和內(nèi)容并沒有改變;
混合的數(shù)據(jù)越大，混合程度越高

shuffle_data =data.shuffle(4)#4表示每次混合的buffer size，因?yàn)槲覀冎挥兴膫€數(shù)據(jù)，直接混合所有數(shù)據(jù)

混合后的結(jié)果為：
可以跟from_tensor_slices數(shù)據(jù)進(jìn)行比較下，原來的數(shù)據(jù)順序?yàn)閇0,1,2,3],混合后為[3,2,1,0]

----------shuffle--------------
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.00394827, 0.51219226],
       [0.81262096, 0.61252607]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.71457578])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.169121084, 0.08833981],
       [0.68535982, 0.95339335]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.91777412])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.49850701, 0.22479665],
       [0.19806286, 0.76053071]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.29187607])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.77132064, 0.02075195],
       [0.63364823, 0.74880388]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.72175532])>)

batch##

batch_data =data.batch(1)

對shuffle處理后的數(shù)據(jù)進(jìn)行打包，如果為1，則數(shù)據(jù)內(nèi)容和格式跟shuffle的數(shù)據(jù)相同，相當(dāng)于沒有處理

----------batch(1)--------------
(<tf.Tensor: shape=(1, 2, 2), dtype=float64, numpy=
array([[[0.77132064, 0.02075195],
        [0.63364823, 0.74880388]]])>, <tf.Tensor: shape=(1, 1), dtype=float64, numpy=array([[0.72175532]])>)

(<tf.Tensor: shape=(1, 2, 2), dtype=float64, numpy=
array([[[0.00394827, 0.51219226],
        [0.81262096, 0.61252607]]])>, <tf.Tensor: shape=(1, 1), dtype=float64, numpy=array([[0.71457578]])>)

(<tf.Tensor: shape=(1, 2, 2), dtype=float64, numpy=
array([[[0.16911084, 0.08833981],
        [0.68535982, 0.95339335]]])>, <tf.Tensor: shape=(1, 1), dtype=float64, numpy=array([[0.91777412]])>)

(<tf.Tensor: shape=(1, 2, 2), dtype=float64, numpy=
array([[[0.49850701, 0.22479665],
        [0.19806286, 0.76053071]]])>, <tf.Tensor: shape=(1, 1), dtype=float64, numpy=array([[0.29187607]])>)

如果batch(2)，則每個節(jié)點(diǎn)信息中包含兩張圖片數(shù)據(jù)，以此類推

(<tf.Tensor: shape=(2, 2, 2), dtype=float64, numpy=
array([[[0.00394827, 0.51219226],
        [0.81262096, 0.61252607]],

       [[0.77132064, 0.02075195],
        [0.63364823, 0.74880388]]])>, <tf.Tensor: shape=(2, 1), dtype=float64, numpy=
array([[0.71457578],
       [0.72175532]])>)


(<tf.Tensor: shape=(2, 2, 2), dtype=float64, numpy=
array([[[0.49850701, 0.22479665],
        [0.19806286, 0.76053071]],

       [[0.16911084, 0.08833981],
        [0.68535982, 0.95339335]]])>, <tf.Tensor: shape=(2, 1), dtype=float64, numpy=
array([[0.29187607],
       [0.91777412]])>)

總結(jié)一下：

  1. from_tensor_slices對原圖片數(shù)據(jù)進(jìn)行切片，有多上張圖片就切成多少個數(shù)據(jù)
  2. shuffle,對切好后的數(shù)據(jù)進(jìn)行混合，交換下順序
  3. batch對數(shù)據(jù)進(jìn)行打包，便于批量化處理；batch后的節(jié)點(diǎn)數(shù)為（數(shù)據(jù)集大小/batch大?。?，兩個數(shù)字可能不是整除，會導(dǎo)致一個batch大小可能小于等于batch size

完整代碼：shuffle_and_batch.py

import tensorflow as tf
import numpy as np
 
from    tensorflow.keras import datasets

np.random.seed(10)#固定每次的隨機(jī)數(shù)
features, labels = (np.random.sample((4, 2, 2)),  # 模擬6組數(shù)據(jù)，每組數(shù)據(jù)3個特征
                    np.random.sample((4, 1)))  # 模擬6組數(shù)據(jù)，每組數(shù)據(jù)對應(yīng)一個標(biāo)簽，注意兩者的維數(shù)必須匹配

print((features))  #  打印feature數(shù)據(jù)
print((labels))  #  打印標(biāo)簽數(shù)據(jù)

print('----------from_tensor_slices--------------') 

for element in features: 
  print(element) 

#切片轉(zhuǎn)換，將數(shù)據(jù)轉(zhuǎn)化成tesor節(jié)點(diǎn)數(shù)據(jù)
data = tf.data.Dataset.from_tensor_slices((features, labels))

for element in data: 
  print(element) 

print('----------shuffle--------------') 

#shuffle_data = tf.data.Dataset.from_tensor_slices((features,labels)).shuffle(1000).batch(128)
shuffle_data =data.shuffle(4)

for element in shuffle_data: 
  print(element) 
  
  
print('----------batch--------------') 
batch_data =shuffle_data.batch(2)
  
for element in batch_data: 
  print(element) 
  
  
batch_data.repeat(2)

output:

runfile('G:/GitHub/tensorflow/Spyder/TF2/shuffle_and_batch.py', wdir='G:/GitHub/tensorflow/Spyder/TF2')
[[[0.77132064 0.02075195]
  [0.63364823 0.74880388]]

 [[0.49850701 0.22479665]
  [0.19806286 0.76053071]]

 [[0.16911084 0.08833981]
  [0.68535982 0.95339335]]

 [[0.00394827 0.51219226]
  [0.81262096 0.61252607]]]
[[0.72175532]
 [0.29187607]
 [0.91777412]
 [0.71457578]]
----------from_tensor_slices--------------
[[0.77132064 0.02075195]
 [0.63364823 0.74880388]]
[[0.49850701 0.22479665]
 [0.19806286 0.76053071]]
[[0.16911084 0.08833981]
 [0.68535982 0.95339335]]
[[0.00394827 0.51219226]
 [0.81262096 0.61252607]]
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.77132064, 0.02075195],
       [0.63364823, 0.74880388]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.72175532])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.49850701, 0.22479665],
       [0.19806286, 0.76053071]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.29187607])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.16911084, 0.08833981],
       [0.68535982, 0.95339335]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.91777412])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.00394827, 0.51219226],
       [0.81262096, 0.61252607]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.71457578])>)
----------shuffle--------------
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.49850701, 0.22479665],
       [0.19806286, 0.76053071]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.29187607])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.16911084, 0.08833981],
       [0.68535982, 0.95339335]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.91777412])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.77132064, 0.02075195],
       [0.63364823, 0.74880388]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.72175532])>)
(<tf.Tensor: shape=(2, 2), dtype=float64, numpy=
array([[0.00394827, 0.51219226],
       [0.81262096, 0.61252607]])>, <tf.Tensor: shape=(1,), dtype=float64, numpy=array([0.71457578])>)
----------batch--------------
(<tf.Tensor: shape=(2, 2, 2), dtype=float64, numpy=
array([[[0.77132064, 0.02075195],
        [0.63364823, 0.74880388]],

       [[0.16911084, 0.08833981],
        [0.68535982, 0.95339335]]])>, <tf.Tensor: shape=(2, 1), dtype=float64, numpy=
array([[0.72175532],
       [0.91777412]])>)
(<tf.Tensor: shape=(2, 2, 2), dtype=float64, numpy=
array([[[0.00394827, 0.51219226],
        [0.81262096, 0.61252607]],

       [[0.49850701, 0.22479665],
        [0.19806286, 0.76053071]]])>, <tf.Tensor: shape=(2, 1), dtype=float64, numpy=
array([[0.71457578],
       [0.29187607]])>)
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡書系信息發(fā)布平臺，僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容