樹莓派攝像頭運(yùn)行物體檢測 - tensorflow with SSD

環(huán)境

sudo pip uninstall tensorflow
sudo pip install --upgrade tensorflow-1.4.1-cp27-none-linux_armv7l.whl

準(zhǔn)備模型

  • 下載tensorflow提供的models API并解壓，我這里解壓后的目錄為models_master，下載路徑：
    https://github.com/tensorflow/models/tree/master/research/object_detection/models
  • 下載訓(xùn)練好的模型并放到上一步models_master下的object_detection/models目錄，下載路徑：
    https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md
    這里下載幾個(gè)典型的：ssd_mobilenet_v1_coco_2017_11_17、faster_rcnn_resnet101_coco和mask_rcnn_inception_v2_coco
    注：做物體檢測的網(wǎng)絡(luò)有很多種，如faster rcnn，ssd，yolo等等，通過不同維度的對(duì)比，各個(gè)網(wǎng)絡(luò)都有各自的優(yōu)勢。
    畢竟樹莓派計(jì)算能力有限，我們這里先選擇專門為速度優(yōu)化過最快的網(wǎng)絡(luò)SSD，以及經(jīng)典的faster-rcnn作對(duì)比，再加上能顯示mask的高端網(wǎng)絡(luò)，，，
    事實(shí)上yolo v3剛出來，比SSD更快，而faster rcnn相對(duì)來說運(yùn)行慢的多了，后面可以都嘗試對(duì)比一下，目前先把基線系統(tǒng)搭建好。

Protobuf 安裝與配置

  • 說明
    protobuf是Google開發(fā)的一種混合語言數(shù)據(jù)標(biāo)準(zhǔn)，提供了一種輕便高效的結(jié)構(gòu)化數(shù)據(jù)存儲(chǔ)格式，可以用于結(jié)構(gòu)化數(shù)據(jù)序列化。很適合做數(shù)據(jù)存儲(chǔ)或 RPC 數(shù)據(jù)交換格式?？捎糜谕ㄓ崊f(xié)議、數(shù)據(jù)存儲(chǔ)等領(lǐng)域的語言無關(guān)、平臺(tái)無關(guān)、可擴(kuò)展的序列化結(jié)構(gòu)數(shù)據(jù)格式。目前提供了 C++、Java、Python 三種語言的 API。
    下載地址：https://github.com/google/protobuf/releases
    我們這里下載最新版本 protobuf-all-3.5.1.tar.gz
  • 安裝
tar -xf  protobuf-all-3.5.1.tar.gz  
cd protobuf-3.5.1  
./configure   
make   
make check   ->這一步是檢查編譯是否正確，耗時(shí)非常長，可略過
sudo make install  
sudo ldconfig  ->更新庫搜索路徑，否則可能找不到庫文件

如果運(yùn)行了make check，結(jié)果如下，可以看到所有的測試用例都PASS了，說明編譯正確：

============================================================================
Testsuite summary for Protocol Buffers 3.5.1
============================================================================
# TOTAL: 7
# PASS:  7
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
  • 配置
    配置的目的是將proto格式的數(shù)據(jù)轉(zhuǎn)換為python格式，從而可以在python腳本中調(diào)用，進(jìn)入目錄models-master/research，運(yùn)行：
protoc object_detection/protos/*.proto --python_out=.

轉(zhuǎn)換完畢后可以看到在object_detection/protos/目錄下多了許多*.py文件。

代碼

這里的代碼很簡單，因?yàn)榛緦?shí)現(xiàn)都已經(jīng)有了，我們只是調(diào)用一下接口實(shí)現(xiàn)功能即可。

import numpy as np
import os
import sys
import tarfile
import tensorflow as tf
import cv2
import time

from collections import defaultdict

# This is needed since the notebook is stored in the object_detection folder.
sys.path.append("../..")

from object_detection.utils import label_map_util
from object_detection.utils import visualization_utils as vis_util

# What model to download.
MODEL_NAME = 'ssd_mobilenet_v1_coco_2017_11_17'
#MODEL_NAME = 'faster_rcnn_resnet101_coco_11_06_2017'
#MODEL_NAME = 'ssd_inception_v2_coco_11_06_2017'
MODEL_FILE = MODEL_NAME + '.tar.gz'

# Path to frozen detection graph. This is the actual model that is used for the object detection.
PATH_TO_CKPT = MODEL_NAME + '/frozen_inference_graph.pb'

# List of the strings that is used to add correct label for each box.
PATH_TO_LABELS = os.path.join('/home/yinan/object_detect/models-master/research/object_detection/data', 'mscoco_label_map.pbtxt')

#extract the ssd_mobilenet
start = time.clock()
NUM_CLASSES = 90
#opener = urllib.request.URLopener()
#opener.retrieve(DOWNLOAD_BASE + MODEL_FILE, MODEL_FILE)
tar_file = tarfile.open(MODEL_FILE)
for file in tar_file.getmembers():
   file_name = os.path.basename(file.name)
   if 'frozen_inference_graph.pb' in file_name:
      tar_file.extract(file, os.getcwd())
end= time.clock()
print('load the model',(end-start))
detection_graph = tf.Graph()
with detection_graph.as_default():
  od_graph_def = tf.GraphDef()
  with tf.gfile.GFile(PATH_TO_CKPT, 'rb') as fid:
    serialized_graph = fid.read()
    od_graph_def.ParseFromString(serialized_graph)
    tf.import_graph_def(od_graph_def, name='')

label_map = label_map_util.load_labelmap(PATH_TO_LABELS)

categories = label_map_util.convert_label_map_to_categories(label_map, max_num_classes=NUM_CLASSES, use_display_name=True)
category_index = label_map_util.create_category_index(categories)

cap = cv2.VideoCapture(0)
with detection_graph.as_default():
  with tf.Session(graph=detection_graph) as sess:
      writer = tf.summary.FileWriter("logs/", sess.graph)
      sess.run(tf.global_variables_initializer())
      while(1):
        start = time.clock()
        ret, frame = cap.read()
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
        image_np=frame
        # the array based representation of the image will be used later in order to prepare the
        # result image with boxes and labels on it.
        # Expand dimensions since the model expects images to have shape: [1, None, None, 3]
        image_np_expanded = np.expand_dims(image_np, axis=0)
        image_tensor = detection_graph.get_tensor_by_name('image_tensor:0')
        # Each box represents a part of the image where a particular object was detected.
        boxes = detection_graph.get_tensor_by_name('detection_boxes:0')
        # Each score represent how level of confidence for each of the objects.
        # Score is shown on the result image, together with the class label.
        scores = detection_graph.get_tensor_by_name('detection_scores:0')
        classes = detection_graph.get_tensor_by_name('detection_classes:0')
        num_detections = detection_graph.get_tensor_by_name('num_detections:0')
        # Actual detection.

        (boxes, scores, classes, num_detections) = sess.run(
          [boxes, scores, classes, num_detections],
          feed_dict={image_tensor: image_np_expanded})
        # Visualization of the results of a detection.
        vis_util.visualize_boxes_and_labels_on_image_array(
          image_np,
          np.squeeze(boxes),
          np.squeeze(classes).astype(np.int32),
          np.squeeze(scores),
          category_index,
          use_normalized_coordinates=True,
          line_thickness=6)
        end = time.clock()
        #print('frame:',1.0/(end - start))
        print 'One frame detect take time:',end - start

        cv2.imshow("capture", image_np)
        print('after cv2 show')
        cv2.waitKey(1)
cap.release()
cv2.destroyAllWindows()

保存為 detect.py，到目錄models-master/research/object_detection/models下。

運(yùn)行

命令：

sudo chmod 666 /dev/video0
python detect.py

效果

SSD模型

下圖可以看到，SSD模型加載模型花了8s，差不多一張圖識(shí)別時(shí)間在5s：


image.png

PS. 為什么把房間識(shí)別成了book...

faster-RCNN模型

faster-RCNN，加載模型83s，內(nèi)存不夠，跑不起來。。。


image.png

mask SSD模型

mask模型可以描繪出輪廓，看起來更高端，加載模型25s，遇到個(gè)問題：


image.png

接下來查一下
CPU占用率100%，內(nèi)存占用60%多

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容