1. 應(yīng)用背景

之前使用adb的指令
adb shell uiautomator dump --compressed /data/local/tmp/uidump.xml
來(lái)獲取布局文件，然后識(shí)別控件的坐標(biāo)位置，但發(fā)現(xiàn)會(huì)報(bào)
ERROR: could not get idle stateorcould not get idle state的錯(cuò)誤，效率很低。因此后來(lái)采用了先截屏，然后通過(guò)圖片匹配識(shí)別控件位置，返回控件的坐標(biāo)，即是本文要介紹的內(nèi)容，由于開(kāi)發(fā)用java，順其自然的使用了javaCV，但目前這方面的資料較少。

2. 先來(lái)看具體效果

截屏得到的原圖

需要識(shí)別點(diǎn)贊按鈕圖標(biāo)

匹配結(jié)果效果圖

重要：一定要保證原圖與目標(biāo)圖的分辨率一致，不能壓縮，簡(jiǎn)單的辦法是使用電腦自帶的畫(huà)圖工具來(lái)?yè)溉ツ繕?biāo)圖。

3. 引入maven

<dependency>
      <groupId>org.bytedeco</groupId>
      <artifactId>javacv-platform</artifactId>
      <version>1.5.3</version>
</dependency>

用的是最新版，只引入這個(gè)包即可，但下載需要好久，后來(lái)更換為阿里云的倉(cāng)庫(kù)地址，快了很多，后期考慮精簡(jiǎn)依賴。

4. 具體實(shí)現(xiàn)

package com.hilbp.web.controller;

import static org.bytedeco.opencv.global.opencv_imgproc.cvtColor;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ThreadLocalRandom;

import org.bytedeco.javacpp.DoublePointer;
import org.bytedeco.javacpp.indexer.FloatIndexer;
import org.bytedeco.opencv.global.opencv_core;
import org.bytedeco.opencv.global.opencv_highgui;
import org.bytedeco.opencv.global.opencv_imgcodecs;
import org.bytedeco.opencv.global.opencv_imgproc;
import org.bytedeco.opencv.opencv_core.Mat;
import org.bytedeco.opencv.opencv_core.Point;
import org.bytedeco.opencv.opencv_core.Rect;
import org.bytedeco.opencv.opencv_core.Scalar;
import org.bytedeco.opencv.opencv_core.Size;

import lombok.extern.slf4j.Slf4j;

@Slf4j
public class ImageTest {
    
    
    
    public void test() {
        String[] args = new String[2];
        args[0] = "log/screen.png"; //截屏圖片
        args[1] = "log/1.png"; //點(diǎn)贊的圖標(biāo)
        newStyle(args);
    }
    
    public void newStyle(String[] args){
        
        //read in image default colors
        Mat sourceColor = opencv_imgcodecs.imread(args[0]);
        Mat sourceGrey = new Mat(sourceColor.size(), opencv_core.CV_8UC1);
       cvtColor(sourceColor, sourceGrey, opencv_imgproc.COLOR_BGR2GRAY);
       
       //load in template in grey 
       Mat template = opencv_imgcodecs.imread(args[1], opencv_imgcodecs.IMREAD_GRAYSCALE);//int = 0
       
       //Size for the result image
       Size size = new Size(sourceGrey.cols()-template.cols()+1, sourceGrey.rows()-template.rows()+1);
       Mat result = new Mat(size, opencv_core.CV_32FC1);
       opencv_imgproc.matchTemplate(sourceGrey, template, result, opencv_imgproc.TM_CCORR_NORMED);
//       opencv_imgproc.threshold(src, dst, thresh, maxval, ThresholdTypes.Tozero);
//       opencv_imgproc.floodFill(image, seedPoint, newVal)
       
       DoublePointer minVal= new DoublePointer();
       DoublePointer maxVal= new DoublePointer();
       Point min = new Point();
       Point max = new Point();
       opencv_core.minMaxLoc(result, minVal, maxVal, min, max, null);
//       log.info("[{}, {}]", max.x(), max.y());
//       opencv_imgproc.rectangle(sourceColor,new Rect(max.x(),max.y(),template.cols(),template.rows()), randColor(), 2, 0, 0);
       
       int centerWith = template.cols() / 2;
       int centerHeight = template.rows() / 2;
       getPointsFromMatAboveThreshold(result, 0.9999f).stream().forEach((point) -> {
           log.info("[{}, {}]", point.x(), point.y());
           log.info("[{}, {}]", point.x() + centerWith, point.y() + centerHeight);
           opencv_imgproc.rectangle(sourceColor, new Rect(point.x(), point.y(), template.cols(), template.rows()), randColor(), 2, 0, 0);
       });

//       List<Point> points = this.getPointsFromMatAboveThreshold(result, 0.99f);
//       for(Point point : points) {
//         opencv_imgproc.rectangle(sourceColor,new Rect(point.x(), point.y(), 30, 30), randColor(), 2, 0, 0);
//         
//       }
       opencv_highgui.imshow("Original marked", sourceColor);
//       imshow("Ttemplate", template);
//       imshow("Results matrix", result);
       
       opencv_imgcodecs.imwrite("log/res.png", sourceColor);
       
       opencv_highgui.waitKey(0);
       
       
       opencv_highgui.destroyAllWindows();
        
    }

    // some usefull things.
    public Scalar randColor(){
       int b,g,r;
       b= ThreadLocalRandom.current().nextInt(0, 255 + 1);
       g= ThreadLocalRandom.current().nextInt(0, 255 + 1);
       r= ThreadLocalRandom.current().nextInt(0, 255 + 1);
       return new Scalar (b,g,r,0);
    }
    
    public List<Point> getPointsFromMatAboveThreshold(Mat m, float t){
        
        List<Point> matches = new ArrayList<Point>();
        FloatIndexer indexer = m.createIndexer();
        for (int y = 0; y < m.rows(); y++) {
            for (int x = 0; x < m.cols(); x++) {
                if (indexer.get(y,x) > t) {
                    //System.out.println("(" + x + "," + y +") = "+ indexer.get(y,x));
                    matches.add(new Point(x, y));
       
                }
            }
        }
        return matches;
    }
    
}

代碼把最佳匹配的代碼的注釋了，很重要的一點(diǎn)是
getPointsFromMatAboveThreshold(result, 0.9999f)
中的0.9999f的閾值的設(shè)置，這個(gè)很重要，多試幾次。調(diào)低了的話結(jié)果可能不準(zhǔn)。
代碼的一下語(yǔ)句：打印目標(biāo)圖左上角的坐標(biāo)和計(jì)算后的目標(biāo)圖的中心點(diǎn)坐標(biāo)

log.info("[{}, {}]", point.x(), point.y());
log.info("[{}, {}]", point.x() + centerWith, point.y() + centerHeight);

5. 后記

本文重在功能邏輯的實(shí)現(xiàn)，關(guān)于javacv的學(xué)習(xí)，由于篇幅限制不予展開(kāi)。