ARKit中提取到的CVPixelBuffer為YUV420格式，很多時候我們需要把它轉(zhuǎn)換為RGB格式，然后再進(jìn)行各種后續(xù)操作。這里我們利用Accelerate中提供一個函數(shù)來完成這種轉(zhuǎn)換：

Declaration

func vImageConvert_420Yp8_CbCr8ToARGB8888(_ srcYp: UnsafePointer<vImage_Buffer>, _ srcCbCr: UnsafePointer<vImage_Buffer>, _ dest: UnsafePointer<vImage_Buffer>, _ info: UnsafePointer<vImage_YpCbCrToARGB>, _ permuteMap: UnsafePointer<UInt8>!, _ alpha: UInt8, _ flags: vImage_Flags) -> vImage_Error

Parameters

srcYp
A pointer to the vImage buffer that references the source Yp plane.

srcCbCr
A pointer to the vImage buffer that references the source CbCr plane.

dest
A pointer to the vImage buffer that references 8-bit ARGB interleaved destination pixels.

info
A pointer to vImage_YpCbCrToARGB, which contains info coefficient and prebias values.

permuteMap
An array of four 8-bit integers with the values 0, 1, 2, and 3, in some order. Each value specifies a channel from the source image that should be copied to that channel in the destination image. 0 denotes the alpha channel, 1 the red channel, 2 the green channel, and 3 the blue channel. 

alpha
A value for the alpha channel in dest.

flags
The options to use when performing the operation. If you plan to perform your own tiling or use multithreading, pass kvImageDoNotTile.

Return Value

kvImageNoError; otherwise, one of the error codes described in Data Types and Constants.

完整代碼：

func pixelBufferToImage(pixelBuffer:CVPixelBuffer) -> UIImage {
    //01
    CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly)
    //02
    defer {
       CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly)
    }
        
    //03
    let lumaBaseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0)
    //04
    let lumaWidth = CVPixelBufferGetWidthOfPlane(pixelBuffer, 0)
    let lumaHeight = CVPixelBufferGetHeightOfPlane(pixelBuffer, 0)
    let lumaRowBytes = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0)
    var sourceLumaBuffer = vImage_Buffer(data: lumaBaseAddress, height: vImagePixelCount(lumaHeight), width: vImagePixelCount(lumaWidth), rowBytes: lumaRowBytes)
        
    let chromaBaseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 1)
    let chromaWidth = CVPixelBufferGetWidthOfPlane(pixelBuffer, 1)
    let chromaHeight = CVPixelBufferGetHeightOfPlane(pixelBuffer, 1)
    let chromaRowBytes = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 1)
    var sourceChromaBuffer = vImage_Buffer(data: chromaBaseAddress, height: vImagePixelCount(chromaHeight), width: vImagePixelCount(chromaWidth), rowBytes: chromaRowBytes)
        
    //05
    var rawRGBBuffer: UnsafeMutableRawPointer = malloc(lumaWidth * lumaHeight * 4)
    var rgbBuffer: vImage_Buffer = vImage_Buffer(data: rawRGBBuffer, height: vImagePixelCount(lumaHeight), width: vImagePixelCount(lumaWidth), rowBytes: lumaWidth * 4)
        
    //06
    guard var conversionInfoYpCbCrToARGB = _conversionInfoYpCbCrToARGB else {
        return UIImage()
    }
        
    //07
    guard vImageConvert_420Yp8_CbCr8ToARGB8888(&sourceLumaBuffer, &sourceChromaBuffer, &rgbBuffer, &conversionInfoYpCbCrToARGB, nil, 255, vImage_Flags(kvImageNoFlags)) == kvImageNoError else {
        return UIImage()
    }
      
    //08
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let ctx = CGContext(data: rgbBuffer.data, width: lumaWidth, height: lumaHeight, bitsPerComponent: 8, bytesPerRow: rgbBuffer.rowBytes, space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)

    let imageRef = ctx!.makeImage()!
    let uiimage = UIImage(cgImage: imageRef)
       
    //09 
    rawRGBBuffer.deallocate()
    return uiimage
}

private var _conversionInfoYpCbCrToARGB: vImage_YpCbCrToARGB? = {
    var pixelRange = vImage_YpCbCrPixelRange(Yp_bias: 16, CbCr_bias: 128, YpRangeMax: 235, CbCrRangeMax: 240, YpMax: 235, YpMin: 16, CbCrMax: 240, CbCrMin: 16)
    var infoYpCbCrToARGB = vImage_YpCbCrToARGB()
    guard vImageConvert_YpCbCrToARGB_GenerateConversion(kvImage_YpCbCrToARGBMatrix_ITU_R_601_4!, &pixelRange, &infoYpCbCrToARGB, kvImage422CbYpCrYp8, kvImageARGB8888, vImage_Flags(kvImageNoFlags)) == kvImageNoError else {
        return nil
    }
    return infoYpCbCrToARGB
}()

代碼解析

01、在訪問buffer內(nèi)部裸數(shù)據(jù)的地址時（讀或?qū)懚家粯樱枰葘⑵滏i上，用完了再放開。

CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly) 
CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly)

02、defer block 里的代碼會在函數(shù) return 之前執(zhí)行

defer {
       CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly)
}

03、ARKit中通過ARFrame獲取的CVPixelBuffer為YUV420格式，YUV420格式Y(jié)通道（Luminance）與UV通道（Chrominance）分開儲存數(shù)據(jù)，可通過下面語句獲取地址

let lumaBaseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0)
let chromaBaseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 1)

04、創(chuàng)建vImage_Buffer，參數(shù)中包含三個vImage buffer，前兩個參數(shù)需要基于Y通道與UV通道地址初始化：
srcYp：

let lumaWidth = CVPixelBufferGetWidthOfPlane(pixelBuffer, 0)
let lumaHeight = CVPixelBufferGetHeightOfPlane(pixelBuffer, 0)
let lumaRowBytes = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0)
var sourceLumaBuffer = vImage_Buffer(data: lumaBaseAddress, height: vImagePixelCount(lumaHeight), width: vImagePixelCount(lumaWidth), rowBytes: lumaRowBytes)

srcCbCr：

let chromaWidth = CVPixelBufferGetWidthOfPlane(pixelBuffer, 1)
let chromaHeight = CVPixelBufferGetHeightOfPlane(pixelBuffer, 1)
let chromaRowBytes = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 1)
var sourceChromaBuffer = vImage_Buffer(data: chromaBaseAddress, height: vImagePixelCount(chromaHeight), width: vImagePixelCount(chromaWidth), rowBytes: chromaRowBytes)

05、參數(shù)中第三個vImage buffer需要初始化一個新的buffer，dest：

var rawRGBBuffer: UnsafeMutableRawPointer = malloc(lumaWidth * lumaHeight * 4)
var rgbBuffer: vImage_Buffer = vImage_Buffer(data: rawRGBBuffer, height: vImagePixelCount(lumaHeight), width: vImagePixelCount(lumaWidth), rowBytes: lumaWidth * 4)

06、第四個參數(shù)info，

guard var conversionInfoYpCbCrToARGB = _conversionInfoYpCbCrToARGB else {
      return UIImage()
}

private var _conversionInfoYpCbCrToARGB: vImage_YpCbCrToARGB? = {
    var pixelRange = vImage_YpCbCrPixelRange(Yp_bias: 16, CbCr_bias: 128, YpRangeMax: 235, CbCrRangeMax: 240, YpMax: 235, YpMin: 16, CbCrMax: 240, CbCrMin: 16)
    var infoYpCbCrToARGB = vImage_YpCbCrToARGB()
    guard vImageConvert_YpCbCrToARGB_GenerateConversion(kvImage_YpCbCrToARGBMatrix_ITU_R_601_4!, &pixelRange, &infoYpCbCrToARGB, kvImage422CbYpCrYp8, kvImageARGB8888, vImage_Flags(kvImageNoFlags)) == kvImageNoError else {
        return nil
    }
    return infoYpCbCrToARGB
}()

07、利用函數(shù)進(jìn)行轉(zhuǎn)換

guard vImageConvert_420Yp8_CbCr8ToARGB8888(&sourceLumaBuffer, &sourceChromaBuffer, &rgbBuffer, &conversionInfoYpCbCrToARGB, nil, 255, vImage_Flags(kvImageNoFlags)) == kvImageNoError else {
      return UIImage()
}

至此已完成RGB格式的轉(zhuǎn)換

08、這里我們把RGB格式的buffer轉(zhuǎn)換為UIImage，注意bitmapInfo的值

let colorSpace = CGColorSpaceCreateDeviceRGB()
let ctx = CGContext(data: rgbBuffer.data, width: lumaWidth, height: lumaHeight, bitsPerComponent: 8, bytesPerRow: rgbBuffer.rowBytes, space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue)

let imageRef = ctx!.makeImage()!
let uiimage = UIImage(cgImage: imageRef)

09、最后別忘了釋放rawRGBBuffer。

rawRGBBuffer.deallocate()

耗時測試

ARKit可以在ARWorldTrackingConfiguration中設(shè)定攝像頭采集分辨率

worldTrackingSessionConfiguration.videoFormat = ARWorldTrackingConfiguration.supportedVideoFormats[0]

對于iphoneX有3種分辨率可供選擇

ARWorldTrackingConfiguration.supportedVideoFormats[0]   1920*1440
ARWorldTrackingConfiguration.supportedVideoFormats[1]   1920*1048
ARWorldTrackingConfiguration.supportedVideoFormats[2]   1024*768

耗時統(tǒng)計

絕大多數(shù)時候我們只需要完成01-07步即可，不需要執(zhí)行08步。

其他

當(dāng)我們使用Vision框架中的Machine-Learning Image Analysis時，CVPixelBuffer格式會自動轉(zhuǎn)換為RGB格式。

如果我們需要的是灰度圖數(shù)據(jù)，直接使用Y通道數(shù)據(jù)即可。