A video image, with position-tracking information, captured as part of an AR session.

具有位置跟蹤信息的視頻圖像作為AR Session的一部分被捕獲。

Overview

A running AR session continuously captures video frames from the device camera. For each frame, ARKit analyzes the image together with data from the device's motion sensing hardware to estimate the device's real-world position. ARKit delivers this tracking information and imaging parameters in the form of an?ARFrame?object.

概述

正在運(yùn)行的AR會(huì)話不斷捕捉設(shè)備攝像頭的視頻幀。 對(duì)于每一幀，ARKit都會(huì)分析圖像以及來(lái)自設(shè)備運(yùn)動(dòng)傳感硬件的數(shù)據(jù)，以估算設(shè)備的實(shí)際位置。 ARKit以ARFrame對(duì)象的形式提供此跟蹤信息和成像參數(shù)。

? ? ? ? ? ? ?Accessing Captured Video Frames

capturedImage
A pixel buffer containing the image captured by the camera.
包含相機(jī)捕獲的圖像的像素緩沖區(qū)。

Discussion

ARKit captures pixel buffers in a planar YCbCr format (also known as YUV) format. To render these images on a device display, you'll need to access the luma and chroma planes of the pixel buffer and convert YCbCr values to an RGB format according to the ITU_R_601_4 standard.?

The following matrix (shown in Metal shader syntax) performs this conversion when multiplied by a 4-element vector (containing Y', Cb, Cr values and an "alpha" value of 1.0):

討論

ARKit以平面YCbCr格式（也稱(chēng)為YUV）格式捕獲像素緩沖區(qū)。 要在設(shè)備顯示器上顯示這些圖像，您需要訪問(wèn)像素緩沖區(qū)的亮度和色度平面，并根據(jù)ITU_R_601_4標(biāo)準(zhǔn)將YCbCr值轉(zhuǎn)換為RGB格式。

以下矩陣（以Metal著色器語(yǔ)法顯示）在乘以4元素向量（包含Y'，Cb，Cr值和“alpha”值為1.0）時(shí)執(zhí)行此轉(zhuǎn)換：

For more details, see?Displaying an AR Experience with Metal, or use the Metal variant of the AR app template when creating a new project in Xcode.

有關(guān)更多詳細(xì)信息，請(qǐng)參閱在metal中顯示AR體驗(yàn)，或者在Xcode中創(chuàng)建新項(xiàng)目時(shí)使用AR應(yīng)用程序模板的metal變體。

timestamp
The time at which the frame was captured.
捕獲幀的時(shí)間。

capturedDepthData
The depth map, if any, captured along with the video frame.
深度圖（如果有）隨視頻幀一起捕獲。

Discussion

Face-based AR (see?ARFaceTrackingConfiguration) uses the front-facing, depth-sensing camera on compatible devices. When running such a configuration, frames vended by the session contain a depth map captured by the depth camera in addition to the color pixel buffer (see?capturedImage) captured by the color camera. This property’s value is always?nil?when running other AR configurations.

The depth-sensing camera provides data at a different frame rate than the color camera, so this property’s value can also be?nil?if no depth data was captured at the same time as the current color image.

討論

基于人臉的AR（請(qǐng)參閱ARFaceTrackingConfiguration）使用兼容設(shè)備上的前置深度感應(yīng)相機(jī)。 運(yùn)行此配置時(shí)，Session 給出的幀除了包含由彩色攝像機(jī)捕獲的彩色像素緩沖區(qū)（參見(jiàn)capturedImage）之外，還包含深度相機(jī)捕獲的深度圖。 運(yùn)行其他AR配置時(shí)，此屬性的值始終為零。

深度感測(cè)攝像機(jī)以不同于彩色攝像機(jī)的幀率提供數(shù)據(jù)，因此如果沒(méi)有深度數(shù)據(jù)與當(dāng)前彩色圖像同時(shí)被捕獲，則該屬性的值也可以為零。

capturedDepthDataTimestamp
The time at which depth data for the frame (if any) was captured.
捕獲幀（如果有）的深度數(shù)據(jù)的時(shí)間。

Discussion

Face-based AR (see?ARFaceTrackingConfiguration) uses the front-facing, depth-sensing camera on compatible devices. When running such a configuration, frames vended by the session contain a depth map captured by the depth camera in addition to the color pixel buffer (see capturedImage) captured by the color camera. This property’s value is always zero when running other AR configurations.

The depth-sensing camera provides data at a different frame rate than the color camera, so this property’s value may not exactly match the?timestamp?property for the image captured by the color camera, and can also be zero if no depth data was captured at the same time as the current color image.

討論

基于人臉的AR（請(qǐng)參閱ARFaceTrackingConfiguration）使用兼容設(shè)備上的前置深度感應(yīng)相機(jī)。 運(yùn)行此配置時(shí)，Session 給出的幀除了包含由彩色攝像機(jī)捕獲的彩色像素緩沖區(qū)（參見(jiàn)capturedImage）之外，還包含深度相機(jī)捕獲的深度圖。 運(yùn)行其他AR配置時(shí)，此屬性的值始終為零。

深度感應(yīng)攝像機(jī)以與彩色攝像機(jī)不同的幀率提供數(shù)據(jù)，因此該屬性的值可能與彩色攝像機(jī)捕獲的圖像的時(shí)間戳屬性不完全匹配，并且如果沒(méi)有捕獲到深度數(shù)據(jù)，也可能為零 與當(dāng)前彩色圖像相同的時(shí)間。

? ? ? ? ? ? ? ? ? ? ? ? Examining Scene Parameters

camera
Information about the camera position, orientation, and imaging parameters used to capture the frame.
有關(guān)用于捕獲幀的相機(jī)位置，方向和成像參數(shù)的信息。

lightEstimate
An estimate of lighting conditions based on the camera image.
基于相機(jī)圖像估計(jì)照明條件。

Discussion

If you render your own overlay graphics for the AR scene, you can use this information in shading algorithms to help make those graphics match the real-world lighting conditions of the scene captured by the camera. (The?ARSCNView?class automatically uses this information to configure SceneKit lighting.)

This property's value is?nil?if the?lightEstimationEnabled?property of the session configuration that captured this frame is?NO.

討論

如果您為AR場(chǎng)景渲染自己的疊加圖形，則可以在著色算法中使用此信息，以幫助使這些圖形與相機(jī)捕捉的場(chǎng)景的真實(shí)照明條件相匹配。 （ARSCNView類(lèi)自動(dòng)使用此信息配置SceneKit照明。）

如果捕獲此幀的會(huì)話配置的lightEstimationEnabled屬性為NO，則此屬性的值為零。

-?displayTransformForOrientation:viewportSize:
Returns an affine transform for converting between normalized image coordinates and a coordinate space appropriate for rendering the camera image onscreen.
返回一個(gè)仿射變換，用于在歸一化圖像坐標(biāo)和適合于在屏幕上渲染相機(jī)圖像的坐標(biāo)空間之間進(jìn)行轉(zhuǎn)換。

Parameters

orientation

The orientation intended for presenting the view.?

旨在呈現(xiàn)視圖的方向。

viewportSize

The size, in points, of the view intended for rendering the camera image.

用于渲染相機(jī)圖像的視圖的大?。ㄒ渣c(diǎn)為單位）。

Return Value

A transform matrix that converts from normalized image coordinates in the captured image to normalized image coordinates that account for the specified parameters.

一個(gè)轉(zhuǎn)換矩陣，可將捕獲圖像中的歸一化圖像坐標(biāo)轉(zhuǎn)換為規(guī)范化圖像坐標(biāo)，從而解釋指定參數(shù)。

Discussion

Normalized image coordinates range from?(0,0)?in the upper left corner of the image to?(1,1)?in the lower right corner.

This method creates an affine transform representing the rotation and aspect-fit crop operations necessary to adapt the camera image to the specified orientation and to the aspect ratio of the specified viewport. The affine transform does not scale to the viewport's pixel size.

The?capturedImage?pixel buffer is the original image captured by the device camera, and thus not adjusted for device orientation or view aspect ratio.

討論

標(biāo)準(zhǔn)化的圖像坐標(biāo)范圍從圖像左上角的（0,0）到右下角的（1,1）。

此方法創(chuàng)建一個(gè)仿射變換，該仿射變換表示將攝像機(jī)圖像調(diào)整為指定方向和指定視口的高寬比所需的旋轉(zhuǎn)和寬高比擬的裁剪操作。 仿射變換不會(huì)縮放到視口的像素大小。

被捕獲的圖像像素緩存是由設(shè)備相機(jī)捕獲的原始圖像，因此未針對(duì)設(shè)備方向或視圖寬高比進(jìn)行調(diào)整。

? ? ? ? ? ? ? ? ? ?Tracking and Finding Objects

anchors
The list of anchors representing positions tracked or objects detected in the scene.
表示跟蹤的位置或在場(chǎng)景中檢測(cè)到的對(duì)象的錨點(diǎn)列表。

Discussion

You can manually add or remove anchors to track locations in the scene using the?ARSessionclass. Depending on session configuration, ARKit may also add anchors, such as the origin of the world coordinate system or automatically detected planes.

討論

您可以使用ARSession類(lèi)手動(dòng)添加或刪除錨點(diǎn)以跟蹤場(chǎng)景中的位置。 根據(jù)Session配置的不同，ARKit還可能會(huì)添加錨點(diǎn)，例如世界坐標(biāo)系的原點(diǎn)或自動(dòng)檢測(cè)到的平面。

-?hitTest:types:
Searches for real-world objects or AR anchors in the captured camera image.
搜索捕獲的相機(jī)圖像中的真實(shí)世界對(duì)象或AR定位點(diǎn)。

Parameters

point

A point in normalized image coordinate space. (The point?(0,0)?represents the top left corner of the image, and the point?(1,1)?represents the bottom right corner.)

歸一化圖像坐標(biāo)空間中的一個(gè)點(diǎn)。 （點(diǎn)（0,0）代表圖像的左上角，點(diǎn)（1,1）代表右下角）。

types

The types of hit-test result to search for.

搜索的結(jié)果的類(lèi)型。

Return Value

A list of results, sorted from nearest to farthest (in distance from the camera).

返回值

結(jié)果列表，從最近到最遠(yuǎn)（與相機(jī)的距離）排序。

Discussion

Hit testing searches for real-world objects or surfaces detected through the AR session's processing of the camera image. A 2D point in the image coordinates can refer to any point along a 3D line that starts at the device camera and extends in a direction determined by the device orientation and camera projection. This method searches along that line, returning all objects that intersect it in order of distance from the camera.

Note

If you use ARKit with a SceneKit or SpriteKit view, the?ARSCNView?hitTest:types:?or?ARSKView?hitTest:types:?method lets you specify a search point in view coordinates.

討論

通過(guò)AR Session處理相機(jī)圖像搜索的真實(shí)世界對(duì)象或表面。 圖像坐標(biāo)中的二維點(diǎn)可以指沿著三維線上的任何點(diǎn)，該三維線開(kāi)始于設(shè)備相機(jī)并且在由設(shè)備方向和相機(jī)投影確定的方向上延伸。 該方法沿著該線搜索，按距離相機(jī)的順序返回與其相交的所有對(duì)象。

注意

如果您使用帶有SceneKit或SpriteKit視圖的ARKit，則可以使用ARSCNView hitTest：types：或ARSKView hitTest：types：方法指定視圖坐標(biāo)中的搜索點(diǎn)。

? ? ? ? ? ? ? ? ? ? ? Debugging Scene Detection

rawFeaturePoints
The current intermediate results of the scene analysis ARKit uses to perform world tracking.
ARKit用于執(zhí)行世界跟蹤的場(chǎng)景分析的當(dāng)前中間結(jié)果。

Discussion

These points represent notable features detected in the camera image. Their positions in 3D world coordinate space are extrapolated as part of the image analysis that ARKit performs in order to accurately track the device's position, orientation, and movement. Taken together, these points loosely correlate to the contours of real-world objects in view of the camera.

ARKit does not guarantee that the number and arrangement of raw feature points will remain stable between software releases, or even between subsequent frames in the same session. Regardless, the point cloud can sometimes prove useful when debugging your app's placement of virtual objects into the real-world scene.

If you display AR content with SceneKit using the?ARSCNView?class, you can display this point cloud with the?ARSCNDebugOptionShowFeaturePoints?debug option.

Feature point detection requires a?ARWorldTrackingConfiguration?session.

討論

這些點(diǎn)表示在相機(jī)圖像中檢測(cè)到的顯著特征。 它們?cè)?D世界坐標(biāo)空間中的位置被推斷為ARKit為了精確跟蹤設(shè)備的位置，方向和移動(dòng)而執(zhí)行的圖像分析的一部分。 總之，考慮到相機(jī)，這些點(diǎn)與真實(shí)世界物體的輪廓松散相關(guān)。

ARKit不保證原始特征點(diǎn)的數(shù)量和排列在軟件版本之間，甚至在同一會(huì)話中的后續(xù)幀之間保持穩(wěn)定。 無(wú)論如何，?point cloud?在調(diào)試應(yīng)用程序?qū)⑻摂M對(duì)象放置到真實(shí)世界場(chǎng)景時(shí)有時(shí)會(huì)證明是有用的。

如果使用ARSCNView類(lèi)使用SceneKit顯示AR內(nèi)容，則可以使用ARSCNDebugOptionShowFeaturePoints調(diào)試選項(xiàng)顯示此?point cloud?。

特征點(diǎn)檢測(cè)需要ARWorldTrackingConfiguration會(huì)話。

ARPointCloud
A collection of points in the world coordinate space of the AR session.
AR Session世界坐標(biāo)空間中的一組點(diǎn)。

Overview

Use the?ARFrame?rawFeaturePoints?property to obtain a point cloud representing intermediate results of the scene analysis ARKit uses to perform world tracking.

概述

使用ARFrame rawFeaturePoints屬性獲取表示ARKit用于執(zhí)行世界跟蹤的場(chǎng)景分析的中間結(jié)果的point cloud。

Identifying Feature Points

count
The number of points in the point cloud.
point cloud的點(diǎn)數(shù)。

points
The list of detected points.
檢測(cè)點(diǎn)的列表。

identifiers
A list of unique identifiers corresponding to detected feature points.
與檢測(cè)到的特征點(diǎn)對(duì)應(yīng)的唯一標(biāo)識(shí)符列表。

Discussion

Each identifier in this list corresponds to the point vector at the same index in the?points?array.

討論

該列表中的每個(gè)標(biāo)識(shí)符對(duì)應(yīng)于points數(shù)組中相同索引處的點(diǎn)向量。

Inherits From?NSObject

Conforms To?NSSecureCoding

Inherits From?NSObject

Conforms To?NSCopying