人脸跟踪（C/C++）
===========

#### 约束与限制

从6.1.0(23)开始，人脸识别与跟踪能力支持部分Phone、部分Tablet设备、TV设备。请参考[硬件要求](https://developer.huawei.com/consumer/cn/doc/harmonyos-guides/arengine-preparations#硬件要求)判断设备是否支持人脸识别与跟踪特性（[ARENGINE\_FEATURE\_TYPE\_FACE](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#arengine_featuretype)）。

#### 接口说明

以下接口为AR Engine人脸跟踪相关接口，详细接口和说明，请参考[AR Engine API参考](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine)。

| 接口名 | 描述 |
| :-- | :-- |
| [HMS\_AREngine\_ARConfig\_GetCameraLensFacing](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arconfig_getcameralensfacing) | 获取相机镜头朝向。 |
| [HMS\_AREngine\_ARConfig\_GetMultiFaceMode](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arconfig_getmultifacemode) | 获取多人脸检测模式。 |
| [HMS\_AREngine\_ARConfig\_SetCameraLensFacing](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arconfig_setcameralensfacing) | 设置相机镜头朝向。 |
| [HMS\_AREngine\_ARConfig\_SetMultiFaceMode](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arconfig_setmultifacemode) | 设置多人脸检测模式。 |
| [HMS\_AREngine\_ARFace\_AcquireBlendShapes](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_acquireblendshapes) | 获取人脸表情信息。 |
| [HMS\_AREngine\_ARFace\_AcquireGeometry](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_acquiregeometry) | 获取人脸几何信息。 |
| [HMS\_AREngine\_ARFace\_AcquireViewMatrix](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_acquireviewmatrix) | 获取当前人脸的面视图矩阵。 |
| [HMS\_AREngine\_ARFace\_GetCenterPose](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_getcenterpose) | 获取从人脸中心点位姿信息。 |
| [HMS\_AREngine\_ARFaceBlendShapes\_AcquireData](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfaceblendshapes_acquiredata) | 获取微表情数据的集合。 |
| [HMS\_AREngine\_ARFaceBlendShapes\_AcquireTypes](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfaceblendshapes_acquiretypes) | 获取所有微表情参数类型数组。 |
| [HMS\_AREngine\_ARFaceBlendShapes\_GetCount](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfaceblendshapes_getcount) | 获取人脸微表情数据的个数。 |
| [HMS\_AREngine\_ARFaceBlendShapes\_Release](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfaceblendshapes_release) | 释放当前人脸的blendShapes对象，即由[HMS\_AREngine\_ARFace\_AcquireBlendShapes](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_acquireblendshapes)创建的对象。 |
| [HMS\_AREngine\_ARFaceGeometry\_AcquireIndices](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_acquireindices) | 获取人脸Mesh中的三角形索引集合。 |
| [HMS\_AREngine\_ARFaceGeometry\_AcquireTexCoord](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_acquiretexcoord) | 获取人脸Mesh中的纹理坐标集。 |
| [HMS\_AREngine\_ARFaceGeometry\_AcquireTriangleLabels](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_acquiretrianglelabels) | 获取人脸Mesh中的三角形标签集合。 |
| [HMS\_AREngine\_ARFaceGeometry\_AcquireVertices](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_acquirevertices) | 获取人脸Mesh中的顶点集合。 |
| [HMS\_AREngine\_ARFaceGeometry\_GetIndicesSize](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_getindicessize) | 获取人脸Mesh中三角形索引的大小。 |
| [HMS\_AREngine\_ARFaceGeometry\_GetTexCoordSize](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_gettexcoordsize) | 获取人脸Mesh中纹理坐标的大小。 |
| [HMS\_AREngine\_ARFaceGeometry\_GetTriangleCount](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_gettrianglecount) | 获取人脸Mesh中三角形的大小。 |
| [HMS\_AREngine\_ARFaceGeometry\_GetTriangleLabelsSize](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_gettrianglelabelssize) | 获取人脸Mesh中三角形标签的大小。 |
| [HMS\_AREngine\_ARFaceGeometry\_GetVerticesSize](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_getverticessize) | 获取人脸Mesh中顶点的大小。 |
| [HMS\_AREngine\_ARFaceGeometry\_Release](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arfacegeometry_release) | 释放当前人脸Mesh对象，即由[HMS\_AREngine\_ARFace\_AcquireBlendShapes](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_acquireblendshapes)创建的对象。 |

#### 开发步骤

#### \[h2\]创建UI界面

创建一个UI界面，使用XComponent组件用于显示相机预览画面，并定时触发每一帧绘制。

// 此代码可参考示例代码：ARSample/entry/src/main/ets/pages/ARFace.ets
import { display } from '@kit.ArkUI';
import { resourceManager } from '@kit.LocalizationKit';
import { systemDateTime } from '@kit.BasicServicesKit';
import arEngineDemo from 'libentry.so';

@Builder
export function ARFaceBuilder() {
  ARFace();
}

@Component
struct ARFace {
  pageInfos: NavPathStack = new NavPathStack();
  @State context: Context = this.getUIContext().getHostContext() as Context;
  private xComponentId = 'ARFace';
  private resMgr: resourceManager.ResourceManager = this.context.resourceManager;
  private interval: number = -1;
  @State rotation: number = display.getDefaultDisplaySync().rotation;

  build() {
    NavDestination() {
      RelativeContainer() {
        XComponent({ id: this.xComponentId, type: XComponentType.SURFACE, libraryname: 'entry' })
          .width('100%')
          .height('100%')
          .alignRules({
            center: { anchor: '\_\_container\_\_', align: VerticalAlign.Center },
            middle: { anchor: '\_\_container\_\_', align: HorizontalAlign.Center }
          })
          .onLoad(() => {
            this.interval = setInterval(() => {
              // 调用更新Native API来更新AR Engine每帧的计算结果
              arEngineDemo.update(this.xComponentId);
            }, 33) // // 将帧速率设置为30fps（每33ms刷新一次帧）
          })
          .onDestroy(() => {
            clearInterval(this.interval);
          })

      }
    }
    .onAppear(() => {
      arEngineDemo.init(this.resMgr);
      let config: Int32Array = new Int32Array(\[1, this.rotation\]);
      arEngineDemo.start(this.xComponentId, config);
    })
    .onWillDisappear(() => {
      arEngineDemo.stop(this.xComponentId);
    })
    .onShown(() => {
      arEngineDemo.show(this.xComponentId);
    })
    .onHidden(() => {
      arEngineDemo.hide(this.xComponentId);
    })
    .onReady((context: NavDestinationContext) => {
      this.pageInfos = context.pathStack;
    })
    .hideTitleBar(true)
    .hideBackButton(true)
    .hideToolBar(true)
  }
}

#### \[h2\]引入AR Engine

开发者可参考AR物体摆放章节的[引入AR Engine](https://developer.huawei.com/consumer/cn/doc/harmonyos-guides/arengine-c-arworld#引入ar-engine)。

#### \[h2\]创建AR会话并配置为开启人脸跟踪模式

使用人脸识别与跟踪能力时请使用[HMS\_AREngine\_ARSession\_Create\_Human\_Perception](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arsession_create_human_perception)创建AR会话。

AREngine\_ARSession \*arSession = nullptr;
// 创建AR会话。
HMS\_AREngine\_ARSession\_Create\_Human\_Perception(nullptr, nullptr, &arSession);
AREngine\_ARConfig \*arConfig = nullptr;
// 创建AR会话配置器。
HMS\_AREngine\_ARConfig\_Create(arSession, &arConfig);
// 设置ARType为FACE开启人脸跟踪模式。
HMS\_AREngine\_ARConfig\_SetARType(arSession, arConfig, ARENGINE\_TYPE\_FACE);
// （可选）设置为前置相机
HMS\_AREngine\_ARConfig\_SetCameraLensFacing(arSession, arConfig, ARENGINE\_CAMERA\_FACING\_FRONT);
// （可选）设置为多人脸模式
HMS\_AREngine\_ARConfig\_SetMultiFaceMode(arSession, arConfig, ARENGINE\_MULTIFACE\_ENABLE);
// 配置器设置给AR会话。
HMS\_AREngine\_ARSession\_Configure(arSession, arConfig);

#### \[h2\]获取当前环境中的人脸信息

1.  创建一个可追踪对象列表faceList，用于存放人脸跟踪模式下AR Engine运行过程中检测到的所有人脸。
    
    AREngine\_ARTrackableList \*faceList = nullptr;
    HMS\_AREngine\_ARTrackableList\_Create(arSession, &faceList);
    // 调用HMS\_AREngine\_ARSession\_GetAllTrackables函数，检测当前环境中的所有人脸，并将结果存放在faceList中。
    AREngine\_ARTrackableType faceTrackedType = ARENGINE\_TRACKABLE\_FACE;
    HMS\_AREngine\_ARSession\_GetAllTrackables(arSession, faceTrackedType, faceList);
    
2.  调用[HMS\_AREngine\_ARTrackableList\_GetSize](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_artrackablelist_getsize)函数获取可追踪对象数量，结果存放在faceListSize中。
    
    int32\_t faceListSize = 0;
    HMS\_AREngine\_ARTrackableList\_GetSize(arSession, faceList, &faceListSize);
    // 未设置多人脸模式时，最多同时跟踪1个人脸信息，设置后最多同时跟踪3个人脸信息
    
3.  转化为人脸信息对象[AREngine\_ARFace](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#arengine_arface)。
    
    for (int i = 0; i < faceListSize; ++i) {
        // 遍历所有人脸信息对象，根据您的应用进行处理。
        AREngine\_ARTrackable \*arTrackable = nullptr;
        HMS\_AREngine\_ARTrackableList\_AcquireItem(arSession, faceList, i, &arTrackable);
        AREngine\_ARFace \*arFace = reinterpret\_cast<AREngine\_ARFace\*>(arTrackable);
    }
    

#### 获取当前人脸的位姿信息

1.  先通过[HMS\_AREngine\_ARPose\_Create](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arpose_create)接口创建一个[ARPose](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#arengine_arpose)对象，然后调用[HMS\_AREngine\_ARFace\_GetCenterPose](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_getcenterpose)，获取当前人脸的位姿信息。
    
    AREngine\_ARPose \*facePose = nullptr;
    HMS\_AREngine\_ARPose\_Create(arSession, nullptr, 0, &facePose);
    HMS\_AREngine\_ARFace\_GetCenterPose(arSession, arFace, facePose);
    
2.  获取当前人脸的视图矩阵。
    
    调用[HMS\_AREngine\_ARFace\_AcquireViewMatrix](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_acquireviewmatrix)函数，获取当前人脸的视图矩阵，该矩阵用于后续生成MVP矩阵实现渲染。
    
    float \*viewMatrix = new float\[16\];
    int size = 16;
    auto result = HMS\_AREngine\_ARFace\_AcquireViewMatrix(arSession, arFace, viewMatrix, size);
    
3.  获取当前人脸的几何信息。
    
    调用[HMS\_AREngine\_ARFace\_AcquireGeometry](https://developer.huawei.com/consumer/cn/doc/harmonyos-references/arengine-capi-arengine#hms_arengine_arface_acquiregeometry)，获取当前人脸的几何信息，并将结果存放在arFaceGeometry中。
    
    AREngine\_ARFaceGeometry\* arFaceGeometry = nullptr;
    HMS\_AREngine\_ARFace\_AcquireGeometry(arSession, arFace, &arFaceGeometry);
    
4.  获取人脸的几何信息中的三角形顶点。
    
    // 调用HMS\_AREngine\_ARFaceGeometry\_GetTriangleCount函数，获取人脸几何信息中的三角形数量
    int triangleCount = 0;
    HMS\_AREngine\_ARFaceGeometry\_GetTriangleCount(arSession, arFaceGeometry, &triangleCount);
    // 调用HMS\_AREngine\_ARFaceGeometry\_GetVerticesSize函数，获取人脸几何信息中的三角形顶点数量
    int verticesSize = 0;
    HMS\_AREngine\_ARFaceGeometry\_GetVerticesSize(arSession, arFaceGeometry, &verticesSize);
    // 调用HMS\_AREngine\_ARFaceGeometry\_AcquireVertices函数，获取人脸几何信息中的三角形顶点集合
    const float \*meshVertices = nullptr;
    HMS\_AREngine\_ARFaceGeometry\_AcquireVertices(arSession, arFaceGeometry, &meshVertices);
    
5.  获取人脸的几何信息中的三角形面片。
    
    // 调用HMS\_AREngine\_ARFaceGeometry\_GetIndicesSize函数，获取三角形面片对应顶点的索引个数，每三个顶点索引表示一个三角形面片
    int indicesSize = 0;
    HMS\_AREngine\_ARFaceGeometry\_GetIndicesSize(arSession, arFaceGeometry, &indicesSize);
    // 调用HMS\_AREngine\_ARFaceGeometry\_AcquireIndices函数，获取三角形面片对应顶点的索引列表
    int32\_t \*meshTriangleIndices = nullptr;
    HMS\_AREngine\_ARFaceGeometry\_AcquireIndices(arSession, arFaceGeometry, &meshTriangleIndices);
    
6.  获取人脸的几何信息中的三角形面片的语义标签。
    
    // 调用HMS\_AREngine\_ARFaceGeometry\_GetTriangleLabelsSize函数，获取三角形面片语义标签数量
    int triangleLabelsSize = 0;
    HMS\_AREngine\_ARFaceGeometry\_GetTriangleLabelsSize(arSession, arFaceGeometry, &triangleLabelsSize);
    // 调用HMS\_AREngine\_ARFaceGeometry\_AcquireTriangleLabels函数，获取三角形面片语义标签集合
    const AREngine\_ARAnimojiTriangleLabel\* triangleLabels = nullptr;
    HMS\_AREngine\_ARFaceGeometry\_AcquireTriangleLabels(arSession, arFaceGeometry, &triangleLabels);
    
7.  获取人脸几何信息中的UV纹理坐标。
    
    // 调用HMS\_AREngine\_ARFaceGeometry\_GetTexCoordSize函数，获取UV纹理坐标数量
    int texCoordSize = 0;
    HMS\_AREngine\_ARFaceGeometry\_GetTexCoordSize(arSession, arFaceGeometry, &texCoordSize);
    // 调用HMS\_AREngine\_ARFaceGeometry\_AcquireTexCoord函数，获取UV纹理坐标集合
    const float\* texCoords = nullptr;
    HMS\_AREngine\_ARFaceGeometry\_AcquireTexCoord(arSession, arFaceGeometry, &texCoords);
    
8.  获取当前人脸的微表情信息。
    
    // 调用HMS\_AREngine\_ARFace\_AcquireBlendShapes，获取当前人脸的微表情信息，并将结果存放在arFaceBlendShapes中。
    AREngine\_ARFaceBlendShapes\* arFaceBlendShapes = nullptr;
    HMS\_AREngine\_ARFace\_AcquireBlendShapes(arSession, arFace, &arFaceBlendShapes);
    // 调用HMS\_AREngine\_ARFaceBlendShapes\_GetCount，获取当前人脸的微表情的数量
    int count = 0;
    HMS\_AREngine\_ARFaceBlendShapes\_GetCount(arSession, arFaceBlendShapes, &count);
    // 调用HMS\_AREngine\_ARFaceBlendShapes\_AcquireTypes，获取当前人脸的微表情的标签集合
    const AREngine\_ARAnimojiBlendShape\* blendShapesTypes = nullptr;
    HMS\_AREngine\_ARFaceBlendShapes\_AcquireTypes(arSession, arFaceBlendShapes, &blendShapesTypes);
    // 调用HMS\_AREngine\_ARFaceBlendShapes\_AcquireData，获取当前人脸的微表情的数据集合，集合中的元素表示该位置在标签集合中表示的微表情的变化程度
    const float \*blendShapesData = nullptr;
    HMS\_AREngine\_ARFaceBlendShapes\_AcquireData(arSession, arFaceBlendShapes, &blendShapesData);