Skip to content

# Game Engine Integration

The Game Engine Integration Sample demonstrates how to integrate the Beam Eye Tracker SDK into a game engine architecture. To avoid dealing with the many complexities of a real game-engine, this sample uses a dummy game engine architecture. However, the coding patterns are compatible with major game engines like Unity and Unreal Engine 5. It demonstrates:

- Wrapping the `API` as a custom game engine input device;

- Implementation of In-game camera control;

- Implementation of an Immersive HUD as in Dynamic HUDs;

- Holding game settings parameters and mapping them to input device parameters;

- Viewport geometry management;

- Camera recentering functionality, and;

- Auto-start capability for seamless user experience.

Note

Requirements: Windows 10 or higher, CMake, C++ compiler, Beam Eye Tracker application (v2.4 or higher).

Language: `C++`.

## Compiling and running the sample

As part of the C++ samples, you can find the `game_engine_integration` sample in the `cpp\samples` folder.

Prebuilt binaries are available for Windows in the `cpp\samples\bin` folder. But you can also compile the sample yourself. To do so, run the `build_samples.bat` script in the `cpp\samples` folder. The script assumes you have CMake installed and that CMake can find a suitable C++ compiler/generator on your system. The `CMakeLists.txt` integrates the Beam Eye Tracker SDK as explained in Project configuration.

The program runs for 30 seconds. The console displays:

- Real-time camera position and rotation (only z translation and yaw are shown for simplicity);

- Current HUD opacity values which change whether you are looking at the top-left corner of the viewport, and;

- Recentering status updates (when pressing SPACE).

Moreover, you can observe that the camera’s z translation continuously increases to simulate the `reference` camera pose changes through the 3D world scene, while the immersive camera pose changes are additive to this base pose.

Warning

The current viewport geometry is hardcoded in `get_rendering_area_viewport_geometry_from_engine` and thus you won’t experience realistic behavior out of the box when running the sample. It assumes the game is rendering on the middle screen of three 1920x1080 monitors, with the left-most screen being the Windows main display.

If you want realistic behavior, for example, that the opacitiy output changes when you indeed look at the top-left corner of YOUR display, you need to modify the `get_rendering_area_viewport_geometry_from_engine` function to return a rectangle geometry that matches your setup (e.g. a given screen or window that you want to pretend is game-rendering area), but do respect the Unity-like viewport definition (bottom-left corner at `(0, 0)`) which is assumed for this specific sample.

## Sample explained

The source code is distributed into three files:

### my_game_engine.h

This file implements dummy classes that emulate a typical game engine object-oriented pattern:

- Base interfaces with common functions like `BeginPlay`, `Tick`, `Update`

- Simplified game engine architecture similar to Unity/Unreal

- Meant to be replaced by your engine’s actual functionality

my_game_engine.h

     1#ifndef MY_GAME_ENGINE_H
     2#define MY_GAME_ENGINE_H
     3
     4#include <vector>
     5/**
     6 * Copyright (C) 2025 Eyeware Tech SA.
     7 *
     8 * All rights reserved.
     9 *
    10 * This file defines an extremely simplified game engine using OOP, similar to Unity and UE5
    11 * OOP paradigm. In real life we assume all of this is already defined in your engine.
    12 *
    13 * The good stuff is in files main.cpp and bet_game_engine_device.cpp.
    14 */
    15
    16struct MyGameEngineTransform {
    17    // For this sample's purpose, we assume Unity's camera coordinate system which is the same as
    18    // Beam, except that x is inverted, and the rotations are left-handed, not right-handed.
    19    float rotation_x_degrees;
    20    float rotation_y_degrees;
    21    float rotation_z_degrees;
    22    float translation_x_inches;
    23    float translation_y_inches;
    24    float translation_z_inches;
    25
    26    MyGameEngineTransform operator+(MyGameEngineTransform other) {
    27        return MyGameEngineTransform{this->rotation_x_degrees + other.rotation_x_degrees,
    28                                     this->rotation_y_degrees + other.rotation_y_degrees,
    29                                     this->rotation_z_degrees + other.rotation_z_degrees,
    30                                     this->translation_x_inches + other.translation_x_inches,
    31                                     this->translation_y_inches + other.translation_y_inches,
    32                                     this->translation_z_inches + other.translation_z_inches};
    33    }
    34};
    35
    36class MyGameEngineObjectInterface {
    37  public:
    38    MyGameEngineObjectInterface(MyGameEngineObjectInterface *parent) : parent(parent) {}
    39    // Called frequently and periodically. delta_time in seconds.
    40    virtual void Tick(float delta_time) {};
    41
    42    // Called when the rendering loop starts.
    43    virtual void BeginPlay() {};
    44
    45    // Called when the rendering loop stops.
    46    virtual void EndPlay() {};
    47
    48    void set_local_transform(MyGameEngineTransform local_transform) {
    49        this->local_transform = local_transform;
    50        if (this->parent) {
    51            this->world_transform = this->parent->world_transform + this->local_transform;
    52        } else {
    53            this->world_transform = this->local_transform;
    54        }
    55    }
    56
    57    MyGameEngineTransform local_transform{0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f};
    58    MyGameEngineTransform world_transform{0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f};
    59
    60    MyGameEngineObjectInterface *parent;
    61};
    62
    63class MyGameEngineHUDElement : public MyGameEngineObjectInterface {
    64  public:
    65    enum class Type {
    66        TOP_LEFT,
    67        TOP_RIGHT,
    68        BOTTOM_LEFT,
    69        BOTTOM_RIGHT,
    70    };
    71
    72    MyGameEngineHUDElement(MyGameEngineHUDElement::Type type, MyGameEngineObjectInterface *parent)
    73        : MyGameEngineObjectInterface(parent), type(type) {}
    74
    75    Type type;
    76    float opacity = 1.0f;
    77};
    78
    79#endif // MY_GAME_ENGINE_H

### bet_game_engine_device.h

Implements the `MyGameEngineBeamEyeTrackerDevice` which is the key class that you would typically implement for your specific game engine. You will see that it:

- manages the API instance;

- uses asynchronous data access using itself as the `eyeware::beam_eye_tracker::TrackingListener`;

- shows how to launch the Beam Eye Tracker at `BeginPlay`;

- computes HUD opacity values based on whether the user is looking at the HUD element or not, making said opacities part of the device output parameters;

- processes the camera pose parameters, mapping them to the game engine coordinates, and setting the mapped `MyGameEngineTransform` (yaw, pitch, roll, x, y, z) as a device output parameter;

- processes how game settings affect the device behavior and output.

Note

The sample uses asynchronous data access, but if you can’t guarantee that the `TrackerListener` will be alive until the handle is explicitly deregistered, you can use polling data access instead, which also does not block the main thread.

bet_game_engine_device.h

      1/**
      2 * Copyright (C) 2025 Eyeware Tech SA.
      3 *
      4 * All rights reserved.
      5 */
      6
      7#ifndef BET_GAME_ENGINE_DEVICE_H
      8#define BET_GAME_ENGINE_DEVICE_H
      9
     10#include "eyeware/beam_eye_tracker.h"
     11#include "my_game_engine.h"
     12#include <algorithm>
     13#include <memory>
     14
     15namespace {
     16
     17constexpr float M_PI = 3.14159265358979323846;
     18constexpr float M_RADIANS_TO_DEGREES = 180.0f / M_PI;
     19constexpr float M_METERS_TO_INCHES = 39.3700787;
     20
     21float update_hud_opacity(float prev_opacity, bool looking_at_hud, float delta_time) {
     22    // Implements asymmetric linear opacity update. You can use a nicer animation curve.
     23    const float opacity_rate_on_looking_at_hud = 10.0f;     // Fully visible in max 0.1 seconds
     24    const float opacity_rate_on_not_looking_at_hud = -1.0f; // Fully invisible in max 1 seconds
     25    const float min_opacity = 0.2f; // In case the HUD should not fully disappear
     26
     27    const float opacity_update_rate =
     28        looking_at_hud ? opacity_rate_on_looking_at_hud : opacity_rate_on_not_looking_at_hud;
     29    return std::min(std::max(prev_opacity + opacity_update_rate * delta_time, min_opacity), 1.0f);
     30}
     31
     32} // namespace
     33
     34// Note, for this sample, we keep explicit mentions to the namespace eyeware::beam_eye_tracker
     35// to make it easier to separate API related code.
     36class MyGameEngineBeamEyeTrackerDevice : public eyeware::beam_eye_tracker::TrackingListener,
     37                                         public MyGameEngineObjectInterface {
     38  public:
     39    MyGameEngineBeamEyeTrackerDevice(MyGameEngineObjectInterface *parent)
     40        : MyGameEngineObjectInterface(parent) {
     41        // We only need one instance of the API. You can also create it on "Begin Play" if you
     42        // want to, but here it is created in the constructor for simplicity and not to check
     43        // on pointer validity.
     44        m_bet_api = std::make_unique<eyeware::beam_eye_tracker::API>(
     45            "Game Engine Integration Sample", get_rendering_area_viewport_geometry_from_engine());
     46    }
     47
     48    ~MyGameEngineBeamEyeTrackerDevice() { stop_bet_api_tracking_data_reception(); }
     49
     50    void BeginPlay() override {
     51        if (m_auto_start_tracking) {
     52            // If auto start is toggled on, this will request the Beam app to launch and
     53            // or to start the webcam and initialize the tracking.
     54            // HEADS UP! Be wise when you call this. Ideally you want to call it when the game
     55            // rendering starts and accepts device input, as otherwise may start the webcam
     56            // at a random time and confuse the user.
     57            m_bet_api->attempt_starting_the_beam_eye_tracker();
     58        }
     59
     60        // Register itself as the listener to receive tracking data from the Beam Eye Tracker
     61        // application on the on_tracking_state_set_update method asynchronously.
     62        if (m_listener_handle == eyeware::beam_eye_tracker::INVALID_TRACKING_LISTENER_HANDLE) {
     63            m_listener_handle = m_bet_api->start_receiving_tracking_data_on_listener(this);
     64        }
     65    }
     66
     67    void EndPlay() override { stop_bet_api_tracking_data_reception(); }
     68
     69    void Tick(float delta_time) override {
     70        // For the purpose of this sample, we assume a custom device output is the HUD opacity,
     71        // which is updated here.
     72
     73        // Animate the opacity change depending on whether the user is looking at HUD elements.
     74        device_output_top_left_hud_opacity = update_hud_opacity(
     75            device_output_top_left_hud_opacity, m_is_user_looking_at_top_left_corner, delta_time);
     76        device_output_top_right_hud_opacity = update_hud_opacity(
     77            device_output_top_right_hud_opacity, m_is_user_looking_at_top_right_corner, delta_time);
     78        device_output_bottom_left_hud_opacity =
     79            update_hud_opacity(device_output_bottom_left_hud_opacity,
     80                               m_is_user_looking_at_bottom_left_corner, delta_time);
     81        device_output_bottom_right_hud_opacity =
     82            update_hud_opacity(device_output_bottom_right_hud_opacity,
     83                               m_is_user_looking_at_bottom_right_corner, delta_time);
     84
     85        // Update viewport every 3 seconds in case the rendering area geometry changed.
     86        // In the Beam Eye Tracker application API, this operation is light-weight
     87        // so you could call it more frequently, but 3 seconds balances the trade-off between
     88        // slight-overhead (inc. game engine retrieval of the geometry) and responsiveness in case
     89        // the game was resized or moved. If you have an specific event for game window geometry
     90        // changes, may be better suited for this purpose.
     91        m_time_since_last_viewport_geometry_update += delta_time;
     92        if (m_time_since_last_viewport_geometry_update >= 3.0f) {
     93            m_bet_api->update_viewport_geometry(get_rendering_area_viewport_geometry_from_engine());
     94            m_time_since_last_viewport_geometry_update = 0.0f;
     95        }
     96    }
     97
     98    // Functions for recentering the camera. Likely mapped to a hotkey press/release event.
     99    void recenter_camera_start() { m_bet_api->recenter_sim_game_camera_start(); }
    100    void recenter_camera_end() { m_bet_api->recenter_sim_game_camera_end(); }
    101
    102    eyeware::beam_eye_tracker::ViewportGeometry get_rendering_area_viewport_geometry_from_engine() {
    103        // Implement here your Game Engine specific logic where you retrieve the rendering area
    104        // geometry, i.e., the viewport. We need to keep the eyeware::beam_eye_tracker::API up to
    105        // date with changes in the viewport geometry.
    106
    107        // For this demo, assume this configuration: three physical monitors from left to right of
    108        // resolutions 1920x1080, 1920x1080, 1920x1080. The left-most monitor is configured in
    109        // Windows settings as the "Main display" (thus, it defines the (0, 0) coordinates in the
    110        // Windows Virtual Screen), and the game is rendering full screen in the center monitor.
    111        // Moreover, lets assume this game engine follows Unity's viewport standard, where the the
    112        // viewport (0, 0) coordinates are at the bottom-left corner of the rendering area.
    113        // Thus this coordinates would represent that configuration:
    114        eyeware::beam_eye_tracker::Point point_00 = {1920, 1079};
    115        eyeware::beam_eye_tracker::Point point_11 = {1920 + 1920 - 1, 0};
    116
    117        return {point_00, point_11};
    118    }
    119
    120    void on_tracking_data_reception_status_changed(
    121        eyeware::beam_eye_tracker::TrackingDataReceptionStatus status) override {
    122
    123        // See TrackingDataReceptionStatus for explanation on all possible statuses.
    124        if (status ==
    125            eyeware::beam_eye_tracker::TrackingDataReceptionStatus::NOT_RECEIVING_TRACKING_DATA) {
    126            // If the tracking data reception status is NOT_RECEIVING_TRACKING_DATA is because the
    127            // Beam app is not open, the webcam is not active, the client was rejected from using
    128            // the API, the user is not signed in, etc. etc.. To "fix" this, manual intervention
    129            // from the user is required. Note that this state could be reached shortly after a call
    130            // to attempt_starting_the_beam_eye_tracker, which failed to achieve the auto-start.
    131            // You can try calling attempt_starting_the_beam_eye_tracker again, but it is a question
    132            // of user experience, as the user may be manually toggling off, but the game insists on
    133            // toggling on.
    134
    135            // In this situation, makes sense to reset the device output to default values.
    136            // Animation curves could be used for a smoother transition.
    137            device_output_camera_local_transform = {0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f};
    138
    139            device_output_top_left_hud_opacity = 1.0f;
    140            device_output_top_right_hud_opacity = 1.0f;
    141            device_output_bottom_left_hud_opacity = 1.0f;
    142            device_output_bottom_right_hud_opacity = 1.0f;
    143
    144            m_is_user_looking_at_top_left_corner = true;
    145            m_is_user_looking_at_top_right_corner = true;
    146            m_is_user_looking_at_bottom_left_corner = true;
    147            m_is_user_looking_at_bottom_right_corner = true;
    148        }
    149    }
    150
    151    void on_tracking_state_set_update(
    152        const eyeware::beam_eye_tracker::TrackingStateSet &tracking_state_set,
    153        const eyeware::beam_eye_tracker::Timestamp timestamp) override {
    154        // Async callback to retrieve the tracking data.
    155
    156        update_device_viewport_gaze_state_from_bet_api_input(tracking_state_set.user_state());
    157
    158        update_device_sim_game_camera_state_from_bet_api_input(
    159            tracking_state_set.sim_game_camera_state());
    160
    161        update_device_game_immersive_hud_state_from_bet_api_input(
    162            tracking_state_set.game_immersive_hud_state());
    163    }
    164
    165    void update_device_viewport_gaze_state_from_bet_api_input(
    166        const eyeware::beam_eye_tracker::UserState &user_state) {
    167
    168        // Eye tracking coordinates referred to the viewport area.
    169        if (user_state.timestamp_in_seconds != eyeware::beam_eye_tracker::NULL_DATA_TIMESTAMP &&
    170            eyeware::beam_eye_tracker::cast_confidence(user_state.viewport_gaze.confidence) !=
    171                eyeware::beam_eye_tracker::TrackingConfidence::LOST_TRACKING) {
    172
    173            // Normalized gaze coordinates in the viewport. Normalized as it is in the range [0, 1],
    174            // however, values outside this range are possible.
    175            device_output_viewport_normalized_gaze_x =
    176                user_state.viewport_gaze.normalized_point_of_regard.x;
    177            device_output_viewport_normalized_gaze_y =
    178                user_state.viewport_gaze.normalized_point_of_regard.y;
    179        }
    180    }
    181
    182    void update_device_sim_game_camera_state_from_bet_api_input(
    183        const eyeware::beam_eye_tracker::SimGameCameraState &sim_game_camera_state) {
    184
    185        if (sim_game_camera_state.timestamp_in_seconds !=
    186            eyeware::beam_eye_tracker::NULL_DATA_TIMESTAMP) {
    187
    188            // Mapping sensitivity (default 0.5) to weight (default 1.0). Note this mapping
    189            // could be more complex, but the assumption is that a weight of 1.0 would make the
    190            // signal as configured by the user within the Beam Eye Tracker application.
    191            const float sim_game_camera_eye_tracking_weight =
    192                2.0 * m_sim_game_camera_eye_tracking_sensitivity;
    193            const float sim_game_camera_head_tracking_weight =
    194                2.0 * m_sim_game_camera_head_tracking_sensitivity;
    195
    196            // This combines the signals into one transform.
    197            eyeware::beam_eye_tracker::SimCameraTransform3D bet_camera_local_transform =
    198                eyeware::beam_eye_tracker::API::compute_sim_game_camera_transform_parameters(
    199                    sim_game_camera_state, sim_game_camera_eye_tracking_weight,
    200                    sim_game_camera_head_tracking_weight);
    201
    202            // Now, we need to map the beam eye tracker coordinates to the game engine
    203            // coordinates. See the documentation of the API for SimCameraTransform3D explaining
    204            // the API in detail. Assuming the game engine is using Unity's coordinate system,
    205            // which is the same as Beam, except that x is inverted, and the rotations are
    206            // left-handed, not right-handed. The rotation order for roll, pitch, yaw is
    207            // consistent with Beam's.
    208
    209            // Rotations going from right-handed to left-handed coordinate system.
    210            device_output_camera_local_transform.rotation_x_degrees =
    211                bet_camera_local_transform.pitch_in_radians * M_RADIANS_TO_DEGREES;
    212            device_output_camera_local_transform.rotation_y_degrees =
    213                -bet_camera_local_transform.yaw_in_radians * M_RADIANS_TO_DEGREES;
    214            device_output_camera_local_transform.rotation_z_degrees =
    215                -bet_camera_local_transform.roll_in_radians * M_RADIANS_TO_DEGREES;
    216
    217            // Translations going from right-handed to left-handed coordinate system.
    218            device_output_camera_local_transform.translation_x_inches =
    219                -bet_camera_local_transform.x_in_meters * M_METERS_TO_INCHES;
    220            device_output_camera_local_transform.translation_y_inches =
    221                bet_camera_local_transform.y_in_meters * M_METERS_TO_INCHES;
    222            device_output_camera_local_transform.translation_z_inches =
    223                bet_camera_local_transform.z_in_meters * M_METERS_TO_INCHES;
    224
    225        } else {
    226            // There could be multiple reasons to receive a NULL_DATA_TIMESTAMP in the callback.
    227            // But in general it means an interruption of the normal tracking, the feature
    228            // itself, or other.
    229
    230            // For user experience, the camera should NOT be reset to the default position
    231            // immediately (m_latest_sim_game_camera_transform being 0.0f), as that would be
    232            // confusing: imagine the user going briefly off-frame to connect a cable, but
    233            // suddenly the camera snaps to zero. Instead, we suggest to keep the
    234            // m_latest_sim_game_camera_transform as is with the latest valid data.
    235            //
    236            // However, you may also choose to set it to zeros after a reasonable time, and
    237            // perhaps even slowly. But that's your choice.
    238        }
    239    }
    240
    241    void update_device_game_immersive_hud_state_from_bet_api_input(
    242        const eyeware::beam_eye_tracker::GameImmersiveHUDState &game_immersive_hud_state) {
    243        if (game_immersive_hud_state.timestamp_in_seconds !=
    244            eyeware::beam_eye_tracker::NULL_DATA_TIMESTAMP) {
    245
    246            // Note: the input values are interpreted a "likelihood" or as a "probability", so
    247            // you can simply threshold it.
    248            m_is_user_looking_at_top_left_corner =
    249                game_immersive_hud_state.looking_at_viewport_top_left > 0.5f;
    250            m_is_user_looking_at_top_right_corner =
    251                game_immersive_hud_state.looking_at_viewport_top_right > 0.5f;
    252            m_is_user_looking_at_bottom_left_corner =
    253                game_immersive_hud_state.looking_at_viewport_bottom_left > 0.5f;
    254            m_is_user_looking_at_bottom_right_corner =
    255                game_immersive_hud_state.looking_at_viewport_bottom_right > 0.5f;
    256
    257        } else {
    258            // There could be multiple reasons to receive a NULL_DATA_TIMESTAMP in the callback.
    259            // But in general it means an interruption of the normal tracking, the feature
    260            // itself, or other.
    261
    262            // In this case, it makes sense to "reset" at set all the HUD as visible. For
    263            // example, assume the user is off-camera.
    264            m_is_user_looking_at_top_left_corner = true;
    265            m_is_user_looking_at_top_right_corner = true;
    266            m_is_user_looking_at_bottom_left_corner = true;
    267            m_is_user_looking_at_bottom_right_corner = true;
    268        }
    269    }
    270
    271    //*********  VARIABLES ASSUMED TO BE LINKED TO IN-GAME SETTINGS ****************** */
    272    // Note: we don't define the setters to avoid clutter.
    273    /**
    274     * @brief Implement a user interface that allows to change this value.
    275     */
    276    bool m_auto_start_tracking = true;
    277
    278    // For the in-game camera controls, assumed to be in the range [0, 1].
    279    float m_sim_game_camera_eye_tracking_sensitivity = 0.5f;
    280    float m_sim_game_camera_head_tracking_sensitivity = 0.5f;
    281
    282    //************ VARIABLES REPRESENTING THE DEVICE STATE/OUTPUT ************ */
    283    // Note: we don't define the getters to avoid clutter.
    284
    285    // Sim game camera local/additive transform
    286    MyGameEngineTransform device_output_camera_local_transform{0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f};
    287
    288    // Immersive HUD state. Here it is default opaque,  i.e., all the HUD is visible until
    289    // tracking data says otherwise. It's different than the boolean counterparts, as the
    290    // opacity changes are smooth across frames.
    291    float device_output_top_left_hud_opacity = 1.0f;
    292    float device_output_top_right_hud_opacity = 1.0f;
    293    float device_output_bottom_left_hud_opacity = 1.0f;
    294    float device_output_bottom_right_hud_opacity = 1.0f;
    295
    296    // Normalized gaze coordinates in the viewport.
    297    float device_output_viewport_normalized_gaze_x = 0.0f;
    298    float device_output_viewport_normalized_gaze_y = 0.0f;
    299
    300    //************************************************************************************ */
    301
    302  private:
    303    bool m_is_user_looking_at_top_left_corner = true;
    304    bool m_is_user_looking_at_top_right_corner = true;
    305    bool m_is_user_looking_at_bottom_left_corner = true;
    306    bool m_is_user_looking_at_bottom_right_corner = true;
    307
    308    void stop_bet_api_tracking_data_reception() {
    309        if (m_listener_handle != eyeware::beam_eye_tracker::INVALID_TRACKING_LISTENER_HANDLE) {
    310            m_bet_api->stop_receiving_tracking_data_on_listener(m_listener_handle);
    311            m_listener_handle = eyeware::beam_eye_tracker::INVALID_TRACKING_LISTENER_HANDLE;
    312        }
    313    }
    314
    315    std::unique_ptr<eyeware::beam_eye_tracker::API> m_bet_api;
    316    eyeware::beam_eye_tracker::TRACKING_LISTENER_HANDLE m_listener_handle{
    317        eyeware::beam_eye_tracker::INVALID_TRACKING_LISTENER_HANDLE};
    318    float m_time_since_last_viewport_geometry_update{0.0f};
    319};
    320
    321#endif // BET_GAME_ENGINE_DEVICE_H

### main.cpp

This is the main function that manages the scene objects, the `MyGameEngineBeamEyeTrackerDevice` instance, and the game loop, in particular, it:

- Creates a HUD `MyGameEngineImmersiveHUD` consisting of a `MyGameEngineHUDElement` instance on each of the 4 corners of the viewport;

- Creates a scene in which a camera `MyGameEngineImmersiveCamera` is positioned as a child of a character head `MyGameEngineCharacterHead`, which provides a reference pose (see Control the in-game camera movement);

- Both the `MyGameEngineImmersiveCamera` and `MyGameEngineCharacterHead` instances hold pointers of the `MyGameEngineBeamEyeTrackerDevice` instance so that they react to eye and head tracking data at the `Tick` function;

- Implements a `MyGameEngineHotkeysMapper` which monitors if the user presses the SPACE key to recenter the camera, following Implement the camera recentering;

main.cpp

      1#include "bet_game_engine_device.h"
      2#include "my_game_engine.h"
      3#include <chrono>
      4#include <iomanip>
      5#include <iostream>
      6#include <thread>
      7#include <windows.h>
      8namespace {
      9std::vector<MyGameEngineObjectInterface *> devices;
     10}
     11
     12// See bet_game_engine_device.h for showing the "integration" of the Beam API as a device in the
     13// engine See this file to see how it interacts with the other objects in the engine.
     14class MyGameEngineImmersiveHUD : public MyGameEngineObjectInterface {
     15  public:
     16    MyGameEngineImmersiveHUD(MyGameEngineObjectInterface *parent)
     17        : MyGameEngineObjectInterface(parent) {}
     18
     19    void BeginPlay() {
     20        // Get a reference to the Beam Eye Tracker device.
     21        beam_eye_tracker_device = dynamic_cast<MyGameEngineBeamEyeTrackerDevice *>(devices.at(0));
     22
     23        // Dummy hud ui_elements added to all corners
     24        ui_elements.push_back(MyGameEngineHUDElement{MyGameEngineHUDElement::Type::TOP_LEFT, this});
     25        ui_elements.push_back(
     26            MyGameEngineHUDElement{MyGameEngineHUDElement::Type::TOP_RIGHT, this});
     27        ui_elements.push_back(
     28            MyGameEngineHUDElement{MyGameEngineHUDElement::Type::BOTTOM_LEFT, this});
     29        ui_elements.push_back(
     30            MyGameEngineHUDElement{MyGameEngineHUDElement::Type::BOTTOM_RIGHT, this});
     31    }
     32    // Implemented in bet_game_engine_device.cpp to identify
     33    void Tick(float delta_time) override {
     34        for (auto &element : ui_elements) {
     35            switch (element.type) {
     36            case MyGameEngineHUDElement::Type::TOP_LEFT:
     37                element.opacity = beam_eye_tracker_device->device_output_top_left_hud_opacity;
     38                break;
     39            case MyGameEngineHUDElement::Type::TOP_RIGHT:
     40                element.opacity = beam_eye_tracker_device->device_output_top_right_hud_opacity;
     41                break;
     42            case MyGameEngineHUDElement::Type::BOTTOM_LEFT:
     43                element.opacity = beam_eye_tracker_device->device_output_bottom_left_hud_opacity;
     44                break;
     45            case MyGameEngineHUDElement::Type::BOTTOM_RIGHT:
     46                element.opacity = beam_eye_tracker_device->device_output_bottom_right_hud_opacity;
     47                break;
     48            }
     49        }
     50    }
     51
     52    std::vector<MyGameEngineHUDElement> ui_elements;
     53
     54    MyGameEngineBeamEyeTrackerDevice *beam_eye_tracker_device{nullptr};
     55};
     56
     57class MyGameEngineImmersiveCamera : public MyGameEngineObjectInterface {
     58  public:
     59    MyGameEngineImmersiveCamera(MyGameEngineObjectInterface *parent)
     60        : MyGameEngineObjectInterface(parent) {}
     61
     62    void BeginPlay() override {
     63        // Get a reference to the Beam Eye Tracker device.
     64        beam_eye_tracker_device = dynamic_cast<MyGameEngineBeamEyeTrackerDevice *>(devices.at(0));
     65    }
     66
     67    void Tick(float delta_time) override {
     68        // Updates the local pose.
     69        // What is critical to notice is that this updates the world_transform by adding up the
     70        // parent's world_transform with the now given local_transform.
     71        this->set_local_transform(beam_eye_tracker_device->device_output_camera_local_transform);
     72    }
     73
     74    MyGameEngineBeamEyeTrackerDevice *beam_eye_tracker_device{nullptr};
     75};
     76
     77class MyGameEngineHotkeysMapper : public MyGameEngineObjectInterface {
     78  public:
     79    MyGameEngineHotkeysMapper(MyGameEngineObjectInterface *parent)
     80        : MyGameEngineObjectInterface(parent) {}
     81
     82    void BeginPlay() override {
     83        // Get a reference to the Beam Eye Tracker device.
     84        beam_eye_tracker_device = dynamic_cast<MyGameEngineBeamEyeTrackerDevice *>(devices.at(0));
     85    }
     86    void Tick(float delta_time) override {
     87        // Check if R key is pressed
     88        bool recenter_key_pressed = GetAsyncKeyState(VK_SPACE) & 0x8000;
     89        if (recenter_key_pressed != was_recenter_key_pressed) {
     90            if (recenter_key_pressed) {
     91                beam_eye_tracker_device->recenter_camera_start();
     92            } else {
     93                beam_eye_tracker_device->recenter_camera_end();
     94            }
     95            was_recenter_key_pressed = recenter_key_pressed;
     96        }
     97    }
     98
     99    bool was_recenter_key_pressed = false;
    100    MyGameEngineBeamEyeTrackerDevice *beam_eye_tracker_device{nullptr};
    101};
    102
    103class MyGameEngineCharacterHead : public MyGameEngineObjectInterface {
    104  public:
    105    MyGameEngineCharacterHead(MyGameEngineObjectInterface *parent)
    106        : MyGameEngineObjectInterface(parent) {}
    107
    108    void Tick(float delta_time) override {
    109        // Just pretend the character is moving forward very slowly.
    110        this->world_transform.translation_z_inches += 0.01f * M_METERS_TO_INCHES * delta_time;
    111    }
    112};
    113
    114int main() {
    115    std::unique_ptr<MyGameEngineBeamEyeTrackerDevice> beam_eye_tracker_device =
    116        std::make_unique<MyGameEngineBeamEyeTrackerDevice>(nullptr);
    117    devices.push_back(beam_eye_tracker_device.get());
    118
    119    // Basic components, whose parent will be ignored as that's irrelevant in this sample.
    120    MyGameEngineCharacterHead character_head(nullptr);
    121    MyGameEngineImmersiveHUD immersive_hud(nullptr);
    122    MyGameEngineHotkeysMapper hotkeys_mapper(nullptr);
    123    MyGameEngineImmersiveCamera immersive_camera(&character_head);
    124
    125    // We could put all in a list, but will be made explicit for clarity.
    126
    127    //============= INITIALIZING THE GAME RENDERING =============
    128    for (auto &device : devices) {
    129        // Initializes the Beam device and API
    130        device->BeginPlay();
    131    }
    132    hotkeys_mapper.BeginPlay();
    133    immersive_hud.BeginPlay();
    134    immersive_camera.BeginPlay();
    135    character_head.BeginPlay();
    136
    137    //============= FRAME LOOP at 60 FPS =============
    138    auto prev_frame_time = std::chrono::system_clock::now();
    139    const auto end_time = prev_frame_time + std::chrono::seconds(30); // Run for 30 seconds.
    140    while (std::chrono::system_clock::now() < end_time) {
    141        auto frame_start = std::chrono::system_clock::now();
    142        const float delta_time =
    143            std::chrono::duration<float>(frame_start - prev_frame_time).count();
    144        prev_frame_time = frame_start;
    145
    146        hotkeys_mapper.Tick(delta_time);
    147        // We assume that devices are updated before the HUD and camera.
    148        for (auto &device : devices) {
    149            device->Tick(delta_time);
    150        }
    151        // In theory, here the parent-child relationship would drive the ordering, but we'll just
    152        // fake it by updating the character head first, then the camera, then the HUD.
    153        character_head.Tick(delta_time);
    154        immersive_hud.Tick(delta_time);
    155
    156        immersive_camera.Tick(delta_time);
    157
    158        // Note: if you want to see "real" responses for the top-left HUD element opacity when you
    159        // look to the top-left corner of your display, please edit
    160        // MyGameEngineBeamEyeTrackerDevice::get_rendering_area_viewport_geometry_from_engine and
    161        // hard-code the correct geometry of your display.
    162
    163        // Note: you should see the printed z values to grow slowly as the character head is moving
    164        // forward slowly, but also increase or decrease in values as you move towards or away from
    165        // the webcam. Press SPACE to recenter the camera.
    166        std::cout << "[Game cam: z_pos_inches=" << std::fixed << std::setprecision(2)
    167                  << immersive_camera.world_transform.translation_z_inches
    168                  << " ; yaw_degrees=" << std::fixed << std::setprecision(2)
    169                  << immersive_camera.world_transform.rotation_y_degrees << "] and ["
    170                  << "HUD top left opacity=" << std::fixed << std::setprecision(2)
    171                  << immersive_hud.ui_elements.at(0).opacity << "]";
    172        if (hotkeys_mapper.was_recenter_key_pressed) {
    173            std::cout << " Recentering!" << std::endl;
    174        } else {
    175            std::cout << std::endl;
    176        }
    177
    178        // Sleep for 16ms to simulate 60-ish FPS
    179        std::this_thread::sleep_for(std::chrono::milliseconds(16));
    180    }
    181    //============= SHUTTING DOWN THE GAME RENDERING =============
    182    for (auto &device : devices) {
    183        device->EndPlay(); // Shuts down the Beam device and API
    184    }
    185    immersive_hud.EndPlay();
    186    immersive_camera.EndPlay();
    187    character_head.EndPlay();
    188}

## Other highlights

- The sample assumes that the game engine is using a Unity-like coordinate system for the camera pose (yaw, pitch, roll, x, y, z definitions) and viewport geometry (origin `(0, 0)` at bottom-left corner of the rendering area).

- The demonstrated game settings are head tracking sensitivity, eye tracking sensitivity and auto-start behavior toggle, which would typically be exposed in the game’s UI.

- Note how the opacity fade in and fade out is animated to make a smoother experience, but fade in is much faster than fade out.

Plugins and samples
Unity Package