Skip to content

Eye Tracker

Overview

Maverick AI glasses include an on-board eye tracking sensor. The glasses run the eye-tracking neural network themselves and stream a compact feature vector to the phone; the SDK turns that into per-frame eye geometry and feeds it to a graph of eye-feature analyzers.

The SDK handles this via M2EyeTrackerService (accessed from Evs.eyeTrackerService).

Capabilities

Feature Description
Eye-feature results Per-frame iris and pupil ellipses, eye corners, LED glint, eyelid apexes and pupil visibility (M2EyeTrackerResult)
Analyzers Pluggable M2EyeFeatureAnalyzer implementations that consume the results and emit higher-level conclusions
Device control Enable / disable the tracker, observe its on/off state, receive errors

Enabling the Eye Tracker

Evs.eyeTrackerService.enableEyeTracker()

// later
Evs.eyeTrackerService.disableEyeTracker()

isEyeTrackerOn() reports whether the device is currently streaming. The tracker needs the glasses to be connected; enabling while disconnected reports M2EyeTrackerErrors.NotConnected.

Receiving Results

Register an IM2EyeTrackerEvents listener via Evs.eyeTrackerService.registerListener(...).

val listener = object : IM2EyeTrackerEvents {
    override fun onEyeTrackerStateChanged(isOn: Boolean) { }

    override fun onM2EyeTrackerResult(result: M2EyeTrackerResult) {
        if (!result.isValid) return          // eye closed / geometry not extracted
        val (pupilX, pupilY) = result.filteredPupilCenter
        val visibility = result.pupilVisibility
    }

    override fun onError(type: M2EyeTrackerErrors, message: String) { }
}
Evs.eyeTrackerService.registerListener(listener)

Callbacks are delivered on the SDK application thread.

M2EyeTrackerResult

Property Type Description
isValid Boolean true when eye geometry was extracted for this frame
iris, pupil FloatArray Ellipse parameters (cx, cy, a, b, angle) in eye-camera coordinates
earCorner, noseCorner Pair<Float, Float> Eye corner positions
ledPosition, filteredLedPosition Pair<Float, Float>? LED glint, raw and filtered; null when not detected
filteredPupilCenter Pair<Float, Float> Smoothed pupil centre
pupilVisibility Float Fraction of the pupil visible (0–1); low values mean a blink or closed eye
eyelidTop, eyelidBottom Pair<Float, Float> Eyelid apex estimates

Analyzers

An analyzer is a small class that receives every frame, keeps whatever state it needs and publishes a typed M2EyeFeatureResult when it has something to say. Analyzers can depend on other analyzers, and the SDK runs them on a dedicated thread in dependency order.

Implementing an analyzer

class BlinkResult(
    override val timestamp: Long,
    override val isValid: Boolean,
    override val confidence: Float,
    val blinkRatePerMinute: Float,
) : M2EyeFeatureResult() {
    override val conclusion get() = M2AnalysisConclusion("Blink rate", "%.0f / min".format(blinkRatePerMinute))
}

object BlinkKey : M2AnalyzerKey<BlinkResult>("blink")

class BlinkAnalyzer : M2EyeFeatureAnalyzer<BlinkResult>(BlinkKey) {
    override val description = "Counts blinks from pupil visibility"

    override fun onFrame(input: M2AnalyzerInput) {
        val closed = input.isEyeClosed()
        // ... track transitions, then:
        emit(BlinkResult(input.timestamp, isValid = true, confidence = 1f, blinkRatePerMinute = rate))
    }
}

M2AnalyzerInput carries the raw result, a normalised frame, baseline statistics once they are available, and the frame timestamp. Use dependsOn(otherKey) in the constructor to read a predecessor analyzer's latest result.

Registering and reading analyzers

val blink = BlinkAnalyzer()
blink.setResultListener { result -> /* app thread */ }
Evs.eyeTrackerService.registerAnalyzer(blink)

// or lazily, by key
Evs.eyeTrackerService.registerAnalyzerFactory(BlinkKey) { BlinkAnalyzer() }

// polling
val latest = Evs.eyeTrackerService.getAnalyzer(BlinkKey)?.lastResult

Evs.eyeTrackerService.unregisterAnalyzer(BlinkKey)

Analyzers are independent of the device lifecycle: they stay registered across tracker enable/disable and reconnects, and receive onStart / onPause / onResume / onStop hooks.

Error Handling

Error Description
ModelInitFailed The eye-tracking model failed to initialise
DecodeFailed Eye camera frame could not be decoded
NotConnected Glasses are not connected
DeviceError Hardware-level error (e.g. eye camera stream stopped after repeated restarts)

Notes

  • Eye tracking runs on the glasses hardware and is decoupled from BLE latency.
  • The SDK power-cycles the tracker automatically when the stream goes silent; the app only sees onEyeTrackerStateChanged and, after repeated failures, a DeviceError.
  • getFps() reports the rate at which feature frames arrive.

See Also