← Design + Engineering

Wearable · Augmentation Lab 2022

CONTROL XR

A universal haptic interface for AR/VR

RoleDesign Engineer · Haptics, Firmware, Interaction
Year2022
TeamAlice Cai + four
HardwareMeta Quest · ESP32 · piezo actuators

One pair of gloves standing in for four inputs: VR controller, haptic glove, virtual keyboard and physical keyboard.

Overview

01

XR still borrows its input from the game pad. ControlXR asks for one wearable with the text bandwidth of a keyboard, the pointing of a controller, and fingertip feedback that can be tuned to feel like different materials.

Touch + type

Try it
QUEST · XR HANDS
GLASSMETALFOAM
Touch somethingeach material, its own pulse
Frequency
·
Amplitude
·
Waveform
·

The two ideas, hands-on. Pulse values and the key layout are illustrative, not the ones tuned and tested in 2022. On a phone, the pulse also vibrates.

System

Architecture
CONTROLXR · SYSTEMAUGMENTATION LAB · 202201 / THE GLOVEFingertips that answerPiezo capon each fingertipBoardESP32, BOS190102 / SIGNAL PATHContact out, readings backA · HEADSETQuest · UnityXR Hands: contact,which fingerB · GLOVEESP32both waysC · DRIVERBOS1901piezo driveD · TIPPiezoFREQUENCY · AMPLITUDE · WAVEFORMSENSOR READINGS COME BACK TO THE HEADSET03 / FOUR INPUTSOne pair of gloves01Controllerpoint and select02Haptic glovefingertip feedback03Virtual keyboardkeys on your fingers04Physical keyboardtext inputKEYS ON THE FINGER SEGMENTS, SEEN BY THE HEADSET’S CAMERAS. SEVERAL LAYOUTS TESTED.CONTROLXR202201 / THE GLOVEFingertips that answerPiezo capeach fingertipBoardESP32, BOS190102 / SIGNAL PATHContact out, readings backA · HEADSETQuest · UnityXR Hands: contact, which fingerWIRELESS, BOTH WAYSB · GLOVEESP32C · DRIVERBOS1901, piezo driveD · FINGERTIPPiezofrequency · amplitude · waveformSENSOR READINGS COME BACK03 / FOUR INPUTSOne pair of gloves01Controllerpoint and select02Haptic glovefingertip feedback03Virtual keyboardkeys on your fingers04Physical keyboardtext inputKEYS ON THE FINGERS, SEEN BY THEHEADSET’S CAMERAS. LAYOUTS TESTED.

How it works

02

Haptics: Unity’s XR Hands detects contact with a virtual object and fires the matching fingertip. Each pulse is shaped by frequency, amplitude and waveform.

Link: an ESP32 on the glove talks to the Quest wirelessly in both directions, actuator commands out and sensor readings back.

Input: several keyboard layouts tested on people, including one that maps keys onto the fingers, with keys identified through the headset’s computer vision.

The prototype

Augmentation Lab · 2022

The real glove: a piezo cap on the fingertip, wired to the board, answering a touch in Unity.

Result

03

Touch a virtual object and the right finger answers. The early prototype was demoed at the Harvard SEAS Design Fair, where visitors tried it.

Components

Built with
HapticsPiezo actuators, Boreas BOS1901 driver, custom PCB
FirmwareESP32, two-way wireless link to Quest
SoftwareUnity, XR Hands