VR Eye Tracking: What Is It? Which VR Headsets Have It?

VR eye tracking is an advanced technology that’s quickly enhancing the way we experience digital content. It’s a key component in immersion, or the feeling of presence we have while we’re in extended reality (XR)—which includes virtual reality (VR), augmented reality (AR), and mixed reality (MR).

But what exactly is eye tracking and why does having it make a difference? In this blog post, HTC VIVE will show you the ins and outs of eye tracking and explain how you can get started with it using a VR headset, like VIVE Focus Vision.

Let’s dive into the eye-opening world of VR eye tracking.

Virtual reality becomes even more lifelike when you incorporate eye tracking.

What is eye tracking in VR?

Eye tracking is the monitoring and analysis of eye movement in real time. It is how a VR headset or peripheral accessory captures where you’re looking at any given moment.

Eye tracking also detects the movements of your eyelids—such as winking and blinking.

Data about what your eyes are doing can be used to make your interactions in virtual reality more seamless and believable. We’ll elaborate on this shortly.

How does eye tracking work?

Eye tracking for VR is powered by a combination of hardware and software. This is either included in a VR headset or added onto one via an accessory.

VIVE Focus 3 Eye Tracker is a VR headset accessory that includes eye-tracking cameras.

A typical eye-tracking system involves:

• Infrared illuminators: LEDs emit infrared light, which is reflected off your eyes but is invisible to you.
• Sensors: Cameras sensitive to this infrared light capture several images of each eye per second.
• Data processing: Computer vision algorithms analyze the infrared images to identify each eye’s position and orientation in real time.

This works because the outer surface of the eye, called the cornea, is both transparent and refractive. Here’s another, more technical explanation of what’s happening:

The cornea’s transparency allows most of the infrared light to pass through and reach the layers underneath, which include a colored area called the iris and a circular black opening called the pupil. At the far back of the pupil is another inner layer called the retina.

There’s more to eye tracking than meets the eye. (Source: Wikimedia)

Both the iris and retina absorb a portion of the light and reflect back the remainder, but the retina absorbs more and reflects less. This causes the pupil to appear much darker than the iris, and the contrast between light and dark helps us determine where the pupil’s edges are.

At the same time, infrared light passing through the cornea also refracts (bends), producing a shiny spot on the surface known as a corneal reflection, or glint.

The white speck of light reflected on the surface of this eye is called a glint.

The pupil’s position changes as the eye looks around, but the glint remains fixed in place. This makes sense given that the source of the infrared light (the LED) is stationary.

As cameras capture images of each eye, the ever-changing locations of the pupils relative to the fixed glints are used to calculate where the eyes are looking. That involves a lot of math and some estimation, but sophisticated tracking algorithms are designed to do just that.

Learn more about how tracking in virtual reality works.

Why does eye tracking matter in VR?

Eye tracking makes the overall experience of virtual reality more enjoyable. Here are some of the benefits you can get by incorporating VR eye tracking into your setup:

• Avatar expressiveness: Having your avatar’s eyes move in sync with your own adds realism to social interactions, allowing you to maintain eye contact with someone while speaking to them in VRChat, for example.
• Optimized performance: Eye tracking enables dynamic foveated rendering, a VR headset feature that focuses the highest-resolution graphics in the direction of where you’re looking. This cuts down on lag and offers smoother gameplay. 
• Gaze controls: Navigating menus or VR experiences with your eyes instead of with controllers can feel more intuitive—and get you where you want faster. 
• Improved comfort: Reduced visual latency also helps prevent motion sickness, which can occur when there’s a mismatch between what your eyes see and how your body moves.
• Better focus: Eye tracking can be used to measure your interpupillary distance (IPD) and align your VR headset’s lenses with the centers of your pupils, letting you instantly find that visual sweet spot.

   

  

Auto-IPD adjustment is a handy, time-saving feature made possible by eye tracking.

The virtual reality content you’re using has to support these eye-tracking features for you to be able to enjoy them, but with so much potential to offer, it’s no surprise that eye tracking has already started making its way into popular VR titles, like VRChat.

What is eye tracking used for in VR?

Eye tracking in virtual reality has both personal and professional applications. Some VR eye tracking use cases include:

• Social: Synchronize your avatar’s eyes with your own to let other people in shared VR environments see where you’re looking and engage in more natural interactions.   
• VTubing: Control your avatar’s gaze precisely to better animate your VTuber character and deliver more convincing virtual performances.
• Gaming: Enable foveated rendering to make your VR games look their best and to never miss a beat because of issues like visual lag.
• Training: Follow the focus of participants in VR simulations to better assess their performance and offer more specific feedback.
• Research: Gain behavior insights and refine UI with recordings of where users of an app, game, video, or other experience are looking.

   

  

Eye tracking helps more of your personality shine through in VR.

Which VR headsets have eye tracking?

Only a few VR headsets currently support eye tracking as a native feature or through an add-on accessory. If you’re looking for a good VR headset with eye-tracking capabilities, consider these options:

• VIVE Focus Vision: A PC VR-standalone hybrid that offers built-in eye tracking right out of the box.
• VIVE Focus 3: A standalone VR solution for businesses with eye tracking enabled by VIVE Focus 3 Eye Tracker—a separately sold accessory.
• VIVE XR Elite: A lightweight, glasses-like mixed reality device that supports eye tracking through the VIVE Full Face Tracker accessory (sold separately).
• VIVE Pro Eye: A PC VR headset with integrated eye tracking (no longer widely available).

   

  

VIVE Focus Vision offers built-in eye tracking, along with several other great features.

How do I get VR eye tracking?

Getting eye tracking for VR is fairly straightforward. Here’s how to begin enjoying this technology:

1. Find a VR headset that supports eye tracking, such as VIVE Focus Vision. 
2. If eye tracking is only available on your device using an eye tracker accessory (as with VIVE XR Elite, which requires VIVE Full Face Tracker), pick up that accessory as well.
3. Update the VR headset’s software to ensure it is running the latest version. 
4. If you’re using an eye tracking accessory, connect it to the VR headset and make sure its firmware is up to date.
5. Turn on the VR headset and locate eye tracking in the system settings. You may be prompted to go there automatically.
6. Calibrate eye tracking to ensure your eye movements are tracked with optimal precision. This usually involves following a series of prompts that ask you to focus on specific points in your field of view.

How do I get eye tracking in VRChat?

Even though VRChat doesn’t have native eye-tracking integration, it supports eye tracking via the OpenSoundControl (OSC) specification and a program called VRCFaceTracking (VRCFT).

VRCFT reads tracking data from a VR headset or VR eye tracker and then uses OSC to send it to VRChat in a way that can be translated into avatar eye data. This eye-tracking data includes:

• Eyelook: The direction that each eye is looking in 3D space.
• Eyelid: How open or closed each eye is.

The specific steps to getting VRChat eye tracking vary slightly depending on the hardware you’re using. Typically, they require you to:

1. Download and install the VRCFaceTracking software.
2. Follow the setup instructions using a VR headset that supports OSC.
3. Enable OSC in VRChat by selecting Action Menu > Options > OSC > Enabled.
4. Use a VRCFT-compatible VRChat avatar.

Following these steps can get you not only eye tracking but facial tracking, too! And for maximum control over your avatar, consider also getting full-body tracking for VRChat.

If you need help setting up eye tracking with an HTC VIVE VR headset, check out our support pages for more information.

How is eye tracking different from face tracking?

Eye tracking only registers the movements of your eyes. For even better expressiveness in VR, you can use eye tracking in tandem with facial tracking, also known as face tracking.

Facial tracking captures expressions made by the combination of your lips, teeth, tongue, cheeks, and chin. These combinations—called blend shapes—can be used to animate the corresponding parts of your VR avatar.

When eye tracking and facial tracking are used together, the result is known as full-face tracking.

Both eye tracking and facial tracking can bring VR avatars to life.

Is VR with eye tracking worth it?

Eye tracking significantly improves the experience of virtual reality. It’s no surprise that many VR users consider it a must-have.

Now that you understand what eye tracking is and how you can use it, it’s time to experience this incredible technology for yourself. Get immersed with HTC VIVE today and discover the many wonders of VR eye tracking.