Text Entry in Virtual Reality

Abstract

Typing is a major interface with computers, tablets and phones. We need to type when we write a document, or a mail, when we use a spreadsheet, when we program, when we share over social apps or when we browse the internet. Still, text entry remains a challenge for Virtual Reality (VR) applications and, so far, it is not well understood how typing on standard keyboards transfers into VR.
Further, VR enables new capabilities, impossible in the real world, such as an unobstructed view of the keyboard, without occlusion by the user’s physical hands. Specifically, several hand representations have been proposed for typing in VR on standard keyboards. However, to date, these hand representations have not been compared regarding their performance for VR text entry. We present two studies, with a total of 48 hours of typing in VR that shed light on these issues.

In a first study, we investigated the effects of four hand representations (no hand representation, inverse kinematic model, fingertip visualization using spheres and video inlay) on typing in VR using a standard physical keyboard with 24 participants. We found that the fingertip visualization and video inlay both resulted in statistically significant lower text entry error rates compared to no hand or inverse kinematic model representations.

In a second study with another 24 participants we found that novice users are able to retain about 60% of their typing speed on a desktop keyboard and about 40–45% of their typing speed on a touchscreen virtual keyboard, compared to typing in the real world. We also find no significant learning effects, indicating that users can transfer their typing skills fast into VR. Besides investigating baseline performances, we study the position in which keyboards and hands are rendered in space. We find that this does not adversely affect performance for desktop keyboard typing and results in a performance trade-off for touchscreen keyboard typing.

See also the Microsoft Research Blog entry.

Media

30 Second Teaser

 

Main Videos

 

Images


Hand Visualizations for VR

Views on the conditions studied in the experiment on effects of hand representations for typing in VR. From left to right: NoHand, IKHand, Fingertip and VideoHand.


reposition study

From left to right: VR views on the conditions DesktopKeyboard+NoReposition, DesktopKeyboard+Reposition,
TouchscreenKeyboard+NoReposition, TouchscreenKeyboard+Reposition for our experiment on typing on standard keyboards.

Publications

  • Jens Grubert, Lukas Witzani, Eyal Ofek, Michel Pahud, Matthias Kranz & Per Ola Kristensson. Effects of Hand Representations for Typing in Virtual Reality. In Proceedings of IEEE Virtual Realiy (VR) 2018, Reutlingen, Germany, to appear.
    preprint | arxiv version | teaser video | main video
  • Jens Grubert, Lukas Witzani, Eyal Ofek, Michel Pahud, Matthias Kranz & Per Ola Kristensson. Text Entry in Immersive Head-Mounted Display-based Virtual Reality using
    Standard Keyboards
    . In Proceedings of IEEE Virtual Realiy (VR) 2018, Reutlingen, Germany, to appear.
    preprint | arxiv version | teaser video | main video
Advertisements