Recently, Spielberg's "The Number One Player" has made a comedy. In the play, men can interact with the virtual world through touch-sensitive gloves and the X1 somatosensory suit. Outside of the movie, how to get a sense of touch in virtual reality and how to interact more naturally is a topic of most concern to current researchers. Recently, Microsoft's research team has released four latest virtual reality research results: CLAW, Haptic Wheel, Haptic Links, and Canetroller, which can better help users “touch” objects in virtual scenes and feel the wonder of the virtual world.
Although VR and AR have made tremendous progress in the past 30 years - they have been able to provide users with a fantastic visual and 3D audio world, but when we reach out and touch those virtual objects, the illusion will suddenly burst and the user will touch. Only air came.
There are many reasons why virtual touch is difficult to achieve. People who have watched movies know that eyes and ears can be deceived by a movie of 24 frames per second, but the sense of touch is very different and it is more complicated. Part of the challenge of implementing virtual tactile sensations comes from hardware. In the laboratory, designing a handheld device suitable for personal user wear is often difficult. Many prototype devices today are only capable of simulating specific sensations such as texture, temperature, or weight, and such devices are often unattractive. More sophisticated mechanical technologies may result in equipment that is too expensive, too large or too fragile to appear in the consumer market.
The Microsoft research team has been exploring how to use existing technology to simulate the generation of multiple haptic sensations on a handheld VR controller, enabling users to touch and grab virtual objects and feel the fingertips slide across the surface of the object. Their dream is to allow users to interact more naturally and more with the virtual digital world.
CLAW
CLAW is the first new multi-function haptic controller developed by the Microsoft research team. Using a single motor, CLAW extends the VR controller concept to a versatile haptic feedback tool. The CLAW has a unique robotic arm that simulates force feedback when the user's hands are holding hands and turning their fingers (see below).
As a multi-function controller, CLAW contains all the functions of a normal VR controller (thumbs and joysticks, 6DOF (Degree of Freedom) control, index finger triggers) and various touches that are enabled for the most common hand interactions. Rendering: grab objects, touch virtual surfaces, and receive force feedback.
What's unique about CLAW is that it can adjust haptic rendering by sensing the difference between the user's grip and the virtual context environment. When the user tries to hold the virtual object (as shown below), the device creates resistance between the index finger and the thumb, simulating the feeling of the object being held. The force sensor embedded in the index finger holder allows the user to "feel" the different materials of the object by changing the feedback of the motor.
If the user holds the handle and makes a pointing gesture (as shown below), CLAW provides a touch feeling. As the fingertip moves toward the surface of the virtual object, the CLAW creates resistance, pushing the finger back and preventing it from penetrating the virtual surface. In addition, when the index finger slides along the virtual surface, the voice coil mounted below the tip of the index finger generates a small vibration to mimic the surface texture of the object.
Perceiving the power exerted by the user can also help the user interact with the virtual object. For example, in a virtual painting program, the program can adjust the intensity of the brush by sensing the strength of the user's hand.
Haptic Wheel
To further simulate the tactile experience of the index finger on the virtual object's material and surface friction, Microsoft has also developed another new haptic controller - the Haptic Wheel. The driving wheel used by the Haptic Wheel simulates a finger's touch with the surface of the virtual object by moving up and down, and it simulates the shear forces and sliding sensations generated when the user slides along the virtual surface by rotating.
When the user touches the surface of the virtual object, the wheel will rise and touch the fingertip, and then begin to rotate to simulate the friction between the fingertip and the surface of the virtual object.
The wheel of the device is replaceable and can contain various physical tactile elements to provide users with different feelings when the user explores the virtual environment. When a user explores a virtual environment, the rendering engine passes the appropriate haptic sensations to the fingers based on the environment. For example, in a virtual card game, when the user touches a card, a poker chip, or a table, the device rotates the wheel to present a proper texture perception under the fingertip. As the user slides along the surface, the wheel rotates under the finger to create shear and sliding sensations.
Haptic Links
Another problem that is difficult to solve in tactile exploration is how to simulate the use of both hands in VR or AR applications. For example, feel the size of a box with both hands, or use your hands to feel the tension on the bow.
Haptic Links consists of several types of connectors that provide different stiffness perceptions between two handheld VR controllers. Haptic Links can dynamically change the power felt by the user's hands to support the rendering of various types of two-handed objects and human interactions, making the controller behave like a two-handed tool. They can constrain the specific degrees of freedom or direction of motion between controllers, and can also be set within a range of stiffnesses to simulate different frictions, viscosities, or tensions. Through these methods, Haptic Links makes the interaction in the VR scene more realistic and gives people an immersive feeling.
Microsoft made three prototype devices for Haptic Links. Each design has its own compromise and advantages, making it the best fit for a particular application. The Chain prototype device uses a highly articulated chain consisting of ball and socket elements. A powerful cable runs through the chain and connects to linear actuators on each end. The movement of the push rod motor controls the tightness of the chain and further controls the user's operating space.
Ayer-Hinge (above left) uses a ball joint to adjust the controller's rotation and uses a hinge to control the distance between them. Its advantage is that it can selectively lock the freedom of movement of the controller. In addition, it can control the friction force of each joint relatively accurately, so that the device simulates a continuous range of stiffness values in the hinge and ball joints.
The Ratchet-Hinge (above right) uses a similar ball joint at the bottom of the controller but replaces the hinge with a double ratchet structure and can independently brake inward or outward movement. When two ratchets are engaged, the gears are fixed; when both ratchets are disengaged, the gears can rotate freely. When a ratchet is disengaged, the gear can move freely in one direction but not in the opposite direction. Directional selectivity enables unique force feedback interactions.
Haptic Links can improve the perceived realism of objects requiring two-handed operation in a virtual environment without overly affecting the normal interaction of the two controllers.
Canetroller
The "traditional" virtual reality experience is strongly visualized in nature, so visually impaired people usually cannot use it. The Canetroller prototype device developed by Microsoft Research hopes that visually impaired people can also use a walking stick for blind guidance in virtual environments.
Canetroller offers three types of feedback: (1) physical resistance from wearable, programmable brakes - simulation of a virtual cane hitting a virtual object; (2) vibrational haptic feedback - simulation of a cane hitting an object or across a variety of surfaces (3) 3D auditory feedback - simulates the sound of a real world cane.
Canetroller allows low vision and blind people to perform simulated blindness training in different virtual spaces, reducing their risk in the daily environment. Microsoft hopes that this work will motivate researchers and designers to design more effective tools to make VR more inclusive, taking into account the different groups around the world.
The Microsoft research team's innovations in virtual touch, on the one hand, hope to inspire the imagination of other researchers. On the other hand, they hope to bring new applications and hopes for special groups such as visual impairment and help them improve their quality of life. In addition, researchers hope that these innovations will encourage more consumer products to use haptic rendering technology. I believe tactile products will soon make VR and AR products more realistic and immersive.