I am the controller now: MIT engineers have created a 'second skin' that could one day track whole bodies in VR

Remember when they did motion capture on a cube? Well, more specifically, they put actor Jack Quaid in a motion capture suit and then they put him inside a cube for his role in God of War Laufey. As charming as that cutting edge cuboid is, I think VR gamers and Vtubers are gonna be all over fresh research out of MIT.

Engineers at MIT have developed a sheet full of optical fibres that can digitally reconstruct its own shape even as it is flexed and folded. According to MIT News, the team reckons their tech could be fashioned into a garment that could then be used to accurately mirror your real-world movements in a game.

Though currently a bulky prototype, the flexible sheet could one day be worn like a second skin that can more accurately track the transformation of a surface, rather than the movement of, say, fixed points on a traditional motion capture suit. What's arguably even cooler though is that the sheet is shown in a demo video to remain functional even as an engineer pierces it with a hole puncher. Basically, this tech stands a good chance of surviving any clumsiness experienced while in VR.

Light from the bendy optical fibres inside the sheet is tracked and then interpreted by a specially developed algorithm. As the sheet bends, so do the fibres, which also dictates how much light can pass through them. This kind of structure is known as a waveguide.

Qifan Yu is a mechanical engineering graduate student at MIT, and one of the co-authors of the paper based on this project. Yu explains, "People have used waveguides to sense the shape of a line, which they have applied to a robot arm to see how it curves, but they haven’t been applied to surface shape sensing, to reconstruct the 3D shape of a surface. That’s what we’re trying to do here."

According to Yu, the team's algorithm reconstructs the movement of the actual sheet with a margin of error that's "less than 0.4 centimeters." He adds, "Existing designs, which are based on rigid sensors, have errors of around 1 to 2 centimeters."

Besides potentially more detailed performance capture, this tech could be used to control in-game movements or even aid patients undergoing physical therapy. Yu says, "In physical therapy, you often need to gather patient data on how they are moving and stretching their arm during rehabilitation, for example. Surface shape sensing could be very useful in that way."

The team is planning to trim down the prototype. Currently, the optical fibres measure 1 millimetre thick but alternate fabrication methods could potentially make these fibres thinner than a human hair at mere tens of micrometres. Let's just say, no matter how dweeby it makes me look, I'm looking forward to donning a shape-sensing garment at some point in the future and jumping into VR with both feet.