Facebook Building New 360 Video Technology.

New algorithm can supposedly stabalize herky-jerky 360 degree videos.

Last week, President Barack Obama dipped his toes into the world of virtual reality.

He starred in a short film during a recent trip to Yosemite National Park that was shot with high-end cameras that capture video in full 360 degrees. Viewers wearing virtual reality headsets could crane their necks to gawk at Yosemite’s majestic granite cliffs and tranquil meadows as if they had tagged along with the president.

It was the next best thing to being there.

President Obama’s trip was a showcase of what is possible by filming video in 360 degrees. But it’s also a reminder of how much work remains to be done before tourists are able to film their own vacations like that of President Obama by a professional VR studio for Facebook’s Oculus Rift virtual reality business.

Facebook  FB 0.15%  has been a big proponent of 360-degree video (in addition to regular, old video in general), and over the past year has added several data center and video processing technologies so that it can more efficiently show 360 degree videos to users without annoying delays.

By improving the technology for consumers to both watch and create videos in 360 degrees, the social network hopes to benefit by keeping its over 1 billion users more engaged and active. As of now, however, most amateur 360-degree video being shot is “super shaky and not so watchable,” said Johannes Kopf, a research scientist in Facebook’s computational photography research group.

Watching one of these shaky 360 videos using a headset can be disorienting and can even cause people with sensitive stomachs to feel queasy, Kopf explained. To deal with these wobbly videos, Facebook built a new algorithm that Kopf said can automatically improve amateur videos by removing some of the unwanted unsteadiness that is typical when non-professionals film using a camera in their untrained hands.

The algorithm is part of what’s known as electronic image stabilization, a type of video processing that can help smooth the jerkiness in videos even when filmed in an unsteady environment, like from a dirt bike cruising down a bumpy trail.

Currently, many different automatic video stabilization tools are available for standard, flat video, but there are not many designed for 360 video. Companies like VideoStitch and Mettle as well as academic researchers have all been working on ways to smooth bumpy 360 videos, but the technology is not yet widespread.

Unlike some video technology companies, Facebook does not want to sell video-processing software to consumers. Instead, it wants to bundle its homegrown 360-degree video smoothing technology into its social networking service so that when users upload videos, the stabilization technology works automatically behind the scenes to remove some of the jittery imagery, Kopf explained.

Facebook’s video stabilization tool involves slicing a video into chunks, or key frames. The engineers used an existing image tracking technology to pinpoint specific objects within a video, which allows them to keep track of those objects in the film over time.

So in the case of a 360-degree video of a yellow ball filmed with shaky hands, Kopf said, Facebook can lock onto that ball and keep track of its original position.

As the film progresses and the ball starts moving away from its center position of the image because of poor camera work, the algorithm can essentially rotate each key frame so that the ball magically appears in the same spot in every frame. Once the key frames are rotated, the algorithm can then re-orient all the other video frames in relation to the stabilized key frames, which helps if there happens to be multiple balls in the scene.

“It’s like a balance, you can’t smooth them out all at once,” said Kopf. “Different parts of the scene might be moving in slightly different ways, and by having more localized information you can smooth them out as well.”

The algorithm can also slightly deform each frame as well, so that in the case of an extreme wobbles the actual frame gets a little tweaked to conform to where the ball should be. Once stabilized, the film should appear less amateurish than before and less nauseating of an experience.

Kopf and his team started working on the technology in January and is still testing and improving it. Facebook hasn’t decided when to actually start rolling it out to all of its users.