Mixed reality

Capturing the infinite experiences of the sky

See a conference room transform into a canvas for changing skyscapes using a RoomAlive Toolkit, which morphs the experience of space via projectors, cameras, and audiovisuals.

How can technology transform a space—even a non-descript room—into an experience that’s immersive, sensory, and above all, emotional?

This was the question that brought together Hrvoje Benko, a human-computer interaction research at Microsoft Research, and Maja Petrić, an artist exploring the interface between technology and art. They collaborated to create A Panorama of the Skies, an immersive audiovisual installation that transformed an everyday conference room into an ever-changing skyscape.

Petrić and Benko’s focus was on designing an audiovisual experience that engages perception, emotion, and imagination, and morphs how people experience space. To do this, Petrić turned to the sky for inspiration.

“What interests me is the multitude of experiences that people have when looking at the sky. We have one sky, but the experiences are infinite,” says Petrić. “They depend on the location from which you’re looking at the sky, the time of day, the climate. Most of all, it depends on your eyes—your subjective view.”

What interests me is the multitude of experiences that people have when looking at the sky. We have one sky, but the experiences are infinite.

Maja Petrić

So how did they transform an otherwise normal conference a room? They used audiovisual content, along with a multi-projector, multi-depth-camera framework called the RoomAlive Toolkit. The system uses 5 projectors and 8 Kinect cameras to acquire a detailed map of the room, register the projectors and cameras into a single coordinate system, and then enable real-time projection mapping to create an immersive scene.

See how the RoomAlive Toolkit immersed viewers in the ephemeral sky, morphing a conference from dawn to sunrise to a lightning storm and beyond.

Normal conference room at Microsoft
Conference room lit like the sun rising
Conference room lit as if the sun were rising
Conference room lit with storm clouds
Conference room lit like the calm after the storm
Woman sitting on conference table looking calm

Normal conference room at Microsoft is transformed…

into a place at dawn…

where the sun rises...

and the storm takes over...

until the calm after the storm...

in the purpose of complete immersion of a viewer into an elevated emotional space.