Nov 15 2017

Views: 624

Virtual reality or virtual realities (VR), also known as immersive multimedia or computer-simulated reality, is a computer technology that replicates an environment, real or imagined, and simulates a user’s physical presence and environment to allow for user interaction. Virtual realities artificially create sensory experience, which can include sight, touch, hearing, and smell.

Most up-to-date virtual realities are displayed either on a computer monitor or with a virtual reality headset (also called head-mounted display), and some simulations include additional sensory information and focus on real sound through speakers or headphones targeted towards VR users. Some advanced haptic systems now include tactile information, generally known as force feedback in medical, gaming and military applications. Furthermore, virtual reality covers remote communication environments which provide virtual presence of users with the concepts of telepresence and telexistence or a virtual artifact (VA) either through the use of standard input devices such as a keyboard and mouse, or through multimodal devices such as a wired glove or omnidirectional treadmills. The immersive environment can be similar to the real world in order to create a lifelike experience—for example, in simulations for pilot or combat training—or it can differ significantly from reality, such as in VR games.

As Cinematographer, we’re used to making immersive experiences for people already. There’s a reason someone comes out of a movie theater thinking they’re in the movie. Compelling storytelling and incredible cinematography ‘pull you in’, not with standing the 60 degree FOV.

With VR, we now have 360 FOV. It’s no less of a development for our industry than the introduction of sound or color, and requires new methods of working, although storytelling remains as important as ever.

Currently there is huge demand from brands to create live-action VR experiences. Everyone is experimenting in this nascent medium, from live 360 streaming of sports and concerts, to 360 music videos, short films, documentaries, current affairs and hybrids of CG and live action.

The jury is still out whether consumers will watch long-form cinematic content in VR, as some people experience nausea after prolonged exposure, and turning your head constantly can literally be a pain in the neck. It also tends to go against the fundamentally passive nature of film and television, where we let content flow over us. But being aware of this issue is the first step towards creating content that works.

To film in VR means rethinking a lot of what we’re used to as Cinematographers & camera operators. We’re used to literally being behind the camera all the time, but that’s not possible with VR. We’re used to spending a lot of time thinking about camera angles, depth of field and lenses, whereas with VR these factors are far less relevant.

Allowing people to see literally anywhere, means hiding the crew while shooting, and finding creative ways to focus the viewer where you would ideally like them to look. Sound, lighting, and talent cues, which direct the viewer in subtle ways, are all techniques that have been used successfully to date.

All-in-one cameras from the likes of Jaunt, GoPro and Nokia, which are being touted as professional cameras, have fixed lenses and little in the way of manual control. While they have their uses, they leave a lot to be desired for professionals.

It’s this reason that has led a few within the industry to design their own 360 VR rigs, typically around Red Dragon Epic’s, Codex action cams, Sony A7S II and others.

Selecting a camera means giving careful thought to whether the output is 2D or stereoscopic, and which kind of stitching software you are likely to use.


360° IMAGES:

A 360° image is one that captures an entire scene in all directions. When just dealing with 360° horizontally these images are better known as panoramas as they tend to miss out the lower and upper portions of a scene, thus making them 360° (horizontal) x180° (vertical) images at best, but usually they’re 360° x 120° or even 360°x100°. For a true 360° image that’s 360°x360° you’re actually talking about “photospheres“. Even if you’re not familiar with the term, you’ve all seen them before. For example, when you switch to Google Street View and pan around on the spot in all directions – that’s a photosphere.

360° VIDEO:

Just like 360°x360° images (photospheres) a 360° video is one that captures the entire field of vision (FoV) in all directions, both horizontally and vertically, giving you a full 360°x360° immersive video.

(In these videos you can pan around using your mouse or keyboard when viewing them on a desktop or laptop, or you can exploit the gyroscope sensor on your phone or tablet to pan around by simply moving the device itself)


VR (Virtual Reality):

The term “VR” (Virtual Reality) originally referred to an artificial reality that not only covered the viewer’s entire field of vision (FoV), but also enabled them to interact with that virtual world by moving around it. True VR achieves this whilst also appearing three-dimensional through what’s known as a stereoscopic projection. VR headsets accomplish this by splitting the screen in two. The headset lenses then focus and reshape the original two-dimensional view for each eye by angling two images to mimic how each of our eyes view the world slightly differently. Finally, the interactive element comes from using a gyroscopic sensor and accelerometer to track the users’ head movements and orientation and then render stereoscopic images to reflect these changing values in real time. Additional interactivity is then down to the software you’re using. For example, interactive VR games will allow you to walk or even fly around and your movements and actions in that virtual world can further alter the virtual environment you’re in.



However, a lot of videos viewed with VR headsets are not strictly speaking VR experiences – they’re merely 360° videos. There are two reasons for this: the first is that they’re not properly 3D as they weren’t recorded to create a true stereoscopic (i.e. three-dimensional) experience (more on this later). The second is that they’re not truly interactive: you can rotate on the spot, but you cannot move from that spot or interact with the virtual environment in any way. In short, you’re at the mercy of the videographer/director, passively observing from whatever fixed vantage point the camera rig was positioned. Now, I’m not saying that these sorts of 360° videos aren’t immersive or great to experience with a VR headset. I’m just pointing out that strictly speaking they’re immersive 360° videos, not VR videos. This is a rather important point as the confusion stems from how 360° videos are viewed. When played back on a desktop or laptop then they’re clearly just 360° videos and nothing more. But when viewed on a VR headset like Google Cardboard or the Samsung Gear VR, then people suddenly start referring to them as VR videos – but they’re not, for the same reasons mentioned above. For a 360° video to be a VR video it also needs to be stereoscopic (i.e. three-dimensional) at the very least. While VR headsets can simulate a stereoscopic projection by splitting the screen in two and trick your brain into seeing something in 3D, it can’t work miracles. So ultimately the three-dimensionality and DoF required for a fully immersive VR experience will always be limited by the quality of the original 360° footage (more on this in a moment). And that’s why the terms “360° video” and “VR video” are not always interchangeable.


Truly immersive VR:

At the risk of repeating myself, the point about a video being stereoscopic (3D) is extremely important here, as that’s determined by how the original video was shot. Currently every 360° video recording device on the market employs more than one lens and the recorded footage is stitched together using software to create a single 360° video. For example, the Ricoh pocket 360° cameras have two spherical lenses opposite each other, while more advanced 360° rigs employ an array of cameras (usually GoPro’s). If it was created by stitching together 2-6 different video cameras in different positions then it won’t be fully 3D when viewed on a VR headset because as the cameras in the rig aren’t close enough together you often lose that extra depth of field (DoF). That’s why the Google Jump rig uses sixteen GoPro’s in a circular array to minimize the warping effects of geometrical distortion, and in the process gives you a better DoF for a truly immersive VR experience.



Currently there are two options if you want to try out VR. You can either buy a complete headset that includes a built-in screen like the Occulus Rift. Or you can go for the more affordable route by buying a headset that uses your own mobile phone. For this you simply slot the phone into the headset and run apps on it that split the screen. Needless to say, this method is dependent on how good your phone is. The key requirements are a high resolution screen (minimum 1080p HD), a fast processor (quad core minimum) with lots of RAM (minimum 2GB) and a good gyroscope sensor and accelerometer to keep track of head movement and orientation. Ideally you’ll also need a phone with a good battery life, as you’ll find that many of the apps drain power like nothing else you’ve seen before!

I am text block. Click edit button to change this text. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.


360° photography: images are only truly 360° if they’re 360°x360° (horizontal and vertical), otherwise they’re just panoramas (i.e. 360° horizontally but miss out capturing a full 360° vertically).

360° videos: these are created by stitching more than one video stream (with different cameras/lenses aimed at different angles from the same centre-point) together. For spherical lenses you need two separate video streams recording footage in opposite directions , but this tends to result in geometrical distortion and lacks the truly stereoscopic (i.e. 3D) feel which can only be achieved by using a lot more cameras/lenses closer together to give the video a better depth of field (DoF).

VR headsets: these are wearable headsets that cover your entire FoV (field of vision) and split the screen to create a stereoscopic projection that gives the viewer a 3D-like experience. By using gyroscope sensors and accelerometers they can track your head movement and orientation to adjust what you’re seeing in real time, thus making it feel as if you’re somewhere else.

True VR and mere 360° video experiences: the split screen stereoscopic trick employed by VR headsets will only ever be as good as the original footage you’re viewing. While video resolution is important, what’s equally neglected is how the original footage was recorded. More cameras in a rig results in less geometrical distortion and a greater DoF (depth of field), both of which massively enhance the authenticity of the virtual environment.

Ref / Thanks: