what resolution does vr need to be to look like reality

The challenges to achieving a quality VR video experience.

We call it "virtual reality," but the quality of the video nosotros watch in VR headsets still doesn't look real. Why is quality important? The feeling of really being there, called "spatial immersion," is undermined by poor-quality video. Today, a great deal of VR video struggles to achieve immersion, because the poor quality distracts the viewer from the feeling of realism.

A true feeling of immersion comes from a combination of narrative and spatial immersion. Narrative immersion occurs when the viewer becomes emotionally invested in the story. Remember about reading a novel or watching your favorite flick. You get so caught upwardly in the story that yous lose track of fourth dimension. Spatial immersion occurs when the viewer becomes convinced past their senses that they are in a new space. When we are experiencing the sights and sounds of a new space and engaged with a compelling story, we can exist powerfully moved. When nosotros get these two ingredients right, nosotros feel like nosotros are truly in virtual reality.

There are no hard and fast rules for achieving narrative immersion. But nosotros can define how to make a great-looking video that tin fool your senses to create spatial immersion.

Many things that are called VR today fall far short of this goal. Services such equally Google'south YouTube 360° and Facebook 360° have started to regularly offering 360-degree video content that simulates VR past displaying spherical video in a standard video actor. 360-degree video is normally delivered through a web browser or on a simple device like Google Cardboard. But moving a mouse or holding up a cardboard box to your face requires hand-eye coordination that makes your brain take on tasks that interfere with immersion. Holding a smartphone up to your head is fine for a few minutes, only who wants to do that for a two-60 minutes movie? A ameliorate experience comes from a caput-mounted brandish that accurately and naturally tracks your movement.

A mobile phone-based headset like the popular Samsung Gear VR has good motion sensors and fast response times, and is capable of creating an immersive feel. In that location are many new headsets coming to the market place in 2016. The Oculus Rift and HTC Vive are both tethered by cables to a PC, making use of the PC's GPU capabilities to render 3D data quickly, and using additional sensors to track user motion in more directions than the Gear VR. Both the mobile and PC-tethered headsets take loftier-resolution displays with plenty of pixels, but when those displays are used to play video, the video quality tends to exist poor.

Even with good display hardware in a headset, low video quality can break the sense of true immersion. The catchy part is delivering a dandy looking image to the device at the correct moment. Since the Gear VR is the most widely bachelor set today, let's look at its technical specs and the challenges of using it to achieve a quality VR video experience.

The challenges of VR video content

While all the headsets take cutting-border high-resolution displays, the shortfall comes from content delivered at low resolution.

File size. With Gear VR, many of the video-based experiences in applications from VRSE and others require an initial download. The downloads are very large—more 1 GB in size— and tiresome to load onto the device. That is the first ding confronting the experience.

Video quality. The 2d and bigger effect is the quality of the videos. Even professionally produced VR media is alarmingly soft. Viewing information technology feels like jumping back in fourth dimension to the 1990s when computers struggled to play anything more than standard-definition video. The VR videos that seem to endure the virtually are in stereo format.

Certainly the video content must have been captured at loftier quality, but in the headset information technology appears very mushy. Why? The answer is resolution.

Resolution

The Gear VR can support Ultra High Definition (UHD)-size playback. In pixel terms, that UHD video has a frame size that is 3840 pixels wide x 1920 pixels tall. The video often plays dorsum at 30 frames per second (higher frame rates can be supported at lower resolution). Conspicuously the Galaxy S6 phone is a little powerhouse that is able to easily play 4K video. Just this turned out to be too depression a resolution to make a good-looking image.

To understand why, we need to answer ii different questions about resolution. What is the brandish resolution that nosotros are looking at on the Galaxy'southward screen? Let's phone call this screen resolution. The second question is, how much of that big UHD size image do I come across inside the headset? Allow'due south telephone call this the field-of-view resolution.

The screen resolution on the Milky way S6 is 2560 x 1440 pixels. So each eye gets 1280 ten 1440 pixels to view through 1 lens of the Gear VR.

[A brusque digression: Gear VR lenses misconstrue the image quite a bit, so the practical resolution is actually 1280 x 1280. Each centre gets to look at a 1280 10 1280 paradigm. Non bad, but not exactly "Full HD." But at over 500 pixels per inch, this should be plenty to make the image await precipitous.]

Let'south return to the field-of-view resolution. Recall that each frame of UHD video is 3840 x 1920 pixels, merely this video frame has to fill 360° of the horizontal view and 180° of the vertical view. On the Gear VR, the field of view is 96°—a bit over one-fourth of the full 360° view. When we view the epitome, we are merely looking at a piffling square section of the whole frame. As nosotros plow our head, the area of the video we are pointed at has to update, and the software shows just the small section we need to see from moment to moment every bit we turn our heads. That piddling foursquare is divers as the field of view.

The field of view is a small-scale part of the complete 360° video frame.

Simple math shows that each degree of view corresponds to 10.6667 pixels/degree (1920 pixels/180 degrees = 10.6667 pixels/degree). So multiply ten.6667 x 96 and yous get 1024.

So the image that we are shown when looking through the headset is 1024 ten 1024 pixels (at near), simply the size of the display is 1280 x 1280 pixels. In that location is our showtime clue to why the video looks soft. The little piece of video that the software cuts out of the frame is SMALLER than the display resolution. So the software has to stretch the video to map the field-of-view crop to the display resolution. This is done with a relatively low-quality scaler, since the Galaxy telephone has to also simultaneously decompress and warp the image to project it on the screen. The result is a little bit of softening. But a 20% scale shouldn't be that big of a trouble.
Stereo Makes it Worse

All of the brandish issues discussed so far are just related to "mono" video—where each eye sees the aforementioned image. To increase immersion, stereo display is preferred, where each eye sees an offset image to trick your brain into perceiving depth. On the Gear VR, stereo display can really boost the sense of realism. A stereo image just looks more than lifelike because the showtime provides depth cues that aid to split up objects in a scene. Some 3D games on the Gear VR can testify the virtual surroundings of the game in stereo, calculation to the realism.

Each eye should ideally get a full-resolution 360° view. That would crave that we double the resolution in i dimension, for example vertically, to have separate images for each middle.

If regular UHD video is 3840 x 1920, a frame that is twice equally tall would be a giant 3840 x 3840, which is besides big for the hardware to back up. Instead, we must pack this super-tall video into the smaller 3840 x 1920 frame. The left and correct eye share the frame, and each eye gets a 3840 ten 960 pixel image that covers the full spherical view horizontally, but must be stretched vertically to cover the full view.

A stereo image with top lesser views packed into 3840 x1920 pixels. The field of view is squeezed vertically by l%. Prototype Credit Bob Dass – Flickr.

The result of packing two image views into a single frame is a much smaller resolution for each middle. Scaling 20% is not great. Merely scaling 150% (512 pixels vertically stretched to 1280 display pixels) is going to consequence in a big perceived loss of resolution. Whether the format is packed top/lesser or left/right in the frame, you lot lose half the resolution.

Smaller and smaller streams

When we look at streaming video, the situation gets much worse. Services like YouTube 360° and others stream video in a lower resolution than 4K because few viewers take the bandwidth to accommodate UHD video. Because the video is being played from a server on the internet and not locally on the device, the services reduce the resolution to maintain a smooth streaming feel. Typically the epitome is streamed in a 2048 x 1024 frame size. Using the same math to a higher place, the field-of-view resolution works out to 540 x 540 pixels for a YouTube size frame. Streaming Hard disk-resolution 360° video really does effect in a resolution comparable to old television (720 x 480 or 720 x 576) and it must exist stretched more than 230% to fit the brandish. That is why streaming stereo VR video looks so soft.

Much bigger is improve

For electric current displays similar the Galaxy S6 (or fifty-fifty the forthcoming Oculus Rift), a field-of-view resolution that is slightly larger than the display resolution would help. Scaling downward a slightly larger prototype will make the image look smoother and will reduce distracting artifacts such every bit aliasing and racket.

A resolution of 1536 x 1536 pixels would be a good solution.  This is well-nigh 17% bigger than the display, and in our tests it shows a pregnant improvement in the experience. Using the math from earlier, if nosotros want to cover a 96° field of view with 1536 pixels, we need a resolution of 16 pixels/degree. Multiply that past 360° x 180°, and you get a resolution of 5760 x 2880. And so it turns out we need images that are nearly 6000 pixels beyond to get better coverage for mono video.

What about stereo? Call back that for stereo, we ideally want a full resolution image for each eye (or a clever manner to pack left and correct together). So a rough measure would mean sending video files that are at least 5760 pixels broad past 5760 pixels tall. That works out to 33 megapixels per frame. Even with next-generation gigabit or 5G wireless networks and better pinch, streaming 33-megapixel videos to a headset is going to be a large challenge.

The Pixvana Organisation

At Pixvana, we think there are other clever ways to solve these resolution and streaming problems. Information technology's possible to create streams that are closer to Hard disk resolution to permit the viewer to experience VR video at an platonic resolution. Facebook is trying an arroyo for their own platform that tries to minimize the bandwidth and increase the quality by focusing the quality on the front view with Dynamic Streaming. You lot tin can read a chip more on their coding blog.

Nosotros are investigating methods for covering the field of view with greater resolution. 1 strategy is to cover the current field of view with higher-resolution imagery and add actress padding to account for head motion. Switching between multiple streams as the viewer turns could generate perfect coverage for headsets like the Gear VR (or the PC-tethered systems—HTC Vive, or the Oculus Rift), or even future headsets that will offer much college brandish resolution. A improve approach might be to have multiple means to pack and encode the data based on content. Video shot in a closed studio surroundings with a fixed camera could have vastly unlike encoding from a VR video shot from a moving vehicle, for example.

Pixvana intends to build an open system that gives content creators all the tools they need for creating beautiful loftier-resolution VR video. Such a system would be responsive to changes in headset position, changes in bandwidth, and headset display characteristics, and even so deliver the right image for the viewer at any moment. The Pixvana organization volition work on a variety of platforms, from mobile to PC. With our system, future video streamed to the Gear VR will be significantly amend than the video nosotros view now, and VR video will accomplish the immersion we desire.

fowleryousiolind.blogspot.com

Source: https://vrscout.com/news/vr-video-look-soft/

Share this post