Virtual reality (VR) video is a popular trend in the virtual reality industry. Unlike ordinary videos, virtual reality videos totally immerse viewers in an artificial environment by rendering 3D images. Many people confuse virtual reality videos with 360-degree videos, but there’s a significant difference between them – namely, the fact that VR viewers can fully move around and explore inside their virtual environments. Virtual reality headsets like Oculus Rift and Oculus Gear VR allow for full interactivity between users and virtual environments. This article describes the experience of Apriorit in building an Android video player for Oculus Gear VR.
The popularity Google Cardboard VR video made an increasing number of projects paid attention to virtual reality. The reason for this was the variety of applications of virtual reality technology, including in media, tourism, education, healthcare, and entertainment.
For entertainment purposes, there’s a growing number of 360-degree videos that you can watch in VR mode on YouTube. Additionally, Google Street View has already made it possible to explore many historical destinations in VR mode. Some platforms offer live performances and educational videos in VR. Virtual reality games are also increasing in popularity among players. And that’s not to mention how useful VR has proven for assisting operators of complex equipment in fields like healthcare, construction, and aviation.
Consequently, there’s demand to create a virtual reality video app that meets all the requirements of the sophisticated VR user.
Software Developer in System Programming Team
Our developers got the task of making their own mobile 360 video player for Oculus Gear VR. These were the requirements:
- The Android video viewer should be optimized for Oculus VR.
- Google VR Video Player should be used.
- Three 360-degree videos, each recorded on a smartphone from different angles, should start playing at the same time.
- The viewer should be able to switch from one video to another seamlessly and continue watching from the same point (for instance, if the first video is at 9:31, the second should start at 9:31 immediately).
- The video player should have an additional UI element, for example a compass that shows the direction in which the user is looking. A compass is necessary to allow users to orient themselves and navigate in the virtual environment.
- A video that’s playing should have an overlay with a video number.
- It should be possible to switch between videos by swiping on the Gear VR controller.
The Google 360 Video Player allows you to play 360-degree videos and is suitable for many VR devices including Oculus Gear VR.
We used VrWidgetView from the package
com.google.vr.sdk.widgets.video for playing our videos. However, we couldn’t easily modify the executable with standard methods. We also couldn’t display anything on top of the player since it runs in full screen mode on top of everything else.
The following methods are available for modifying the executable:
VR video player development involves displaying two instances of a 360-degree video at once – one for each eye. In order to achieve this, we need to start VR mode after our app runs. Then we need to overlay our UI elements on top of the video and hide unnecessary buttons.
There’s no API for calling VR mode, so we just find enterVrModeButton and activate it in the code of our app.
It’s also possible to enter VR mode using Java Reflection. We can do this by calling it manually with onClick(). The following function finds all the private variables we need:
After we enter VR mode (enterFullscreenButton.performClick), we need to display our UI elements (for instance, a compass) somewhere. However, this isn’t possible because of the fullscreen video player that’s always on top of the UI. In order to add UI elements, we need to find a parent UI element and place our view on it. Thus, look for the field which is called FullScreenDialog:
In addition, we need to hide the standard buttons: Settings, Exit VR mode, and About.
We’ve already found all these buttons in the method initPrivateFields(). Now, we need to hide them after entering VR mode by using the method setVisibility(). However, this doesn’t work, as the buttons do not appear immediately after switching to VR mode. We can solve this problem by creating a thread that will wait for a certain time and then hide these buttons. This can be achieved using the JavaRX library. For example:
Here, we wait three seconds in an additional thread (to avoid blocking the UI thread), then call the button-hiding function in the UI thread.
Anyone who wants to create a 360-degree video player will need to start with a good understanding of what 360-degree video is. A 360-degree video is also known as a spherical video. These videos have views in different directions, which is necessary for playback on a VR headset. There are two types of videos for VR players: stereoscopic 360-degree and monoscopic 360-degree.
A stereoscopic 360-degree video is recorded with a camera for each eye, achieving the perception of depth. However, this is quite hard to accomplish, as you need to unite the separate footage of two cameras and then create a side-by-side mapping for 3D video. The difficult structure of stereoscopic videos negatively impacts their resolution. Thus, the maximum resolution for this type of VR videos is 1920x960 at 30 frames per second (fps) or 1440x670 at 60 fps.
In contrast, a monoscopic 360-degree video is easier to produce, supports higher resolution, and is therefore usually chosen for creating VR content. A monoscopic video is filmed with one camera per field of view, and these videos are then combined to create a spherical video. To create a monoscopic video, you should have at least six cameras that simultaneously record six different fields of view. The maximum resolution for 2D monoscopic video on Gear VR is 3480x1920 at 30 fps or 2880x1340 at 60 fps.
In our case, we have a 360-degree monoscopic video.
Options.TYPE_MONO specifies that we have a monoscopic video and that the Oculus Gear video player should split it for two eyes with the image offset. If we choose Options.TYPE_STEREO_OVER_UNDER, then the player will consider this video stereoscopic and will play it without splitting. It’s very important to set the right option. Otherwise, the video will be displayed incorrectly.
Previously, we discussed how to add an overlay to our view (for example, an image or text):
Each type of video requires a special approach for creating the overlay view. For instance, if we have a stereoscopic video, we should do the following: create a layout, then place a null view in the center, then place the views that will be duplicated for each eye. For example:
If we have a monoscopic video, then it’s better to place one image. If we add two images, the final picture may get blurry. Also, we made sure to add transparency to our overlay image so it doesn’t look like a sticker.
To play a VR video on Gear VR, we need to use the Gear VR SDK and a device-specific key generated on the Oculus Store for a single device. You can find the SDK for Gear VR and examples of basic apps for Gear VR on GitHub. If you need clear instructions on how to generate a key on the Oculus Store, click here.
Below, you can see the base process for running your app on the GearVR.AndroidManifest:
Run GVRActivity for easy integration with a VR headset:
To run the app in Gear VR mode, we need to opt for setMain(main) in VrVideoActivity, where main is the following:
Then we need to overlay our player on top of GVRActivity. The standard method setContentView() doesn’t work, but GVRActivity has another method, registerView(view), which we can use for placing our overlay view:
The ability to scroll video with the help of the Gear VR sensor was another requirement of our task to create an Android 360 video app. We discovered that in order to do this, there’s no need to use any additional interface for the Gear VR sensor. We can simply add SwipeGestureDetector to the highest view, which is just like swiping the screen of the device with your finger.
In this article, we’ve shared our experience developing a VR video app for Oculus Gear VR. We’ve also discussed key problems that developers may face while working on similar projects. If you’re considering developing your own virtual reality video app, our Apriorit team can assist you with our knowledge of video delivery and mobile development.