Space making the virtual a reality

The answer is virtual and augmented reality, or VR and AR.

Usually associated with video games, this ultra-immersive technology is quickly being recognised for its real-world potential.

VR is a computer-generated environment that simulates physical presence and interaction, artificially creating sensory experiences like sight, hearing, touch, and smell.

Today, ESA’s astronauts use it to help them prepare for life on the International Space Station, practicing space walks and operating payloads in zero-gravity – all while never leaving the ground.

In comparison, AR actively works with the user’s circumstances; superimposing digital information directly on real objects, settings and maps and allowing people to process the digital and physical simultaneously, improving their ability to absorb it, make decisions and act more quickly.

It was used to great effect in the mobile app game Pokémon GO, but is also creeping into more practical applications, such as projecting information onto car windshields directly in the driver’s field of vision.

As our world becomes increasingly digitalised, both technologies’ ability to transform how the user perceives their surroundings is attracting growing interest, and ESA is examining how space can help.

Reality augmented with space

With satellite communications, Earth observation, and navigation data, VR and AR experiences can be extended beyond the reach of terrestrial networks, feature up-to-the-minute live mapping and provide specific location-based information.

For example, the ESA-supported CapeOutdoors3D, a 3D virtual reality mobile app.

It overlays Sentinel-2 maps of South Africa with both location-tracked recreational information for tourists, like hiking trails and weather information, and fire risk spatial data for wildlife park management.

Both are important features for the popular but wildfire-plagued Western Capetown, which can have limited terrestrial network coverage. The app circumvents this, showing the very latest satellite-generated information on top of a 3D, HD and realistic representation of the landscape, regardless of when or where it is accessed.

Another ESA co-funded project, Wildwego, takes a similar approach, with the purpose of providing visitors to national parks with a more engaging experience.

It intends to encourage a greater appreciation for nature conservation with an AR-based game that offers information about the flora and fauna surrounding the player as they travel through the park, with the option to share the experience with friends.

The healthcare applications of this technology also look like they can be greatly enhanced by space.

Telesurgery fuses both telemedicine and AR technologies, enabling surgeons to operate on patients remotely.

It uses a combination of advanced technologies to precisely track the surgeon’s movements, a realistic and realtime VR environment display for the physician, and telecommunications to transmit the data between the two locations.

AR can even make the patient’s veins’ easier to locate, by converting the circulatory system’s heat signature into an image and superimposing it onto the skin.

Additionally, VR’s training uses are of course not limited just to astronauts. VR can also be used to help organisations prepare for dangerous situations and practice how to respond, such as emergency response teams, or the MIDAS demining simulator, which was developed with ESA backing.

For these, various data sets can be integrated, overlaid on satellite imagery and georeferenced for a true-to-life exercise in which various scenarios and potential mitigating actions can be tested and rehearsed in a safe environment, without any danger to life.

Space bringing VR/AR closer

It goes without saying that – regardless of whether a VR/AR application uses satellite information – this is still a nascent technology.

Nick Appleyard, Head of ESA’s Business Applications department, said: “The VR/AR market is estimated to reach $150 billion by 2020, with VR representing around $30 billion and AR $120 billion. Both the consumer and the professional market look promising and capable of flourishing if offered the right products and services in a technically-supported environment.

“We think space can add a great deal of value here, both in terms of helping industry capitalise on the business opportunities, and helping the technology and its positive applications for people develop further. Our Business Application programme is offering support to companies for business case assessment and development of space-enabled services based on virtual and augmented reality applications.”

Another reason these applications are not in widespread use now is that current 4G networks lack sufficient capacity, responsiveness and coverage.

Here, too, ESA is looking into how space can assist.

The next generation of data services, 5G, can offer more capacity, security and lower latency, and with the help of satellites, can spread the always-on, ultra-fast connectivity across the globe, supplementing terrestrial networks.

Antonio Franchi, Head of ESA’s Satellite for 5G and Future Programmes Acquisition, said: “Satellite services are becoming increasingly vital in all aspects of daily life, and part of the digitalisation of industry and society.

“5G will change the way we communicate, work and interact with technology, and space has an invaluable role to play, which is why we are working with industrial partners to develop market solutions and support space-enabled 5G trials and service demonstrations to help realise the technology’s potential and the many applications it will enable.”