Help us boost human-machine interfaces to make science fiction a reality

By enhancing other space technologies, XR could improve how we build spacecraft, train astronauts, and operate distant missions. XR usually refers to virtual or augmented reality, but there are many other ways that humans can interact with computers. This call is open to all of them.

"Right now, XR technologies – especially augmented reality – are seeing lots of hype, and more people are working on them," explains ESA engineer David Martinez Oliveira, who is leading the search for ideas. "The hardware has reached a stage where we can use it for new applications that were previously not possible. We are launching this call for ideas now to take advantage of this push."

Until now, XR development has focused mainly on sight and hearing. Humanity has come a long way in digitally simulating these two senses: think smart glasses, realistic-looking virtual worlds and the sound of footsteps sneaking up behind you through a VR headset. But there are three more senses to be explored.

XR touch (or haptics), for example, could be useful for training astronauts to use machinery that does not yet exist. XR smell could be interesting for triggering memories – combining it with visuals and sound could help astronauts confined in a small spacecraft enter a convincing virtual Earth-like landscape.

"We are looking for really novel ideas for activities and projects that can help us integrate and adopt XR technologies in the space sector," continues David. "We want ideas for new technologies to be developed, but also suggestions for new space applications of existing technologies."

So how else could XR be used in the space sector?

"XR technology is a 'supporter' or 'enabler' technology – rather than solving problems, it enriches existing technologies and enhances existing tools," explains David. "There are many potential applications. It will enable new digital engineering concepts, enhance how we explore and interact with data, and support the assembly, integration and testing of spacecraft."

Currently, we see four key applications of XR: maintenance, training, digital design and telepresence.

As a prime example of maintenance, astronauts and engineers could get instructions for fixing and building equipment through AR. Training on the other hand, could be supported by VR so that astronauts can learn how to carry out tasks in advance. XR could be used to design equipment. And the benefits of telepresence are easy to imagine in our world where remote working is becoming the norm.

David explains where we're at right now: "XR for space is currently very much in the R&D phase. ESA has been exploring how it can be used by astronauts on the ISS, for example via the development of a procedure viewer. The Agency has also started working on using it to train astronauts. For this, a big challenge is the 'authoring' of XR content; somebody has to produce the content, which can be complicated. It's one of the main hurdles that we’re looking to overcome."

David concludes: "OSIP allows us to open this call for ideas to everyone: researchers, industry partners, even the public. This makes it an easy way for experts or anybody to submit their ideas and for us to engage with these ideas in one place. The platform also encourages a more dynamic and transparent way of interacting with ESA experts at the idea stage."

This call for ideas will be open until 4 November 2021. Ideas will then be evaluated and the people behind the best ones will be invited to mature their ideas into proposals for research projects, studies or early technology development projects.

This call for ideas is run by the Discovery element of ESA's Basic Activities, ESA's first contact point for promising new concepts.