Handling ‘big data’ is no small feat

Data from Earth observing satellites and other remote sensing technologies are growing in volume and diversity at an exceptionally fast rate. This poses challenges and opportunities for their quick access, stewardship and applications. 

Diversity of space data, the combined use of data from different space missions and the fusion of satellite data with non-space data lead to new types of user applications. This affects the way such data are collected, processed, delivered and preserved.

These new combinations of data, coupled with today’s latest technologies, open new opportunities for their handling and exploitation, and for collaboration among scientists.

Not all applications require expert understanding of Earth observation data. There is an increasing need to serve a wider user community looking to retrieve simple, understandable and immediate information from remotely sensed data that have undergone complex processes and analyses.

To explore this idea, some 250 science, industry and policy-making representatives and national delegates from Europe, the US, Australia, China and Africa met at ESA’s ESRIN centre in Frascati, Italy last week for ESA’s first ‘Big Data from Space’ event.

Over 50 presentations during the three-day conference stimulated discussion between the different communities involved in the business of providing and manipulating very large-scale data and complex analyses of satellite and in situ Earth observations. 

Selected talks covered aspects like the challenges of data access, policies for their dissemination, data capture, search, sharing, transfer, mining, analysis, fusion and visualisation.

A broad range of application domains was discussed, including situational awareness, maritime and land cover monitoring, oceanography, climatology, meteorology and geology.

Representatives from ESA and NASA opened the event together with the European Commission. European Commission Directorates-General for Enterprise and Industry, Research and Innovation and Communications Networks, Content and Technology, along with representatives from the European Environment Agency, the National Oceanic and Atmospheric Administration and the Open Geospatial Consortium acted as session chairs. 

The event concluded with a strong call by all parties for the ability to handle and use big Earth observing data. This could potentially open new opportunities for research and international cooperation schemes such as programmatic and industrial coordination.

There was also unanimous support to promote the development of processing capabilities, and making data more accessible to users, complementing more traditional web service approaches.

The excellent feedback and contributions received during last week’s event will strongly influence ESA’s plans for managing future Earth observation data and form the basis for discussion among Earth observation data owners and suppliers.

The broad attendance and the high interest call for a follow-on event in the near future. Discussions on big data will continue at ESA’s Living Planet Symposium in September.