Earth Server 2: Big Earth Data Analytics at your fingertips

,  Jacobs University is at the helm of the EU-funded Earth Server project which unites partners* from Greece, the United Kingdom, Italy, the USA, Australia and Germany. With an overall volume of €2.84m supported by the European Union’s H2020 program, the project – now in its second phase – aims at making gigantic Big Data sets generated from Earth Science observation more readily and easily available to a wide range of users across different platforms by using and also establishing standardized methods. At their recent kick-off meeting in Munich, the global consortium of researchers has established the new framework for the second phase of Earth Server project which will run over three years until April 2018. Making use of the award-winning »rasdaman« technology developed at Jacobs University under the lead of Peter Baumann, Professor of Computer Science – the scientists have set themselves a number of ambitious goals with the all-encompassing vision of eventually having “Big Earth Data at your fingertips”. Their plan is to continue work on the “disruptive paradigm shift in technology”, as reviewers of the first phase characterized rasdaman. This will see isolated silos of data with disparate functionality moving towards a single, uniform information space.  Furthermore, the current difficult, artificial differentiation between data and metadata access is to make way for a system of unified retrieval with zillions of files being integrated into just a few datacubes each between a dozen Terabyte and over a Petabyte in size. Prof. Baumann explains: “With our intercontinental initiative we intend to move away from today’s limited functionality of communication between servers to the freedom of asking anything from any server in a peer network of data centers worldwide at any given time.” This latest stage of the EarthServer project is based on the solid footing of the successful first phase of the undertaking, which followed a two-pronged approach.  Firstly, Earth Server 1 established open ad-hoc analytics of massive geoscientific and planetary data by employing the “rasdaman” array database, which enables the storage and processing of multi-dimensional arrays ("raster data") of unlimited size with a high-level, user-oriented query language. The technology can visualize and analyze four-dimensional climate data and also help in the exact location of wildfires or ocean currents. One-dimensional series of measurements, two-dimensional satellite images, three-dimensional data from ocean research and four-dimensional climate data can be selected, connected and analyzed on request. Secondly, the researchers significantly advanced the worldwide standardization and interoperability of Big Earth Data by working closely with international associations such as the Open Geospatial Consortium (OGC) and the International Organization for Standardization (ISO). “The continued financial support from the EU for Earth Server shows that within the first stage of the project we have already significantly changed the way in which scientists from different fields of Earth Science will be able to access and use data,” says  Dr. Angelo Pio Rossi, coordinator of the EarthServer-2 project. “We have now set out to redefine the Big Data service landscape even more providing real-time scaling of Petabyte data cubes, and intercontinental fusion. This power of data handling will be wrapped into direct visual interaction based on multi-dimensional visualization techniques, in particular the open source virtual globe, NASA WorldWind.” * Partners in the EU-funded Earth Server 2 project:Jacobs University (Germany; project coordination) CITE s.a. (Greece) European Centre for Medium-Range Weather Forecast (UK/intl.) Meteorological Environmental Earth Observation S.R.L. (Italy) NASA Ames Research Center (USA) National Computational Infrastructure (Australia) Plymouth Marine Laboratory (UK) rasdaman GmbH (Germany)  ====================================================# Data Source:Model results from ECMWF, MACC-AOD, http://www.copernicus-atmosphere.euMonitoring Atmospheric Composition & Climate (now Copernicus Atmosphere Monitoring Service) - Forecast of Aerosol Optical Depth # Provived by:MEEO (http://www.meeo.it)Earth Observation Data Service (http://earthserver.services.meeo.it) # Description:Aerosol Species: desert dust (red), biomass burning (green), sea salt (blue), sulphates (white) # Background:NASA Blue Marble - http://visibleearth.nasa.gov/view.php?id=74518====================================================