European platform for Earth Observation data processing
CREODIAS is a public computing cloud, seamlessly integrated with an Earth Observation data repository, augmented with Earth Observation focused applications for browsing, selecting and processing Earth Observation products. Its design grants third party users an easy access to a wide selection of data with on-site processing and guarantees simplicity, scalability and repeatability of user's applications and services.
In 2017 the European Space Agency (ESA) – an intergovernmental organization of 22 member states and a major stakeholder of space industry in Europe – acting on behalf of the European Commission signed DIAS contracts, as a part of Copernicus initiative.
Copernicus is a major GEO intelligence and Earth Observation data provider, offering full, free and open data, managed by European Commission in partnership with ESA. It creates new opportunities and poses complex logistical challenges in utilizing the available data. Previously, value chains taking advantage of EO data were inaccessible to many stakeholders due to high infrastructural and manpower costs and excessive demand for storage and computing resources.
In order to overcome those challenges in a big data-enabled environment the Commission decided to launch Copernicus Data and Information Access Services. The main goal was to make it possible to use Copernicus data and information within production chains without the need of extensive on-site infrastructure and expertise. CloudFerro, as a major participant in a consortium, has been selected to develop and manage its own DIAS, known as CREODIAS.
What is CREODIAS
CREODIAS is a big data-enabled platform designed for Earth Observation data processing, utilizing EO data storage cluster and a dedicated IaaS cloud infrastructure for the platform’s Users. Earth Observation data within the repository as well as the tools are free, full and publicly available for everyone. Some actions require signing in.
The main concept behind CREODIAS is to lower the entry level for entities pursuing business opportunities in EO data processing sector. Financial, infrastructural, network and personal requirements confronting vast quantities of potential stakeholders have been a decisive deterrent until now. Bringing processing capabilities to the data, utilizing flexible open-source technology allows Third Party Users to research value chains and prototype and build products and applications at a fraction of the cost (manpower, financial and time). As a result of CloudFerro's successful deployment of the platform:
- Copernicus data and information is easily available to all stakeholders.
- Data is collected and indexed within flexible, seamless, open source-based environment.
- On demand production chain scaling is easy, fast and cost-effective.
- Experienced and passionate staff, helps users through the help desk.
- Variety fo platforms, applications and third aprty services is integrated with the CREODIAS to bolster user's capabilities.
CloudFerro has developed the following applications as the cornerstones for users’ activities within the CREODIAS environment, which makes the value chain management much easier:
- CREODIAS Portal - the main hub for account management and information handling, with FAQs, knowledgebase, forum and news feed available.
- Horizon Dashboard - Horizon Dashboard is an OpenStack web-based graphical user interface dashboard.
- CREODIAS Finder - Finder is designed for searching for specific Earth Observation products, either using Finder graphical interface or Finder JSON-based API.
- CREODIAS Browser - Browser is tool for visualization of different EO products, on different processing levels. It provides users’ capability to perform semantic searches, using SPARQL.
- CREODIAS Jupyter Notebooks - Jupyter Notebook is an online, web-based programming environment, seamlessly connected with Earth Observation data repository. Perfect platform for prototyping and developing new ideas for algorithms and services. Available free of charge for registered Creodias users, it supports multiple programming languages.
External applications – created and managed by third party users – are supplementing CREODIAS environment. When user needs are highly specific and complex, but they may lack resources to proceed with value chain on their own. Ever growing set of external application and platforms operating on and with CREODIAS platform, may grant solution and add required pace. Application marketplace is envisioned in the future, drawing stakeholders closer to each other.
Cloud infrastructure behind CREODIAS platform operates on open-source Openstack software, that consists of multiple interrelated components that control diverse, multi-vendor hardware pools of processing, storage and networking resources. Users can either manage it through web-based dashboard, through command-line interface or RESTful web services. The infrastructure and services offered are optimized for Earth Observation data handling, acquiring and processing. Optimization supports scientific, research and commercial application developed by Third Party Users.
The Earth Observation data repository can be accessed using multiple interfaces including S3 object interface compatible with Amazon S3 standard, REST API or standard POSIX-type access. Solution for data discovery, processing and indexing is part of the CREODIAS platform. Data discovery is made easy via CREODIAS Browser and CREODIAS Finder.
CREODIAS data repository is covering all the base products from the ESA’s Sentinel satellite family – full repositories of Sentinel 1, 2, 3 and 5p. Historical archive of Envisat/Meris, Landsats 5 and 7 and rolling archive of Lansat 8 are stored locally. All collections from four of the Copernicus services – Atmosphere, Emergency, Land and Marine – are available as well.