Copernicus Data Space Ecosystem explained
CloudFerro, together with its partners, has been selected by ESA as an industrial team to implement and operate Copernicus Data Space Ecosystem. The goal is to provide full spatial and temporal Copernicus EO data coverage available immediately and free for the users.
On Tuesday, 24th January 2022, the new Copernicus Data Space Ecosystem was launched. The service is taking the European Union Copernicus programme to the next level, ensuring the data make the greatest possible impact on users, all the citizens, and – ultimately – on our planet. The new Copernicus Data Space Ecosystem builds on existing data distribution services (incl. CREODIAS and others), ensuring their continuity and introducing significant improvements.
Evolution of Copernicus data access
The European Union Copernicus programme is among the biggest providers of Earth Observation data in the world and its launch was one of the most important events in the remote sensing community over the past decade. With systematic monitoring over large areas, good quality of data, resolution fitting its objectives, ensured the longevity of the program, and, most importantly, a clear and simple open data policy it took the world by storm. This, however, is not something to be taken for granted. When Copernicus started, remote sensing was a niche field occupying mostly researchers and the intelligence community. There was no specific reason to expect that it will be any different with Sentinel data. But it happened.
The data was picked by enthusiasts, then by companies and institutions. Fast forward a couple of years and suddenly there are hundreds of applications helping farmers to better manage their fields, financial markets forecast prices of corn, journalists regularly use it to observe or validate news, the European Common Agriculture Policy relies on its data to monitor practically all agriculture fields for sustainable agriculture practices, researchers are building digital twins of the environment, security organizations predict migration patterns... There are many more examples.
Nobody expected Copernicus data to start such a revolution in this field. It was (and still is) a tremendous validation of ESA's operational capacity, but at the same time also significant pressure on the data distribution systems as they were not expecting such uptake. Still, the Copernicus Data Hubs have already distributed an order of magnitude more data than is published. With user uptake growing further, as well as the recognized importance of Earth Observation data for the monitoring of climate change, the EU decided to invest in the next level of user data processing and distribution infrastructure. The new Copernicus Data Space Ecosystem was born.
The future of European user data processing and distribution is powered by experienced players - T-Systems and CloudFerro, with their well-used cloud infrastructure, Sinergise and VITO with Sentinel Hub and OpenEO data discovery and processing tools, and DLR, ACRI-ST, and RHEA taking care of on-demand processing and Copernicus Contributing Missions.
What does it mean to CREODIAS users?
This is CloudFerro long-term commitment to grow and maintain the Copernicus EO data repository available in CREODIAS. Within the next few years, one of the largest online repositories worldwide will grow from 35PB to over 85 PB of free and immediately available EO data. Data from all over the world is available directly from the cloud through efficient, cloud-native access mechanisms (both API and GUI). Data is available for current users, new users, and federated platforms.
Copernicus Data Space Ecosystem - what’s in it for users?
- The data offer available for the user will increase. The data repository includes full up-to-date Copernicus data as well as complimentary commercial data offering (EO data+)
- Data that can be processed directly in a cloud environment
- Expanded search and visualisation of the data with Jupyter notebooks for testing and development
- Federation with other EO services – Sentinel Hub, OpenEO, JupyterHub, and others
- Serverless, on-demand processing services – for Sentinel products and other data
- Many services will be free of charge with easy access to a payable but easily scalable extension
Data offer
First and foremost, it's about the data we guarantee - instant data availability. Just about all the data ever acquired by Sentinel satellites, with minor exceptions, will be available online, instantaneously including e.g. Sentinel-1 SLC and GRD and L2 OCN, Sentinel-2 L1C, and L2A (reprocessed to Collection 1, as reprocessed data becomes gradually available), Sentinel-3 and Sentinel-5P L1 and L2 data as well as Copernicus Contributing Missions data. Full archive, and always up-to-date.
There is currently no infrastructure where all of this would be available in one place. The new Copernicus Data Space Ecosystem will be it. Data will be available through various interfaces – from old-fashioned download, direct S3 access with STAC items, and cloud-optimized formats to streamlined access APIs, which are able not just to fetch the data but also to process it.
EO services
There is a web-based application built on top of a very popular EO Browser technology to allow for data visualisation and as a user interface to access the data. Several on-demand processors are capable of building non-default formats and derived products, such as Sentinel-1 coherence and CARD4L products, MAJA-powered atmospheric processing, and similar. The data can be accessed via the openEO and JupyterHub for serverless processing. Last but not least, there will be a special focus put on traceability. For all data managed within the Copernicus Data Space Ecosystem, it will be possible to trace where it came from and what operations were performed on it.
Very important – the vast majority of these capabilities will be available free for use, funded by the European Union. The quotas designated for users should be more than sufficient for the individual's use - personal, research, or commercial. For those interested in the larger-scale operation, there will be practically unlimited resources available under commercial terms in CREODIAS allowing them to build applications on a world scale. Furthermore, there will be significant credits made available, in the form of extra free resources, to be used for research and pre-commercial exploitation opportunities.
In addition to that, users will be able to add and disseminate their collections and processors. What is more, there will be a set of additional services available for the CREODIAS users, such as streamlined access to data, and on-demand processing.
It is all about timing, responsibilities, and commitments
The initiative has quite an intense phase-in plan, in order to allow hundreds of thousands of existing users of the Copernicus Data Hubs to migrate their workflows to the new service. First, a limited but stable, roll-out has just happened (24th January 2023) with continuous upgrades over the upcoming months until full service will be made available by the end of June 2023.
This is a very challenging yet super exciting opportunity for everyone in the remote sensing community - from beginners to experts, from researchers, companies, and institutions, as well as individuals. Just about anyone can benefit from being aware of what is happening with our planet. And we should all take interest in it, and act. For the consortium partners, however, this is especially important - we were given an opportunity to build a new ecosystem for everyone to use. An ecosystem that strives to provide a significant upgrade over existing tools and services and is open to welcoming new partners and service providers in the future! This comes with huge responsibility towards everyone - not just to the ESA and European Commission, but for the worldwide EO community. We are confident we can execute it, so stay tuned for further information.
Discover Copernicus Data Space Ecosystem.
Read more in our news and case studies.
How can satellite remote sensing help analyse forests?
Forests cover 31 percent of the world's land surface, that's over 4 billion hectares[1]. According to the research published in Nature Climate Change in 2021, forests are a natural ’carbon sink’ that absorbs a net 7.6 billion metric tonnes of CO2 per year. The degradation of timberland around the world has prompted scientists not only to seek ways to halt these processes but also to develop objective diagnostic methods to assess the status and changes occurring in forests. After all, there is a lot at stake – as the United Nation's Global Forest Goals Report 2021 estimates, 1.6 billion people worldwide depend directly on forests for food, shelter, energy, medicines, and income[2].
The development of technology and the integration of satellite data with environmental models have given us the ability to observe the Earth on a large scale. One of the tools used to monitor the state of the environment, especially complex ecosystems such as forests, is satellite remote sensing.
How satellites for remote sensing work
Satellite remote sensing includes a whole set of procedures involving the acquisition, processing, and interpretation of satellite images, from which spatial information is extracted. Images of a given area can be taken regularly with successive satellite overpasses, which makes it possible to observe changes that have occurred in each area even over a long period of time. These methods allow us to observe changes in land cover, analyse the condition of vegetation over large areas, or monitor the extent of natural disasters, such as droughts, floods, and others. Moreover, remote sensing methods effectively reduce costs and research time. The CREODIAS platform, for example, has more than 30 PB Copernicus EO data available free of charge and off-the-shelf. This platform and similar platforms allow for data to be processed in the cloud without the need for time-consuming downloads to local computers or arrays. Moreover, the public cloud for their processing that CloudFerro provides is a much more cost-effective solution than having infrastructure on one's own.
Observing forests from space
Applying remote sensing techniques to monitor such complex ecosystems as forests can be a laborious undertaking. The complexity lies mainly in the multi-layered structure of forests, their species composition, and spatial variability. Therefore, when analysing images from several months, we can see completely different objects, despite the fact that the characteristics of the forest have remained unchanged.
However, this can bring some advantages to the process. For instance, if we dispose of multi-temporal data, we can better classify the species because their evolution during the growing season is specific. The growing popularity of remote sensing can also be attributed to its detail and comprehensiveness. Today, satellites are able to capture high-resolution images, making it possible to detect even individual trees.
The process of forest area analysis
Today, remote sensing is mainly used for ongoing monitoring and forest inventories. A satellite image is a unique way to do it on a global scale. Even at regional, national, or local scales, remote sensing is often the only way to obtain data on forests located in hard-to-reach places or dangerous regions of the world. Such data can be used by local forest management institutions and global environmental agencies to monitor the condition of the area, and manage environmental emergencies.
The first stage of forest monitoring is determining its spatial extent, where different remote sensing data, are used. This information, combined with the documentation on land use, allows us to create accurate maps, but also to indicate areas of succession.
Knowing the extent of forests, satellite data can also be used for forest analysis to:
- determine the size (volume) of the forest, i.e., the amount of biomass present in a given area. This allows us to estimate the amount of wood that can be harvested, as well as determine the amount of carbon in the ecosystem
- rapidly and accurately identify changes on the forest surface caused by wind, fires, floods, or sustained or human activities
- to map all the impacts of human activities, even in the most remote areas of the planet.
Processing forest data in a cloud environment
One of the most advanced forest analyses performed on a global scale is Forest Digital Twin Earth Precursor developed by the European Space Agency, as part of the Digital Twin Earth Precursor (DTEP) initiative. Its aim is to create a digital replica of all the forest ecosystems, which involves integrating complex environmental models with a vast amount of Earth observation data combined with artificial intelligence solutions. CloudFerro, the operator of the CREODIAS platform, was a partner in the Precursor phase of the Digital Twin Earth project, delivering technical expertise and resources.
Large-scale research projects require advanced competencies and vast IT resources because they need to collect, store and process large volumes of data in an easy, cost-effective, and timely manner. CREODIAS’ powerful cloud infrastructure and its large repository of current and historic Earth observation data greatly facilitate conducting this type of research helping meet computation and storage requirements for processing growing volumes of data.
For more on species classification of forests, go to an article by a forest expert published on CREODIAS.
[1] FAO. The State of the World’s Forests 2022. Forest pathways for green recovery and building inclusive, resilient and sustainable economies. Rome, FAO (2022). https://doi.org/10.4060/cb9360en
[2] United Nations Department of Economic and Social Affairs, United Nations Forum on Forests Secretariat. The Global Forest Goals Report (2021). https://doi.org/10.18356/9789214030515
Computing Cloud as a perfect environment for Big EO data repositories
Copernicus as a flagship European space programme
Copernicus is the European Union's Earth Observation (EO) programme managed by the European Commission. The main goal of the programme is to observe our planet and its environment to benefit all European citizens. The programme has revolutionized the European space industry by making access to sattelite data is easier than never before. The enormous amount of EO data available as a public good for anyone at no cost.Huge amount of archival data and the continuous growth of large EO data sets made it necessary not only to create a cloud environment, but also to develop the usability and exploit the potential of EO data - the DIAS platforms (Data and Information Access Services). For this reason, EO platforms were launched, thanks to which we have gained a powerful tool for EO data access and scalable processing.
DIAS and EO platforms
EO platforms are an essential element to enhance the EO data usability, making it possible to change data into information. The platforms provide users with a computing power that enables fast and efficient processing and allow users to access, process and analyse Copernicus products in the cloud directly connected to an EO archive. The availability of data, storage, and processing in a single place makes it easier to develop and distribute scalable digital services based on the EO data. Below you can see consumption model for DIAS and EO platforms.
We rely on an open- source technology
We have built and now operate several EO platforms based on a hybrid cloud in an Infrastructure-as-a-service model. Our services consist of open-source components, such as Open-Stack for cloud orchestration, Ceph for storage management, Prometheus and Graphana for monitoring, K8s for processing, and others.
EO platform as a service
Taking as an example CREODIAS, WEkEO, CODE-DE and other platforms built and operated by CloudFerro, we have created the main building blocks and architecture of entirely European cloud platforms based on open-source software.
We have managed to create a new service - an EO platform as a service.
Federation and multi- platform approach
Federation and a multi-platform approach allows users and operators to gain benefits from the synergy between different platforms, such as a shared multi PB data repository available from various platforms and elasticity cloud components for additional processing needs. At the same time, it generates a technical challenge that we need to address. We have met challenges focusing on three main topics:
- Interoperability of the environments: how to support easy migration between different clouds and provide direct access to the repositories available from different computing clouds, with multi-cloud processing enabled and dynamically adaptable usage and payment models.
- Data access and dissemination: the multi-petabyte, scalable EO data repositories should provide fast, instantaneous access to many heterogeneous users and algorithms with different needs. At the same time, the data needs to be discoverable and ready for analysis in a context of constantly evolving data offers (new collections, commercial data, products), and users should be supported in data dissemination.
- Cloud efficiency: how to optimize data access, storage, and processing costs and energy usage both for individual users and for the entire ecosystem.
Why is it so important to use EO platform
Using the EO Platform is connected with numerous benefits. The most significant are:
How satellite data can be used to improve comfort of city residents
The use of Earth Observation (EO) data contributes to smart city management helping to reduce harmful effects of urban heat island phenomenon and improve the quality of life.
The density of buildings and urban infrastructure causes changes in the microclimate of urban areas. This has a direct impact on the health and quality of life of the inhabitants. The solution may be Copernicus satellite data, which will allow for a better understanding of spatial phenomena occurring in densely populated places.
What is urban heat island phenomenon and how does it affect cities
Heatwaves boosted by urban heat island phenomenon are the most dangerous weather extremes, which not only damage our health, but also lead to the increase of energy consumption. To reduce or prevent their detrimental effects we need to track urban climate. Urban surface heat island (USHI) is an obvious consequence of urban processes and it is associated with the increase of temperature of urbanized areas in relation to undeveloped rural areas.
The phenomenon results from city morphology* and it is influenced by many factors. Even a single building can increase the ambient temperature. The accumulation of concrete buildings and urban infrastructure causes the temperature in the city to rise from 2 to even 8ºC, and in extreme cases, even above 12ºC. The strongest temperature amplitudes are observed on summer nights.
Urban Heat Island affects human health and energy consumption and that's why the issue of the urban heat island is extremely important in the management and planning of a space comfortable for residents to live.
The most important factors contributing to the urban heat island phenomenon are:
- The environmental context of city
- City size
- City topography
- Water resources
- Urban green areas
- Anthropogenic heat sources (e.g. air conditioning, cars)
* City morphology - structure of the inner and outer urban space and the origin of its parts. The spatial conditions of the city influence the scale of the urban heat island phenomenon. The amount of heat accumulated is determined by factors such as the city's environmental context, the number of inhabitants, the amount of greenery, the height of buildings, width of streets, the predominant color of the surface (albedo), the presence of ventilation corridors or even the roughness of materials used in the city.
The most dangerous weather extreme
Heatwaves are the most dangerous weather extremes that cause more casualties than hurricanes and floods combined. On the average, 1,500 American city dwellers die each year because of the heat, which is many more deaths than those caused by all other natural disasters combined. The compounding effect of heatwaves and UHI could prove to be even deadlier to urban populations, and are extremely dangerous especially for people who suffer from chronic diseases (such as asthma, heart condition), elderly people, manual workers, children and athletes.
Threats to cities caused by the urban heat island phenomenon:
- Health hazard for the elders and risk groups
- Higher energy consumption
- Intensification of convective processes
- Increased probability of heavy rainfall and greater exposure of cities to local flooding.
Land Surface Temperature Monitoring on CREODIAS
The CREODIAS platform allows users to search and download satellite scenes of the Sentinel, Landsat and MODIS programmes at no cost (repository of ready-to-use 30 PB of current and archive Earth observation data) which allows you to analyse the data in order to find the areas that are most endangered by heat waves and the urban heat island phenomenon.
Cloud workflow with EO data repository allows researchers to automate processes and calculate large amounts of data and changes over time. These operations combined with data repository are crucial in identifying critical areas for the microclimatic functioning of cities. Analysts can find spatial patterns to better understand the phenomena occurring in urban space, and consequently help local governments to take necessary steps to improve the quality of life of the residents. As we can see, spatial analyses are crucial for a better understanding of the urban climate.
The best solution - urban green areas
The easiest and most effective way to cool down urban climate is to introduce urban green areas. Vegetation contributes to the formation of shade, reflects solar radiation, and thanks to evapotranspiration processes, it effectively reduces the temperature in cities. Urban vegetation also contributes to the reduction of air and water pollution, reduces noise and accumulates large amounts of water, creating opportunities to reduce energy consumption.
Copernicus data can be applied to analyse urban climate and improve urban greenery in the most neglected areas by new investments in green infrastructure, for instance:
- Planting street greenery (stormwater street design)
- Protecting ecological urban areas
- Green rooftops
- Relocation of industrial areas outside the city
The extensive usage of EO data provides faster and more sustainable economic growth through better use of resources. This is key to achieving sustainable development goals driven by space innovations.
Questions?
Should you have any questions, please contact us at communication@cloudferro.com
How does Horizon Europe support innovative EO projects?
Get funding for sustainable innovative solutions based on EO data, AI, machine learning and Big data analysis with Horizon Europe programme.
The start-up and technology sector in Europe is growing stronger, and the appetite for innovation is effectively fuelled by grants from the European Union. Horizon Europe programme has a budget of almost 95 billion euros to distribute until 2027, and the main areas of financing include climate protection, energy transformation, development of the digital sector, and artificial intelligence.
Horizon Europe focuses on progress through innovation
In a webinar about technologies that support the development of innovative solutions based on EO data, Chiara Solimini from EUSPA (European Union Agency for the Space Programme) indicated key areas of interest for building applications and advised on the use of the synergy of data and technologies. “The proposals to the Horizon Europe programme should focus on the development of innovative EGNSS and Copernicus applications that support the implementation of the European Green Deal objectives and its related policies. They should use the synergy of the Copernicus data or products, EGNSS differentiators and advanced technologies such as blockchain, internet of things or 5G applications ” – said Chiara Solimini, Space Downstream Market Officer at EUSPA.
Innovative applications based on massive amounts of satellite data, using Big Data, artificial intelligence or machine learning technologies can only be developed with fast and reliable online access to a rich data repository. This is provided by the EU Copernicus Earth Observation Programme that provide users with free and open access to Earth Observation (EO) data and information services, such as monitoring of the atmosphere, marine environment, land areas, climate change, security or crisis management.
Research with Copernicus EO data on CREODIAS
Satellite data analysis is far easier with free and open access to the Copernicus EO data. This is provided by CREODIAS platform, built and operated by CloudFerro for the European Space Agency. The platform offers cloud computing resources with an integrated powerful repository of imagery captured by Sentinel, Envisat and Landsat satellites as well as other Earth observation data.
The CREODIAS infrastructure is adequately prepared to process large Earth observation data sets in a cloud computing environment, which are available online immediately. Cloud computing resources are scalable and can be adapted to user needs depending on the size and timing of their project. Tools available on the platform allow users to design prototypes, as well as build their own services and products and tailor all the relevant parameters of the cloud environment to specific topics, data types and project stages.
"We want to be a strong contributor to the Horizon Europe and European initiatives, and together build a better future for European society based on digital sovereignty and innovation. The CREODIAS repository currently stores more than 28 PB of Earth observation data. For our partners and the platform users, we have prepared cloud resources in different locations in Poland and Europe, including data centres with German BSI 200-1 and C5 certification, proving the highest quality and security level"
– says Przemysław Mujta, Sales Support Director at CloudFerro, the provider and operator of CREODIAS.
Horizon as a driver for innovations
Together EO platform tools and computing capabilities allow monitoring natural resources, precise irrigation, satellite monitoring of agriculture under the EU Common Agricultural Policy, automatic control, or pasture management. State-of-the-art technologies combined with satellite data and services also support renewable energy and its distribution. These include RES (renewable energy sources) assessment and forecasting, risk assessment for energy support equipment, power plant design optimization, power grid monitoring, or environmental impact assessment of power plants and mineral resources. Satellite data can form a basis for the analysis of snow cover maps for hydropower plants or biomass maps for bioenergy, determining sunshine areas for solar panels, or analyse wind speed for wind turbines – onshore and offshore. It is also interesting how satellite data can be applied for analysis of mining damages, the occurrence of urban heat islands, or study of bee activity to plan crop spraying.
The above mentioned examples are only a small part of a wide array of areas where innovations based on EO data can make a real contribution to climate neutrality and reduced consumption of natural resources in Europe. Any idea of using this technology can certainly be supported under Horizon Europe.
Who can join the programme?
Micro, small and medium-sized enterprises, and the so-called "small mid-cap" employing up to 500 people can benefit from Horizon Europe grants. Funding applications may be submitted by entities that develop a product, process or service of high risk, based on scientific discoveries, critical thinking or technological breakthroughs (so-called "deep tech")
Questions?
Should you have any questions, please contact us at communication@cloudferro.com
Why do we need European digital sovereignty?
The EU has attempted to regulate how industrial data can be shared and used to innovate for the better. In the past years we have seen an explosion of legislation and projects pursuing this dream. From the publication of the EU Data Strategy and the Data Governance Act or the launching of Gaia-X and a variety of upcoming sectoral data spaces there is a whirlwind of policy that one needs to keep track of to stay abreast. But that could all change…
European Alliance for Industrial Data and Cloud
It is rightly pointed out that the COVID-19 pandemic has brought about massive societal and economical shifts in how our societies function, furthermore exposing our deep reliance on digital infrastructure to conduct our daily lives. The surge in cyberattacks only highlights how precarious the foundations of our digital architecture are. As a result, we are witnessing a historic period for the tech industry as a whole. Competition is increasingly becoming a point of friction as governments worldwide are waking up to the realities that exist in the digital market.
It is amidst this backdrop that another milestone has been achieved by the European Commission. That is the launch of the European Alliance for Industrial Data and Cloud. An initiative that aims at further crystallising the ambitions set out in the EU’s Data Strategy when it comes to how Europe can boost its own cloud capacity.
CloudFerro is excited to be among the first participants of this initiative and we are eager to share our expertise, resources, and enthusiasm to build a strong, competitive, and resilient European data and cloud service. As a truly European company, we at CloudFerro are very pleased to see the Commission launch the European Alliance on Industrial Data and Cloud. This is the next step forward for the EU’s plans to be competitive and spur innovation across sectors while aspiring to build European digital sovereignty"
– says Maciej Krzyżanowski, CEO of CloudFerro.
Unblocking the Cloud Market with the Data Act
The publication of the Data Act provides a much-needed legislative push for the cloud sector to unlock Europe’s full competitiveness. As the file now falls under the responsibility of the Council and Parliament, it will be paramount that the political aspirations of the Data Act are not watered down and a strong emphasis on fairness and interoperability is maintained. We can only encourage the Commission to stand by its objectives and defend these principles during the negotiations of the co-legislatures.
Hearkening back to the Council of the European Union’s October 2020 declaration on cloud we can only echo that a common approach is needed in order to build Europe’s cloud supply, reinforce European digital sovereignty and increase competition.
The Alliance states in its declaration that one of its objectives is to provide support for defining common norms and requirements for cloud. We hope that amongst the Alliance we will be able to assist in providing the needed rationale and argumentation so such measures as interoperability and switching are encouraged and maintained among the wider cloud agenda and that policymakers do not lose sight of the bigger picture.
Fostering Common European Data Spaces and Norms
Integral to that bigger picture of a European sovereign cloud ecosystem, we see data governance as critical. While in December both European Parliament and Council concluded their trilogue negotiations on the Data Governance Act, it will take time for the effects of this Act to be fully established. What we know though is that, while the Act was reportedly ‘unloved’, it plays such a fundamental role in shaping the future framework for launching Europe into the era of cloud, that being the Common European Data Spaces. Only with the technical support of the cloud industry will these Common European Data Spaces manifest and be brought to scale in such a way that makes them viable.
It is our aim that the Alliance takes the lead among the variety of stakeholder formats in the conversation about cloud and data. Especially on operationalising the Alliance’s goal of creating a platform that can synergise with the Common European Data Spaces. Indeed the Alliance declaration notes how important close coordination will be on this topic with the European Data Innovation Board, which is established under the Data Governance Act.
But governance is only one piece of the puzzle, as with many great projects, this will take funding and investments. The format of the Alliance has the potential to open up so many sectors to each other and bring new investment opportunities to the forefront.
Earth Observation Data – a Call for Common European Data Spaces
Europe has a plethora of private and public organisations that can serve as a source for rich datasets. For example, at CloudFerro we work with the European Space Agency and the European Commission under the remit of the Copernicus program to provide our CREODIAS platform - which allows open, free and easy access to Earth Observation Data and its processing in the cloud. CREODIAS cloud services are sovereign and are based on open source technology. The platform allows third-parties to create their own applications and services to fit their needs and share them for the benefit of others.
Using our platform allows for a wide application of Earth Observation Data collected from the Copernicus satellites and other sources. What our platform is able to do is render and provide these images tailored to a variety of needs, together with an integrated cloud computing services to process this data. Need to monitor crop yield? How about water consumption? Or what about observing the changes in ice along the Alps? We have the possibility to analyse that, but what Europe needs now is an effective way to help synergize this analysis.
Our work with Copernicus has taught us that a rapid and effective set up of Common European Data Spaces – an ecosystem of public and private organisations, standards and technologies that enable shared value creation via the exchange of data and data-driven solutions – is key to exchanging data effortlessly between economic actors and creating value.
We would encourage you to see the results of our competition #SeizeTheBeautyOfOurPlanet - the incredible Earth Observation imagery demonstrating climate change, its negative effects on our planet, and the need to counteract them together.
Conclusion
At CloudFerro our expertise lies in providing specialized cloud services and in storing, indexing, delivering and processing big data volumes. Our key specialty is Earth Observation and other specialised domains such as meteorology and climate data. The wide range of specialised data needs our customers have means we as CloudFerro see Europe’s opportunity to provide a new way of cloud services, capitalising on the yet untouched realm of industrial data. The promise of Common European Data Spaces is close to becoming a reality, but it will take political courage and ambition to see it done.
As members of the Alliance on Industrial Data and Cloud, we whole heartedly call for more action and more participation on behalf of all cloud services so that Europe can develop its digital sovereignty in a way that is grounded and fair.
Questions?
Should you have any questions, please contact us at communication@cloudferro.com
Power of Large-scale data storage
Storage of European and global research industry
Large research projects require collecting, storing, processing and analysing huge amounts of data. Now thanks to the advancements in large data storage and access technologies, it is possible to provide large volumes of data in an instant manner. CloudFerro has carried out tests that proved its CREODIAS platform is now able to serve over 2PB daily.
Scientific exploration - new paradigm
The Fourth Paradigm of discovery is a concept that assumes the science and scientific exploration processes will be redefined by an open or increased access to data and new methods of sharing this data. In today’s world where we are overwhelmed by large amount of information, it is necessary to supplement classic scientific paradigms: observation, theory and simulation with the fourth one: large-scale Data Exploration. This will unify observation, theory and simulation in an extensive system.
Recent research projects and initiatives highlight the importance of making data, tools, technologies and platforms more accessible and easier to use. Initiatives like Digital Twin Earth, Human Brain Project, Digital Twin Manufacturing, and many others are frameworks for understanding, modelling and forecasting the behaviour of extremely complex systems. For such frameworks to work it is indispensable to store and manage huge amounts of heterogenous data and to make it available through unified, flexible, streamlined interfaces to multiple user communities.
Storing and dissemination of peta- or exabytes of heterogenous data in an open and flexible manner poses a serious technical challenge. Object storage provides a solution that is cost-effective, easily scalable and accessible. It allows for storing unstructured or extremely diversely structured data, thanks to the lack of hierarchical directory structure. Instead, object storage uses a unique identifier for each object. This convention and “flat” architecture allows also for massive, dynamic scaling of the storage where the scale is nearly infinite. The matter is to employ it in a real life, commercially viable scenario. Something that requires advanced cataloguing and network services altogether guaranteeing an automated, fast and easy access to data stored online for immediate use
The solution
CREODIAS, operated by CloudFerro, currently stores almost 21 PB of Earth Observation data. On average, it ingests 25 TB of data daily and disseminates it to more than 6 thousand registered users of the portal and countless non-registered ones.
Using opensource CEPH software for building storage – thanks to its ability to build and manage object storage for OpenStack – and an advanced cataloguing solution, CREODIAS platform can serve data inside and outside of its cloud via graphical application (CREODIAS Finder) and a variety of access interfaces. This machine2machine interfacing enables stakeholders to leverage the data in their processing chains in an automated manner, both on CREODIAS cloud and on any other infrastructure of their choice.
Why CloudFerro's cloud?
CloudFerro builds on expertise and lessons learned from our previous projects. This is why we are able to ingest, store, index and disseminate massive amounts of EO data, tens and hundreds of Petabytes.
- CloudFerro has recently conducted tests proving the ability to serve 2PB of data daily from our repositories. We are even able to double this rate.
- CloudFerro’s experience and technology proved excellent performance in benchmarking and tests results.
- We are currently storing in our cloud over 21 PB of EO data and it can grow to 50PB if necessary in the near future
- CloudFerro has gained broad experience in building and operating numerous large cloud platforms - Climate Data Store, CODE-DE, WEkEO, EO IPT and others - with a combined storage of over 100PB.
All the above allows us to operate at a scale required by such complex projects as Destination Earth.
With our services, our clients can have an easy, remote, broadband and scalable access to online, granular data in a cost-efficient manner. Those are vital capabilities when the fourth paradigm is in force.
