Enhancing Earth Observation capabilities - satellite data applications and implications

By Maciej Myśliwiec, Planet Partners

The view from space offers an expanded perspective on Earth. It is remarkable how data can be obtained and comprehended through satellite-based Earth observation. Sixty-five years after the launch of the first artificial satellite, space technology continues to provide an immense volume of data utilized by analysts and scientists worldwide, enabling the development of solutions across various disciplines. Let us explore the breadth of observations that can be done from space!

Currently, over 4,500 artificial satellites orbit our planet. Among them, there are the satellites of the European Union’s Copernicus programme – the leading provider of satellite data in Europe. Each day Copernicus delivers over 12 terabytes of data that is freely accessible to all users. With a simple internet connection, one can access the extensive collection of satellite images gathered since the programme's inception in 2014, with the launch of the first Sentinel-1A satellite.

Managing and processing such vast amounts of data necessitates substantial computational and storage resources. Earth Observation (EO) platforms like CREODIAS provide users with the means to transform EO data into valuable information. CREODIAS currently houses a repository of over 34 petabytes of data and an integrated cloud computing environment, enabling efficient processing of EO data.

Presently, half a million users of the Copernicus programme process more than 16 terabytes of data daily. To put this into perspective, 1 terabyte equates to approximately 250,000 photos captured by a 12-megapixel camera or 500 hours of high-definition video footage. Hence, we speak of millions of images and substantial quantities of other data (e.g., from satellite sensors) generated within the European programme every single day. The easy accessibility of satellite data fuels innovation across various industries and scientific domains. The current market turnover for Earth observation data is estimated at €2.8 billion, with over 41% of the European industry relying on data obtained from Earth observation programmes.

In recent times, several initiatives have emerged to facilitate data access and processing for scientists, researchers, public administrations, the private sector, and the general public. One of these initiatives is the Copernicus Data Space Ecosystem, which aims to provide comprehensive spatial and temporal coverage of Copernicus EO data, immediately accessible to all users free of charge, along with cloud computing resources for further data processing. Another notable initiative is Destination Earth, the flagship project of the European Commission that strives to develop an advanced digital model of the Earth powered by European supercomputers (HPC) and state-of-the-art artificial intelligence technologies.

The convenient access to current and historical Earth Observation data enhances planning and decision-making across diverse fields. How can we leverage these vast resources of satellite data?

Environmental research and climate change

One of the primary applications that come to my mind is the study of climate change and meteorology. EO data plays a crucial role in developing meteorological models for weather forecasting, preparing for and mitigating natural disasters, as well as monitoring and addressing climate change.

Accurate knowledge of current land cover and land use is pivotal for effective planning and management by local and regional authorities. Areas under the jurisdiction of public administration institutions, such as national parks, wetlands, lakes, riverbeds, and coastlines, benefit immensely from open satellite data captured by Sentinel-1 and Sentinel-2 satellites. These data offer extensive possibilities for managing such areas with minimal effort. Even 10-meter multispectral data is sufficient for a range of environmentally significant applications, including drought monitoring, flood management, freshwater ecosystem health assessment, forest cover analysis, biodiversity monitoring, air quality monitoring, and land surface temperature analysis.

Sentinel satellite missions, part of the European Copernicus programme, have revolutionized Earth research since 2014. The ubiquitous availability of freely accessible satellite data is complemented by specialized tools that enable their processing and analysis, converting raw data into actionable information.

Illustrative examples of climate change analysis tools include:

• "Global temperature trend monitor": an application that monitors global air temperature changes on Earth's surface over recent decades.
• "Climate monitoring and volcanic eruptions": an application studying volcanic eruptions, which can cause short-term climate effects on a global and regional scale, contributing to natural climate variability.
• "European temperature statistics derived from climate projections": an application providing European temperature statistics for historical periods (1976-2005) and projected 30-year periods (2031-2060 and 2071-2100) under different climate change scenarios.

These applications represent a fraction of the vast array of possibilities available to users of satellite data programs, tailored to address their specific research inquiries.

Forestry

Most forests are managed by state-owned enterprises for timber production, which significantly contributes to national budgets. Effective forestry management requires comprehensive knowledge of soil characteristics, water dynamics, climate conditions, and microclimates across expansive and sometimes challenging-to-access areas. Satellite data offers valuable insights into forest dynamics and their influence on climate patterns. Real-time spaceborne data serves as an invaluable tool for locating forest fires, monitoring plant diseases, and assessing the impact of drought on tree health.

Agriculture

Earth Observation supports various agricultural processes. At the local level, data assists in identifying optimal locations for cultivation, predicting crop yields, and maintaining records of agricultural parcels. Advanced remote sensing techniques, such as Synthetic Aperture Radar (SAR) and hyperspectral analysis, are increasingly being applied in precision agriculture. At regional or national scales, EO data facilitates land cover classification and supports the implementation of programs aimed at agricultural development and direct subsidies for farmers.

Spatial planning

Satellite data plays a vital role in spatial planning for both urban and rural landscapes. Leveraging EO data, one can conduct detailed surveys and complex analyses of areas of interest. Currently, imagery serves as a primary source of information for formulating up-to-date spatial development plans, considering changes in land use and cover, identifying new areas for investment, and detecting wetland areas. Copernicus Sentinel-2 data, for instance, provides rapid and comprehensive insights into current and historical phenomena. Investors seeking to expand their activities in remote regions greatly benefit from information on the status of potential investment areas.

Urban heat islands

Satellite-based Earth observation data also contributes to urban management, including the monitoring and mitigation of urban heat islands, characterized by significantly higher air temperatures within urban areas compared to adjacent rural regions. Unlike traditional point-based air temperature measurements from monitoring stations, satellite data enables the measurement of surface temperatures at any location within a city. This capability facilitates the development of spatial patterns critical for understanding the phenomenon and aids local authorities in taking necessary measures to improve residents' quality of life.

Maritime transport and logistics

Satellite imagery plays a crucial role in the monitoring and detection of vessel traffic on seas and oceans. It also supports the management of maritime economies and ensures the safe transportation of valuable commodities such as liquefied chemicals and crude oil. In the event of accidents, EO images provide timely assistance in mapping the consequences. Combining EO data with Automatic Identification System (AIS) information yields a powerful tool for monitoring marine objects and phenomena, including coastal monitoring and analysis of changing water depths.

Crisis management

Natural disasters like floods, wildfires, and storms often have a wide spatial extent. Open satellite data forms the foundation for the detection and monitoring of such events. Specialized methods, relying on environmental and atmospheric satellite data, are employed to detect favourable conditions and early warning signs of potential natural hazards like droughts or algal blooms in lakes or seas. Synthetic Aperture Radar (SAR) image processing is recommended for monitoring land movement, particularly for applications such as landslide risk detection, vital for mining and mountain tourism. Very high-resolution data plays a crucial role in assessing building disasters, managing mass events, and ensuring the security of government and military facilities.

Natural disasters like floods, wildfires, and storms often have a wide spatial extent. Open satellite data forms the foundation for the detection and monitoring of such events. Specialized methods, relying on environmental and atmospheric satellite data, are employed to detect favourable conditions and early warning signs of potential natural hazards like droughts or algal blooms in lakes or seas. Synthetic Aperture Radar (SAR) image processing is recommended for monitoring land movement, particularly for applications such as landslide risk detection, vital for mining and mountain tourism. Very high-resolution data plays a crucial role in assessing building disasters, managing mass events, and ensuring the security of government and military facilities.

Satellite imagery provides an insightful perspective on the locations and magnitudes of flooding events. These visual data enable the identification of inundated regions, assessment of soil and vegetation moisture levels, and the prediction of areas potentially vulnerable to flash floods. Earth Observation platforms like CREODIAS house an extensive array of such images, inclusive of superior-resolution datasets from esteemed programs such as Copernicus. Users have the opportunity to peruse diverse images, categorize them chronologically or geographically, and thus acquire a more profound comprehension of flood dynamics. 

When combined with ground-level measurements, satellite-based remote sensing offers an in-depth perspective on global drought trends. Meteorological satellites play a crucial role in monitoring key environmental indicators such as humidity, temperature, and wind, which help predict periods of limited rainfall that can subsequently lead to dry soil conditions and stressed vegetation, commonly referred to as agricultural droughts. To support this analysis, the Copernicus programme provides an array of tools tailored to drought assessment. When it comes to hydrological droughts, which signify water shortages within certain regions, advanced satellite imagery becomes indispensable. One of the standout techniques in this domain is SAR – Synthetic Aperture Radar, which is a type of radar imaging that uses radio waves to capture detailed images of the Earth's surface, making it possible to detect even subtle changes in water levels and providing valuable insights into drought conditions.

Assessment of war-related environmental damage

Satellite data proves invaluable during wars and armed conflicts. However, its utility extends beyond military applications. By processing data, it becomes possible to estimate the scale of environmental destruction or predict actions necessary for post-war reconstruction. An example of such an initiative is EO4UA (Earth Observation for Ukraine), which aims to support Ukrainian institutions and international organizations in assessing the environmental damage caused by war activities within Ukraine. EO4UA activities are conducted in a cloud computing environment integrated with a substantial repository of Earth observation data, encompassing various datasets, such as satellite imagery, crop classifications, forest fire extents, and more, necessary for comprehensive environmental analysis.

The above-mentioned areas provide only a glimpse into the diverse applications of EO data. As we witness rapid technological advancements in space technologies, we can anticipate gaining unprecedented insights into the Earth's ecosystems. We believe that the future will bring a deeper understanding of our planet, facilitated by rapidly evolving satellite data technologies. Equipped with these advancements, we will be better prepared to address environmental challenges on our planet, fostering a more optimistic outlook for the future of Earth.

Enhancing Earth Observation capabilities - satellite data applications and implications

By Maciej Myśliwiec, Planet Partners

The view from space offers an expanded perspective on Earth. It is remarkable how data can be obtained and comprehended through satellite-based Earth observation. Sixty-five years after the launch of the first artificial satellite, space technology continues to provide an immense volume of data utilized by analysts and scientists worldwide, enabling the development of solutions across various disciplines. Let us explore the breadth of observations that can be done from space!

Currently, over 4,500 artificial satellites orbit our planet. Among them, there are the satellites of the European Union’s Copernicus programme – the leading provider of satellite data in Europe. Each day Copernicus delivers over 12 terabytes of data that is freely accessible to all users. With a simple internet connection, one can access the extensive collection of satellite images gathered since the programme's inception in 2014, with the launch of the first Sentinel-1A satellite.

Managing and processing such vast amounts of data necessitates substantial computational and storage resources. Earth Observation (EO) platforms like CREODIAS provide users with the means to transform EO data into valuable information. CREODIAS currently houses a repository of over 34 petabytes of data and an integrated cloud computing environment, enabling efficient processing of EO data.

Presently, half a million users of the Copernicus programme process more than 16 terabytes of data daily. To put this into perspective, 1 terabyte equates to approximately 250,000 photos captured by a 12-megapixel camera or 500 hours of high-definition video footage. Hence, we speak of millions of images and substantial quantities of other data (e.g., from satellite sensors) generated within the European programme every single day. The easy accessibility of satellite data fuels innovation across various industries and scientific domains. The current market turnover for Earth observation data is estimated at €2.8 billion, with over 41% of the European industry relying on data obtained from Earth observation programmes.

In recent times, several initiatives have emerged to facilitate data access and processing for scientists, researchers, public administrations, the private sector, and the general public. One of these initiatives is the Copernicus Data Space Ecosystem, which aims to provide comprehensive spatial and temporal coverage of Copernicus EO data, immediately accessible to all users free of charge, along with cloud computing resources for further data processing. Another notable initiative is Destination Earth, the flagship project of the European Commission that strives to develop an advanced digital model of the Earth powered by European supercomputers (HPC) and state-of-the-art artificial intelligence technologies.

The convenient access to current and historical Earth Observation data enhances planning and decision-making across diverse fields. How can we leverage these vast resources of satellite data?

Environmental research and climate change

One of the primary applications that come to my mind is the study of climate change and meteorology. EO data plays a crucial role in developing meteorological models for weather forecasting, preparing for and mitigating natural disasters, as well as monitoring and addressing climate change.

Accurate knowledge of current land cover and land use is pivotal for effective planning and management by local and regional authorities. Areas under the jurisdiction of public administration institutions, such as national parks, wetlands, lakes, riverbeds, and coastlines, benefit immensely from open satellite data captured by Sentinel-1 and Sentinel-2 satellites. These data offer extensive possibilities for managing such areas with minimal effort. Even 10-meter multispectral data is sufficient for a range of environmentally significant applications, including drought monitoring, flood management, freshwater ecosystem health assessment, forest cover analysis, biodiversity monitoring, air quality monitoring, and land surface temperature analysis.

Sentinel satellite missions, part of the European Copernicus programme, have revolutionized Earth research since 2014. The ubiquitous availability of freely accessible satellite data is complemented by specialized tools that enable their processing and analysis, converting raw data into actionable information.

Illustrative examples of climate change analysis tools include:

• "Global temperature trend monitor": an application that monitors global air temperature changes on Earth's surface over recent decades.
• "Climate monitoring and volcanic eruptions": an application studying volcanic eruptions, which can cause short-term climate effects on a global and regional scale, contributing to natural climate variability.
• "European temperature statistics derived from climate projections": an application providing European temperature statistics for historical periods (1976-2005) and projected 30-year periods (2031-2060 and 2071-2100) under different climate change scenarios.

These applications represent a fraction of the vast array of possibilities available to users of satellite data programs, tailored to address their specific research inquiries.

Forestry

Most forests are managed by state-owned enterprises for timber production, which significantly contributes to national budgets. Effective forestry management requires comprehensive knowledge of soil characteristics, water dynamics, climate conditions, and microclimates across expansive and sometimes challenging-to-access areas. Satellite data offers valuable insights into forest dynamics and their influence on climate patterns. Real-time spaceborne data serves as an invaluable tool for locating forest fires, monitoring plant diseases, and assessing the impact of drought on tree health.

Agriculture

Earth Observation supports various agricultural processes. At the local level, data assists in identifying optimal locations for cultivation, predicting crop yields, and maintaining records of agricultural parcels. Advanced remote sensing techniques, such as Synthetic Aperture Radar (SAR) and hyperspectral analysis, are increasingly being applied in precision agriculture. At regional or national scales, EO data facilitates land cover classification and supports the implementation of programs aimed at agricultural development and direct subsidies for farmers.

Spatial planning

Satellite data plays a vital role in spatial planning for both urban and rural landscapes. Leveraging EO data, one can conduct detailed surveys and complex analyses of areas of interest. Currently, imagery serves as a primary source of information for formulating up-to-date spatial development plans, considering changes in land use and cover, identifying new areas for investment, and detecting wetland areas. Copernicus Sentinel-2 data, for instance, provides rapid and comprehensive insights into current and historical phenomena. Investors seeking to expand their activities in remote regions greatly benefit from information on the status of potential investment areas.

Urban heat islands

Satellite-based Earth observation data also contributes to urban management, including the monitoring and mitigation of urban heat islands, characterized by significantly higher air temperatures within urban areas compared to adjacent rural regions. Unlike traditional point-based air temperature measurements from monitoring stations, satellite data enables the measurement of surface temperatures at any location within a city. This capability facilitates the development of spatial patterns critical for understanding the phenomenon and aids local authorities in taking necessary measures to improve residents' quality of life.

Maritime transport and logistics

Satellite imagery plays a crucial role in the monitoring and detection of vessel traffic on seas and oceans. It also supports the management of maritime economies and ensures the safe transportation of valuable commodities such as liquefied chemicals and crude oil. In the event of accidents, EO images provide timely assistance in mapping the consequences. Combining EO data with Automatic Identification System (AIS) information yields a powerful tool for monitoring marine objects and phenomena, including coastal monitoring and analysis of changing water depths.

Crisis management

Natural disasters like floods, wildfires, and storms often have a wide spatial extent. Open satellite data forms the foundation for the detection and monitoring of such events. Specialized methods, relying on environmental and atmospheric satellite data, are employed to detect favourable conditions and early warning signs of potential natural hazards like droughts or algal blooms in lakes or seas. Synthetic Aperture Radar (SAR) image processing is recommended for monitoring land movement, particularly for applications such as landslide risk detection, vital for mining and mountain tourism. Very high-resolution data plays a crucial role in assessing building disasters, managing mass events, and ensuring the security of government and military facilities.

Natural disasters like floods, wildfires, and storms often have a wide spatial extent. Open satellite data forms the foundation for the detection and monitoring of such events. Specialized methods, relying on environmental and atmospheric satellite data, are employed to detect favourable conditions and early warning signs of potential natural hazards like droughts or algal blooms in lakes or seas. Synthetic Aperture Radar (SAR) image processing is recommended for monitoring land movement, particularly for applications such as landslide risk detection, vital for mining and mountain tourism. Very high-resolution data plays a crucial role in assessing building disasters, managing mass events, and ensuring the security of government and military facilities.

Satellite imagery provides an insightful perspective on the locations and magnitudes of flooding events. These visual data enable the identification of inundated regions, assessment of soil and vegetation moisture levels, and the prediction of areas potentially vulnerable to flash floods. Earth Observation platforms like CREODIAS house an extensive array of such images, inclusive of superior-resolution datasets from esteemed programs such as Copernicus. Users have the opportunity to peruse diverse images, categorize them chronologically or geographically, and thus acquire a more profound comprehension of flood dynamics. 

When combined with ground-level measurements, satellite-based remote sensing offers an in-depth perspective on global drought trends. Meteorological satellites play a crucial role in monitoring key environmental indicators such as humidity, temperature, and wind, which help predict periods of limited rainfall that can subsequently lead to dry soil conditions and stressed vegetation, commonly referred to as agricultural droughts. To support this analysis, the Copernicus programme provides an array of tools tailored to drought assessment. When it comes to hydrological droughts, which signify water shortages within certain regions, advanced satellite imagery becomes indispensable. One of the standout techniques in this domain is SAR – Synthetic Aperture Radar, which is a type of radar imaging that uses radio waves to capture detailed images of the Earth's surface, making it possible to detect even subtle changes in water levels and providing valuable insights into drought conditions.

Assessment of war-related environmental damage

Satellite data proves invaluable during wars and armed conflicts. However, its utility extends beyond military applications. By processing data, it becomes possible to estimate the scale of environmental destruction or predict actions necessary for post-war reconstruction. An example of such an initiative is EO4UA (Earth Observation for Ukraine), which aims to support Ukrainian institutions and international organizations in assessing the environmental damage caused by war activities within Ukraine. EO4UA activities are conducted in a cloud computing environment integrated with a substantial repository of Earth observation data, encompassing various datasets, such as satellite imagery, crop classifications, forest fire extents, and more, necessary for comprehensive environmental analysis.

The above-mentioned areas provide only a glimpse into the diverse applications of EO data. As we witness rapid technological advancements in space technologies, we can anticipate gaining unprecedented insights into the Earth's ecosystems. We believe that the future will bring a deeper understanding of our planet, facilitated by rapidly evolving satellite data technologies. Equipped with these advancements, we will be better prepared to address environmental challenges on our planet, fostering a more optimistic outlook for the future of Earth.

15th February 2023

Lessons learned from my Kubernetes Journey

By Paweł Turkowski, Product Manager at CloudFerro

At CloudFerro, one of our guiding principles is Technological Courage. This value has strongly resonated with me from the start of my journey at the company. It is a simple yet powerful statement that gives our team a clear guideline not to be afraid to operate outside our comfort zone.

Kubernetes, by today’s standards, is not a new technology anymore. It has graduated to being an often primary go-to model of application deployment. CloudFerro was quick to jump on the Kubernetes bandwagon. We have multiple applications running on Kubernetes, as well as a massive amount of K8S jobs running daily to process data in our satellite image repositories.

For me, personally, it has been an exciting challenge to be a Product Manager for the Kubernetes product offering. Apart from Kubernetes, there is also a large ecosystem of other technologies required to navigate in the DevOps space. I have worked in the Product Management field for several years, managing a variety of different products. The generic understanding of operating principles is a very useful and transferable skill. Yet I think that taking time to understand the domain well is one of the best investments one can make as a PM.
Some of our clients have been running Kubernetes clusters on CloudFerro clouds for likely as long as this technology has lasted. However, in the past couple of years, we have made some significant commitments to grow our Kubernetes offering as a natively available solution on our clouds.
With OpenStack Magnum, we enable our customers to deploy a functional Kubernetes cluster in literally a couple of minutes. It is a matter of selecting a few options from the Horizon UI. In fact, the only key decision before deploying pods/applications/workloads is to choose your cluster size and underlying Virtual Machine flavors. This setting can also be changed after cluster creation and one can opt for an auto-scalable number of nodes.
Aside from making sure the Kubernetes clusters are quick to set up and reliable to run, we also constantly scan the interaction with various integrations used by our partners, to make sure they work seamlessly with Magnum.

Working with Kubernetes is an exciting journey, where you never stop learning. Some of the lessons I can share from my own perspective:

• Browsing through the documentation for hours might not be unusual for a developer, but it is in fact quite the same for a PM working on a highly technical product. These days there is a lot of high-quality, free supportive content that helps a lot. Recently, Chat GPT is also enabling us all to make some significant shortcuts in various domains, and Kubernetes is no exception.
• Learning from others is valuable, however, it helps to do your own research before asking a question (this refers to colleagues, but also e.g., to online forums). You might be able to answer your own question faster, or, at minimum, narrow down the question to the actual core.
• Speaking to clients is of tremendous value. It helps to get a proper understanding of what features we think might help, and which ones actually move the needle for our partners.

It has been a rewarding experience to see the growth of Magnum and the trust of our clients to run their Kubernetes clusters on the CloudFerro infrastructure. I would like to take the opportunity to invite readers of this post to watch my Kubernetes webinar. It can provide some help to quickly get started with Magnum and Kubernetes overall, with some examples from the Earth Observation domain.

Paweł Turkowski

Paweł has over 15 years of experience in product management. At CloudFerro, he is responsible for the infrastructure offering, currently focusing on the Kubernetes platform.

24th January 2023

Copernicus Data Space Ecosystem explained

CloudFerro, together with its partners, has been selected by ESA as an industrial team to implement and operate Copernicus Data Space Ecosystem. The goal is to provide full spatial and temporal Copernicus EO data coverage available immediately and free for the users. 

On Tuesday, 24th January 2022, the new Copernicus Data Space Ecosystem was launched. The service is taking the European Union Copernicus programme to the next level, ensuring the data make the greatest possible impact on users, all the citizens, and – ultimately – on our planet. The new Copernicus Data Space Ecosystem builds on existing data distribution services (incl. CREODIAS and others), ensuring their continuity and introducing significant improvements.  

Evolution of Copernicus data access

The European Union Copernicus programme is among the biggest providers of Earth Observation data in the world and its launch was one of the most important events in the remote sensing community over the past decade. With systematic monitoring over large areas, good quality of data, resolution fitting its objectives, ensured the longevity of the program, and, most importantly, a clear and simple open data policy it took the world by storm. This, however, is not something to be taken for granted. When Copernicus started, remote sensing was a niche field occupying mostly researchers and the intelligence community. There was no specific reason to expect that it will be any different with Sentinel data. But it happened. 

The data was picked by enthusiasts, then by companies and institutions. Fast forward a couple of years and suddenly there are hundreds of applications helping farmers to better manage their fields, financial markets forecast prices of corn, journalists regularly use it to observe or validate news, the European Common Agriculture Policy relies on its data to monitor practically all agriculture fields for sustainable agriculture practices, researchers are building digital twins of the environment, security organizations predict migration patterns... There are many more examples. 

Nobody expected Copernicus data to start such a revolution in this field. It was (and still is) a tremendous validation of ESA's operational capacity, but at the same time also significant pressure on the data distribution systems as they were not expecting such uptake. Still, the Copernicus Data Hubs have already distributed an order of magnitude more data than is published. With user uptake growing further, as well as the recognized importance of Earth Observation data for the monitoring of climate change, the EU decided to invest in the next level of user data processing and distribution infrastructure. The new Copernicus Data Space Ecosystem was born.  

The future of European user data processing and distribution is powered by experienced players - T-Systems and CloudFerro, with their well-used cloud infrastructure, Sinergise and VITO with Sentinel Hub and OpenEO data discovery and processing tools, and DLR, ACRI-ST, and RHEA taking care of on-demand processing and Copernicus Contributing Missions.  

What does it mean to CREODIAS users?

This is CloudFerro long-term commitment to grow and maintain the Copernicus EO data repository available in CREODIAS. Within the next few years, one of the largest online repositories worldwide will grow from 35PB to over 85 PB of free and immediately available EO data. Data from all over the world is available directly from the cloud through efficient, cloud-native access mechanisms (both API and GUI). Data is available for current users, new users, and federated platforms.

Copernicus Data Space Ecosystem - what’s in it for users? 

• The data offer available for the user will increase. The data repository includes full up-to-date Copernicus data as well as complimentary commercial data offering (EO data+)
• Data that can be processed directly in a cloud environment
• Expanded search and visualisation of the data with Jupyter notebooks for testing and development 
• Federation with other EO services – Sentinel Hub, OpenEO, JupyterHub, and others 
• Serverless, on-demand processing services – for Sentinel products and other data
• Many services will be free of charge with easy access to a payable but easily scalable extension

Data offer

First and foremost, it's about the data we guarantee - instant data availability. Just about all the data ever acquired by Sentinel satellites, with minor exceptions, will be available online, instantaneously including e.g. Sentinel-1 SLC and GRD and L2 OCN, Sentinel-2 L1C, and L2A (reprocessed to Collection 1, as reprocessed data becomes gradually available), Sentinel-3 and Sentinel-5P L1 and L2 data as well as Copernicus Contributing Missions data. Full archive, and always up-to-date. 

There is currently no infrastructure where all of this would be available in one place. The new Copernicus Data Space Ecosystem will be it. Data will be available through various interfaces – from old-fashioned download, direct S3 access with STAC items, and cloud-optimized formats to streamlined access APIs, which are able not just to fetch the data but also to process it. 

EO services

There is a web-based application built on top of a very popular EO Browser technology to allow for data visualisation and as a user interface to access the data. Several on-demand processors are capable of building non-default formats and derived products, such as Sentinel-1 coherence and CARD4L products, MAJA-powered atmospheric processing, and similar. The data can be accessed via the openEO and JupyterHub for serverless processing. Last but not least, there will be a special focus put on traceability. For all data managed within the Copernicus Data Space Ecosystem, it will be possible to trace where it came from and what operations were performed on it.  

It is all about timing, responsibilities, and commitments

The initiative has quite an intense phase-in plan, in order to allow hundreds of thousands of existing users of the Copernicus Data Hubs to migrate their workflows to the new service. First, a limited but stable, roll-out has just happened (24th January 2023) with continuous upgrades over the upcoming months until full service will be made available by the end of June 2023. 

This is a very challenging yet super exciting opportunity for everyone in the remote sensing community - from beginners to experts, from researchers, companies, and institutions, as well as individuals. Just about anyone can benefit from being aware of what is happening with our planet. And we should all take interest in it, and act. For the consortium partners, however, this is especially important - we were given an opportunity to build a new ecosystem for everyone to use. An ecosystem that strives to provide a significant upgrade over existing tools and services and is open to welcoming new partners and service providers in the future! This comes with huge responsibility towards everyone - not just to the ESA and European Commission, but for the worldwide EO community.  We are confident we can execute it, so stay tuned for further information. 

Discover Copernicus Data Space Ecosystem.

Read more in our news.

2nd December 2022

How can satellite remote sensing help analyse forests?

Forests cover 31 percent of the world's land surface, that's over 4 billion hectares^[1]. According to the research published in Nature Climate Change in 2021, forests are a natural ’carbon sink’ that absorbs a net 7.6 billion metric tonnes of CO₂ per year. The degradation of timberland around the world has prompted scientists not only to seek ways to halt these processes but also to develop objective diagnostic methods to assess the status and changes occurring in forests. After all, there is a lot at stake – as the United Nation's Global Forest Goals Report 2021 estimates, 1.6 billion people worldwide depend directly on forests for food, shelter, energy, medicines, and income^[2].

The development of technology and the integration of satellite data with environmental models have given us the ability to observe the Earth on a large scale. One of the tools used to monitor the state of the environment, especially complex ecosystems such as forests, is satellite remote sensing.

How satellites for remote sensing work

Satellite remote sensing includes a whole set of procedures involving the acquisition, processing, and interpretation of satellite images, from which spatial information is extracted. Images of a given area can be taken regularly with successive satellite overpasses, which makes it possible to observe changes that have occurred in each area even over a long period of time. These methods allow us to observe changes in land cover, analyse the condition of vegetation over large areas, or monitor the extent of natural disasters, such as droughts, floods, and others. Moreover, remote sensing methods effectively reduce costs and research time. The CREODIAS platform, for example, has more than 30 PB Copernicus EO data available free of charge and off-the-shelf. This platform and similar platforms allow for data to be processed in the cloud without the need for time-consuming downloads to local computers or arrays. Moreover, the public cloud for their processing that CloudFerro provides is a much more cost-effective solution than having infrastructure on one's own.

Observing forests from space

Applying remote sensing techniques to monitor such complex ecosystems as forests can be a laborious undertaking. The complexity lies mainly in the multi-layered structure of forests, their species composition, and spatial variability. Therefore, when analysing images from several months, we can see completely different objects, despite the fact that the characteristics of the forest have remained unchanged.

However, this can bring some advantages to the process. For instance, if we dispose of multi-temporal data, we can better classify the species because their evolution during the growing season is specific. The growing popularity of remote sensing can also be attributed to its detail and comprehensiveness. Today, satellites are able to capture high-resolution images, making it possible to detect even individual trees.

The process of forest area analysis

Today, remote sensing is mainly used for ongoing monitoring and forest inventories. A satellite image is a unique way to do it on a global scale. Even at regional, national, or local scales, remote sensing is often the only way to obtain data on forests located in hard-to-reach places or dangerous regions of the world. Such data can be used by local forest management institutions and global environmental agencies to monitor the condition of the area, and manage environmental emergencies.

The first stage of forest monitoring is determining its spatial extent, where different remote sensing data, are used. This information, combined with the documentation on land use, allows us to create accurate maps, but also to indicate areas of succession.  

 Knowing the extent of forests, satellite data can also be used for forest analysis to:

• determine the size (volume) of the forest, i.e., the amount of biomass present in a given area. This allows us to estimate the amount of wood that can be harvested, as well as determine the amount of carbon in the ecosystem
• rapidly and accurately identify changes on the forest surface caused by wind, fires, floods, or sustained or human activities
• to map all the impacts of human activities, even in the most remote areas of the planet.

Processing forest data in a cloud environment

One of the most advanced forest analyses performed on a global scale is Forest Digital Twin Earth Precursor developed by the European Space Agency, as part of the Digital Twin Earth Precursor (DTEP) initiative. Its aim is to create a digital replica of all the forest ecosystems, which involves integrating complex environmental models with a  vast amount of Earth observation data combined with artificial intelligence solutions. CloudFerro, the operator of the CREODIAS platform, was a partner in the Precursor phase of the Digital Twin Earth project, delivering technical expertise and resources.

Large-scale research projects require advanced competencies and vast IT resources because they need to collect, store and process large volumes of data in an easy, cost-effective, and timely manner. CREODIAS’ powerful cloud infrastructure and its large repository of current and historic Earth observation data greatly facilitate conducting this type of research helping meet computation and storage requirements for processing growing volumes of data.

For more on species classification of forests, go to an article by a forest expert published on CREODIAS.

^[1] FAO. The State of the World’s Forests 2022. Forest pathways for green recovery and building inclusive, resilient and sustainable economies. Rome, FAO (2022). https://doi.org/10.4060/cb9360en

^[2] United Nations Department of Economic and Social Affairs, United Nations Forum on Forests Secretariat. The Global Forest Goals Report (2021). https://doi.org/10.18356/9789214030515

3rd August 2022

Computing Cloud as a perfect environment for Big EO data repositories

Copernicus as a flagship European space programme 

Copernicus is the European Union's Earth Observation (EO) programme managed by the European Commission. The main goal of the programme is to observe our planet and its environment to benefit all European citizens. The programme has revolutionized the European space industry by making access to sattelite data is easier than never before. The enormous amount of EO data available as a public good for anyone at no cost.Huge amount of archival data and the continuous growth of large EO data sets made it necessary not only to create a cloud environment, but also to develop the usability and exploit the potential of EO data - the DIAS platforms (Data and Information Access Services). For this reason, EO platforms were launched, thanks to which we have gained a powerful tool for EO data access and scalable processing.  

DIAS and EO platforms 

EO platforms are an essential element to enhance the EO data usability, making it possible to change data into information. The platforms provide users with a computing power that enables fast and efficient processing and allow users to access, process and analyse Copernicus products in the cloud directly connected to an EO archive. The availability of data, storage, and processing in a single place makes it easier to develop and distribute scalable digital services based on the EO data. Below you can see consumption model for DIAS and EO platforms. 

We rely on an open- source technology

We have built and now operate several EO platforms based on a hybrid cloud in an Infrastructure-as-a-service model. Our services consist of open-source components, such as Open-Stack for cloud orchestration, Ceph for storage management, Prometheus and Graphana for monitoring,  K8s for processing, and others.

EO platform as a service 

Taking as an example CREODIAS, WEkEO, CODE-DE and other platforms built and operated by CloudFerro, we have created the main building blocks and architecture of entirely European cloud platforms based on open-source software.

We have managed to create a new service - an EO platform as a service.

Federation and multi- platform approach

Federation and a multi-platform approach allows users and operators to gain benefits from the synergy between different platforms, such as a shared multi PB data repository available from various platforms and elasticity cloud components for additional processing needs. At the same time, it generates a technical challenge that we need to address. We have met challenges focusing on three main topics: 

• Interoperability of the environments: how to support easy migration between different clouds and provide direct access to the repositories available from different computing clouds, with multi-cloud processing enabled and dynamically adaptable usage and payment models. 
• Data access and dissemination: the multi-petabyte, scalable EO data repositories should provide fast, instantaneous access to many heterogeneous users and algorithms with different needs. At the same time, the data needs to be discoverable and ready for analysis in a context of constantly evolving data offers (new collections, commercial data, products), and users should be supported in data dissemination. 
• Cloud efficiency: how to optimize data access, storage, and processing costs and energy usage both for individual users and for the entire ecosystem. 

Why is it so important to use EO platform

Using the EO Platform is connected with numerous benefits. The most significant are:

7th July 2022

How satellite data can be used to improve comfort of city residents

The use of Earth Observation (EO) data contributes to smart city management helping to reduce harmful effects of urban heat island phenomenon and improve the quality of life.

The density of buildings and urban infrastructure causes changes in the microclimate of urban areas. This has a direct impact on the health and quality of life of the inhabitants. The solution may be Copernicus satellite data, which will allow for a better understanding of spatial phenomena occurring in densely populated places.

What is urban heat island phenomenon and how does it affect cities

Heatwaves boosted by urban heat island phenomenon are the most dangerous weather extremes, which not only damage our health, but also lead to the increase of energy consumption. To reduce or prevent their detrimental effects we need to track urban climate. Urban surface heat island (USHI) is an obvious consequence of urban processes and it is associated with the increase of temperature of urbanized areas in relation to undeveloped rural areas.  

The phenomenon results from city morphology* and it is influenced by many factors. Even a single building can increase the ambient temperature. The accumulation of concrete buildings and urban infrastructure causes the temperature in the city to rise from 2 to even 8ºC, and in extreme cases, even above 12ºC. The strongest temperature amplitudes are observed on summer nights. 

Urban Heat Island affects human health and energy consumption and that's why the issue of the urban heat island is extremely important in the management and planning of a space comfortable for residents to live. 
 

The most important factors contributing to the urban heat island phenomenon are: 

• The environmental context of city 
• City size 
• City topography 
• Water resources 
• Urban green areas 
• Anthropogenic heat sources (e.g. air conditioning, cars) 

* City morphology - structure of the inner and outer urban space and the origin of its parts. The spatial conditions of the city influence the scale of the urban heat island phenomenon. The amount of heat accumulated is determined by factors such as the city's environmental context, the number of inhabitants, the amount of greenery, the height of buildings, width of streets, the predominant color of the surface (albedo), the presence of ventilation corridors or even the roughness of materials used in the city.

The most dangerous weather extreme 

Heatwaves are the most dangerous weather extremes that cause more casualties than hurricanes and floods combined. On the average, 1,500 American city dwellers die each year because of the heat, which is many more deaths than those caused by all other natural disasters combined. The compounding effect of heatwaves and UHI could prove to be even deadlier to urban populations, and are extremely dangerous especially for people who suffer from chronic diseases (such as asthma, heart condition), elderly people, manual workers, children and athletes.  

Threats to cities caused by the urban heat island phenomenon: 

• Health hazard for the elders and risk groups 
• Higher energy consumption
• Intensification of convective processes 
• Increased probability of heavy rainfall and greater exposure of cities to local flooding. 

Land Surface Temperature Monitoring on CREODIAS 

The CREODIAS platform allows users to search and download satellite scenes of the Sentinel, Landsat and MODIS programmes at no cost (repository of ready-to-use 30 PB of current and archive Earth observation data) which allows you to analyse the data in order to find the areas that are most endangered by heat waves and the urban heat island phenomenon.  

Cloud workflow with EO data repository allows researchers to automate processes and calculate large amounts of data and changes over time. These operations combined with data repository are crucial in identifying critical areas for the microclimatic functioning of cities. Analysts can find spatial patterns to better understand the phenomena occurring in urban space, and consequently help local governments to take necessary steps to improve the quality of life of the residents. As we can see, spatial analyses are crucial for a better understanding of the urban climate.    

The best solution - urban green areas 

The easiest and most effective way to cool down urban climate is to introduce urban green areas. Vegetation contributes to the formation of shade, reflects solar radiation, and thanks to evapotranspiration processes, it effectively reduces the temperature in cities. Urban vegetation also contributes to the reduction of air and water pollution, reduces noise and accumulates large amounts of water, creating opportunities to reduce energy consumption.  

Copernicus data can be applied to analyse urban climate and improve urban greenery in the most neglected areas by new investments in green infrastructure, for instance:   

• Planting street greenery (stormwater street design)  
• Protecting ecological urban areas  
• Green rooftops  
• Relocation of industrial areas outside the city 

The extensive usage of EO data provides faster and more sustainable economic growth through better use of resources. This is key to achieving sustainable development goals driven by space innovations.  

Questions?

Should you have any questions, please contact us at communication@cloudferro.com

9th May 2022

How does Horizon Europe support innovative EO projects?

Get funding for sustainable innovative solutions based on EO data, AI, machine learning and Big data analysis with Horizon Europe programme.

The start-up and technology sector in Europe is growing stronger, and the appetite for innovation is effectively fuelled by grants from the European Union. Horizon Europe programme has a budget of almost 95 billion euros to distribute until 2027, and the main areas of financing include climate protection, energy transformation, development of the digital sector, and artificial intelligence.

Horizon Europe focuses on progress through innovation

In a webinar about technologies that support the development of innovative solutions based on EO data, Chiara Solimini from EUSPA (European Union Agency for the Space Programme) indicated key areas of interest for building applications and advised on the use of the synergy of data and technologies. “The proposals to the Horizon Europe programme should focus on the development of innovative EGNSS and Copernicus applications that support the implementation of the European Green Deal objectives and its related policies. They should use the synergy of the Copernicus data or products, EGNSS differentiators and advanced technologies such as blockchain, internet of things or 5G applications ” – said Chiara Solimini, Space Downstream Market Officer at EUSPA.

Innovative applications based on massive amounts of satellite data, using Big Data, artificial intelligence or machine learning technologies can only be developed with fast and reliable online access to a rich data repository. This is provided by the EU Copernicus Earth Observation Programme that provide users with free and open access to Earth Observation (EO) data and information services, such as monitoring of the atmosphere, marine environment, land areas, climate change, security or crisis management.

Research with Copernicus EO data on CREODIAS

Satellite data analysis is far easier with free and open access to the Copernicus EO data. This is provided by CREODIAS platform, built and operated by CloudFerro for the European Space Agency. The platform offers cloud computing resources with an integrated powerful repository of imagery captured by Sentinel, Envisat and Landsat satellites as well as other Earth observation data.

The CREODIAS infrastructure is adequately prepared to process large Earth observation data sets in a cloud computing environment, which are available online immediately. Cloud computing resources are scalable and can be adapted to user needs depending on the size and timing of their project. Tools available on the platform allow users to design prototypes, as well as build their own services and products and tailor all the relevant parameters of the cloud environment to specific topics, data types and project stages.

"We want to be a strong contributor to the Horizon Europe and European initiatives, and together build a better future for European society based on digital sovereignty and innovation. The CREODIAS repository currently stores more than 28 PB of Earth observation data. For our partners and the platform users, we have prepared cloud resources in different locations in Poland and Europe, including data centres with German BSI 200-1 and C5 certification, proving the highest quality and security level"

– says Przemysław Mujta, Sales Support Director at CloudFerro, the provider and operator of CREODIAS.

Horizon as a driver for innovations 

Together EO platform tools and computing capabilities allow monitoring natural resources, precise irrigation, satellite monitoring of agriculture under the EU Common Agricultural Policy, automatic control, or pasture management. State-of-the-art technologies combined with satellite data and services also support renewable energy and its distribution. These include RES (renewable energy sources) assessment and forecasting, risk assessment for energy support equipment, power plant design optimization, power grid monitoring, or environmental impact assessment of power plants and mineral resources. Satellite data can form a basis for the analysis of snow cover maps for hydropower plants or biomass maps for bioenergy, determining sunshine areas for solar panels, or analyse wind speed for wind turbines – onshore and offshore. It is also interesting how satellite data can be applied for analysis of mining damages, the occurrence of urban heat islands, or study of bee activity to plan crop spraying.

The above mentioned examples are only a small part of a wide array of areas where innovations based on EO data can make a real contribution to climate neutrality and reduced consumption of natural resources in Europe. Any idea of using this technology can certainly be supported under Horizon Europe. 

Who can join the programme?

Micro, small and medium-sized enterprises, and the so-called "small mid-cap" employing up to 500 people can benefit from Horizon Europe grants. Funding applications may be submitted by entities that develop a product, process or service of high risk, based on scientific discoveries, critical thinking or technological breakthroughs (so-called "deep tech")

Questions?

Should you have any questions, please contact us at communication@cloudferro.com

6th April 2022

Why do we need European digital sovereignty?

The EU has attempted to regulate how industrial data can be shared and used to innovate for the better. In the past years we have seen an explosion of legislation and projects pursuing this dream. From the publication of the EU Data Strategy and the Data Governance Act or the launching of Gaia-X and a variety of upcoming sectoral data spaces there is a whirlwind of policy that one needs to keep track of to stay abreast. But that could all change…

European Alliance for Industrial Data and Cloud

It is rightly pointed out that the COVID-19 pandemic has brought about massive societal and economical shifts in how our societies function, furthermore exposing our deep reliance on digital infrastructure to conduct our daily lives. The surge in cyberattacks only highlights how precarious the foundations of our digital architecture are. As a result, we are witnessing a historic period for the tech industry as a whole. Competition is increasingly becoming a point of friction as governments worldwide are waking up to the realities that exist in the digital market.

It is amidst this backdrop that another milestone has been achieved by the European Commission. That is the launch of the European Alliance for Industrial Data and Cloud.  An initiative that aims at further crystallising the ambitions set out in the EU’s Data Strategy when it comes to how Europe can boost its own cloud capacity. 

CloudFerro is excited to be among the first participants of this initiative and we are eager to share our expertise, resources, and enthusiasm to build a strong, competitive, and resilient European data and cloud service. As a truly European company, we at CloudFerro are very pleased to see the Commission launch the European Alliance on Industrial Data and Cloud. This is the next step forward for the EU’s plans to be competitive and spur innovation across sectors while aspiring to build European digital sovereignty"

–  says Maciej Krzyżanowski, CEO of CloudFerro.

Unblocking the Cloud Market with the Data Act

The publication of the Data Act provides a much-needed legislative push for the cloud sector to unlock Europe’s full competitiveness.  As the file now falls under the responsibility of the Council and Parliament, it will be paramount that the political aspirations of the Data Act are not watered down and a strong emphasis on fairness and interoperability is maintained. We can only encourage the Commission to stand by its objectives and defend these principles during the negotiations of the co-legislatures.

Hearkening back to the Council of the European Union’s October 2020 declaration on cloud we can only echo that a common approach is needed in order to build Europe’s cloud supply, reinforce European digital sovereignty and increase competition.

The Alliance states in its declaration that one of its objectives is to provide support for defining common norms and requirements for cloud. We hope that amongst the Alliance we will be able to assist in providing the needed rationale and argumentation so such measures as interoperability and switching are encouraged and maintained among the wider cloud agenda and that policymakers do not lose sight of the bigger picture.

Fostering Common European Data Spaces and Norms

Integral to that bigger picture of a European sovereign cloud ecosystem, we see data governance as critical. While in December both European Parliament and Council concluded their trilogue negotiations on the Data Governance Act, it will take time for the effects of this Act to be fully established. What we know though is that, while the Act was reportedly ‘unloved’, it plays such a fundamental role in shaping the future framework for launching Europe into the era of cloud, that being the Common European Data Spaces. Only with the technical support of the cloud industry will these Common European Data Spaces manifest and be brought to scale in such a way that makes them viable. 

It is our aim that the Alliance takes the lead among the variety of stakeholder formats in the conversation about cloud and data. Especially on operationalising the Alliance’s goal of creating a platform that can synergise with the Common European Data Spaces. Indeed the Alliance declaration notes how important close coordination will be on this topic with the European Data Innovation Board, which is established under the Data Governance Act.

But governance is only one piece of the puzzle, as with many great projects, this will take funding and investments. The format of the Alliance has the potential to open up so many sectors to each other and bring new investment opportunities to the forefront.

Earth Observation Data – a Call for Common European Data Spaces

Europe has a plethora of private and public organisations that can serve as a source for rich datasets. For example, at CloudFerro we work with the European Space Agency and the European Commission under the remit of the Copernicus  program to provide our CREODIAS platform - which allows open, free and easy access to Earth Observation Data and its processing in the cloud. CREODIAS cloud services are sovereign and are based on open source technology. The platform allows third-parties to create their own applications and services to fit their needs and share them for the benefit of others.

Using our platform allows for a wide application of Earth Observation Data collected from the Copernicus satellites and other sources. What our platform is able to do is render and provide these images tailored to a variety of needs, together with an integrated cloud computing services to process this data. Need to monitor crop yield? How about water consumption? Or what about observing the changes in ice along the Alps? We have the possibility to analyse that, but what Europe needs now is an effective way to help synergize this analysis.

Our work with Copernicus has taught us that a rapid and effective set up of Common European Data Spaces – an ecosystem of public and private organisations, standards and technologies that enable shared value creation via the exchange of data and data-driven solutions – is key to exchanging data effortlessly between economic actors and creating value.

We would encourage you to see the results of our competition #SeizeTheBeautyOfOurPlanet - the incredible Earth Observation imagery demonstrating  climate change, its negative effects on our planet, and the need to counteract them together.

Conclusion

At CloudFerro our expertise lies in providing specialized cloud services and in storing, indexing, delivering and processing big data volumes. Our key specialty is Earth Observation and other specialised domains such as meteorology and climate data.  The wide range of specialised data needs our customers have means we as CloudFerro see Europe’s opportunity to provide a new way of cloud services, capitalising on the yet untouched realm of industrial data. The promise of Common European Data Spaces is close to becoming a reality, but it will take political courage and ambition to see it done.

As members of the Alliance on Industrial Data and Cloud, we whole heartedly call for more action and more participation on behalf of all cloud services so that Europe can develop its digital sovereignty in a way that is grounded and fair.

Questions?

Should you have any questions, please contact us at communication@cloudferro.com