CloudFerro has been supplying a hybrid cloud hosting service for Climate Data Store (CDS) and Atmosphere Data Store (ADS) since 2017, and now its contract has been extended to double the capacity of the cloud services, reaching 7 PB of high availability storage. The private cloud hardware was physically installed on customer premises to enable broadband communication with the local infrastructure, and to meet extended security requirements. It is provided in a pay-per-use 'as a service' model and is being fully operated and monitored by CloudFerro.
An on-premises Private Cloud is directly connected to the ECMWF storage infrastructure to provide fast, high-bandwidth access to the Center's huge data store. In order to provide a seamless user experience, both infrastructures are managed by a common cloud system based on OpenStack – a leading open-source software for building cloud systems avoiding vendor lock-in.
The main reason for the infrastructure migration was to help deliver ECMWF’s strategy to move the full ECMWF’s data center to Bologna what implied moving the cloud infrastructure for CDS and ADS services. The cloud infrastructure entirely migrated to the Bologna data centre in June 2022, and its final operational readiness was achieved in Bologna data centre in July 2022.
The migration services provided by CloudFerro included the construction of a new interim scaled-down infrastructure in Warsaw, the remote replication of 3.5PB of data, physical relocation of the Reading infrastructure from Warsaw to Bologna, its re-synchronisation to the newest data, system architecture modernization of the infrastructure in line with the newest best practises, and finally a switch-over to the new production. By now, the entire system has increased twofold in comparison to the previous one. CloudFerro was the first ECMWF’s supplier to have launched production services in the Bologna data centre.
– We are happy with the results of the migration process provided by CloudFerro, who managed to deliver end-to-end service in an efficient and timely manner. With the new data centre, the ECMWF has gained an advanced and efficient infrastructure based on an innovative and highly secure technology. The fast-changing environment requires advanced solutions that optimize the entire ecosystem for enhanced performance. The new cloud infrastructure not only improves the entire service but also enables our further growth.
– says Angel Lopez Alos, Climate Store Manager at ECMWF.
Each day CDS users make over 480,000 requests for data, and daily downloads very often exceed 100 TB (source of statistics). Given such a great scale and a complex nature of the CADS service, the migration of its cloud infrastructure and data had to be conducted in a safe and smooth manner, with minimal efficiency penalty throughout the entire process.
A migration of any cloud infrastructure is always a challenge, let alone an overseas relocation of a large and complex system in unfavourable conditions of post-Brexit hurdles and pandemic restrictions. During the entire process of migration, we managed to fully maintain the continuity of the system operations and properly handle the ongoing ingestion of new data. The existing cloud infrastructure for Climate Data Store and Atmosphere Data Store is currently hosting over 3.5 PB of data and the enhancements we have made guarantee much higher efficiency and reliability of the service. – says Andrzej Stella-Sawicki, the Vice President and Chief Operating Officer of CloudFerro.
- Our cooperation with ECMWF has been very effective. We have been building the relationship upon mutual trust and professional approach for over 5 years now. We are happy to be able to continue the delivery of cloud computing services to ECMWF
adds Andrzej Stella-Sawicki.
About Copernicus Climate and Atmosphere Data Stores
CADS include the Copernicus Climate and Atmosphere Data Stores, high-performance and scalable free and open services that retrieves time-series-based climate and atmospheric data and enables its analysis, aggregation, processing, and delivery. CADS is at the heart of the Copernicus Climate Change Service (C3S) and the Copernicus Atmosphere Monitoring Service (CAMS) infrastructure. C3S and CAMS are two of the operational services of the EU’s Copernicus Programme.
https://cds.climate.copernicus.eu/#!/home
https://ads.atmosphere.copernicus.eu/#!/home
About Copernicus and ECMWF
Copernicus is a component of the European Union’s space programme, with funding by the EU, and is its flagship Earth observation programme, which operates through six thematic services: Atmosphere, Marine, Land, Climate Change, Security and Emergency. It delivers freely accessible operational data and services providing users with reliable and up-to-date information related to our planet and its environment. The programme is coordinated and managed by the European Commission and implemented in partnership with the Member States, the European Space Agency (ESA), the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), the European Centre for Medium-Range Weather Forecasts (ECMWF), EU Agencies and Mercator Océan, amongst others.
The European Centre for Medium-Range Weather Forecasts (ECMWF) operates two services from the EU’s Copernicus Earth observation programme: the Copernicus Atmosphere Monitoring Service (CAMS) and the Copernicus Climate Change Service (C3S). They also contribute to the Copernicus Emergency Management Service (CEMS), which is implemented by the EU Joint Research Council (JRC).
ECMWF is an independent intergovernmental organisation supported by 35 states. It is both a research institute and a 24/7 operational service, producing and disseminating numerical weather predictions to its Member States. This data is fully available to the national meteorological services in the Member States. The supercomputer facility (and associated data archive) at ECMWF is one of the largest of its type in Europe and Member States can use 25% of its capacity for their own purposes.
ECMWF has expanded its location across its Member States for some activities. In addition to an HQ in the UK and Computing Centre in Italy, offices with a focus on activities conducted in partnership with the EU, such as Copernicus, are in Bonn, Germany.
The Copernicus Atmosphere Monitoring Service website can be found at https://atmosphere.copernicus.eu/
The Copernicus Climate Change Service website can be found at https://climate.copernicus.eu/
More information on Copernicus: www.copernicus.eu
The ECMWF website can be found at https://www.ecmwf.int/
We will answer every your questions, as soon as possible.
