Unleashing the power of Earth Foundation Models with ThinkingEarth

Importance of Earth Foundation Models

Earth Foundation models are crucial for advancing our understanding and monitoring of the planet’s complex systems. These foundation models, which are trained on vast amounts of Earth Observation (EO) data, enable researchers and policymakers to make informed decisions regarding environmental management, disaster response, and climate change mitigation. By providing high-resolution, real-time insights into land, ocean, and atmospheric conditions, these models can help in predicting natural disasters, tracking deforestation, and monitoring agricultural changes.

The recent study “On the Foundations of Earth and Climate Foundation Models”, with contribution from ThinkingEarth Work Package 2 coordinator Yi Wang from the Technical University of Munich (TUM), explores the future of earth foundation models and has pinpointed several features that should be implemented to create an ideal foundation model going forward.

The features include geolocation embedding, balanced geographic representations, scale awareness, wavelength embeddings, the time variable, multisensory, task-agnostic and carbon minimised, while these eight features have been identified as “must-haves”, the study also identified features such as uncertainty quantification, physical consistency, and Machine Learning assistants, ensuring a more human-focused approach.

Challenges with Earth Foundation Models

Despite their potential, developing Earth Foundation models presents several significant challenges. One major hurdle is the vast nature of EO data, which comes from various sources like satellites, drones, and ground sensors, each with different resolutions and formats. Integrating and harmonising this data to train robust models is complex. Additionally, the models must be capable of generalising to different geospatial and temporal contexts, which requires sophisticated domain adaptation techniques to avoid overfitting to specific datasets. Computational resources are another challenge; training these models demands significant processing power and storage, often necessitating high-performance computing (HPC) environments. Finally, ensuring the models’ transparency and interpretability is crucial for gaining trust from stakeholders and effectively integrating these tools into decision-making processes.

ThinkingEarth’s Role

While the challenges with foundation models and big data are vast and complex, ThinkingEarth is taking a unique approach to tackle these challenges. The main aim of the ThinkingEarth project is to create the first Copernicus Foundation Models. Viewing the earth as a complex and interconnected system, ThinkingEarth will create a data-driven graph representation of essential earth system variables (ESVs), modelling Earth as a Graph to showcase their applications. We’ll use self-supervised learning to enhance data compression and generalisation for Sentinel-1, 2, and 3 data, and develop spatio-temporal graph neural networks (GNNs) to analyse both static and dynamic Earth representations.

This involves integrating self-supervised graph learning to capture ESV interactions, contributing to the DestinE initiative with a prototype Earth graph model visualised on NVIDIA’s Omniverse platform. Combining physics-aware machine learning, causal methods, and explainable AI (xAI), we’ll tackle challenges in earth system variables, improve model transparency, and reveal cause-and-effect relationships in climate variability. Ultimately, leveraging these models, ThinkingEarth will create scalable products for renewable energy, biodiversity management, and food security, fostering new value chains and enhancing forecasting and anticipatory actions.

Follow us!

To learn more about ThinkingEarth and stay up to date on the latest news, follow us on X and LinkedIn. You can also get updates directly to your inbox by signing up to our newsletter.

Get in touch with us directly here.