FDL Europe 2023 - Live Twin for Space Weather

Can machine learning onboard space observatories and on Earth help build a near real-time warning system for potentially dangerous solar weather events?

Published 18 DEC 2023

FDL Europe is a public - private partnership between the European Space Agency (ESA), the University of Oxford, Trillium Technologies and leaders in commercial AI supported by Google Cloud, NVIDIA and Scan AI. FDL Europe works to apply AI technologies to space science, to push the frontiers of research and develop new tools to help solve some of the biggest challenges that humanity faces. These range from the effects of climate change to predicting space weather, from improving disaster response, to identifying meteorites that could hold the key to the history of our universe.

FDL Europe 2023 was a research sprint hosted by the European Space Agency Phi-Lab, taking place over a period of eight weeks. The aim is to promote rapid learning and research outcomes in a collaborative atmosphere, pairing machine learning expertise with AI technologies and space science. The interdisciplinary teams address tightly defined ‘grand challenge’ problems and the format encourages rapid iteration and prototyping to create meaningful outputs for the good of all humanity. This approach involves combining data from various sources, leveraging expertise from multiple fields and utilizing advanced technologies to tackle complex challenges.

Live Twin for Space Weather (Heliophysics Onboard)

Project Background

Early detection of solar events is crucial for responding to strong solar activity that could potentially impact Earth. One significant event to consider is a Coronal Mass Ejection (CME), involving the expulsion of large amounts of plasma into the outer heliosphere, where it interacts with the stellar wind and electromagnetic systems near Earth or other celestial bodies. A CME affecting terrestrial infrastructure, such as a power grid, can cause severe damage due to large currents induces in the metal conductors. Detecting a CME as early as possible is essential to allow for sufficient time to develop and implement counterstrategies.

Currently, data required for early CME detection is collected by a network of strategically positioned Sun-orbiting satellites. However, these satellites are not ideally positioned to maximise the warning time between detection and potential CME impact. This situation is expected to change with the launch of the ESA VIGIL mission, planned for 2029. The VIGIL spacecraft will assume a stable orbit at the fifth Lagrangian point (L5), which provides an early ‘side view’ of the Sun, as shown in the diagram below. L5 is located 60 degrees behind Earth in its orbit, as seen from the Sun, providing VIGIL a short window of opportunity to process what it sees and predict the impact of dynamic solar weather on Earth’s environment. However, estimating the direction and propagation speed of CMEs poses a difficult task and the distance from L5 to Earth further imposes further challenges - limited computing power and low telemetry bandwidth. Deploying AI onboard the L5 solar observatory has great potential to address these shortcomings in a novel way, enabling near real-time space weather awareness.

In 2022, FDL Europe applied AI to images from the STEREO (Solar Terrestrial Relations Observatory) and SOHO (Solar and Heliospheric Observatory) spacecraft to reconstruct a 4-pi Steradian model of the full Sun called SPI3S. This 4-pi Steradian image represents the full spherical surface of the Sun, created at high-fidelity, even at the solar poles.

The SPI3S project provided the foundation for the FDL Europe research team to build a state-of the-art CME detection and characterization system that utilises data from existing heliospheric imagers as proxies for VIGIL data. The new system can precisely track CMEs throughout the heliosphere, leveraging intelligence onboard the spacecraft and the greater processing power in Earthbound computers. This approach could significantly reduce downlink bandwidth requirements and alert latency, providing a novel approach to rapidly detect CMEs, study the underlying physics of solar eruptive events and determine more precisely their impact on Earth.

Project Approach

The FDL Europe team proposed an AI pipeline broken down in to two interconnected components: onboard and ground-based processing. The onboard processing module would offer the advantage of real-time data access from diverse instruments without latency, cadence or quality constraints, but suffer from limited computing and memory resources. Conversely, ground-based operations have access to substantial computing capabilities and extensive historical data archives, with multiple sources, and therefore viewpoints, available, but suffer from data latency due to communication constraints. The proposed pipeline is illustrated in the diagram below.

The research team’s ML-based approach was shaped by the data currently available, so synthetic data was used as a proxy to emulate the anticipated VIGIL mission. To properly benchmark the new method, they used images from the STEREO spacecrafts’ chronograph Cor1 and Cor2 instruments, and simulations of solar eruptive events developed by researchers at Predictive Science Inc. (PSI) and the High Altitude Observatory (HAO). The PSI dataset provided a static image of the inner corona, meaning there was no CME present, however, reconstructing the 3D density distribution from a single snapshot was essential to estimating the suitability of their method to the inner corona. The HAO data focused on the outer corona and provided a temporal series of CME evolution - this dataset was essential to evaluate the performance of their method for the 3D reconstruction of CME density distributions, as well as the CME’s temporal evolution.

For the onboard phase, the team focused on two sub-areas, both based on using coronagraphic white-light imagery. Firstly, compression: using ML-based algorithms to enable faster downlinking and lower latency; and secondly detection: - by deployment of an onboard classifier to detect the onset of CME events, triggering both an increase in sampling cadence and an algorithm to isolate the parts of the image containing the CME. This approach minimises the amount of data transmitted during ordinary monitoring operations and maximises the amount of useful data aquired after the onset of a CME. Images from the simulated datasets were fed into the proposed onboard classifier - a ResNet-18 pre-trained convolutional neural network (CNN) which the team anticipate will be compatible with VIGIL's planned onboard computing resources. The advantages of using an ML model for the classification task is that it could act as a detector and only process the important sections on an image directly into the VQ-VAE (vector-quantised variational encoder) compression algorithm for downlinking to the ground station, as illustrated in the diagram below.

For the ground phase, the team adapted the neural radiance field (NeRF model) methodology using the decompressed downlinked images to reconstruct the electron density of a CME in 3D. The new NeRF model built this density model using only coronagraph images from two viewpoints: a spacecraft such as VIGIL and from the Earth. The team also incorporated a Physics Informed Neural Networks (PINNs) into the NeRF, in what they called a PINNeRF approach (as shown below) to increase the reliability of the prediction.

Project Results

In terms of the quality of onboard compression performance, the research team plotted multi-scale structural similarity (ms-ssim) versus bit rate (bpp = bit-per-pixel), as shown in the graph below. Ms-ssim is a compression quality measurement capturing the structural information, whereas bit-rate is an absolute measure and represents the average number of bits needed to encode the information in each image pixel. A higher bit rate means more bits are used to encode the image (i.e., less compression), while a higher value for ms-ssim corresponds to less information lost due to the compression. The team’s preliminary results show that VQ-VAE variants are comparable to JPEG, and at low bit-rates (less than 0.5 bpp) they exhibited higher compression quality.

The team’s PINNeRF approach utilised physical equations for continuity and radial velocity profiles in the 3D reconstruction of the NeRF approach. Qualitatively, the reconstruction on simulated data from only two viewpoints (L1 and L5) show results that are close to the ground-truth density distribution of the CME simulation. Quantitatively, their study on physics constraints indicated that incorporating continuity and radial constraints improves density reconstruction performance, with further performance enhancements expected after additional model tuning.

This entire two stage pipeline combining the onboard and ground-based results, was christened ICARUS and was used to produce 3D CME reconstructions like one shown below. In conclusion, the FDL Europe project team demonstrated that AI methods can vastly improve 3D reconstructions of the CME profiles, which provides critical information for the estimation of space weather hazards and enables novel insights into the topology and propagation of CMEs.

As more hardware specifications are disclosed regarding the VIGIL mission, the FDL Europe team will seek to fine-tune their models ahead of launch. You can learn more about this case study by reading the FDL Europe 2023 Results Booklet, where a summary, poster and full technical memorandum can also be viewed and downloaded.

The Scan Partnership

Scan AI is a major supporter of FDL Europe 2023, building on its participation in the previous three years events. Acting as a technology partner of NVIDIA to provide access to a DGX cluster in order to facilitate much of the machine learning and deep learning development and training required. The FDL Europe Live Twin for Space Weather team made use of virtual NVIDIA V100 and A100 GPU accelerators in DGX systems accessed via the Scan Cloud platform, feeding back that the user interface was straightforward and that it didn’t take long to begin working with it, producing great results right away.

“Scan’s DGX system was absolutely essential for performing model ablation experiments during the second half of the FDL Europe period. It was used by the Space Weather team to help create the ICARUS pipeline which makes detection and characterisation of coronal mass ejections on the Sun more efficient, accurate and reliable.”

Catherine Ballarin

AI Project Manager, FDL Europe

“We are proud to work with NVIDIA to support the FDL Europe research sprint with GPU-accelerated systems for the fourth year running. It is a huge privilege to be associated with such ground-breaking research efforts in light of the challenges we all face when it comes to climate change and extreme weather events.”