AI in Space - FDL Europe 2021 - Extreme Weather

FDL Europe is a public - private partnership between the European Space Agency (ESA), the University of Oxford, Trillium Technologies and leaders in commercial AI supported by Google Cloud, NVIDIA and Scan AI. FDL Europe works to apply AI technologies to space science, to push the frontiers of research and develop new tools to help solve some of the biggest challenges that humanity faces. These range from the effects of climate change to predicting space weather, from improving disaster response, to identifying meteorites that could hold the key to the history of our universe.

FDL Europe 2021 was a research sprint hosted by the University of Oxford that took place over a period of eight weeks in order to promote rapid learning and research outcomes in a collaborative atmosphere, pairing machine learning expertise with AI technologies and space science. The interdisciplinary teams address tightly defined problems and the format encourages rapid iteration and prototyping to create meaningful outputs to the space program and humanity.

Extreme Event Hot Spots using ML Onboard

Project Background

Satellite observations of the Earth's surface provide a vital data source for diverse environmental applications, including disaster management, landcover change detection and ecological monitoring. Currently, sensors on satellites collect and downlink all data points for further processing on the ground; yet, limitations in downlink capacity and speed result in delayed data availability and inefficient use of ground stations. These limitations affect time-sensitive applications such as disaster management where data is required with as low latency as possible to inform decision making in real time. This problem is set to worsen as the sensing resolution and number of satellites in orbit increase together with further restrictions on radio-frequency spectrum sharing licensing.

A solution is to apply processing onboard to identify the most useful data for a particular scenario, and prioritise this for rapid downlink. There has been considerable recent interest in using machine learning onboard for this processing. Existing work has focused on the deployment of supervised classifiers for applications such as identifying clouds or floods. Supervised learning has a significant drawback: only events of a particular type determined at training time will be flagged and prioritised for downlink, with no generalisation to new event types, imager specifications and lighting or local features.

Project Approach

In this work the ML Onboard project team proposed a new fully unsupervised novelty-detection model suitable for deployment on remote sensing platforms using a Variational AutoEncoder (VAE) to generate a latent representation of incoming sensor data over a particular region. A novelty score is assigned to this data using the distance in the latent space between representations from consecutive passes. This offers a substantial advantage over existing supervised methods as any change between passes can be detected onboard at inference time, regardless of the availability of training data.

From these measurements, the team developed the RaVÆn library for unsupervised change detection for use onboard satellites. As shown in the table below, the team found that the VAE models (blue background) outperform a random baseline for all classes of event - landslides, hurricanes, wildfires and floods; and a custom baseline constructed by taking the cosine distance in pixel space for all classes except landslides. The scores in bold represent the best outcome for each class of event depending on the dimensions of the latent space.

Percentage of events identified	Landslides	Hurricanes	Wildfires	Floods
Random Baseline	33.62%	43.69%	65.70%	30.06%
Custom baseline: Cosine distance	60.19%	59.58%	87.50%	38.18%
VAE₁₀₀	59.02%	74.96%	90.83%	43.11%
VAE₁₀₂₄	58.91%	74.85%	89.91%	46.04%

Project Results

Traditionally, data analysis is performed on the ground after all data is transferred – downlinked – to a ground station. Constraint on the downlink capabilities therefore affects any downstream application. In contrast, RaVÆn pre-processes the sampled data directly on the satellite and flags changed areas to prioritise for downlink, shortening the response time. The team verified the efficacy of the system on a dataset composed of time series of catastrophic events, demonstrating that RaVÆn outperforms pixel-wise baselines.

The results from the VAE model allows the creation of change-score maps, that show a qualitative comparison between, for example, before a river that floods or a wildfire occurs and afterwards showing a change in colour in the after image. In these examples - the ML Onboard method – the cosine embedding produces a change map that is crisper than the cosine baseline; for instance, the small flooded canal can be seen in the cosine embedding image but not in the baseline.

Flooding Example

Wildfire Example

In conclusion, the ML Onboard team introduce a new method - RaVÆn - for unsupervised change detection in remote sensing data using a VAE. The team's method was evaluated on a new dataset of remote sensing images of disasters. The proposed model outperforms a classical computer vision baseline in all of the tested disaster classes. This demonstrates that RaVÆn is a robust change detection method and suitable for application in improving data acquisition for disaster response. You can learn more about this case study by visiting the FDL EUROPE 2021 RESULTS PAGE, where a summary, poster and full technical memorandum can also be viewed and downloaded.

The Scan Partnership

Scan is a major supporter of FDL Europe, building on its participation in the previous year’s events. As an NVIDIA Elite Solution Provider Scan contributes multiple DGX supercomputers in order to facilitate much of the machine learning and deep learning development and training required during the research sprint period.

Project Wins

satellite_alt

Demonstration that the team’s RaVÆn model outperforms a classical computer vision baseline in all of the tested disaster classes

speed

Time savings generated during eight-week research sprint due to access to GPU-accelerated DGX systems

James Parr

Founder, FDL / CEO, Trillium Technologies

"FDL has established an impressive success rate for applied AI research output at an exceptional pace. Research outcomes are regularly accepted to respected journals, presented at scientific conferences and have been deployed on NASA and ESA initiatives - and in space."

Dan Parkinson

Director of Collaboration, Scan Computers

"We are proud to work with NVIDIA to support the FDL Europe research sprint with GPU-accelerated systems for the second year running. It is a huge privilege to be associated with such ground-breaking research efforts in light of the challenges we all face when it comes to climate change and extreme weather events"

Speak to an expert

You’ve seen how Scan continues to help FDL Europe further its research into the climate change and space. Contact our expert AI team to discuss your project requirements.

phone_iphone Phone: 01204 474210

mail Email: [email protected]

FDL Europe 2021 - Extreme Event Hot Spots using ML Onboard

FDL Europe 2021