Self-Supervised Representations of Geo-located Weather Time Series - an Evaluation and Analysis

Abstract

Self-supervised learning (SSL) is gaining traction in various domains, and demonstrated their potential particularly in tasks where labelled data is limited and costly to collect. In this work, we evaluate the performance existing self-supervised multivariate time series learning algorithms on weather-driven applications involving regression, classification and forecasting tasks. We experiment with a two-step protocol. In the first step, we employ and SSL algorithm and learn generic weather representations from multivariate weather data. Then, in the next step, we use these representations and fine-tune them for multiple downstream tasks. Through our initial experiments on air quality prediction and renewable energy generation tasks, we highlight the benefits of self-supervised weather representations, including improved performance in such tasks, ability to generalize with limited in-task data, and reduction in training time and carbon emissions. We expect such a direction to be relevant in multiple weather-driven applications supporting climate change mitigation and adaptation efforts.