One day, your weather app may ping you with a local alert: rain in five minutes.
That is one of the tantalising prospects raised by artificial intelligence, according to Kirstine Dale, the Met Office’s Chief AI Officer and Principal Fellow for Data Science.
Based on a proof-of-concept study done last summer, she believes AI could one day enable ultra-local, real-time forecasts—tailored not just to your town, but to your street or even garden. ‘We are still working it through,” she said, declining to specify when such forecasts might become widely available.
I spoke to Dale last week in Exeter at the unveiling of the Met Office’s new Strategic Plan. There, CEO Penny Endersby introduced the organisation’s 14th supercomputer – a milestone six decades in the making since the Met’s first – which offers the prospect of making detailed 14-day forecasts routine.
Housed in two southern England data centres (their location has not been disclosed) and accessed via the internet, it is thought to be the world’s first cloud-based supercomputer dedicated to weather and climate science.
It will be around six times more powerful than its 14 petaflop (a quadrillion calculations per second) predecessor and will enable more detailed and accurate weather forecasts together with new capabilities in climate science research, setting new standards for industry.
The cloud-based supercomputer took four years to implement. Ben Fitzpatrick, Head of Science IT Data Management and Processing, called it the dawn of a new era—especially in the integration of AI into forecasting.
Until now, meteorologists have relied on physics-based numerical models, such as the Met Office’s Unified Model, which use billions of equations to make predictions, and date back to when scientists began to describe the behaviour of the atmosphere mathematically.
These ingest up to 300 terabytes of data daily—more than 50 billion observations from satellites, sensors, and weather stations—to simulate the atmosphere’s behaviour.
AI models, such as the FastNet model developed through a partnership between the Met Office and the Alan Turing Institute, can perform to a similar level when compared with physics-based models and reality, based on pooled data from national European forecasters, explained Steve Ramsdale, one of the Met Office’s Chief Operational Meteorologists.
Rather than simulating from first principles, AIs learn from torrents of historical and real-time data, detecting patterns and generating forecasts with remarkable speed.
To create the AI requires computer chips called GPUs, graphic processing units, and can take tens of thousands of GPU hours to trains. But once trained, the AI can run on modest hardware—democratising access to advanced forecasting.
However, machine learning has its limits. AI can struggle with rare or unprecedented events – for instance, the climate disruption caused by the Pinatubo volcanic eruption of 1991 – and can often not extrapolate beyond what it has already seen. For such extremes, physics-based forecasting still rules.
Moreover, said Ramsdale, while AI provides answers, it lacks ‘explainability’—the ability to show how it reached its conclusions. Physics-based models, by contrast, offer transparency. ‘I have huge trust in them—I need that from AI models as well,’ he said.
That is why the Met Office is pursuing a hybrid strategy—melding the statistical muscle of AI with the theoretical rigour of physics-based models, notably in ensemble forecasts, created by slightly varying the initial conditions of a weather model to weigh up the probability of different weather events.
However, Ramsdale emphasised that the experience of human meteorologists remains central to forecasting for the near future.
The stakes are more than meteorological, however. Met Office forecasts guide aviation, defence, shipping, infrastructure, and even Sunday cricket. By one estimate, by London Economics, the value of the Met Office to the UK economy will reach £56 billion over the next decade.
Unlike the UK’s high performance computer strategy, which is back on track after being thrown into disarray last year, the Met Office is already planning the next generation of the supercomputer in 2027, which will require exabyte-scale storage.
‘Imagine something like 40 million Blu-ray discs, enough to fill every bit of a very large house from floor to attic,’ commented Ben Fitzpatrick, adding that that both AI and physics-based modelling have benefited from improved hardware, software, and datasets in recent years.
‘One strength of the Met Office is that our physical modelling can create new and higher resolution training data, so we can keep pushing the frontiers of what AI can learn from.’
The forecast, in short, is for smarter weather.
Discover more about the history of weather forecasting and climate modelling on our website.