Beyond Weather: On-demand visualizations of fine tuned AIFS models over NetCDF workarounds in one week

The Company: Long Range AI Weather Prediction

Beyond Weather is an Amsterdam-based AI weather forecasting startup with its roots in academic research. Founded in 2023 by a master’s student, a PhD researcher, and a professor from the Vrije Universiteit Amsterdam’s climate extremes and societal risk research group, the company set out to solve a problem they believed was genuinely underserved: accurate forecasting beyond the standard 14-day horizon.

“We saw that the predictability on longer than two weeks out just wasn’t good enough yet for many customers in the energy sector, agricultural sector, but also NGOs and humanitarian aid,” said Jannes von Ingen, co-founder and CEO. “We were very motivated to get our insights out there and make an impact in the world.”

Beyond Weather fine-tunes open-source AI models, primarily AIFS from ECMWF, to create specialized forecasts tailored to specific client needs. Whether that’s wind conditions over the North Sea, temperature anomalies in Europe, or drought risk in Spain, the company delivers purpose-built models to customers who need better visibility into long-range weather risk. Their clients are primarily in energy (traders, utilities, and grid operators) and agri-food, all of whom face exposure to weather risk on forward markets.

The Amsterdam based Beyond Weather team recently secured additional funding to focus on AI foundation models for tailored, long-range weather forecasts for the energy industry.

The Challenge: Beautiful Data, No Way to Show It

For a company whose value is delivering precise, long-range forecasts, the ability to visualize that data for clients was core to the product experience. But for most of Beyond Weather’s early life, it was their biggest unsolved problem.

“Back when we were starting Beyond Weather, we were just working with NetCDFs on a local computer,” said Jannes. “To expose a NetCDF to a customer in a web interface — that was just too difficult. We did workarounds which weren’t ideal. Converting it to some weird format and then exposing it, it was never like I wanted it to be.” Meanwhile, Jannes could see what was possible — polished, interactive weather visualizations on platforms like Windy — but knew the infrastructure they would have to build was likely out of reach for their early-stage startup. “I saw weather data visualizing nicely online. It should be possible — but the infrastructure behind it was a lot for our company to build at the time,” Jannes said.

The team tried various options, like hosting NetCDFs on a server and building their own bespoke solutions. Each iteration got a little better, but never quite met the requirements an operational client-facing product requires. One of the main bottlenecks in the previous setups was the loading time of data in the web interface. The converted formats that are readily visualisable in Mapbox (GeoTIFF, PNGs, vector formats, etc.) loaded and visualised slowly, sometimes taking seconds. With Earthmover’s Flux API and some optimisations, the Beyond Weather team brought that down to milliseconds.

Courtesy: Beyond Weather

Delivering the data behind the visualizations was also a consideration for Beyond Weather. If they built and maintained their own visualization and data delivery infrastructure, they’d be on the hook to maintain (and improve) it forever. Arraylake provides a great solution for that: it’s as easy to expose a dataset to external stakeholders, allowing seamless data delivery to customers and partners alike.

Finding Earthmover

Beyond Weather had been loosely aware of Earthmover for some time. Jannes and the team were already working with Zarr and Xarray, open source projects that Earthmover actively develops and supports. Earthmover CEO Dr. Ryan Abernathey and CTO Dr. Joe Hamman had strong reputations in the open-source scientific data community. “If you’re in the weather data open-source software space, it’s hard to miss Earthmover,” said Jannes.

They had also explored Earthmover’s commercial Arraylake product earlier in their stack design process, but weren’t ready to adopt it at that time. The turning point came when Jannes watched an Earthmover webinar and saw a concrete demonstration of Zarr visualization in a web interface through Flux, Earthmover’s tile and visualization API. “I was like, ‘Hey, this is something I can concretely use right now, because I know this is quite hard to set up. You need quite a lot of people to know how to do this. And if we had to do it ourselves, it would be very costly — and we’d have to let people maintain it for years. This was a much better ‘total cost of ownership.’ I’d been looking for something like this for years.”

“I thought it was quite a unique solution. I didn’t know of any other things like Earthmover and Flux. I was just curious to see what it would bring — and it convinced me.”

Courtesy: Beyond Weather

Implementation: One Developer, One Week

The implementation was led by AI Data Engineer Isa Rethans, working closely with the Earthmover team. Getting started required adopting Icechunk, Earthmover’s open-source tensor storage format, alongside Flux. Beyond Weather was already working in Zarr and had data on Google Cloud Storage buckets, which meant the learning curve was manageable. “We were already a little bit there with Zarr,” said Jannes.

There were some friction points along the way. The documentation was still evolving in places, and Isa occasionally hit error messages that weren’t descriptive enough to pinpoint issues quickly. “The docs weren’t always complete yet, which is understandable in the first versions of such a product, but honestly the team more than made up for it,” said Isa. “Whenever I hit a wall, they were there on Slack — even from completely different time zones. That’s what kept us moving fast.” The payoff came when the visualization finally clicked into place. “That was a magic moment,” said Isa. “Seeing the weather data actually render beautifully in the browser — that’s what we’d been working toward.”

One tile service feature for reduced Gaussian grids, the native grid format for ECMWF’s AIFS model, the weather model Beyond Weather was tuning, was particularly challenging — but still resolved within the week. “You could see it was designed and coded by people who know how to do this,” said Jannes.

The Earthmover Stack for AI

Jannes and the Beyond Weather team see Earthmover only becoming more deeply embedded in their infrastructure. They started using Earthmover for the map tiles service delivered through Flux, to replace the slow, unsatisfying raster visualizations they had before, but that experience exposed them to Arraylake, Earthmover’s data management platform for large-scale scientific array data. Beyond Weather has now fully adopted Icechunk as the storage engine for their training workflows, putting Earthmover at the core of their weather prediction business.

The launch of the Earthmover Data Marketplace has accelerated Beyond Weather even further. “It’s great that Earthmover has the IFS initialization data in Arraylake already,” says Jannes. “We can run our models operationally using the ECMWF data that’s already there, in Icechunk, already well-chunked. Just one repository where the initial data lives. That makes our life much easier.”

Beyond Weather is now a provider in Earthmover’s data marketplace, listing an improved wind forecast beyond 7 days for the energy sector: produced via a fine-tuned ECMWF AIFS model optimised for wind accuracy, inclusive of 100m zonal wind (u100) and 100m meridional wind (v100), with more datasets to come. Earthmover enables Beyond Weather to produce their data, serve their visualizations, and make their specialized model outputs and data products directly discoverable and usable to end-users.

Beyond Weather on the Earthmover Data Marketplace

Results

• Replaced slow, manual NetCDF-based visualization workflows with a production-ready map tile service
• Eliminated the need to build and maintain custom visualization infrastructure
• Gained a scalable path to client-facing data delivery via API
• Implemented in one week with one developer
• Chose Earthmover’s IFS open data for operational model runs
• Discovered Icechunk for optimizing their AI model training workflows and GPU utilization