Old format, no problem!: Cloud-optimizing the GOES-16 archive as Virtual Zarr
Access billions of chunks of satellite imagery as a single Zarr store, without copying any data!
Tom Nicholas
Software Engineer
blog · tag
#fundamentals
5 posts
From Files to Datasets: FM-301 and the Future of Radar Interoperability
An introduction to the WMO FM-301 standard for weather radar data and how open-source tools like Xradar are turning fragmented binary radar files into structured, analysis-ready datasets.
Alfonso Ladino-Rincon
Data Scientist
Fundamentals: What Is Zarr? A Cloud-Native Format for Tensor Data
Zarr is an open-source, cloud-native protocol for storing chunked, compressed N-dimensional arrays. This guide covers how Zarr works, its ecosystem of tools like Xarray and Icechunk, and when to use it for large-scale scientific and ML data.
Lindsey Nield
Senior Engineer
Fundamentals: What is Cloud-Optimized Scientific Data?
Why traditional scientific file formats like NetCDF perform poorly on cloud object storage, and how cloud-optimized formats like Zarr and Icechunk solve the problem by separating metadata and chunking data.
Tom Nicholas
Software Engineer
Fundamentals: Tensors vs. Tables
Multidimensional array data about the physical world is fundamentally incompatible with the tabular data model. Benchmarks show that array-native tools like Xarray and Zarr outperform DuckDB and Parquet by up to 10x for common weather data queries.
Ryan Abernathey
CEO & Co-founder
Related Posts
Blog Tags
access controls access-control ai AI weather AIFS announcement arraylake Arraylake aurora Big Data blog post blog-post case study case-study chat cirrus climate risk cloud cloud native compliance compute conference data cube data infrastructure data workflows dev note dynamical earth observation earth-observation energy engineering enterprise era5 event FAIR feature release financial-services flux fundamentals garbage collection geospatial GOES-R governance grib guest-post hrrr hugging-face hypothesis iceberg icechunk Icechunk IFS LangChain LangGraph marketplace mcp NASA natural language processing NCEI netCDF NetCDF noaa NOAA nws oceanography open data open science open source performance product property-testing public-sector python Python Radar rust Rust security serverless shuttle soc2 sql tables tabular data teams technical tensorops tensors testing third-wheel toxiproxy trust vector data virtual-zarr virtualizarr weather wildfire WOD xarray Xarray zarr Zarr zax