Old format, no problem!: Cloud-optimizing the GOES-16 archive as Virtual Zarr
Access billions of chunks of satellite imagery as a single Zarr store, without copying any data!
Tom Nicholas
Software Engineer
Tag
#fundamentals
5 posts
From Files to Datasets: FM-301 and the Future of Radar Interoperability
An introduction to the WMO FM-301 standard for weather radar data and how open-source tools like Xradar are turning fragmented binary radar files into structured, analysis-ready datasets.
Alfonso Ladino-Rincon
Data Scientist
Fundamentals: What Is Zarr? A Cloud-Native Format for Tensor Data
Zarr is an open-source, cloud-native protocol for storing chunked, compressed N-dimensional arrays. This guide covers how Zarr works, its ecosystem of tools like Xarray and Icechunk, and when to use it for large-scale scientific and ML data.
Lindsey Nield
Software Engineer
Fundamentals: What is Cloud-Optimized Scientific Data?
Why traditional scientific file formats like NetCDF perform poorly on cloud object storage, and how cloud-optimized formats like Zarr and Icechunk solve the problem by separating metadata and chunking data.
Tom Nicholas
Software Engineer
Fundamentals: Tensors vs. Tables
Multidimensional array data about the physical world is fundamentally incompatible with the tabular data model. Benchmarks show that array-native tools like Xarray and Zarr outperform DuckDB and Parquet by up to 10x for common weather data queries.
Ryan Abernathey
CEO & Co-founder
Related Posts
-
Multiscale overviews in Arraylake: GeoZarr-Powered Pyramids in Flux - and How to Generate Them in Icechunk Jun 16, 2026 -
The University of Southern Mississippi makes 90 million ocean profiles conversational: AQUAVIEW's approach to the World Ocean Database Jun 15, 2026 -
Virtual chunks that just work — but securely Jun 10, 2026 -
Icechunk Adopted by the National Weather Service: Earthmover Joins Booz Allen on NWS CIRRUS Jun 4, 2026
Blog Tags
access controls ai AI weather AIFS announcement arraylake Arraylake Big Data blog post blog-post case study case-study chat cirrus climate risk cloud cloud native conference data cube data infrastructure data workflows dev note earth observation earth-observation energy engineering enterprise era5 event FAIR feature release flux fundamentals garbage collection geospatial GOES-R hypothesis icechunk Icechunk IFS LangChain LangGraph marketplace mcp NASA natural language processing NCEI netCDF NetCDF noaa NOAA nws oceanography open data open science open source performance property-testing python Python Radar rust Rust security serverless shuttle tabular data technical tensorops tensors testing third-wheel toxiproxy vector data virtual-zarr virtualizarr weather wildfire WOD xarray Xarray zarr Zarr