Ship fast, break nothing: Engineering rigor in Icechunk with concurrency & fault injection testing
We describe our approach to rigor in Icechunk 2 including concurrency permutation testing and more!
Deepak Cherian
Forward Deployed Engineer
Deepak Cherian
Forward Deployed Engineer
5 posts
Ship fast, break nothing: Engineering rigor in Icechunk with property and stateful testing
How we engineer rigor into Icechunk and its upstream brethren using property and stateful testing techniques. Also, a cautionary tale.
Deepak Cherian
Forward Deployed Engineer
Sebastian Galkin
Staff Engineer
Wicked smaht dynamic map tile rendering of Icechunk/Zarr data with xpublish-tiles
Earthmover is launching a new open-source library xpublish-tiles that powers our new Flux Tiles service, which allows Earthmover Platform users to view their data on a slippy map with dynamically rendered tiles at lower zoom levels than was possible previously.
Deepak Cherian
Forward Deployed Engineer
Matt Iannucci
Engineering
Ergonomic seasonal grouping and resampling in Xarray
Xarray introduces SeasonGrouper and SeasonResampler, two new Grouper objects that enable custom, overlapping, and variable-length seasonal aggregations without workarounds.
Deepak Cherian
Forward Deployed Engineer
Cloud-native platforms are the natural evolution of atmosphere-ocean open-data practice
Open-data practice in ocean/atmosphere sciences is approximately 170 years old! While it is easy to exclaim, "weather/climate are global, of course we must share data", the actual story is anything but. That story holds valuable inspiration that we can draw from as we face a significant reduction in US climate science research.
Deepak Cherian
Forward Deployed Engineer
Related Posts
-
Beyond Weather: On-demand visualizations of fine tuned AIFS models over NetCDF workarounds in one week May 7, 2026 -
Variable length chunks in Zarr May 5, 2026 - Ship fast, break nothing: Engineering rigor in Icechunk with concurrency & fault injection testing Apr 28, 2026
- Ship fast, break nothing: Engineering rigor in Icechunk with property and stateful testing Apr 22, 2026
Blog Tags
access controls AI weather AIFS announcement arraylake Big Data blog post blog-post case study case-study climate risk cloud cloud native conference data cube data workflows dev note earth observation earth-observation energy engineering event FAIR feature release flux fundamentals garbage collection geospatial GOES-R hypothesis icechunk IFS marketplace NASA netCDF NetCDF NOAA oceanography open data open science open source performance property-testing Python Radar rust Rust serverless shuttle tabular data technical tensorops tensors testing third-wheel toxiproxy vector data weather wildfire xarray Xarray zarr Zarr