Your data layer matters more than ever
Agents can build the app. Turning raw files into a dataset still takes a human, so share the dataset, and do it without copying the data.
Ian Hunt-Isaak
Xarray Community Developer
blog · author
Ian Hunt-Isaak
Xarray Community Developer
8 posts
Fundamentals: What is zero-copy ingest?
You want to use Zarr, but your data is in HDF5, TIFF, or ND2. Zero-copy ingest lets you read it as Zarr without converting it.
Ian Hunt-Isaak
Xarray Community Developer
A Data Layer for Scientific AI: Icechunk for Agentic Single-Cell Biology Workflows
Agents work best when they can branch, explore in parallel, and converge. Using examples from single-cell biology, we show how Icechunk turns a shared cloud Zarr store into a versioned, transactional system of record for agentic science.
Ryan Abernathey
CEO & Co-founder
Luiz Irber
Software Engineer
Ian Hunt-Isaak
Xarray Community Developer
Ship fast, break nothing: Engineering rigor in Icechunk with cross-version stateful compatibility testing
How we used third-wheel and Hypothesis to interleave Icechunk 1.x and 2.x in a single stateful test — and the bugs that fell out.
Ian Hunt-Isaak
Xarray Community Developer
Cursed venvs, Confident Releases: Testing Icechunk Across Major Versions
Earthmover built third-wheel, an open-source tool that rewrites Python wheels to install multiple versions of a library in one environment, enabling cross-version compatibility testing for the Icechunk V2 release.
Ian Hunt-Isaak
Xarray Community Developer
Ditch the Data Pipeline: A Snow Alert Bot in an Afternoon
How we built a Slack bot that alerts our team when it's snowing at their location, using Earthmover's Marketplace and Flux APIs to skip the data pipeline entirely.
Ian Hunt-Isaak
Xarray Community Developer
Matt Iannucci
Staff Engineer
Scientific Data Visualization with Xarray and Napari
A roadmap for integrating Xarray and napari to deliver named-dimension-aware, metadata-rich scientific data visualization across biology and geosciences.
Ian Hunt-Isaak
Xarray Community Developer
Xarray for Biology
Xarray's labeled, multidimensional data structures can solve common pain points in biological data analysis, from tracking microscopy metadata to managing complex genomic datasets. Adoption has been limited by awareness, technical rough edges, and lack of tool integration, but the community is actively working to change that.
Ian Hunt-Isaak
Xarray Community Developer
Related Posts
Blog Tags
access controls access-control agents ai AI weather AIFS announcement architecture arraylake Arraylake aurora Big Data biology blog post blog-post case study case-study chat cirrus climate risk cloud cloud native compliance compute conference data archives data cube data infrastructure data workflows dev note dynamical earth observation earth-2 earth-observation energy engineering enterprise era5 event FAIR feature release financial-services flux forecast fundamentals garbage collection geospatial GOES-R governance grib guest-post hrrr hugging-face hypothesis iceberg icechunk Icechunk IFS LangChain LangGraph marketplace mcp NASA natural language processing NCEI netCDF NetCDF noaa NOAA nowcasting nvidia nws oceanography open data open science open source performance product property-testing public-sector python Python Radar rust Rust scverse security serverless shuttle single-cell soc2 sql tables tabular data teams technical tensorops tensors testing third-wheel toxiproxy trust vector data virtual-zarr virtualizarr virtualization weather wildfire WOD xarray Xarray zarr Zarr zax zero-copy-ingest