blog

Note: This post was originally published on the Zarr developer blog. After more than a year of development, we’re thrilled to announce the release of Zarr-Python 3! This major release brings full support for the Zarr v3 specification, including the new chunk-sharding extension, major performance enhancements, and a thoroughly modernized codebase. Whether you use Zarr to managing large multi-dimensional datasets in the cloud or for high-performance machine learning applications, we’ve built Zarr-Python 3 to help you. Let’s dive into some of the details of this release! Zarr-Python 3 is available today on PyPI and Conda-Forge. It is compatible with Python 3.11 and above. pip install --upgrade zarr # or conda install --channel conda-forge zarr Support for Zarr’s v3 specification The most n…

TLDR We are excited to announce the release of the Icechunk storage engine, a new open-source library and specification for the storage of multidimensional array (a.k.a. tensor) data in cloud object storage. Icechunk works together with Zarr, augmenting the Zarr core data model with features that enhance performance, collaboration, and safety in a multi-user cloud-computing context. With the release of Icechunk, powerful capabilities such as isolated transactions and time travel, which were previously only available to Earthmover customers via our Arraylake platform, are now free and open source. Head over to icechunk.io to get started! This is a blog version of a webinar that took place on October 22, 2024. View the presentation slide deck or check out the video of that webinar: The …