a data lake for tensors and tables

A multi-cloud data catalog for scientific data: Icechunk repositories and Iceberg tables side by side, in your own buckets, under one set of permissions. Built for humans and agents.

scientific data is invisible

At most organizations, scientific data lives in random buckets with no discoverability or governance. Nobody can say what exists, who may use it, or which version an analysis ran on.

Lakehouse catalogs understand tables, not the multidimensional arrays most scientific data comes in — so teams with both end up running two systems, or building their own. Each one pays the same costs:

  • No catalog — nobody can say what data exists, where it lives, or whether it is current.

  • Credentials everywhere — cloud storage keys spread across laptops, notebooks, and CI jobs.

  • No version history — datasets change silently underneath the analyses that depend on them.

  • DevOps drag — scientists spend their time operating infrastructure instead of doing science.

one catalog for tensors and tables

Your data lives in open formats in your own bucket — arrays in Icechunk repositories, tables in Apache Iceberg. Arraylake holds the metadata about them and the record of who may see and change them.

Data catalog

One catalog for every dataset in your organization: Icechunk repositories and Iceberg tables side by side, with a native understanding of arrays. Browse groups, arrays, dimensions, and attributes in the web app, or let an agent explore them over MCP.

Iceberg tables

A standard Iceberg REST catalog for the data that belongs in rows. DuckDB, Spark, Polars, pandas, Trino, Snowflake, and ClickHouse read and write your tables with no Earthmover-specific connector. In public beta.

Access control and teams

Sign in with Google, GitHub, Microsoft, or your Active Directory. Grant roles on the organization or a single repository, group users and API keys into teams that inherit them, and give automation its own service accounts.

Credential vending

Arraylake brokers scoped, short-lived cloud credentials on every request — to the Python client and to Iceberg engines alike — so users and pipelines never hold long-lived storage keys.

Bring your own bucket

Configure S3, Google Cloud Storage, Azure, Cloudflare R2, or any S3-compatible endpoint. Your bytes stay in your account under your policies — or let us manage storage for you.

Version control

Every Icechunk repository has one shared version history: branches for staging changes, tags for versions that will never move, and time travel to any prior snapshot.

Zero-copy ingestion

Catalog existing NetCDF, HDF5, GRIB, and TIFF archives as virtual datasets. The bytes stay where they are; Arraylake presents them as analysis-ready Zarr.

Webhooks

Trigger downstream work on every commit: send an HTTPS webhook to your own service, or a notification to a Slack channel, configured per repository.

Background optimization

Scheduled jobs expire old snapshots and garbage-collect data nothing references any more, in an optimization window you choose, at no extra compute cost.

one identity, every cloud

Your data can live in AWS, Google Cloud, and Azure at once, and your people and pipelines still sign in once. Arraylake checks who is asking, then vends short-lived credentials scoped to exactly the dataset they asked for — so nobody ever holds a long-lived storage key, and access is revoked in one place.

Now: AWS

  1. 1Your tool sends its request to Arraylake with an Arraylake token — from single sign-on or an API key.
  2. 2Arraylake checks the caller’s roles and teams for that repository or table.
  3. 3Arraylake asks the IAM of whichever cloud holds the data — AWS, Google Cloud, or Azure — for credentials scoped to just that dataset.
  4. 4The short-lived credential goes back to your tool. No long-lived storage keys are ever handed out.
  5. 5Your tool reads and writes that bucket directly. The data never passes through Arraylake. One identity, every cloud.

familiar tools, enterprise-grade data governance

Arraylake meets you in the tools you already use. Open a repository by name and read it with Xarray, writing inside a transaction that either lands completely or not at all — or attach the Iceberg catalog to DuckDB and query your tables with SQL.

arrays · Python

import arraylake as al
import xarray as xr

client = al.Client()
repo = client.get_repo("my-org/weather-forecast")

# Read the current state of the data — credentials vended by Arraylake,
# no cloud keys in your environment
session = repo.readonly_session("main")
ds = xr.open_zarr(session.store, group="forecast", consolidated=False)

# Write a new forecast cycle as one atomic transaction
with repo.transaction("main", message="2026-08-23 00Z cycle") as store:
    ds_new.to_zarr(store, group="forecast", mode="a", append_dim="time")

tables · DuckDB

-- Point DuckDB at Arraylake's Iceberg REST catalog
INSTALL iceberg; LOAD iceberg;
CREATE SECRET al (
    TYPE ICEBERG,
    TOKEN 'ema_...',
    ENDPOINT 'https://api.earthmover.io/iceberg');
ATTACH 'my-org' AS warehouse (TYPE iceberg, SECRET al);

SELECT station, avg(temp_c) AS mean_temp
FROM warehouse."my-tables".observations
GROUP BY station
ORDER BY mean_temp DESC;

More in the quickstart guide and the Iceberg engines guide.

Science data comes in tables and tensors

Science data comes in tables and tensors

Apache Iceberg comes to Arraylake. Tables now live alongside Icechunk arrays in the same catalog, under the same permissions, in your own buckets — queryable from DuckDB, Spark, Trino, Snowflake, and Zax-SQL.

Announcing the Arraylake Community Tier

Announcing the Arraylake Community Tier

Arraylake's refreshed Community Tier unlocks powerful developer capabilities and open-science workflows for free: create your own repos, bring your own object store or use 10 GB of managed storage, and serve data through Earthmover Compute with 50 compute credits per month.

Announcing the Arraylake MCP Server

Announcing the Arraylake MCP Server

The Arraylake MCP server is now publicly available. Connect Claude, ChatGPT, Cursor, or any MCP-compatible client to your Arraylake data and let an AI assistant discover repositories, inspect schemas, query Flux, and render interactive maps.

interface

Humans and agents

Same interfaces, same permissions

Humans
web appCLIPython
Agents
MCPskillsCLIPython
compute

Powered by Zax tensor query engine

Zax-SQLtilesEDRDAP2openEO Zax Server · soon

Bring Your Own Compute

Read and write the open formats directly

NumPyXarrayPyTorchTensorFlowDaskPolarsApache SparkDuckDB and more
catalog

Data catalog and governance

catalogaccess controlteams and SSOcredential vendingIceberg REST catalogwebhooksbackground optimization

Exchange for scientific data

free and paid datazero-copy subscriptionsprivate exchanges
open formats

Zarr-compatible open tensor format

ACID transactionsversion controlzero-copy ingestfast cloud I/O

Apache Iceberg

Industry-standard open table format

ACID transactionstime travelany SQL engine

Open source, in your own bucket — readable with or without Earthmover.

cloud

Bring your own bucket

Or use storage we manage for you

AWSGoogle CloudAzureCloudflareCoreWeaveBackblazeWasabiTigrisHugging FaceVAST DataCeph generic S3-compatible

ready to give your data a home?