Millisecond analytics on Apache Iceberg used to mean giving something up somewhere. Copy data into a serving store. Accept stale reads. Or hand one vendor the whole stack. Mid-2026 looks different, though: Snowflake now has GA support for Snowflake storage for Apache Iceberg tables and a GA Horizon Catalog REST API, while Databricks is promising millisecond query latency on Apache Iceberg through Lakehouse//RT.
That combination changes the design space for a cross-engine Apache Iceberg layer. It also makes one architectural mistake much easier to make: treating catalog choice as an afterthought. I think that’s the wrong default. In a Snowflake-plus-Databricks design, the catalog is the control plane, the storage layout is the failure domain, and the file format is just the starting point.
The useful question isn’t whether Iceberg is open. It is. The real question is whether you can keep it open while pushing latency toward milliseconds and keeping governance intact across engines with very different assumptions. That’s where Snowflake Horizon Catalog, Snowflake-managed Iceberg storage, Databricks Lakehouse//RT, and the June–July 2026 Apache Iceberg v4 discussions start to line up.
Apache Iceberg architecture changed when Snowflake stopped being just a reader
On June 1, 2026, Snowflake announced general availability of Snowflake storage for Apache Iceberg tables. Quiet release. Big consequence. An Iceberg table no longer has to be treated as an external artifact first and a Snowflake object second. Snowflake can now manage the storage and metadata while still exposing the table in Iceberg’s open format for external engines.
That’s a very different posture from the older pattern where Snowflake consumed externally managed Iceberg tables sitting in S3, Azure, or GCS under somebody else’s catalog. If Snowflake is both the governance plane and the storage manager for Iceberg, it starts to behave less like a warehouse bolted onto open tables and more like an Iceberg-native data platform with a strong SQL engine attached.
Snowflake reinforced this on May 7, 2026 by taking Apache Iceberg version 3 support to GA and making Snowflake-managed Iceberg v3 tables readable from external engines through the Horizon Iceberg REST Catalog API. That’s the part many teams will underestimate. Open files without an open-ish catalog story are awkward. Open files plus a catalog that implements Iceberg REST semantics are operationally useful.
What Horizon Catalog changes in practice
Horizon is not just metadata hosting. It is a REST catalog endpoint compatible with Iceberg’s catalog specification. Snowflake’s docs even show the shape clearly: https://<account_identifier>.snowflakecomputing.com/polaris/api/catalog/v1/config?warehouse=<database_name>. That matters because it tells you where the integration boundary is. External engines are not speaking generic Snowflake SQL APIs here; they are speaking an Iceberg-style catalog API against a dedicated surface.
The operational details matter just as much. Each user or service principal needs its own access token tied to a Snowflake role with permissions on the Iceberg tables. API calls are billed at 0.5 credits per million requests. Cheap enough that metadata chatter won’t dominate most budgets, but not free, and not something you should ignore if you have many short-lived clusters constantly refreshing table state.
If I were designing this from scratch, I’d keep Snowflake as the authoritative catalog for the shared Iceberg layer only when governance is the top priority and write coordination is tightly controlled. I would not let every engine write everywhere just because Iceberg allows multiple writers in theory. Cross-engine writes are where clean diagrams go to die.
A cross-platform Apache Iceberg design that actually holds up
Picture the architecture in prose. At the bottom sits object storage: S3, Azure Blob, GCS, or an S3-compatible target. Above that is the Iceberg table layer: data files, manifests, manifest lists, snapshots, metadata JSON. Snowflake either manages those tables directly using Snowflake storage for Apache Iceberg tables or reads and writes through an external volume. Because Horizon speaks the Iceberg REST catalog spec, an external engine like Databricks could in principle read the same logical tables through catalog integration and run low-latency queries with Lakehouse//RT — but as of mid-2026 neither vendor documents a verified Lakehouse//RT-plus-Horizon path, so treat that link as a design target to validate, not a shipped feature. Spark Structured Streaming handles ingestion. Trino or Dremio may show up later for federated access. The catalog is the switchboard.
Snowflake’s external volume model gives you two concrete modes. A read-only external volume for externally managed Iceberg tables. A read/write external volume for Snowflake-managed tables. You can verify connectivity with SYSTEM$VERIFY_EXTERNAL_VOLUME('my_s3_external_volume'), and defaults can be set at the account, database, or schema level through the Scope field in Snowsight. Those are not cosmetic details. They define who is allowed to mutate files and where accidental writes can happen.
The cleanest pattern for a shared data layer is boring by design:
Use one principal writer per table family. Keep Snowflake as catalog authority when the table must be broadly shared. Let Spark streaming jobs write to controlled paths only. Let Databricks Lakehouse//RT query those tables for serving workloads where that cross-engine path is actually supported. Reserve cross-engine writes for narrow, tested cases such as append-only event streams with deterministic compaction windows.
I’d rather accept a bit of write centralization than debug snapshot conflicts between two vendors with different retry behavior. That’s not anti-open. Just practical engineering.
Reference flow: streaming into Iceberg, querying from Snowflake and Databricks
A realistic flow looks like this. A Spark Structured Streaming job ingests Kafka topics, writes micro-batches into Iceberg tables on S3, and commits snapshots at a bounded cadence. Snowflake governs its own managed Iceberg tables and exposes them to external engines through the Horizon REST catalog — the documented direction is external reads of Snowflake-managed v3 tables, so if you want those Spark-written tables under the same governance you either register them with Snowflake or keep them in a separate externally managed catalog rather than assuming Horizon re-exports them on its own. Databricks Lakehouse//RT, powered by Reyden, could then query those governed Apache Iceberg tables for millisecond-latency dashboards and APIs — assuming that Lakehouse//RT-to-Horizon path is supported in your accounts, which as of mid-2026 isn’t vendor-verified and should be proven before you design around it.
curl -H "Authorization: Bearer $TOKEN"
"https://<account_identifier>.snowflakecomputing.com/polaris/api/catalog/v1/config?warehouse=<database_name>"That one request is enough to show the pattern. Your compute engines need a token lifecycle, a role mapping model, and a cache invalidation policy for metadata. If those are hand-waved away, the architecture isn’t done.
Databricks Lakehouse//RT raises the bar for Iceberg latency
Catalogs and snapshots are the hidden coordination layer that makes open table architectures work.
The June 2026 Databricks on AWS release notes introduced Lakehouse Real-Time, or Lakehouse//RT, in Beta. Databricks’ press release goes further: Reyden is a new compute engine that delivers millisecond query latency and runs real-time analytics directly on governed Delta Lake and Apache Iceberg data. If the claim holds under mixed production workloads, that is a serious shift. It means the serving layer may stop being a separate database for many teams.
The interesting part is not the word “millisecond.” Vendors say that a lot. The interesting part is that Databricks explicitly included Apache Iceberg beside Delta Lake. That implies Reyden has to treat Iceberg snapshot semantics, manifest planning, schema evolution, and delete handling as first-class concerns rather than compatibility leftovers.
For architecture, this creates a useful split. Snowflake can own governed storage and broad SQL access. Databricks can own low-latency read paths where teams would previously have copied data into Pinot, Druid, ClickHouse, or a custom serving tier. Maybe you still keep those systems for specific patterns. But the burden of proof just changed.
What will make or break Lakehouse//RT on Iceberg
Query latency on Iceberg rarely dies in the scan operator first. It dies in metadata planning, small-file amplification, and too many snapshots with poor retention hygiene. Databricks’ June 2026 notes also mention billing changes such as snapshot storage becoming billable for Lakebase Autoscaling starting June 1, 2026. Not directly a Lakehouse//RT feature, but it’s a useful reminder that snapshots are not abstract. They cost money, and real-time systems tend to create more of them.
If you want Snowflake and Reyden to stay fast on the same Iceberg layer, keep file size distribution, manifest fan-out, snapshot frequency, and partition evolution under control. Small files punish everybody. Huge manifests punish planning. Hyperactive snapshots make retention and metadata scans expensive. Partition changes help until they don’t, especially when one engine caches assumptions longer than another. I’d avoid loose retention settings in production.
This is where ClickHouse offers a useful side signal. The 2026 ClickHouse Cloud changelog mentioned expanded ClickPipes controls including Kafka exactly-once delivery, while ClickPipes for Kafka documentation still describes at-least-once delivery semantics. That’s not a contradiction so much as a warning label. Real-time systems improve guarantees gradually, and your Iceberg design has to survive the period where duplicates, retries, and out-of-order commits still exist.
Security, IAM, and catalog discipline in a Snowflake Horizon Catalog setup
Cross-engine Apache Iceberg falls apart faster in identity than in storage. Snowflake’s model is role-centric. Horizon requires access tokens per user or service principal, tied to Snowflake roles with table privileges. External volumes require grants such as USAGE to roles with the right ownership path. Databricks usually approaches the same storage through service principals, instance profiles, or credential passthrough. Same bucket. Different trust model.
The disciplined pattern is straightforward, even if the plumbing isn’t. Put a central identity provider in front. Issue short-lived credentials. Map one platform-neutral data access profile to one Snowflake role and one Databricks principal set. Keep storage paths narrow. Separate read principals from write principals. Don’t let catalog permissions imply storage permissions by accident.
Snowflake’s docs around external volumes matter here more than people expect. You can define specific storage locations for Amazon S3, Microsoft Azure, Google Cloud, or S3-compatible providers, and you can scope defaults at account, database, or schema level. That’s useful for guardrails. It also means a sloppy default can expose far more storage than intended. This is the wrong default for regulated data. Prefer explicit locations and per-domain volumes.
Row-level and column-level controls are trickier. If Snowflake enforces policies that Databricks cannot mirror exactly for the same Iceberg table, then the “shared table” story becomes asymmetric. In that case, treat the table as physically shared but logically distinct by access tier. Engineers hate hearing that because it feels inelegant. Auditors usually don’t.
Designing for Apache Iceberg v4 before Apache Iceberg v4 exists
Alex Merced’s June 2026 piece, The State of Apache Iceberg Catalogs in June 2026, made the right point: catalogs are central to how engines interact with Iceberg tables because they manage metadata, schemas, snapshots, and locations. His July 2026 follow-up, The State of Apache Iceberg v4 in July 2026: What the Dev List Tells Us About the Format’s Next Chapter, sharpened that into concrete themes around improved row-level deletes, more flexible partition specification, and semantics better aligned with mixed streaming and batch workloads.
v4 is not here yet. You still need to design for it now. That means avoiding table patterns that depend on one engine’s quirks around deletes, metadata refresh, or partition transforms. It means assuming catalog behavior will matter more, not less, as transactions become richer and multi-engine concurrency gets more ambitious.
Starburst’s Apache Iceberg vs Delta Lake comparison is a useful anchor here. Delta Lake historically stayed tighter to Databricks and Spark. Iceberg aimed for broad engine compatibility from the start. Mid-2026 muddies that old boundary. Databricks is pushing Iceberg in Lakehouse//RT, and Snowflake is embracing Iceberg storage and Horizon Catalog. My read is simple: use Delta where Spark-native ergonomics and Delta-specific tooling are decisive, but use Iceberg as the shared contract when multiple engines must read the same governed data without format translation.
The deeper opinion is this: if your organization says it wants optionality, then the catalog and table behavior should be reviewed with the same seriousness as network architecture or key management. Anything less is branding, not engineering.
The real decision: who writes, who catalogs, who serves
A strong mid-2026 pattern is emerging. Let one system own the write path for a given Iceberg domain. Let one catalog act as authority. Let multiple engines compete on read performance. Snowflake now makes a credible case to own the governed catalog plus managed Iceberg storage side through Horizon and Snowflake storage for Apache Iceberg tables. Databricks makes a credible case to own the low-latency serving side through Lakehouse//RT and Reyden.
The goal is durable cross-engine calm: governed storage, low-latency access, and room for Iceberg v4.
The open question isn’t whether this can work. It can. The open question is how much cross-engine mutation you really want before Apache Iceberg v4 settles more of the semantics that practitioners are still inferring from vendor behavior. If I had to choose today, I’d keep writes conservative, catalogs explicit, snapshots tidy, and latency ambitions high. Start there, then decide exactly where you’re willing to let a second engine commit.