If you’re running a data lakehouse, you already know that Iceberg point releases tend to be quiet — a dependency bump here, a bugfix there. The 1.11.0 (released on May 19) is not that release. It brings structural changes that touch how you query, how you secure, and — this is key — what JVM you’re allowed to run.
Let’s walk through what really changes for you, and then the migration problem you need to plan before upgrading.
Spark 4.1 and Flink 2.1 are now the default targets
The headline: Spark 4.1 and Flink 2.1 are now the default build targets. And the most practical improvement comes with Spark 4.1’s MERGE INTO.
Now you can run a MERGE with a WITH SCHEMA EVOLUTION clause. If your source data brings columns that your destination table doesn’t yet have, the MERGE adds those columns to the table in the same statement — without a separate ALTER TABLE. For anyone maintaining ingestion pipelines where upstream schemas change all the time, that’s one less moving piece to worry about.
The Spark connector also modernizes against the newer DataSource V2 APIs and adds an asynchronous micro-batch planner that speeds up Structured Streaming.
On the Flink side, the centerpiece is the experimental DynamicIcebergSink: instead of one sink per table, a single sink routes each record to a table chosen at runtime — creating tables on demand and evolving their schemas and partition specs on the fly. It’s experimental, so treat it as such, but it breaks a pattern that’s been frustrating Flink+Iceberg users for years.
Server-side scan planning: metadata work moves out of your engine
Until now, your query engine did the heavy lifting of scan planning — the driver traversed the table’s metadata tree, pulling manifest lists and files from object storage to filter against your partitions.
1.11.0 moves that work to the catalog. The engine sends a single POST .../plan request and the REST catalog returns optimized FileScanTasks. It scales by design: small scans return results instantly, larger ones return a plan-id that you poll, and enormous datasets come back through plan-tasks in parallel.
In plain terms: your query engine stops being the bottleneck for planning, and that work lives where it can be optimized centrally.
Table-level encryption integrated with Google KMS
Bucket-level encryption never cut it for a lakehouse holding PII or financial data — and 1.11.0 finally brings encryption at the table level.
Iceberg uses envelope encryption with a three-level key hierarchy: a table master key lives in your KMS and never touches Iceberg storage; that key wraps the key-encryption keys (KEKs) stored in the table’s metadata; and each KEK wraps a unique data-encryption key (DEK) per file. Each data file and each manifest list is encrypted with AES-GCM under its own DEK.
The part that matters for compliance teams: it also encrypts manifest lists, not just raw data. That closes a real leak — an attacker with bucket access before could read the manifest metadata and extract file paths, column names, partition bounds, and exact null counts without ever touching a data file. Keys also rotate automatically as they age, so you meet rotation mandates without having to rewrite enormous datasets.
The breaking change: Java 11 is gone
Here’s the one you need to plan for. 1.11.0 drops support for Java 11 — you now need Java 17 or 21. Support for Spark 3.4 is also deprecated.
This isn’t something “negative” — it’s the cost of staying current, and you should know about it before you schedule the upgrade, not after. If your platform still runs on Java 11 or got stuck on Spark 3.4, your migration has a prerequisite: upgrade your runtime first, then update Iceberg. Trying to do both things in the same window on a production lakehouse is the recipe for ruining your weekend.
A practical sequence:
- Audit which engines and JVMs your Iceberg workloads currently run on.
- Move to Java 17/21 and get off Spark 3.4 in a staging environment.
- Validate your current pipelines against the new Spark connector (the modernization to DSv2 might surface edge cases).
- Only then upgrade Iceberg to 1.11.0.
Things to keep in mind
A couple of honest caveats: DynamicIcebergSink and parts of the Flink 2.1 support are experimental — useful for piloting, not for betting a production pipeline on yet. The integrated KMS encryption shipped with support for Google KMS, so if you’re on another provider, verify current support before assuming it’s at parity. And the JVM/engine upgrade is real operational work — budget for it.
None of that takes away from the release. For a data platform team, 1.11.0 is one of the most substantial Iceberg drops in a good while: a real shift in query planning architecture, table-level encryption that closes a metadata leak, and a Spark MERGE that finally handles schema drift on its own.
What about you? Do you already have your stack on Java 17/21, or will the jump from Java 11 hold up your migration to 1.11.0? Let us know how your lakehouse is doing.