Aurora PostgreSQL queries Iceberg and Parquet directly with embedded DuckDB
AWS embeds DuckDB inside Aurora PostgreSQL to query operational data and the data lake in a single query, with no ETL pipeline. A turning point for teams that replicated the lake into Postgres.
September 30, 2026. AWS announces that Aurora PostgreSQL can now directly query Apache Iceberg and Parquet data stored in a data lake, from existing PostgreSQL applications and tools. August 2026. Amazon signed the acquisition of DuckLabs, the team behind DuckDB, and it is that engine that is now embedded in Aurora. September 30, 2026. The capability is available in all commercial regions and GovCloud, at no additional charge. Why it matters: this is the first major product integration of DuckDB into an AWS database service, and it erases the replication pipeline between the lake and the operational database.
DuckDB embedded in the database engine
The announcement materializes an acquisition, not a cosmetic integration. The DuckDB engine — the open source analytical database that runs SQL directly over Parquet, CSV or JSON files — is now compiled into Aurora PostgreSQL. Query processing stays inside Aurora, with no additional network hop and no data duplication.
The scope is technical but direct. A single query can join live operational data, including uncommitted writes, with Iceberg or Parquet files sitting in Amazon S3 or S3 Tables. All in familiar PostgreSQL syntax, with the same clients, tools and endpoints as today.
DuckDB is a natural fit for the job: an in-process, columnar engine built to scan Parquet files with minimal overhead, which is why AWS embedded it rather than bolting on a separate query engine.
Setup takes a few commands. Create the extension, then a foreign table pointing at the remote file:
CREATE EXTENSION aurora_analytics;
CREATE FOREIGN TABLE transaction_history ()
SERVER aurora_analytics_server
OPTIONS (
location 's3://<my-bucket>/finance/transaction_history.parquet',
format 'parquet'
);Two details deserve attention. The empty parentheses in CREATE FOREIGN TABLE mean Aurora reads the schema from the Parquet file metadata — no columns to declare by hand. And a single IMPORT FOREIGN SCHEMA bulk-creates foreign tables for every Iceberg or Parquet table in an AWS Glue Data Catalog database.
One query, two data worlds
The official example shows the exact use. A merchant keeps the last 7 days of transactions in an Aurora table, and 5 years of history in a Parquet file on S3. Before, joining the two required a pipeline. Now a single query suffices — here a UNION ALL between the recent table and the foreign table:
SELECT merchant, category, amount, transaction_date, 'recent' AS source
FROM recent_transactions
WHERE customer_id = 'C-1001'
UNION ALL
SELECT merchant, category, amount, transaction_date, 'historical' AS source
FROM transaction_history
WHERE customer_id = 'C-1001'
AND transaction_date >= CURRENT_DATE - INTERVAL '5 years'
ORDER BY transaction_date DESC
LIMIT 15;Under the hood the split is clean: Aurora handles the operational rows, DuckDB runs the analytical scan of the Parquet file. The boundary between transactional database and lake — the thing that justified a whole family of tools — becomes a question of a foreign table.
The capability goes beyond S3. Federation through the Glue Data Catalog lets you query Iceberg REST Catalog (IRC)-compatible catalogs, registered once. A single query can then join Iceberg tables spread across multiple catalogs, without moving data or abandoning existing catalog investments.
The use case that disappears: reverse-ETL
The real change is what you no longer have to build. Until now, combining recent transactions with history in S3 meant reverse ETL pipelines: duplicating data, paying for infrastructure, and keeping everything synchronized. That burden grows as AI agents enter applications, because it becomes impossible to predict and pre-replicate every dataset an agent might need.
The target use case is crisp: real-time dashboards, enriching transactions with historical context, or agents that reason over both live and archived data — all behind a single SQL interface. AWS’s argument is complexity removal, not a new feature to learn.
For queries that demand single-digit-millisecond latency, the reverse gateway exists: materialize lake data into a native Aurora table via CREATE TABLE AS SELECT, INSERT INTO … SELECT or MERGE INTO. The materialized table lives in Aurora and is queried like any PostgreSQL table, with no separate ingestion pipeline.
What it optimizes, and what it costs
The optimizations are built in rather than bolted on. Aurora applies predicate pushdown and column pruning to read only the relevant data, even as the lake grows. Frequently accessed data is cached in the instance, speeding up later queries. An aurora_analytics_stat_statements() function exposes per-query metrics — rows scanned, bytes read from S3, cache hits — enough to measure exactly what you save.
The cost model is simple. No additional charge for the capability itself: you pay the incremental Aurora compute the queries consume and the S3 request costs for reading lake files. The gain comes from no longer paying for the replication pipeline or the duplicated storage.
Availability is broad but versioned: the capability requires Aurora PostgreSQL 17.11 or 18.6 or later. Setup goes through an IAM role carrying the AuroraAnalytics feature, which grants Aurora access to S3 and the Glue Data Catalog — a permission checkpoint worth scoping from day one.
A signal about AWS’s strategy
The announcement matters beyond the product. By embedding DuckDB in Aurora, AWS lays the first stone of a broader strategy: making the embedded analytical engine a standard bridge between the operational database and the lake, rather than a separate product. The official wording is explicit — future improvements to the open source engine “can continue to bring performance and functionality gains to Aurora and other AWS services.”
The competitive reading is simple. Against vendors that sell the lake and the warehouse as two distinct products, AWS chooses to dissolve the boundary on the database side. For a team already committed to Aurora and S3, the switching cost is close to zero; for everyone else, it is one more simplification argument in the balance.
Verdict
If you run reverse-ETL pipelines or Glue jobs today to hydrate Postgres from a lake, this capability removes that pipeline for analytical queries: test it on a real case and measure the gain with aurora_analytics_stat_statements(). If your workloads demand sub-millisecond latency, materialize the hot data into Aurora and keep the lake for cold history — both paths coexist. If you are multi-region or in the public sector, check the deployed Aurora version and scope the AuroraAnalytics role to the minimum: that is the only real risk lever in an otherwise cost-free integration.