Cold data is read-only - that's the assumption baked into basically every tiering solution on the market. ColdFront breaks it.

UPDATE & DELETE on archived rows work through standard SQL. A GDPR deletion on five-year-old events is a single DELETE statement. No restore cycle.

DuckDB in-process, Apache Iceberg on any S3. Stock unpatched #PostgreSQL 16/17/18. Beta now, PostgreSQL License. Led by Jimmy Angelakos.

📖 https://github.com/pgEdge/coldfront

#OpenSource #DataEngineering #ApacheIceberg #DuckDB

pgEdge ColdFront: #PostgreSQL data tiering. Hot data in the heap, cold to Apache Iceberg on S3 - up to 90% lower storage cost.

The cold tier is writable. UPDATE & DELETE on cold rows work in standard SQL. No restore cycle, no rehydration. No app changes.

DuckDB runs in-process. No daemon, no sidecar. PostgreSQL License, beta now. Led by @vyruss.

Press release: 👉 https://www.pgedge.com/press-releases/pgedge-announces-coldfront-for-postgresql

GitHub: 🔗 https://github.com/pgEdge/coldfront

#OpenSource #DataEngineering #ApacheIceberg #DuckDB

I'lll spare you the "I did a thing" cliché and just say I'm really proud to share what I've been building lately: #pgEdge ColdFront is live.

Figuring out how to move #PostgreSQL data to #S3 while keeping it queryable and updatable as if it were still local, without requiring any application code changes, is exactly the kind of hard problem I enjoy.

Check out the link below to see how it all works under the hood.

https://hubs.la/Q04lS0fb0

#Postgres #OpenSource #ApacheIceberg #Analytics #LLM #AI

🧊 pgEdge ColdFront beta is out - transparent data tiering for #PostgreSQL. Fully writable cold tier.

Hot data stays in the heap. Cold data moves to Iceberg on S3 at up to 90% lower cost. UPDATE & DELETE on archived rows, same SQL. No rehydration. No code changes.

DuckDB runs in-process - no daemon, no RPC. C extension routes DML to the correct tier transparently.

Development by Jimmy Angelakos. Blog by Antony Pegg: 📖 https://www.pgedge.com/blog/introducing-coldfront-seamlessly-uniting-oltp-analytics-and-ai-workloads-on-postgresql

#OpenSource #DataEngineering #ApacheIceberg #DuckDB

DuckDB Labs released #DuckLake 1.0 - a data lake format that stores table metadata in a SQL database, rather than spreading it across object storage files.

Key features:
• catalog-stored small updates
• improved sorting and partitioning
• compatibility with Iceberg-style data features

Learn more ⇨ https://bit.ly/48PsPIS

#InfoQ #DuckDB #ApacheIceberg #AI #DataLake #DataStorage

Lakehouse architectures allow multiple engines to run on shared data through open table formats like #ApacheIceberg.

But #SQL identifier resolution and catalog naming rules differ across engines - creating hidden interoperability failures.

In this #InfoQ article, Maninder Parmar explains why enforcing consistent naming conventions and cross-engine validation is critical.

📰 Read now: https://bit.ly/4902zeH

#RelationalDatabases #DataLake

IT'S FINALLY COMPLETE! (35% OFF)

Just submitted the last bits to complete my latest book with Manning.

As always, thanks for your support over the years. It really does mean a lot, and it has been quite the ride!

Find this and all my other books at https://books.alexmerced.com

#ApacheIceberg #DataLakehouse #DataEngineering

The Data Lakehouse Explained: Why Apache Iceberg Is Quietly Running the Show

https://techlife.blog/posts/data-lakehouse-iceberg

#ApacheIceberg #DataLakehouse #DataWarehouse #DataLake #Snowflake #ApacheSpark #DataEngineering