Mastodawn

The Data Lakehouse Explained: Why Apache Iceberg Is Quietly Running the Show

https://techlife.blog/posts/data-lakehouse-iceberg

#ApacheIceberg #DataLakehouse #DataWarehouse #DataLake #Snowflake #ApacheSpark #DataEngineering

The Data Lakehouse Explained: Why Apache Iceberg Is Quietly Running the Show

Data warehouses were expensive. Data lakes turned into swamps. Enter the Lakehouse — and the open table format that makes it actually work.

TechLife — AI, Software Engineering & Emerging Technology

InfoQ Mar 3

#Pinterest launched a next-gen CDC-based ingestion framework.

Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelines

Win: optimized cost & efficiency!

Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H

#SoftwareArchitecture #ChangeDataCapture

arcofai Feb 10

🚀 Big Data meets AI—powered by Iceberg, Spark & LLMs

At #ArcOfAI, Pratik Patel shows how to build a real architecture that lets users query massive datasets with natural language—no dashboards, no SQL, just questions & insights.

https://www.arcofai.com/speaker/1c241471d7f04018a0da70efffd35b32

🎟️ Get tickets: https://arcofai.com

#ArtificialIntelligence #BigData #DataArchitecture #ApacheSpark #ApacheIceberg #LLM #GenAI #EventStreaming #Kafka #Flink #AIEngineering #TechLeadership

Daniel Hinojosa

Jan 26

So, feeling deflated. I put a lot of effort into #ApacheIceberg material and noticed a .palantir folder in the source which I believe is for build repository tooling. But still felt shock that #Palintir, who runs the data analysis for #ICE and #CBP has some dependencies in this project. It also seems that there were some code contributions from Palantir into Iceberg when used in Netflix. What to do about #Opensource when this happens. Source link below.

https://github.com/apache/iceberg

GitHub - apache/iceberg: Apache Iceberg

Apache Iceberg. Contribute to apache/iceberg development by creating an account on GitHub.

GitHub

Alex Merced Jan 17

Go start your free trial of the Dremio Agentic Lakehouse at https://drmevn.fyi/am-get-started

#DataEngineering #DataAnalytics #ApacheIceberg #ApachePolaris #AgenticAI #AgenticAnalytics

InfoQ Jan 12

#AWS announced 2 new capabilities for #S3Tables!

🔹 Intelligent-Tiering storage class that automatically optimizes costs based on access patterns
🔹 Replication support that keeps Apache Iceberg table replicas consistent across AWS regions and accounts - no manual syncing required

Find out more: https://bit.ly/4qgRn3Y

#CloudComputing #S3 #ApacheIceberg #InfoQ

DST Global Jan 11

Преодоление разрыва между озерами данных и хранилищами данных

Системы хранения данных типа «озера данных» сочетают в себе гибкость озер данных с надежностью, производительностью и возможностями управления, характерными для хранилищ данных.

В современных аналитических системах компании в значительной степени полагаются на озера данных...

#DST #DSTGlobal #ДСТ #ДСТГлобал #озёраданных #хранилищаданных #lakehouse #ApacheIceberg #Метаданные #Кэширование

Источник: https://dstglobal.ru/club/1144-preodolenie-razryva-mezhdu-ozerami-dannyh-i-hranilischami-dannyh