🚀 Big Data meets AI—powered by Iceberg, Spark & LLMs

At #ArcOfAI, Pratik Patel shows how to build a real architecture that lets users query massive datasets with natural language—no dashboards, no SQL, just questions & insights.

https://www.arcofai.com/speaker/1c241471d7f04018a0da70efffd35b32

🎟️ Get tickets: https://arcofai.com

#ArtificialIntelligence #BigData #DataArchitecture #ApacheSpark #ApacheIceberg #LLM #GenAI #EventStreaming #Kafka #Flink #AIEngineering #TechLeadership

So, feeling deflated. I put a lot of effort into #ApacheIceberg material and noticed a .palantir folder in the source which I believe is for build repository tooling. But still felt shock that #Palintir, who runs the data analysis for #ICE and #CBP has some dependencies in this project. It also seems that there were some code contributions from Palantir into Iceberg when used in Netflix. What to do about #Opensource when this happens. Source link below.

https://github.com/apache/iceberg

Go start your free trial of the Dremio Agentic Lakehouse at https://drmevn.fyi/am-get-started

#DataEngineering #DataAnalytics #ApacheIceberg #ApachePolaris #AgenticAI #AgenticAnalytics

#AWS announced 2 new capabilities for #S3Tables!

🔹 Intelligent-Tiering storage class that automatically optimizes costs based on access patterns
🔹 Replication support that keeps Apache Iceberg table replicas consistent across AWS regions and accounts - no manual syncing required

Find out more: https://bit.ly/4qgRn3Y

#CloudComputing #S3 #ApacheIceberg #InfoQ

Преодоление разрыва между озерами данных и хранилищами данных

​Системы хранения данных типа «озера данных» сочетают в себе гибкость озер данных с надежностью, производительностью и возможностями управления, характерными для хранилищ данных.

В современных аналитических системах компании в значительной степени полагаются на озера данных...

#DST #DSTGlobal #ДСТ #ДСТГлобал #озёраданных #хранилищаданных #lakehouse #ApacheIceberg #Метаданные #Кэширование

Источник: https://dstglobal.ru/club/1144-preodolenie-razryva-mezhdu-ozerami-dannyh-i-hranilischami-dannyh

#DuckDB now supports end-to-end interaction with Iceberg REST Catalogs directly in the browser - no infrastructure setup required.

With DuckDB-Wasm, users can query, read, and write Iceberg tables seamlessly.

Learn more: https://bit.ly/4qCTYoF

#DataAnalytics #WebAssembly #ApacheIceberg #AI #InfoQ

Will I see you at the Subsurface Lakehouse Conference Nov 13th?

Register at Dremio.com/subsurface

#DataLakehouse #ApacheIceberg #ApacheArrow #ApachePolaris

Are you subscribed?

Subscribe to my blog on medium or substack to get regular updates on the data and AI world. Find all the links at AlexMerced.com/data.

#ApacheIceberg #DataLakehouse #DataEngineering

Cloudflare has just launched the open beta of its Cloudflare Data Platform - a managed service for ingesting, storing & querying analytical data tables using open standards like Apache Iceberg.

🔍 Dive into the key insights on #InfoQ ⇨ https://bit.ly/49y1tIa

#CloudComputing #DataLake #DataAnalytics #ApacheIceberg #Cloudflare

scrapy-contrib-bigexporter 0.6.1 released: https://codeberg.org/ZuInnoTe/scrapy-contrib-bigexporters

Added: You can customize Iceberg table location

#scrapy #webscraping #bigdata #iceberg #apacheiceberg #opensource #python