Schema proliferation builds slowly and gets expensive fast.

One schema per event type seems reasonable - until you're:
• Managing 10+ tables
• Writing union queries across all of them
• Propagating a single field rename everywhere

The Alternative❓ Discriminator-based schema consolidation.

It reduces schema count, simplifies downstream consumption, and makes schema evolution manageable. New variants become additive changes instead of breaking existing consumers.

🔗 Check out the #InfoQ article for a practical look at the pattern and its trade-offs ⇨ https://bit.ly/4uC023n

#Java #ApacheKafka #ApacheFlink

A very nice tool from Tom Cooper for tracking KIPs and FLIPs.

#apachekafka
#apacheflink

https://ossip.dev/

Kafka vs Flink vs Spark Streaming: What Nobody Tells You Before You Pick One

#apachekafka
#apacheflink
#apachespark

https://alper-korukcu.medium.com/kafka-vs-flink-vs-spark-streaming-what-nobody-tells-you-before-you-pick-one-aa83c26a287a

#Pinterest launched a next-gen CDC-based ingestion framework.

Using #ApacheKafka, #ApacheFlink, #ApacheSpark & #ApacheIceberg, they achieved:
• Latency cut from 24+ hours to 15 minutes
• Processing of only changed records
• Support for incremental updates & deletions
• Petabyte-scale data across 1,000+ pipelines

Win: optimized cost & efficiency!

Read the architectural deep dive on InfoQ 👉 https://bit.ly/4rMJB2H

#SoftwareArchitecture #ChangeDataCapture

Estimator - Smart capacity planning for Apache Flink workloads

#apacheflink

https://mcolomerc.github.io/flink-calc/

Why Apache Flink Is Not Going Anywhere

#apacheflink

https://www.streamingdata.tech/p/why-apache-flink-is-not-going-anywhere

Flink Stateful Functions https://nightlies.apache.org/flink/flink-statefun-docs-stable/ hasn't had an update in several years and is still compiled against an old version of Apache Flink.

I spent some time during the holidays uplifting it to Flink 2.2.0 https://github.com/fransking/flink-statefun/commit/25b2dfcc3feafd701eb2fec32d413fab56e1cfe6

Notionally it appears to be working as expected.

#flink #flinkStatefun #apacheFlink

Stop 3 AM Pager Duty: Enforce Data Contracts with Apache Kafka & Apache Flink

#apachekafka
#apacheflink

https://aimindupdate.com/2025/12/03/stop-3-am-pager-duty-enforce-data-contracts-with-apache-kafka-apache-flink/

Agentic Flink - AI Agents on Apache Flink

#apacheflink

https://github.com/Ugbot/Agentic-Flink

Apache Flink 2.2.0: Advancing Real-Time Data + AI and Empowering Stream Processing for the AI Era

#apacheflink

https://flink.apache.org/2025/12/04/apache-flink-2.2.0-advancing-real-time-data--ai-and-empowering-stream-processing-for-the-ai-era/