Is your Kafka lag dashboard lying? Learn how log compaction and retention silently manipulate reported timestamps to understate consumer delays and how to identify these inaccuracies and ensure your monitoring reflects actual system performance.
https://softwaremill.com/compaction-and-retention-edge-cases-that-make-your-kafkas-lag-metrics-inaccurate/

#ApacheKafka #DataEngineering #DistributedSystems #EventStreaming #Observability

Apache Kafka 4.2.0 is here – and it’s a big one. The headliner? Queues for Kafka (KIP-932) is officially GA. This means you can finally use Kafka for classic "competing consumer" queuing patterns without the rigid constraints of partition assignments.
But there is much more:

#ApacheKafka #Kafka42 #EventStreaming #DataEngineering #DistributedSystems

This is a hands-on walkthrough of building a real-time dashboard with Quarkus + Redis TimeSeries.

Live ingestion via WebSockets, automatic downsampling, multi-resolution queries, and a simple browser UI.
Crypto is just the data source. The patterns apply to metrics, IoT, and event streams.

https://www.the-main-thread.com/p/real-time-crypto-dashboard-java-quarkus-redis-timeseries

#Java #Quarkus #Redis #TimeSeries #BackendEngineering #EventStreaming

🚀 Big Data meets AI—powered by Iceberg, Spark & LLMs

At #ArcOfAI, Pratik Patel shows how to build a real architecture that lets users query massive datasets with natural language—no dashboards, no SQL, just questions & insights.

https://www.arcofai.com/speaker/1c241471d7f04018a0da70efffd35b32

🎟️ Get tickets: https://arcofai.com

#ArtificialIntelligence #BigData #DataArchitecture #ApacheSpark #ApacheIceberg #LLM #GenAI #EventStreaming #Kafka #Flink #AIEngineering #TechLeadership

#DDD keeps my mind spinning around messaging patterns.
Should I publish each Aggregate's events to a same named topic partition it by Aggregate Id ?
On the consumer side should I stop event consumption if an event handler fails in order to ensure invariants ?

#DDD #eventstreaming

✔️ YouTube TV Clone Rentals

✨Rental service for #YouTubeTV clone accounts and streaming access, offering temporary access to #PremiumContent, personalized channels, and exclusive streaming packages for events and special occasions.

#StreamingRental #ContentAccess #TemporarySubscriptions #MultiDeviceAccess #EventStreaming

Wrote another blog post on creating scalable, performant and secure serverless applications. This post covers the CQRS and event streaming implementation I used for Sale Nook. As ever, I'm still looking for work or a contract so if you, or someone you know, needs an engineering manager or freelance, fullstack developer hit me up.

https://dav3.co/blog/rust-cqrs-event-streaming

#rust #rustlang #serverless #cqrs #eventstreaming #lookingforwork

"Event Streaming is not Event Sourcing!" is probably the article I'm linking the most from those that I wrote. It's part of my Don Quixote crusade to untangle those terms, as I've seen many significant architectural decisions made without realising those differences. And the consequences were severe.

There's a skewed perspective conflating #EventSourcing with #EventStreaming.

I know those terms sound similar. I know many people tell you that you can use Kafka as an event store, but...

Event Sourcing is about making decisions, capturing their outcomes (so events) and using them to make further decisions (so events are the state).

Event Streaming is about moving information from one place to another and integrating multiple components.

Event stores are databases. They may have similar capabilities as Event Streaming solutions, but the focus is different:
- event stores on consistency, durability and quality of data,
- event streaming solutions (like #Kafka) are focused on delivery, throughput and integration.

So, to give a bold comparison, saying that Kafka is an event store is almost like saying that #RabbitMQ is a database.

I really like Kafka; I've been using it successfully, along with the Event Sourcing tool. That's why I believe it's important to know the difference and how to compose those tooling instead of mixing them and getting a hangover.

Check more in my article 👇

https://event-driven.io/en/event_streaming_is_not_event_sourcing/