VictoriaMetricsMar 3, 2025

🔍 We already know how storage handles data ingestion from #vminsert, turning them into “raw-row shards” in memory. However, they’re not searchable yet.
⚡ vmstorage flushes the raw-row shards data into what’s called “LSM parts”, starting with in-memory parts.
📖 Check Phuong's latest blog post to learn How #vmstorage Processes #Data: Retention, Merging, Deduplication..

https://victoriametrics.com/blog/vmstorage-retention-merging-deduplication/

#VictoriaMetrics #opensource

How vmstorage Processes Data: Retention, Merging, Deduplication,...

vmstorage takes data from vminsert through a concurrency limiter, creates TSIDs for each row, and puts them in memory buffers. Every few seconds, it moves data to storage parts on disk. The system merges parts, removes duplicates, and cleans old data. This turns raw metrics into data users can search.

VictoriaMetrics
VictoriaMetricsJan 9, 2025

🚀 When Metrics Meet vminsert: A Data-Delivery Story
Breaking down how different components of #VictoriaMetrics do their thing: This time, we're focusing on vminsert.
🔍 Here's a sneak peek:
🔌 Accepting connections.
🤝 Handshake.
⏭️ Parse and Relabel.
✂️ Sharding and Buffering.
🚚 Replication and Sending Data to #vmstorage.
Learn more in latest blog post from Phuong Le:

https://victoriametrics.com/blog/vminsert-how-it-works/

#Observability #timeseriesdata

When Metrics Meet vminsert: A Data-Delivery Story

vminsert acts as a gateway for incoming monitoring data. It receives data in different formats, processes it by parsing and adjusting labels, then uses memory buffers to send this data to storage nodes. It’s smart enough to always send the same type of data to the same storage node and can redirect data if a node isn’t working properly.

VictoriaMetrics
VictoriaMetricsDec 11, 2024

🔍 Did you know? In #VictoriaMetrics #vmstorage is the powerhouse managing long-term #storage for your #monitoring data. From receiving raw metrics via vminsert to efficiently organizing them into history-rich structures, it’s built to optimize performance and scalability.
In the latest Phuong Le #blog post, dive deeper into how vmstorage transforms raw data into organized history.

https://victoriametrics.com/blog/vmstorage-how-it-handles-data-ingestion/

#timeseries #databases

How vmstorage Turns Raw Metrics into Organized History

This article walks through how data flows from collection to storage, explaining how vmstorage processes incoming metrics, assigns unique IDs to time series, and organizes everything into different types of storage parts. The whole system is pretty clever - it uses in-memory buffers for speed, smart compression to save space, and has various watchdogs keeping an eye on things like disk space and data retention.

VictoriaMetrics