beyondwatts
Have been experimenting with #signoz in the #homelab as a possible replacement for #grafana #prometheus #loki #fluentbit. Early results across the #kubernetes cluster are positive - metrics, logs, dashboards and alerts tested. Before I start a more thorough test ahead of migration, does anyone have any experiences to share? Really like the idea of queries and logs with SQL, are there any other tools I should be considering?
May 27 at 11:44amWeb
Show threadSlash909ukMay 27

@beyondwatts ooh, that looks interesting!

We ( @phlash ) and I have been using a series of tools for our homelab observability this past year or so. We needed journald and syslog sources plus std host metrics and a few custom metrics to collect.

We are basically Grafana stack (Alloy collector, web UI and alerting tool) but we dumped Loki very quickly and are now looking to dump Promethus for storage in favour of VictoriaMetrics tools:

https://victoriametrics.com/

VictoriaMetrics: Simple & Reliable Monitoring for Everyone

Incredibly fast time series database and easy-to-use high performance monitoring solutions simple and reliable for everyone. VictoriaMetrics Database Solutions.Incredibly fast time series database and easy-to-use high performance monitoring solutions simple and reliable for everyone. VictoriaMetrics Database Solutions.

VictoriaMetrics
Show threadbeyondwattsMay 27
@Slash909uk @phlash VictoriaMetrics does look good although I was somewhat put off by the quick start guide… maybe I need to give it another shot
Show threadSlash909ukMay 27

@beyondwatts @phlash it's only the storage/search/DB part. You still need collector and UI tools. They do have alerting but not used it:

https://docs.victoriametrics.com/opentelemetry/

OpenTelemetry

Documentation for VictoriaMetrics, VictoriaLogs, Operator, Managed VictoriaMetrics and vmanomaly

Show threadbeyondwattsMay 27
@Slash909uk @phlash ahh got it. I quite like that signoz is one tool. Also includes tracing but I haven’t experimented with this
Show threadSlash909ukMay 27

@beyondwatts @phlash

We have been looking for alternatives to the Grafana Alloy collector and alerting server. Signoz appears to have these tools! VictoriaMetrics does not handle the collector part AFAIK...

Show threadSlash909ukMay 27
@beyondwatts @phlash .. reading further, it seems Signoz uses the OpenTelemetry collector, and Grafana Alloy is a port/fork of the same. So is there really only one collector codebase? Alloy has been.. OK. Bit resource hungry. We dumped Telegraf+syslog-ng+prom-node-exporter for Alloy (1 binary, easy to mange.)