@kittylyst "I am, in general, sceptical of micro-benchmarks. For measuring Open Telemetry overhead, if you add it to hello world, the overhead will look enormous, because the app isn't doing anything! You need to measure on a realistic application."
Did I really hear @kittylyst say "sad lamb-da?" ... or was it "sad panda," and I just had my next demo on my mind?
.@kittylyst "With open source, there is still vendor lock-in. With an open standard, people can move from vendor to vendor."
"OpenTelemetry is a successful example of an open standard, that grew out of inter-vendor collaboration."
@kittylyst "Prometheus is a pull-based system; Prometheus has to come and ask your endpoint for the data. But what happens if Prometheus doesn't know where to find all your containers? What happens if your container doesn't live long enough for Prometheus to find it?"
@kittylyst Metrics, unlike traces or logs, don't scale with transaction volume. You can have a huge number of transactions, and a small number of metrics.
A metric has a name, [something], value, and dimension. Dimensions should aggregate in a sensible way, and have low cardinality.
@kittylyst laying out ingredients for distributed tracing.
- Instrumentation - "don't do this manually, use an interceptor."
- Context propagation - w3 has a standard for headers that servers must propagarae
- "Store it somewhere"
- Search + retrieval
- Visualization
Open Telemetry is the best standard for the first two, Jaeger plays with the last three but the space is changing. Tempo is a cool new visualisation tool.
At #LJC Live, hearing Heather Whyte talk about their data challenges.
(Marshmallow have a gorgeous office, but the company name keeps making me hungry.)