Mastodawn

Russell Keith-Magee Jan 8, 2025

One of the most frustrating things about Open Source is that there's almost no way to know if someone is happily using the tools you build. I found out today that someone I know has been using Briefcase for an internal company tool... but I had no idea because there's no need for them to report or register their usage. The only reason I found out is because of weird edge case that caused a bug.

Russell Keith-Magee Jan 8, 2025

The great irony here is that the better your do your job as an Open Source maintainer, the less feedback you get that you're doing anything worthwhile at all. Thousands could be using your tool, and you hear nothing... because the tool Just Works.

I really wish there was a better way to get this sort of usage feedback without earning the ire of the special kind of advocate who insists that registration or user tracking is an evil that must be opposed at all costs.

Hans Olischläger Jan 8, 2025

@freakboy3742 I wonder if it's indicative of some sort of stockholm-syndrome like effect that when I wrote about this — https://blog.glyph.im/2023/03/telemetry-is-not-your-enemy.html — I didn't even *consider* the mental health of maintainers. All I wanted was to be allowed to serve users better, for free, without getting yelled at!

Telemetry Is Not Your Enemy

Not all data collection is the same, and not all of it is bad.

Glyph Jan 8, 2025

@freakboy3742 there was not much discourse on the terrible orange website, but what little there was, was still 👨🏻‍🍳👌🏻 https://news.ycombinator.com/item?id=35329819

I don't care. It's my machine, not yours. | Hacker News

meejah Jan 8, 2025

@glyph @freakboy3742 there's a huge difference between "this app collects no telemetry" and anything else.

In the "else" case I have to understand all the data that's collected (or not) and when and how that's communicated, aggregated and stored and what privacy implications that may have.

I'd way rather be asked to do a short survey. From the other side, the results of surveys seem better than concluding things from telemetry data.

Glyph Jan 8, 2025

@meejah @freakboy3742 if a user were to report a bug, would you like the user's subjective description of the experience of that bug, or a log file with a PoC?

meejah Jan 8, 2025

@glyph @freakboy3742 "The application will now send the following information when reporting this bug" is not what I would really call "telemetry", but instead something like opt-in, automatically collected crash data.

So that's already way, way better than "telemetry". It's still a little harder to audit / understand than "this app never phones home". Maybe there's already a better word for this (vs. "telemetry")?

Glyph Jan 8, 2025

@meejah @freakboy3742 I was trying to make a slightly more general point. Self-reported data is notoriously unreliable; users attempting to describe bugs is just a particularly visible and visceral example of that unreliability that developers almost universally experience.

As I highlighted in the article, there are also huge problems with self-selection bias with opt-in schemes.

Glyph Jan 8, 2025

@meejah @freakboy3742 ultimately what this comes down to is that trustworthy institutions are a superpower. If you hear "telemetry" and think "I do not trust the judgement of the people deciding how the telemetry collection functions, so I will re-do all of their due diligence" then that is a tremendous waste. It may also be an entirely rational and valid thing for you to do, because if *nobody* does it, all we have is "trust me bro" from powerful players in the market.

meejah Jan 8, 2025

@glyph @freakboy3742 this isn't a neutral playing field anymore, though: the people saying "it's fine, it's just telemetry" have pretty consistently proven that they should not be trusted with _any_ user data. IMO ;)

meejah Jan 8, 2025

@glyph @freakboy3742 Also, these days, there _are_ better tools that do not just store a whole bunch of user data (that, even if the institution is 100% trustworthy _now_ can be re-purposed to Evil(tm) later on).

Differential Privacy and related research has had some decent results -- that is, allowing words like "we only want to look at the aggregate Frobnication of the Quux!" to actually have some mathematical truth.

Glyph Jan 8, 2025

@meejah @freakboy3742 case in point: what Russell is describing is a self-selection behavior that demoralizes developers; users report problems, not successes. You can't fix *that* self-selection with telemetry though, because "we know that 2.4 million people successfully used this feature" might just make you feel _worse_ if you know it for sure and you also know that 0 of them said so much as "thanks"

meejah Jan 8, 2025

@glyph @freakboy3742 Yes, conditioning users (and developers!) to only provide the "I had a problem!" button hasn't helped this situation.

I don't believe that "telemetry" (or even "telemetry, but only the good kind") is a good solution to it though?

meejah Jan 8, 2025

@glyph @freakboy3742 the only real "positive" button to push on GitHub is the "star" one -- maybe GitHub should be pushed to have a "report success!" button or similar.

Anyway, I'm not trying to dismiss the original problem / ask here. All I'm saying is that I don't think "more telemetry" helps a lot here. Even where it does, there are better tools than "traditional" telemetry to answer specific questions ("do I have users?") with privacy features.

I'd love to see more ideas / answers too :)

Glyph Jan 8, 2025

@meejah @freakboy3742 the topics are related but not entirely overlapping. the overlap is "user entitlement". There are a lot of people who simultaneously believe that they deserve software which:

1. has all the features they want,
2. has zero cost,
3. has zero bugs,
4. collects zero information to anticipate and address the presence of bugs or the lack of features without them having to speak out about it, and that
5. they never provide feedback for, except to complain that it fails to do 1-4

meejah Jan 8, 2025

@glyph @freakboy3742 I'm not interested in "blaming users" but it could be constructive to explore _why_ some people might believe some or all of these things.

meejah Jan 8, 2025

@glyph @freakboy3742 Looking at "Briefcase" specifically, the "Contributing" section is the last thing in the README and only invites people to report problems or to fork the code and become a developer.

PyPI and GitHub similarly lack mechanisms to say "I used this successfully" or even just "thanks" (beyond the vague signal that "star"-ing a repository brings).

For "users" reading this: please do go ahead and "star" GitHub projects you enjoy in any way. I certainly like it for my projects :)

meejah Jan 8, 2025

@glyph @freakboy3742 There are also the "support this project" links too (my bad!), so if you do like BeeWare or Briefcase in particular also check out https://beeware.org/contributing/membership/ and https://github.com/sponsors/freakboy3742

Membership— BeeWare

meejah Jan 8, 2025

@glyph @freakboy3742 Sure, statistics are hard, I get that. You're also going to have bias in "user selection" if you just have an "always phone home" app though.

meejah Jan 8, 2025

@glyph @freakboy3742 Looking at it for me personally as a user, I am extremely unlikely to turn on any "monitoring" / "telemetry" (and likely to just never install an app that doesn't allow it to be turned off).

I'd be very _likely_ to click "sure, send along relevant crash data" if the app had some "report a problem" feature.

I guess the important part here is "human agency"? Like I might indeed authorize an app that said, "we want to do telemetry for 24 hours on January 9, 2025 (y/n)?"