Dave MillerNov 5, 2023
mcc

Fascinating both for what it says about dev & what it says about statistics:

A gamedev realized Linux users were just 5.8% of their sales, but represented 38% of bug reports.

Then they looked at those numbers closer, and realized. Linux users were not experiencing more bugs. Almost none of the Linux-user bugs were Linux-related. Linux users were simply *more likely to file bugs*.

Their conclusion: A linux port pays for itself bc it nerdsnipes ppl into giving u free QA

https://techhub.social/@ozone89/111337250473454154

Daniele Pantaleo 🦥:verified: (@[email protected])

Attached: 1 image @[email protected] silver lining on devving for :linux:

TechHub
Nov 1, 2023 at 9:50pmWeb
Show threadmccNov 1, 2023

This is the "what if lab rats just get cancer a lot" joke except real

Do you have a correlation in your data? Or is one of your sample groups simply more likely to generate *statistics*?

Show threadmccNov 1, 2023

"A recent study performed on the University of Toronto campus with participants selected by responding to a flier offering $5 for participation, revealed that 92% of all Canadians are students at University of Toronto…"

(This is a joke; I suspect sociologists have some way of correcting for this already)

(EDIT: Note I am not saying I believe the way sociologists have of correcting for it *works*.)

Show threadJason Petersen (he)Nov 1, 2023

@mcc uh IIRC this is a “joke but not really” in psychology and social sciences.

The joke is “add ‘among mostly 18–22yos with the social status to be in college’ to every psychology paper”

Show threadPieNov 2, 2023
@jason @mcc
And show up for appointments
Show threadPaul BernhardtNov 1, 2023
@mcc People that participate in studies are WEIRD https://slate.com/technology/2013/05/weird-psychology-social-science-researchers-rely-too-much-on-western-college-students.html
Social Science is WEIRD, and That’s a Problem.

How did you lose your virginity? Maybe it was in a romantic garden under a full moon with the scent of roses all around, in the arms of your one true...

Slate
Show threadkeithzgNov 2, 2023
@mcc The important thing is to try a number of different "corrections" until you find it creates a statistically significant result you can publish
Show threadmrFred489Nov 2, 2023
@mcc @keithzg Mastodon should really have emoji reactions. I bet it would significantly increase the number of interactions to toots like this.
Here is my reaction: 😬
Show threadkeithzgNov 2, 2023
@mrFred489 Considering nearly every piece of server software on the Fediverse these days other than Mastodon has emoji reactions, I wouldn’t be surprised if Mastodon got them sooner or later!
Show threadSven (retired)Nov 2, 2023
@mcc 98% of people say that they enjoy responding to surveys. 😉
Show threadEel-on-MolluskNov 2, 2023
@mcc yeah, quota sampling and population weights. Standard for pretty much anything where generalizable findings are the aim.
Show threadmoduluxNov 1, 2023
@mcc Sometimes I wonder if Firefox isn't heavily under-measured as a browser because it's run by privacy nerds who set up no telemetry, no js, or other forms of blocking that obscure it, for example.
Show threadabadideaNov 1, 2023
@modulux @mcc you can under-measure privacy nerds who take technically advanced steps to obscure their browser fingerprint but I don’t think it’s mathematically possible to heavily under-measure them on a mainstream website
Show threadmirabilosNov 2, 2023
@0xabad1dea @modulux @mcc it is because we don’t visit those mainstream websites either
Show threadPeter BindelsNov 2, 2023
@0xabad1dea @modulux @mcc You'll trivially heavily undermeasure security-conscious users on the mainstream Facebook.com site.
Show threadkepstinNov 2, 2023
@modulux @mcc any site that simply looks at the user agents recorded in the web server logs will correctly count Firefox. But if you're relying only on a third party JavaScript tracker... Yeah, there's gonna be some biased undercounting .
Show threadChris Combs (he/him)Nov 2, 2023
@kepstin @modulux @mcc yeah--speaking as a former product manager working for media websites in the US, essentially nobody has done access.log at the edge in media for 15+ years. Omniture, GA, Chartbeat ended all that... plus ubiquitous html cdn caching.
Show threadTai xzoNov 2, 2023
@kepstin @modulux @mcc unless you’ve got the user agent set to Chrome for those websites that “only work on chrome” (but actually the devs just never bothered to test on other browsers)
Show threadRaptor Nov 2, 2023
@taixzo @kepstin @modulux @mcc definitely done that, plus anonymizers randomize metadata like user agent on every page load so its definitely not reliable, probably too small a group to make a significant bump though
Show threadBarryPNov 2, 2023
@taixzo @kepstin @modulux @mcc Some of them possibly never test usability on any browser once they’ve tweeked something.
Way back, a hip designer for Bose (I think it was), decided that the page background on their website pages should be black.
Thus becoming the same color as the text, which they didn’t change.
Show threadbassplayerNov 2, 2023
@modulux @mcc there are a few browsers more private than FF https://privacytests.org
Show threadRobert Anderberg 🕹️Nov 2, 2023
@modulux @mcc I don't think so, unless they're all changing their user agents to Chrome.
Show threadmoduluxNov 2, 2023
@bobbydigitales @mcc It's not a very strong hypothesis, but a lot of modern analytics run on the browser and depending how it's configured might not run at all. Many sites don't get data from the webserver logs.
Show threadPeter BindelsNov 2, 2023
@bobbydigitales @modulux @mcc As soon as anything gets close to / beyond 90%, websites will start filtering on it and people will start changing their user agents to imitate it. So yes.
Show threadRobert Anderberg 🕹️Nov 2, 2023

@dascandy42 @modulux @mcc I don't really buy that, we've seen a steady decline of Firefox users on our site, mostly because performance is ~40% worse than Chrome (It's a game platform) and we couldn't find any good tools to investigate that.

At least for our case, I'm pretty sure we just don't have many Firefox users (<1%).

Show threadFlyingManaNov 2, 2023

@modulux @mcc reminds me I once was reporting a usability issue for a niche browser with focus on privacy, to a website.

The website declined the report, as there reporting says noone is using this browser on their website 🙄

Show threadSuz KorbelNov 2, 2023
@modulux @mcc yep, I’ve been a Firefox user since they started and it’s a much saner browser experience
Show threadJohn-Mark GurneyNov 2, 2023
@modulux
It reminds me thinking about browser usage stats where someone is sure to say we don't need Firefox compatibility because none of our users use Firefox, but that's simply because their website doesn't work with Firefox so all the Firefox users have to use another browser to use the site.
@mcc
Show threadmoduluxNov 2, 2023

@encthenet I'm very used to that same line when it comes to #a11y. "We don't have any blind/visually impaired users." "Yeah well, that's because your website is inaccessible."

@mcc

Show threadStarkRGNov 4, 2023
@modulux @mcc I'd think most would still leave the user agent untouched.
Show threadmoduluxNov 4, 2023
@StarkRG @mcc Yes, but a lot of analytics is client-side these days. If blocked by no js, blocked tracker, or some other privacy extension the website owner will never find out.
Show threadStarkRGNov 4, 2023
@modulux @mcc Other than modifying the user agent, I don't think it's possible to block them, It's a necessary part of HTTP requests. That data stands out quite prominently in server logs.
Show threadmoduluxNov 4, 2023
@StarkRG @mcc Right, but lots of sites don't use server logs for analytics. At least that's what I've been told.
Show threadStarkRGNov 4, 2023
@modulux @mcc Oh, really? That seems pretty short-sighted. Why would you get your analytics from sources that can be blocked entirely rather than sources that can't? (User agent is a text string that can be modified to whatever you want it to say, it just usually isn't)
Show threadmoduluxNov 4, 2023
@StarkRG @mcc Complicated reasons, but sometimes the webserver logs are not available. CDNs, caching, all that stuff. Plus aggregating those logs from load balanced servers and so on becomes more complicated. Also for some sites if you don't watch ads you don't generate revenue, so analytics on the browser make more sense to measure what they're actually interested in.
Show threadStarkRGNov 4, 2023
@modulux @mcc I get that, though I'd still expect that the initial request would be available, even if it's then offloaded to other servers. That said, I wouldn't think the browser people use would actually be high on anyone's priority list.
Show threadphii (local floofgirl cutie)Nov 1, 2023

@mcc i might be wrong about this but isn't it actually even the other way round, with rats that were bred to be cute and pet-like having a higher cancer risk than lab rats due to being over-bread?

bc when me and my relatives had rats that were bread to be kept as pets they all died rlly quickly, but then we got lab rats instead (as pets, i mean) and they lived for wayy longer

Show thread🆘Bill Cole 🇺🇦Nov 1, 2023
@mcc #RandomPedantry
Lab rats in fact DO get cancer more than normal. There are tightly-inbred strains of mouse and rat bred for higher cancer susceptibility. For exactly the reason the Linux users are valued...
Show threadSiderea, Sibylla BostoniensisNov 2, 2023
@mcc This is a huge, huge, huge problem in personality science. If you want to get any sort of demographic information, about the distribution of traits through a population, you can't actually do it through voluntary testing, because the selection indexes of so many interesting things on "who is voluntarily willing to take a personality test" are kind of astronomical.
Show threadAndy WoottonNov 2, 2023
@mcc It also seems related to the famous case of the analytical error of deciding where WW2 aircraft needed better armour, based on statistics about the location of holes in returning planes. Windows users learn to expect to be shot down because they have no say in what gets armoured.
Show threadPunnO)))Nov 1, 2023

@mcc This tracks for me. I assume most open-source stuff is run by accessible people who care about the software, otherwise why the heck would they bother with any of this? But most corporate bug reports are shouting into the void.

One exception is when I filed a Radar ticket for some wacky behavior in, I think, an Apple AirPort WiFi base station. And I *got a call* from someone at Apple, who gave me a special diagnostic tool to collect more data. Never had anything like that happen before.

Show thread💬Nov 2, 2023

@ieure @mcc congrats! every radar I file gets closed as a forward dupe*, which I guess means someone intends to work on it but Apple doesn’t want any of that to be visible to a member of the unwashed public?

* “closed as duplicate” of a higher-numbered radar

Show threadBerkubernetusNov 1, 2023
@mcc can support. I used a music player for a while that had a linux port. Desipte their being like a few dozen Linux users, we filed over half of all bug reports, and most of those were UX/server issues.
Show threadSmytheNov 1, 2023
@mcc automatic +1 for coining the term “nerdsnipe”! 👏🏼
Show threadWillow "Wolveric" CatkinNov 1, 2023
@mcc Ah yes... the age old story from the PA:T devs... 😹
Show threadQuinn9282 🖥️🌙✌️Nov 1, 2023

@mcc It's always been my experience that most open source/Linux software devs tend to care more about bug reports than most corporations do. Like you'll submit a bug report and it might actually get fixed.

...Unlike most corporate software, where you'll submit a bug report, and it NEVER gets fixed, even if a lot of people are having the same problem. The only time they might actually care about bug reports is if it prevents you from properly using/opening the software at all...

Show threadpennyNov 2, 2023
@mcc wait i just noticed i got that game from a steam coupon lmao nice
Show threadKevin Granade has movedNov 2, 2023
@mcc "Open source is weaponized Cunningha'sm law." -me
Show threadALNov 2, 2023
@mcc
i'll bet that holds, in general, for people on fediverse stuff. Its not so much nerd and people who care and want to help others.
Show threadScissors Cut Paper...Nov 2, 2023

@mcc @sadmac356

At work, I have a bit of a reputation as the best person to report bugs on our internal tools.

I put #Linux on my desktop when IBM threw in the OS/2 towel.

One of my other oddities is that my first #Windows was 7 because I needed it for the required tech apps class when I went back to college as a non-traditional student.

And here I thought I wrote good reports because I used to do tech support for a dial-up ISP.

"The thing is borken. Here is how you can prove to yourself that I am observing a real issue. You can follow along using the attached screenshots."

Show threadZorro Notorious MEB 😡Nov 2, 2023
@mcc Except I'm tired of doing free QA (looking at you, Microsoft `usbipd` 😉
Show threadJosh TriplettNov 2, 2023
The sad thing is, I've seen people talk about this as a bad thing, not just a good thing. I agree with the perspective of "it's good that you get more bug reports", but some folks view this as "ugh, Linux users complain more".

And there's a small grain of truth to that. Linux users are much more likely to file bugs, and as this developer noted the bugs are generally high-quality, but one or two experiences of That One Kind Of Linux User (why aren't you prioritizing *my obscure use case*) can really spoil a developer's view of Linux users in general.
Show threadmirabilosNov 2, 2023
@mcc I feel seen!
Show threadJubeiNov 2, 2023
@mcc it really depends on the effort required to port to Linux. If your game is written for DX12 and you need to write a whole Vulkan renderer for the Linux port then perhaps not.
Show threadargonautNov 2, 2023
@mcc @lisamelton
My dev colleagues file more bugs than my designer colleagues, due to the nature of their work. Wonder if peeling one more layer would show that Linux users file more bugs because there are more devs using Linux, which would then rewrite the maxim to “port to Linux to get free qa dev work”.
Show threadRaptor Nov 2, 2023
@mcc guilty, lol, definitely submitted my share, generally with a patch and debug info. Also i found long ago when reporting bugs in games to often lie about os (gen by omission) as most devs wouldnt take it seriously otherwise, even with it being a known complained about issue with actual windows users and absolute proof. Windows players in general are so used to bugs with no path to get them fixed they complain, but never report!
Show threadTom ForsythNov 2, 2023
@mcc I guess when you've filed 26 bugs against device drivers just last week, filing 3 bugs against a game is how you kick back and unwind.
Show threadMariaNov 2, 2023
@TomF @mcc the real relaxation is finding existing bug reports with patches stuck in limbo that fix your issue. I'm sure those missing Xbox controller Bluetooth device IDs will get merged in any year now /s
Show threadiheartbooksNov 2, 2023
@mcc
Show threadcalvin 🛋️ Nov 2, 2023
@mcc What an interesting outcome! I remember hearing the computer game Planetary Annihilation have a problem with bug reports being mostly in regards to their Linux port. I guess if this result can be shown to be true for lots of software, then it will be a good problem to have!
Show threadMarkNov 2, 2023
@mcc This is similar to the effect that makes Swiss trains so good.