Cat Hicksoh my fking god. I am TRYING to do research for my book right now but I am getting DISTRACTED by the fact that I followed a citation from a paper into another paper and learned that they're interpreting EEG and eyetracking as a way to sort people into "in the flow" or "not in the flow" I AM GOING TO EFFING LOSE IT
I love that this was published in the same overall goddamn umbrella organization that sent me reviewers that said my work had "too much psychology" cited in it I AM. GOING. TO. LOSE. IT.
eyeblinks as a measure of frustration? I'm going to throw my entire laptop into the ocean
this is UNCRITICALLY cited in a BIG paper with NO reference to it being a biometrics study AT ALL
I mean. Can we measure things with EEG and with eyetracking, yes we can. Can we detect different kinds of waves yes we can. Do blinks correlate with something, I don't know, probably. This is all just so effing strange.
how is engineering the most critical field with the LEAST critical research I've ever seen. Do you guys just think if something is published in Fuck All Journal, 2024, that makes it good?
ML CLASSIFIER ON SIX DATA POINTS WHAT IS HAPPENING
Movement dominates brain activity, and is ALSO a huge artifact in any skin conductance. So like THAT'S a confound in these conditions. Yet no details at all about how this is truly analyzed, because it's an off the shelf EEG "band" someone bought with some research cash I bet. This paper is "we thought this was cool and so we bought a lot of stuff and tried it."
What's the play guys. What's the thing. Are we going to believe developers are human beings if we count enough eyeblinks and say they're systematically different if you focus harder? Are we going to unlock the magical super Computer Brain region if we measure hard enough?
MY WIFE PUBLISHED A PAPER ABOUT HOW MOVEMENT DOMINATES BRAIN ACTIVITY AND NEEDS TO BE ACCOUNTED FOR IN MEASURES AND NONE OF THESE GUYS READ IT
one of the measures is defined as capturing "various emotional states" I am dying
somebody made like twelve tables for this paper. Just like, table after table of everything they could tally up. NOT ONE OF THESE THINGS SHOWS KEY INFORMATION ABOUT THE EEG. This is like the baseball statistics version of analysis. Just tallying up as many numbers as possible. "mean task completion" across like six people which tells us fuck all about fuck all stfu
look, it's not that I wish anything but healing upon these people who wrote this paper and don't know better. it's just that you're so underserved. You're so underserved and you don't even know it. Somebody is serving you up a plate of raw hamburger paired with rat shit in the back alley next to the dumpster and telling you it's fine dining. You (software developers) BUILD THE WORLD. YOU COULD GET A NICE MEAL ONCE IN A WHILE.
You're stuck. We're all stuck. We're stuck in the back alley trying to say please god believe I have emotions look at my eyeballs WHO PUT US HERE
"we deleted all measurements that were marked as invalid by the eye-tracking device"
ahahhahahahahahhahhahahahhahahahahahhaaaaaa
they extracted??????? eyeblink "data"????? FROM THE EEG??????? what
are we treating the artifact AS the measure now
oh, I see, the sensor (purchased from some random website) has "two pre-processed signals" omfffffg YOU GUYS. HOW IT IS PROCESSED IS THE POINT
we're going to say that brain waves are predictive but not say WHICH brain waves we're talking about?
"finding relevant code" vs "not finding relevant code" are two sides of the classification how much do you want to bet this is essentially categorizing scrolling and skimming vs "not scrolling and reading" which is, again, A MOTOR MOVEMENT among many other things
The obsession with the idea that we are detecting "higher order problem-solving" via Secret Signals in the Skin
I forced my neuroscientist wife to look at this paper from a signal processing POV because she's actually published on the complex math of signal processing (and because I am not above outsourcing my complaining) and in her characteristically gentle spirit she said "many things could be extremely misleading, there's no way to know with what is being reported" I am going to translate that for you: GARBAGE IN GARBAGE OUT
Do you all want to know something super funny, the lab I did my PhD in actually did EEG work as well and THIS is why I NEVER talk about it, early in my tech career someone was like "omg you should put on your resume that you can do EEG and eye tracking and your entire career in UX will be set!" And I said no thank you, I will never! And ever since then people have mistakenly said I was "Qualitative" and maybe "not a scientist" but I sleep at night, so
Baffled by the idea that we should divide brain states into two categories that seem to mean "thinking about stuff" and "chilling" and some of these activities under the "chill" category do not sound chill at all unless we're talking about taking a nap
Stress is mentioned as a possible threat to validity when I think it's probably actually the entire ball game if you're forcing people to solve a coding task in a lab setting
Stress+ motor movements, that's what I see
Obscuredavid@grimalkina
As far as I'm concerned six data points is roughly zero data points.
As far as I'm concerned six data points is roughly zero data points.
DeterioratedStucco@grimalkina
I am in a bizarre horrified-giggles state.
My son just came down the stairs to check if I was OK...
I am in a bizarre horrified-giggles state.
My son just came down the stairs to check if I was OK...
Glyph@grimalkina I am sitting in a black belt board test for my kids martial arts class and reading this during a break and desperately trying not to loudly lose it. This is “we did an fMRI on a cheeseburger to see if it has the buddha nature” levels of gobbledygook. How did someone write all this down without realizing what they were doing??
0xC0DEC0DE07EA@glyph @grimalkina “Oh, god, I’ve been with this advisor for [however long] and we have nothing, but I’ve been paying for ‘education’ and I need to publish a thesis to graduate, so maybe if I just put a ton of graphs in, I can escape with a Master’s and go DO something and not feel like a complete failure?”
Dan Sugalski@grimalkina this looks uncomfortably like the time I went looking for foundational economics papers. (Would you like to see a list of foundational papers on economics with clear theses, properly collected data, and clean statistical analyses that support the paper’s conclusions? Yeah, well, so would I, that’d be a nice change)
Jacob Something@grimalkina Which journal?
Eaton@grimalkina oh, you know… waves…
emmatonkin@grimalkina
I get the impression it's a NeuroSky? If so that explains the preprocessed signals and the reporting of eye blinks. The MindWave gives, or at any rate used to give last time I played with their original model in like 2014, "attention and meditation" waves and eye blinks. I think the blinks are just freebies because it has one sensor and that happens to be on the forehead.
I get the impression it's a NeuroSky? If so that explains the preprocessed signals and the reporting of eye blinks. The MindWave gives, or at any rate used to give last time I played with their original model in like 2014, "attention and meditation" waves and eye blinks. I think the blinks are just freebies because it has one sensor and that happens to be on the forehead.
@grimalkina (if it is one of those, back in the day we concluded that it would need serious validation to become relevant. If it isn't, worrying that there are multiple devices in this class.)
Samael 
I once had a coworker (we were both software dev) who refused to discuss unions with me. I don't recall if I could coax a reason out of him, but he also complained about feeling overworked and, I just... couldn't. FUCK!!!
Why are humans? (that's it, that's the question)
@Kishi @grimalkina well now you know your mistake. You should have blinked a lot more, with your brainwaves. You’re welcome
Liz Marley@grimalkina Thank you for trying to help us. I am a software developer, not any kind of doctor or research specialist, and you put the problems with these papers in terms I can understand.
grillchen@grimalkina 6 people? wtf
@grillchen we can learn interesting things about small groups of people sometimes but Not Like This
@grimalkina yeah but it sounds like they confused qualitative with quantitative work.
Rich Puchalsky ⩜⃝It's not a real paper unless they ran correlations between all of the variables in those twelve tables and their research result was that 2 of the variables were strongly correlated.
Morten@grimalkina Does this paper have any comments on PubPeer?
The Witch of Crow Briar@grimalkina 🤦🏼♀️🤦🏼♀️🤦🏼♀️🤦🏼♀️
robryk
Single-trial neural dynamics are dominated by richly varied movements - Nature Neuroscience
The authors use a linear model to reveal how neural activity patterns are related to cognition or movements. They find that uninstructed movements dominate single-cell and population activity throughout the brain, outpacing task-related activity.
@robryk yes that's one!
Wharrrrrrgarbl@grimalkina obviously the solution is to implant depth electrodes until the machine learning model we tape to the headbox tells us that it has decoded The 10x Engineer Coding Flow State
i can taste the citations already
they taste like electrodes jammed into the anterior insula
Stephan Eggermont@grimalkina Don't tease me without the link, please :)
robryk (@[email protected])
@[email protected] Do you mean https://www.nature.com/articles/s41593-019-0502-4?
Joseph Kohlmann@grimalkina Makes you wonder if they’ve tried, uh, measuring skulls or some shit 🫠🤬😭
Maya@grimalkina That was what my research advisor did back in the 2000s.
I was super-pissed off because I couldn't replicate the prior grad student's EEG classifier work at all, even with the source code, and I was under so much stress because it felt like I was being pressured into doing academic fraud if I wanted to complete my thesis.
It was a major contributor to why I ended up ABT.
@Mayabotics oh this breaks my heart to hear this. I know exactly what you mean and I've felt these kind of pressures. This is yet another reason it's so important to have a compassionately rigorous science
@grimalkina And I was extremely uncomfortable by the fact that my research advisor wanted me to collect EEG data without human studies review.
Ted Mielczarek@grimalkina lmao that is absurd. They wrote a whole paper with n=6???
@tedmielczarek actually not the worst thing about this paper, there is a lot you can learn from a small group from bio measures and a lot of measures within individuals, but in this particular case, really really not right for this particular question
jmelesky@grimalkina I mean, I guess that means the vector space could be exhaustively defined? No idea why they’d want to do that, mind you.
cthos 🐱@grimalkina oh gods, yes, I think yes.
@grimalkina having a citation at all is pretty impressive. Usually it's just vibes.
@grimalkina hey, this is computer science! We’re *trained* not to follow pointers and references, they may be null and we’d all segfault!
@wordshaper I have no doubt this is really funny (to y'all) 🥰
Yvonne Lam@grimalkina We live in venture capital land; we don't know from either "critical" or "research".
Irenes (many)@grimalkina honestly it's almost certainly because research is either funded by corporations for Taylorist reasons and for that purpose it doesn't matter whether it's true since it's just propaganda, or by academics for sincere reasons but for that purpose it doesn't matter at all because management will ignore it?
@ireneista I see the larger points and agree re: the difficulty of getting authentic work heard and issues in science reform,
but still there IS good science on all of this. There is amazing science on human behavior, there's good science on how we use EEGs, there's good science on all of this! There are people spending their whole careers studying this with sincerity and bravery. Eng just won't let it in, specifically!
@grimalkina yeah :/ sorry. we have seen people advocate for reforms of business processes based on science, and we have seen the absolute brick wall response from management, who close ranks around it apparently without even considering the substance. we wish that it were otherwise.
@ireneista I totally understand/hear that, it's really painful, I've experienced that so often as well