streetlightsApr 14, 2024

Breaking Down Cass Review Myths and Misconceptions: What You Need to Know – The Quackometer

https://lemmy.world/post/14288447

Breaking Down Cass Review Myths and Misconceptions: What You Need to Know – The Quackometer - Lemmy.World

Seen the “98% of studies were ignored!” one doing the rounds on social media. The editorial in the BMJ put it in much better terms: “One emerging criticism of the Cass review is that it set the methodological bar too high for research to be included in its analysis and discarded too many studies on the basis of quality. In fact, the reality is different: studies in gender medicine fall woefully short in terms of methodological rigour; the methodological bar for gender medicine studies was set too low, generating research findings that are therefore hard to interpret.”

Show threadCogencyApr 14, 2024
advocate.com/…/hilary-cass-nhs-report-debunked#to…
Why Hilary Cass' NHS report is wrong about trans health care

The recent NHS report from Dr. Hilary Cass isn't the take-down of gender-affirming care that conservatives want it to be.

Advocate.com
Show threadstreetlightsApr 14, 2024
Ms Reed is not an objective source. Nor does it appear she has much experience with systematic reviews.
Show threadCogencyApr 14, 2024
It’s not a lie, they were mostly dismissed even according to your own article, they were dismissed and rolled into one conclusion.
Show threadstreetlightsApr 14, 2024

101 of 103 studies were not dismissed. All systematic reviews classofy their source studies based on the quality of the work. Of the 103, two were classed as high quality, 58 as moderate quality and the remaining 43 as low quality. For synthesis, only high and moderate quality studies were drawn on. That’s more than half, not 2%.

So yes, Erin is lying.

Show threadCogency
You can’t say she’s lying until we do a systemic review of why the Cass study dismissed everything but 2 studies for the numbers it used to reach us conclusion. You can’t say she’s lying without that review no more than I can support Erin by reading each study that was dismissed. What I can tell you is that dismissing that many studies is not normal scientific analysis. It reeks of bias.
Apr 14, 2024 at 7:00pmWeb
Show threadstreetlightsApr 14, 2024

You can’t say she’s lying until we do a systemic review of why the Cass study dismissed everything but 2 studies

This is the lie. They didn’t dismiss all but two studies, they actually included 60. More than half of the 103 studies identified for the review.

What I can tell you is that dismissing that many studies is not normal scientific analysis.

It’s key part of synthesising multiple sources into a meta-analysis. Including poor quality studies dilutes the quality of the overall analysis.

It reeks of bias.

By design, it’s biased towards higher quality research.

Show threadCogencyApr 14, 2024
Synthesis is a paragraph summary inclusion only it means they didn’t use data from the study, it is dismissal. I’m done arguing that with you.
Show threadstreetlightsApr 14, 2024

They have absolutely used the data from those 60 studies. You can read where they say explicitly that in the report if you cared to.

You are utterly mistaken and firm on your conviction, these are not the qualities of skepticism.

“Don’t seek refuge in the false security of concensus”

Show threadCogencyApr 14, 2024
That’s not what synthesis means. I’ve written synthesis reports before and the data you include from those reports is once you have dismissed them as inaccurate, is entirely selectivel whatever you want them. We even have a phrase for it in law, Summarily dismissed.
Show threadstreetlightsApr 14, 2024
And of the 103 reviewed they included data from 60. It is a lie to say they “dismissed all but two.”
Show threadCogencyApr 14, 2024
So you don’t know what you are talking about. Gotcha.
Show threadstreetlightsApr 14, 2024

You can read the review for yourself

t.co/82Rjs2L1pA

Let me know where you find the bit where they dismiss 101 out of 103 papers.

Show threadCogencyApr 14, 2024
There only inclusion in the report is to half explain why the were discluded, exactly what I said.
Show threadstreetlightsApr 14, 2024
They did not dismiss 98% of the data.
Show threadCogencyApr 14, 2024
Putting 98% of the data in supplementary table 4 is not including the data.
Show threadstreetlightsApr 14, 2024

Supplementary Table 4 (from the first review) is a list of each of the 53 studies included in the review and how the were scored based on the Newcastle-Ottawa scale.

The “data” is in supplementary tables 6 and 7. Only studies that were scored as low quality were excluded from the synthesis.

“They dismissed 98% of data” is a lie.

Show threadCogencyApr 14, 2024
No it’s not. None of the dismissals are scientifically supported, and their data is incomplete presented in a way that isn’t inclusive of what those studies actually say.
Show threadstreetlightsApr 14, 2024

Nothing was dismissed at all (and “statistics” has nothing to do with it so curious to mention it).

Studies were scored for quality on the well established Newcastle-Ottawa Score. High and Moderate quality studies were included in the synthesis. Low quality studies were not, but their outcomes are still reported.

Outcomes from each study were included in tables 3, 5, 6 and 7.

'They dismissed 98% of the data" remains a lie.

Show threadCogencyApr 14, 2024
You can’t remove a study from your a scientific paper without having statistical analysis to back it up. Each of those removed studies all had statistical analysises of how confident they remained in their data even with the gaps. Because there aren’t completed 100% studies in science it just doesn’t happen so you use the data you have. And the idea that some trans people don’t make it to the completetion of a study due to personal reasons or even suicide isn’t that rare. Not using 98% of the data because of that would be stupid.
Show threadstreetlightsApr 14, 2024

You can’t remove a study from your a scientific paper without having statistical analysis to back it up.

You can of course. Statistics are not required to explain why a self selective Facebook poll is low quality while a multi centre 5 year study with followup and compartor is of a much higher quality.

Each of those removed studies all had statistical analysises of how confident they remained in their data even with the gaps.

Studies are also scored low on quality if, for example, they don’t control for important sociodemographic confounders. Study that do control these, will have more reliable results.

You can read how the scoring works in supplementary material 1.

Not using 98% of the data because of that would be stupid.

“They dismissed 98% of the data” remains a lie. Repeating it doesn’t change anything.

Show threadCogencyApr 14, 2024

You can of course. Statistics are not required to explain why a self selective Facebook poll is low quality while a multi centre 5 year study with followup and compartor is of a much higher quality.

That’s wrong when you are trying to be scientifally correct. A science paper without math isn’t science my dude.

Show threadstreetlightsApr 15, 2024

It’s remarkably common in systematic reviews. You give the impression that this is a new or foreign concept to yourself and are just encountering these ideas for the first time.

Search on pubmed or the bmj or the Cochrane library for other systematic reviews using the Newcastle-Ottawa score. You’ll trip over them.

And comparing trans healthcare data to Facebook polls is ridiculous

One of the studies reviewed recruited patients over Facebook and polled them.

Show threadCogencyApr 15, 2024
Again I’ve written these reports. It is absolutely not common practice to discluded data.
Show threadstreetlightsApr 15, 2024

Again I’ve written these reports.

I am forced to strongly doubt this given your wholly misunderstanding of the basic concepts on assessing methodical quality…

Certainly, you’ve never authored a systematic review for a reputable medical journal.

But don’t take my word for it…

…cochrane.org/…/13_5_2_3_tools_for_assessing_meth…

It is absolutely not common practice to disclude data without scientific reason and analysis.

You mean such as using a method like the Newcastle-Ottawa score to assess data quality?

It is explicitly taught not to do it that way in college.

If your college course covered systematic reviews and didn’t include a review of study assessment methods, ask for a refund.

And is not scientific to do that without a statistical threshold

Statistics are not required to assess that a study without a comparator is weaker than one with.

“They dismissed 98% of data” remains a lie.

13.5.2.3 Tools for assessing methodological quality

Show threadCogencyApr 15, 2024

The Newcastle method is not seen as a scientific basis for dismissal on its own.

98% of the data was dismissed in the synthesis and were not used in the conclusion that there wasn’t enough scientific evidence to support transition when 98% of the science says that is wrong.