Pushback on “Brief Outpatient Rehab” Trial for Long Covid from Norwegian Ideological Brigades

By David Tuller, DrPH

I often find myself responding to crap studies–such as a Norwegian study called “Brief Outpatient Rehabilitation Program for Post–COVID-19 Condition: A Randomized Clinical Trial,” from Nerli et al., published last December by JAMA Network Open. The senior author was Professor Vegard Wyller, the dean of the Norwegian wing of the CBT/GET/Lightning Process ideological brigades. The trial was as bad as one would expect, given its provenance. (Here’s my post about it.)

It is refreshing when other academics issue their own challenges to the kinds of self-evident methodological lapses that mar this study and so much of this research. Richard Aubrey White, a Harvard-trained biostatistician and a researcher at the Norwegian Institute of Public Health (NIPH), has just published a tough take-down of Nerli et al.—to which the lead author has responded with more of the usual blah blah.

The article appeared in Forskning.no, an online publication focused on news about Norwegian and international research, and includes a note that White is not writing on behalf of his employer. (NIPH has been a hotbed of support for the so-called “biopsychosocial” paradigm, so White’s opinion might be unpopular in some quarters.)

White’s critique—called “Promising treatment for long covid without evidence”–highlights similar issues to those I raised. (I am reliant on Google Translate, so don’t hold me responsible for any mistranslations.)

“A recent study by Tom Farmen Nerli and others suggests treatment ‘based on a cognitive and behavioral approach.’ The study has raised hopes, but a closer examination reveals serious methodological weaknesses.

“Clinically significant” is a crucial term when assessing treatment effectiveness. It means that the treatment produces a real, noticeable improvement in the patient’s health—not just a difference that can be measured statistically. It is not enough to demonstrate a change; it must be large enough to have practical significance for quality of life and function.

“The Nerli study uses a physical function scale (SF-36-PF) in which participants self-assess the extent to which their health limits them in everyday activities such as climbing stairs, carrying goods or vacuuming. 

“In the analysis plan, written before the results were known, the authors state that an improvement of at least 10 points on this scale is necessary for a treatment to be considered clinically significant. 

“In this study, the treatment group achieved an average improvement of only 9.2 points – below the threshold value. They have indeed measured an effect, but according to the authors’ own criteria it is not large enough to have practical significance.

“Yet the authors present the treatment as clinically significant and effective in the article. This is clearly misleading.”

White also points out that the design of the study lends itself to bias, and that modestly positive results should not therefore be regarded as evidence of the impact of the intervention itself:

“Since participants self-assessed the outcomes (for example, “Is your health such that it limits you from moderate activities such as vacuuming?”), and knew whether they received the intervention or not, it is unclear how much of the intervention’s modest effect is due to the placebo effect and response bias. 

“This means that participants may have overestimated their improvement because they expected an effect, or wanted to confirm the researchers’ assumptions. Objective measures, such as the number of steps per day, would likely have shown even weaker results.”

According to White, the implications are clear. The study’s findings indicate “the need for biomedical treatment strategies aimed at, among other things, viral persistence, post-acute inflammation, autoimmunity, thrombosis, reactivation of latent viruses, dysbiosis, intestinal translocation and mitochondrial dysfunction.”

Finally, White writes: “Misrepresentation, weak methods and data secrecy are not just technical details, but serious mistakes with major consequences for those struggling with long covid. Only honest and open science can form the basis for informed choices for patients, healthcare professionals and politicians.” 

The response from the corresponding author, Tom Nerli, is unconvincing. He fails to acknowledge the key point behind White’s charge of misrepresentation—that the primary outcome fell below the threshold deemed clinically significant by the investigators themselves. And Nerli doesn’t address the concern about an unblinded trial that relies solely on subjective outcomes. Instead, he cites similar research, as if multiple poorly designed studies can collectively correct the problem.

Finally, one of his suggestions is a real whopper of delusional thinking: “The results from our study are so promising that it should be offered to the patient group that currently has no other documented effective treatment available.”

There’s clearly a huge gap in the two versions of reality being presented in this exchange.

(View the original post at virology.ws)

#norway #wyller

Norwegian Long Covid Rehab Trial Misrepresents Clinically Insignificant Findings As “Effective”

By David Tuller, DrPH

If the results for a trial’s primary outcome do not meet the threshold for what is considered a “clinically significant” benefit, it goes without saying—or at least it should–that investigators have no legitimate grounds for promoting their intervention as “effective.” This is especially true when the trial in question is unblinded and the measure involved is subjective. This combination of elements is a recipe for bias. Given the placebo effect, such trials should be expected to yield modestly positive results solely as an artefact of the study design.

And yet…the die-hard members of the CBT/GET ideological brigades apparently feel free to ignore these basic principles. And so do the high-impact journals that keep publishing their research.

Last month’s example—“Brief Outpatient Rehabilitation Program for Post–COVID-19 Condition: A Randomized Clinical Trial”—was published by JAMA Network Open. The senior author is Norway’s Vegard Bruun Wyller, a professor of pediatrics at the University of Oslo. The study included 314 patients with “mild to moderate” cases meeting the WHO’s broad criteria for Long Covid, or what the organization calls post-COVID-19 condition (PCC). 

Half of the participants received a program of two to eight clinical sessions “based on a cognitive and behavioral approach.” The other half didn’t receive the intervention–just “care as usual.”

The intervention included offering participants “cognitive reassurance that bodily symptoms do not necessarily indicate a disease but rather a disorder that is temporary and amendable” and explaining that “certain infections (eg, COVID-19) could trigger maladaptive responses and diverse, unpredictable, and bothersome symptoms (eg, fatigue, dyspnea, and brain fog).” During sessions, “cognitive behavioral therapy–trained physiotherapists supervised the patients by using nondirective communication, socratic dialogue, and guided discovery, prompting patients to infer that recovery would require an active pursuit of physical and mental tasks, thereby fostering positive stimuli expectancies.”

The trial’s primary outcome was the SF-36 Physical Function Subscale (SF-36-PFS)—a commonly used measure in these studies. Higher scores on the 100-point scale represent better self-reported health. As the paper and the study protocol both noted, a 10-point change on the SF-36-PFS is considered “clinically significant.” Changes less than 10 points are, by definition, considered clinically insignificant—that is, too small to be meaningful or even noticeable to the individual.

In this trial, the difference between the changes in the intervention and non-intervention groups on the SF-36-PFS at the end is 9.2 points–below the 10-point threshold pre-designated as “clinically significant.”  But the abstract reports instead that the SF-36-PFS scores “improved statistically and clinically significantly in the intervention group.” The same phrase is repeated in a highlights box headlined “Key Points.”

Note the clever wording—the phrase apparently refers to the change within the intervention group from baseline to post-intervention and suggests that it exceeded the 10-point threshold for clinical significance. But change within the intervention group is not the metric of interest in a clinical trial. You don’t need to conduct a clinical trial in order to measure change in a group receiving an intervention. You conduct a clinical trial in order to compare the change in the intervention group with the change in some other group.

In a clinical trial, therefore, the metric that matters—as Professor Wyller and editors of JAMA Network Open surely know–is the difference between the changes in the intervention and non-intervention groups. And in this case, the difference was not clinically significant–even in a trial with bias built into the design.

Here’s the claim from the abstract’s conclusion: “In this randomized clinical trial, a brief outpatient rehabilitation program with a cognitive and behavioral approach in patients with PCC was effective and safe.” Given that the primary outcome did not reach the level of clinical significance, it is unwarranted and deceptive to claim the intervention overall was “effective.”

Beyond the abstract, the fact that the difference between the groups fell below the clinically significant threshold is also not mentioned in either the discussion or limitations sections. The only mention of the 10-point threshold itself is in the section on statistical calculations. (The abstract mentions that the difference between the groups is 9.2 points, but the number on its own, and without the needed context, is meaningless to readers.)

In short, the investigators are seriously misrepresenting their findings. And JAMA Network Open is allowing them to do so.

That misrepresentation is far from the only problem with this trial. Among other issues, almost a quarter of the participants—83 of 314, or 74%–did not provide final data. Since they were presumably motivated to get better, the high drop-out rate raises questions about why so many decided not to follow through.

This trial, however unimpressive the findings, has received lots of attention.

Along with the study, JAMA Network Open published an “invited commentary” from Trudie Chalder, King’s College London’s factually and mathematically challenged professor of cognitive behavior therapy. It is full of her usual blah blah, like this bit of PACE-style propaganda: “Over the past few decades, substantial evidence has amassed for the efficacy of CBT for symptoms in the context of somatic conditions, including chronic fatigue syndrome.”

And as often happens with research from these investigators, the news coverage has been glowing and gullible. NRK—the Norwegian Broadcasting Corporation—published a credulous article headlined “New study gives hope.” The headline on a MedPage Today article was “Long COVID Symptoms Improve With Outpatient Intervention.” Of course, neither mentioned that the results on the primary outcome were not clinically significant. I assume this trial’s inflated claims will be included in the next iteration of the recent “living” systematic review of Long Covid interventions, which is itself in need of a major correction.

(View the original post at virology.ws)

#norway #wyller

By David Tuller, DrPH

Earlier this month, I wrote a post about a new study asserting that “exercise does not cause post-exertional malaise in Veterans with Gulf War Illness.” As I explained, the research, led by experts from the University of Wisconsin and published by the journal Brain, Behavior, and Immunity, made no sense. Although it purported to investigate whether an exercise challenge caused Gulf War Illness (GWI) patients to experience post-exertional malaise (PEM), 85% of the trial participants had already reported that PEM was not a factor for them.

In other words, the study was essentially designed to produce self-evident results. Of course GWI patients without PEM did not experience PEM when put through an exercise challenge. I mean, really, University of Wisconsin? Put simply, this piece of research does not pass the smell test.

Along with the paper, the journal published a laughable comment from Professor Vegard Wyller, a professor of pediatrics at the University of Oslo and a dedicated member of the Norwegian arm of the GET/CBT ideological brigades. In his comment, Professor Wyller somehow tried to spin the study’s non-findings into evidence for his misguided theories.

Now an article in the Norwegian tabloid Dagbladet has focused on this study and promoted its bogus claims. And just like the journal, Dagbladet offered Professor Wyller an opportunity to opine about the study’s purported significance. (Sissel Sunde has written about the Dagbladet article in her blog Life with ME.)

I’ve had my own run-around with Dagbladet. Four years ago, the news outlet ran an article about the Lightning Process for ME/CFS that referred to my work while failing to mention my academic credentials. I wrote a letter of protest. In response, Dagbladet published my letter as an opinion piece. (I was told that, under Norwegian press law, I had a right-of-response after having been misidentified.)

My impression is that Dagbladet is generally on the wrong side of these issues. So…the new article on the GWI study is illustrated with a photo of an attractive young woman on an exercise bike, with the following caption: “People who suffer from chronic fatigue syndrome can also tolerate exercise, a new study shows.” Of course, the study was about GWI, not chronic fatigue syndrome. That didn’t stop the investigators—and Professor Wyller—from extrapolating the findings to patients with other conditions characterized by fatigue.

Here are some of what Professor Wyller had to say in Dagbladet:

“Wyller says that a feeling of pain and exhaustion is perceived as a signal that something is wrong, but according to the professor it is entirely possible to feel pain without there being anything wrong with the body.

“He emphasizes that the experience of pain and other symptoms is completely real.

“- This suggests that the symptoms occur in the brain, rather than in the body. That doesn’t mean that there aren’t real feelings, but that the symptom has another explanation – it might be the brain playing a little trick on you…I think we should do more research on the brain to find an explanation for these diseases, rather than trying to find a physiological explanation, says Wyller.”

To its credit, Dagbladet sought out someone to counter Professor Wyller’s non-evidence-based views: Karl Johan Tronstad, a professor in the biomedicine department at the University of Bergen, who is investigating ME as a disease involving auto-immune aspects. Professor Tronstad noted, as I had in my critique, that few of the GWI patients in the study actually experienced PEM as part of their illness presentation. He suggested, therefore, that it was “dangerous to generalize the findings to other patient groups.”

Professor Tronstad also told Dagbladet: “- What this study actually shows is that PEM occurs to a relatively small extent in their study group of 40 patients…Most patients who have PEM will most likely decline to participate. They know that they will not be able to complete the hard test programme, or that it would cause them to deteriorate significantly over a long period of time, he says.”

**********

I decided to submit a formal letter to Brain, Behavior, and Immunity about this core issue with the study. Here is what I wrote:

Dear Editor—

A recent study–“Exercise does not cause post-exertional malaise in Veterans with Gulf War Illness: A randomized, controlled, dose–response, crossover study,” Boruch et al–reported that exercise at different levels of intensity did not cause post-exertional malaise (PEM) in patients with Gulf War Illness (GWI). Even though some participants reported symptom exacerbation, those changes were washed out in the group averages.

According to the investigators, their findings support the argument that “the benefits of exercise outweigh the risks.” However, Table 3 indicates that only 15% of the study sample—6 out of 40 participants–experienced PEM as part of their GWI presentation. Although the paper noted that PEM is “prevalent” in GWI, it is not a required criterion for a diagnosis. Whatever PEM’s overall prevalence among GWI patients, it was apparently not a factor for 85% of participants in the study sample.

It is hard to understand the point of investigating whether exercise causes PEM in GWI patients who have already indicated that they do not experience PEM. Ultimately, this study tells us nothing about the potentially harmful impact of exercise on a cohort of GWI patients who actually suffer from PEM.

David Tuller, DrPH
Senior Fellow in Public Health and Journalism
Center for Global Public Health
School of Public Health
University of California, Berkeley

[1] Boruch A et al. Exercise does not cause post-exertional malaise in Veterans with Gulf War Illness: A randomized, controlled, dose–response, crossover study. 2024. Brain, Behavior, and Immunity. 120, 221-230.

https://trialbyerror.org/2024/09/20/norwegian-tabloid-covers-stupid-gulf-war-illness-pem-study-my-letter-to-journal-editor-on-bogus-claims/

#Dagbladet #wyller

By David Tuller, DrPH

A recent study investigated a question no one seems to have been asking. That can be a good thing—if it’s a question that’s worth investigating. But that’s not the case here.

The study, published in the journal Brain, Behavior, and Immunity and led by researchers at the University of Wisconsin, was called “Exercise does not cause post-exertional malaise in Veterans with Gulf War Illness: A randomized, controlled, dose–response, crossover study.” The title itself indicates a conceptual problem built into the study. I mean, has anyone seriously argued or raised concerns that exercise “causes” PEM among GWI patients even if they do not report having PEM as part of their GWI in the first place?

Post-exertional malaise or a related construct is a required characteristic of ME or ME/CFS, however defined (excluding pre-2000 definitions like the 1991 Oxford criteria and 1994 Fukuda criteria). In contrast, PEM is not required in GWI but is one of many possible complaints that can combine to generate a diagnosis. Like Long Covid, GWI is an umbrella term and it covers a wide range of presntations for those suffering after military service from what is categorized as a chronic, multi-system illness (CMI). Case definitions for GWI require the presence of various symptoms spread across multiple domains but with no specific symptoms universally mandated, and certainly not PEM.

So the study title is hard to understand. You might want to investigate the impact of exercise in GWI patients in general, or you might want to investigate differential effects of exercise in GWI patients with and without PEM. But you don’t need to investigate whether exercise causes PEM in GWI patients who have already reported that they don’t experience PEM. Am I missing something here?

The study included 40 participants with GWI diagnoses. As described in the paper, it was “a dose–response study that evaluated acute effects of light-, moderate-, and vigorous-intensity cycling on three types of psychometric outcomes previously shown to be exacerbated by exercise in CMIs.” The investigators mentioned ME/CFS and Long Covid as other CMIs. As far as I can tell, they did not point out that only in the former is PEM a required symptom.

Here’s the rationale given for the research: “Studies [of CMI patients] which investigate and document the effect of different aerobic exercise intensities on PEM are needed to better inform exercise prescription and ensure risk minimization for these individuals.” And the main hypothesis was that “higher intensity exercise” would elicit greater levels of PEM

One corollary of this rationale and hypothesis is, or should be, that you can’t study anything about the effects of different aerobic exercise intensities on PEM in patients who don’t experience PEM in the first place. It’s like studying whether standing on the edge of a cliff leads to vertigo in people who don’t experience vertigo while standing on the edge of a cliff. It doesn’t make a lot of sense. The investigators do not seem to grasp this basic notion.

The key data point is in Table 3, which lists “baseline symptoms reported during study visit 1.” In other words, this reflects what participants reported as part of their GWI at the start of the study. Almost all—36 out of 40, or 90%–reported fatigue. Not surprising. But looking down the list, only 6 out of the 40, or 15%, met criteria for PEM as measured with the DePaul Symptom Questionnaire.

In other words, the great majority of this cohort of GWI patients—85%–did not report experiencing PEM. It is therefore hard to understand why the investigators would have expected any kind of exercise challenge to trigger PEM in these patients.

Some participants did report a worsening of symptoms, but those effects got washed out in the averages. “Undesirable effects such as symptom exacerbation were observed for some participants, but the group-level risk of PEM from light-, moderate-, or vigorous-intensity exercise was no greater than seated rest,” the paper noted. Of course, that’s exactly what you’d expect with a study sample in which only 15% reported having PEM as part of their GWI.

Beyond that—and there’s not really much point in going “beyond that,” since the study has essentially invalidated its own premise–the main findings were based on symptom reports taken within one hour after the exercise challenge. It is well-known that PEM frequently manifests as a delayed response. So even if the study were kosher in other regards—which it is not–it wouldn’t be possible to make reliable statements regarding PEM from data that would inevitably miss many or most cases of PEM.

And despite all these caveats, a leading light of the Norwegian branch of the CBT/GET ideological brigades, Professor Vergard Wyller, is nonetheless touting the study as proof of something or other in a commentary in the same journal called “Post-exertional malaise – A functional brain aberration?” Since he takes the findings at face value, he makes some absurd points. For example: “The lack of associations between exercise intensity and PEM experience suggest that sensory input cannot be the sole driver of the symptom.”

No, the lack of associations suggests nothing of the kind. It can be explained by this salient detail–the vast majority of the 40 study participants did not experience PEM as part of their GWI. The fact that exercise did not trigger PEM in this cohort says nothing about the impact of exercise in cohorts of GWI patients who do suffer from PEM. This is such an obvious and elemental point that even to have to express it strikes me as a ridiculous state of affairs.

And more from Professor Wyller: “The study…adds to an increasing body of evidence confirming that physical activity is not harmful in conditions characterised by PEM.”

No, it does nothing of the kind. The word “characterised” is doing a lot of work in that sentence. A case of GWI might be “characterised” by PEM; however, GWI is very frequently not “characterised” by PEM. Excessive physical activity leads to PEM in patients susceptible to PEM. That means all patients accurately diagnosed with ME or ME/CFS. And it means patients with GWI who happen to experience PEM–which was not the case for 85% of the participants in the study under review.

In his comment, Professor Wyller is responding to a straw-person argument. Since the rest of his blathering rests on such fundamental misstatements, it doesn’t warrant further scrutiny.

https://trialbyerror.org/2024/09/07/bogus-claims-in-study-of-exercise-and-pem-in-gulf-war-illness/

#wyller