PREreviewLast week our team participated in both #FSCI2024 and #FORCE2024. A short🧵summarizing our experience.
We thank the organizers for inviting us and all the colleagues who made this experience memorable
For the first time, we ran a 3-day in-person #OpenReviewers workshop.
On Day 1 we started with this ice-breaker exercise: Thinking about what you know of peer review today, which of these images would you say best describes the process?
This activity usually prompts participants to think about what they perceive peer review to be like and what they wish it to be.
This is a commonly used cartoon used to depict the peer-review process. What do you see? What would you deduce the process is like from this cartoon?
Then we go on questioning the very definition of peer review which centers the activity around "qualified members of a profession within a relevant field" (Wikipedia). But who decides who is qualified and who is not? What are the traits that make someone qualified?
After learning about alternative ways to engage in peer review and imagining what a better future looks like, we move on to a section many don't expect: we talk about systemic oppression, particularly the systems of racism and colonialism, and how they show up in scholarship.
We discuss "Racial Essentialism", the belief that socially constructed racial categories reflect “inherent” biological differences & that races are fundamentally different".
The reason we talk about this is because many (peer-reviewed) studies published to this day practice racial essentialism making the study not only scientifically wrong, but also harmful as they continue to uphold racist beliefs. In the workshop, we walk through some examples.
As scholars, the idea that academia is immune to systemic oppression is one of the biggest lies we have to work hard to unlearn. Science has worked and continues to work to uphold oppression. We need to know how to interrupt these systems including in our role as reviewers.
On Day 2, we dive into a step-by-step process to guide the process of reviewing a research manuscript.
The Open Reviewers Reviewer Guide is available on Zenodo: https://doi.org/10.5281/zenodo.5484087
In step 1, we go over how our biases and beliefs often stem from systemic oppression and do affect the way we review. The goal is to gain awareness not shaming ourselves. For this step, we practice the IDEA-R2 Method.
Bias reflection guide: https://doi.org/10.5281/zenodo.5484052
Bias reflection guide: https://doi.org/10.5281/zenodo.5484052
In step 4, we practice how to make our feedback constructive, clear, and actionable so that it's most likely to minimize harm, be well-received, and be incorporated by the authors.
Then, together we read and discussed a #preprint with the goal of collaboratively review it applying these steps. For this workshop we selected the @biorxivpreprint Scale-sensitive Mouse Facial Expression Pipeline using a Surrogate Calibration Task https://doi.org/10.1101/2024.05.15.594417
We chose this preprint because the authors had asked for feedback via COAR Notify. Learn more about that process here: https://content.prereview.org/request-a-review
None of us were a subject matter expert in the topic, but we all had feedback that we hope will be helpful to the authors. We pieced together the feedback into a full review and published it on PREreview.org https://prereview.org/reviews/13162263
Our 🌟 Head of Community
@vanessa presented our work at #FORCE2024. Slides are openly available on Zenodo at: https://doi.org/10.5281/zenodo.13174950
Once again we thank
@force11 conference organizers and workshop participants for making it possible for us to learn together!
Transforming Scholarly Publishing: A Community-Led Approach to Peer Review with PREreview
Talk by Vanessa Fairhurst, Head of Community at PREreview, at the FORCE11 conference which took place in-person at the UCLA campus, Los Angeles, California on Thursday 1st August 2024. Abstract: At PREreview, we are dedicated to fostering a more equitable, transparent, and accessible scholarly ecosystem. Our work is a direct response to the flawed way scientific research is evaluated. Behind closed doors, a handful of unpaid reviewers—selected opaquely and mainly through personal connections—use subjective criteria to decide the fate of a research article. Reviewers often undergo minimal training for this crucial task and have a lack of awareness of how to address issues of bias and systemic oppression, leading to a perpetuation of current inequities within scholarly publishing. In this talk, we'll explore PREreview's community-driven approach to transform peer review, rooted in values of accessibility, connectivity, self-awareness, and measurable impact, highlighting key features of our community-informed open preprint review platform, alongside our training programs aimed at fostering fair, constructive feedback while addressing systemic biases. You’ll hear case studies of the impact of creating a more open, equitable, and community-driven scholarly evaluation ecosystem from some of our global community members, as well as learn about key opportunities to join us in revolutionizing the peer-review process.