Our first session of the morning at the EASE conference in Istanbul is a round table discussion in collaboration with ISMTE.
Mary Miskin, George Vousden, Joe Krumpfer, Ilana Kolodkin, and Matthew Woodcock discuss author, editor and publisher perspectives of quality.
What's a good #PeerReview for an author? Ilana Kolodkin says short, positive reviews aren't given as much weight as long, negative reviews, which can be unfortunate. She prefers careful review over speed, even as an author.
George Vousden says reviewer training is variable and to avoid a focus on language you need clear guidelines. Reports should be constructive and guide the authors. If they're positive, the reviewer should outline how the article is suitable.
We shouldn't assume peer reviewers haven't seen a submission before, says Ilana Kolodkin. They usually have submitted and revised a couple of times already.
Joe Krumpfer notes that a reviewer saying "I reviewed this before and recommended rejection" is unhelpful, because the journal criteria may be different.
Are structured #PeerReview forms useful?
Guidelines online are often not read, agree Joe Krumpfer and George Vousden. However, George says he wants every review to discuss methods and so structured questions are helpful.
Ilana Kolodkin finds structured forms really annoying, because she has carefully prepared a coherent review and needs to break it up.
George notes you shouldn't use scales for "impact": prose is better to guide editors, if they use that criterion.
Different #PeerReview models are accepted by different communities, notes Joe Krumpfer. It needs to be tailored to each field.
Ilana Kolodkin thinks double anonymisation is a great idea, but it's hard for authors because you need to detach it from your previous work.
p.s. @bahar notes to use "double anonymisation" vs "double blind" terminology.
Should editors set similarity thresholds? George Vousden cautions against this, because the number "can mean anything". A long article can have the whole introduction plagiarised, but a low overall similarity index.
Altmetrics such as article-level metrics can be useful, says George Vousden, because the Journal Impact Factor is not useful for individual article impact. However, #altmetrics attention such as on social media can be due to negative concerns about errors or integrity as well as scientific quality.
"Quality" can only really be judged qualitatively, but @PLOS is introducing #OpenScience indicators.
EASE