Our first session of the morning at the EASE conference in Istanbul is a round table discussion in collaboration with ISMTE.
Mary Miskin, George Vousden, Joe Krumpfer, Ilana Kolodkin, and Matthew Woodcock discuss author, editor and publisher perspectives of quality.
What's a good #PeerReview for an author? Ilana Kolodkin says short, positive reviews aren't given as much weight as long, negative reviews, which can be unfortunate. She prefers careful review over speed, even as an author.
George Vousden says reviewer training is variable and to avoid a focus on language you need clear guidelines. Reports should be constructive and guide the authors. If they're positive, the reviewer should outline how the article is suitable.
We shouldn't assume peer reviewers haven't seen a submission before, says Ilana Kolodkin. They usually have submitted and revised a couple of times already.
Joe Krumpfer notes that a reviewer saying "I reviewed this before and recommended rejection" is unhelpful, because the journal criteria may be different.
Are structured #PeerReview forms useful?
Guidelines online are often not read, agree Joe Krumpfer and George Vousden. However, George says he wants every review to discuss methods and so structured questions are helpful.
Ilana Kolodkin finds structured forms really annoying, because she has carefully prepared a coherent review and needs to break it up.
George notes you shouldn't use scales for "impact": prose is better to guide editors, if they use that criterion.
Different #PeerReview models are accepted by different communities, notes Joe Krumpfer. It needs to be tailored to each field.
Ilana Kolodkin thinks double anonymisation is a great idea, but it's hard for authors because you need to detach it from your previous work.
p.s. @bahar notes to use "double anonymisation" vs "double blind" terminology.
Should editors set similarity thresholds? George Vousden cautions against this, because the number "can mean anything". A long article can have the whole introduction plagiarised, but a low overall similarity index.
Altmetrics such as article-level metrics can be useful, says George Vousden, because the Journal Impact Factor is not useful for individual article impact. However, #altmetrics attention such as on social media can be due to negative concerns about errors or integrity as well as scientific quality.
"Quality" can only really be judged qualitatively, but @PLOS is introducing #OpenScience indicators.
Background Plagiarism is common and threatens the integrity of the scientific literature. However, its detection is time consuming and difficult, presenting challenges to editors and publishers who are entrusted with ensuring the integrity of published literature. Methods In this study, the extent of plagiarism in manuscripts submitted to a major specialty medical journal was documented. We manually curated submitted manuscripts and deemed an article contained plagiarism if one sentence had 80 % of the words copied from another published paper. Commercial plagiarism detection software was utilized and its use was optimized. Results In 400 consecutively submitted manuscripts, 17 % of submissions contained unacceptable levels of plagiarized material with 82 % of plagiarized manuscripts submitted from countries where English was not an official language. Using the most commonly employed commercial plagiarism detection software, sensitivity and specificity were studied with regard to the generated plagiarism score. The cutoff score maximizing both sensitivity and specificity was 15 % (sensitivity 84.8 % and specificity 80.5 %). Conclusions Plagiarism was a common occurrence among manuscripts submitted for publication to a major American specialty medical journal and most manuscripts with plagiarized material were submitted from countries in which English was not an official language. The use of commercial plagiarism detection software can be optimized by selecting a cutoff score that reflects desired sensitivity and specificity.
EASE
Ksenija Bazdaric
Kate Nyhan is changing servers