Great article reviewing and testing the accuracy of a range of direct measures (i.e., bogus items, requiring the respondent to choose a specific answer) and indirect measures (i.e., unobtrusive post hoc indices) for detecting careless responses in online surveys.

Excellent overview tables and very practical conclusions!

Open Access: https://doi.org/10.3758/s13428-024-02484-3

#psychologicalassessment #psychometrics #researchmethods #psychology #quantitativepsychology

European Journal of #PsychologicalAssessment publishes TOP updates to 'materials, data, code publicly available'

and updated word limits, "the extended word count should primarily go towards the Methods and Results section" 👏

https://econtent.hogrefe.com/doi/full/10.1027/1015-5759/a000926

#Psychometrics #OpenScience

Our guideline on how to report assessments where experts review several sources of (longitudinal) information for a more accurate result has been available as a #PrePrint for a while and is now available at a journal:
https://www.sciencedirect.com/science/article/pii/S0010440X25000318

Led by @oscarkjell and team, the guideline aims to support researchers in planning, reporting, and evaluating research that aims to achieve best-estimate assessments.

#PsychologicalAssessment #Diagnosis #Psychometrics #ClinicalPsychology #Psychiatry

Individualizing personality assessments through humanistic trait-based interventions

New research published in The Humanistic Psychologist introduces the Five-Factor Personality Assessment System (FFPAS), a novel method for individualizing psychological assessments.

PsyPost

Led by @oscarkjell's team, together with @admin and loads of Not-on-Mastodon people we just published a #PrePrint
https://www.medrxiv.org/content/10.1101/2024.03.19.24304526v1

We propose reporting standards for "best-estimate assessments", i.e. where results from multiple experts and multiple sources of (longitudinal) information are integrated (e.g., a diagnosis).

#PsychologicalAssessment
@psychology

The LEADING Guideline Reporting Standards for Expert Panel, Best-Estimate Diagnosis, and Longitudinal Expert All Data (LEAD) Studies

Accurate assessments of symptoms and diagnoses are essential for health research and clinical practice but face many challenges. The absence of a single error-free measure is currently addressed by assessment methods involving experts reviewing several sources of information to achieve a more accurate or best-estimate assessment. Three bodies of work spanning medicine, psychiatry, and psychology propose similar assessment methods: The Expert Panel, the Best-Estimate Diagnosis, and the Longitudinal Expert All Data (LEAD). However, the quality of such best-estimate assessments is typically very difficult to evaluate due to poor reporting of the assessment methods and when it is reported, the reporting quality varies substantially. Here we tackle this gap by developing reporting guidelines for such studies, using a four-stage approach: 1) drafting reporting standards accompanied by rationales and empirical evidence, which were further developed with a patient organization for depression, 2) incorporating expert feedback through a two-round Delphi procedure, 3) refining the guideline based on an expert consensus meeting, and 4) testing the guideline by i) having two researchers test it and ii) using it to examine the extent previously published articles report the standards. The last step also demonstrates the need for the guideline: 18 to 58% (Mean = 33%) of the standards were not reported across fifteen randomly selected studies. The LEADING guideline comprises 20 reporting standards related to four groups: The Longitudinal design ; the Appropriate data ; the Evaluation – experts, materials, and procedures ; and the Validity group. We hope that the LEADING guideline will be useful in assisting researchers in planning, reporting, and evaluating research aiming to achieve best-estimate assessments. Open data (Delphi surveys 1 and 2), code (analyses), and material (surveys): <https://osf.io/fkv4b/> ### Competing Interest Statement All authors have completed the ICMJE uniform disclosure form and declare: V C Eijsbroek, K Kjell, and O Kjell received funding from FORTE (2022-01022); and H A Schwartz from the National Institutes of Health (Grant R01 AA028032-01); O Kjell and K Kjell have co-founded and hold shares in a start-up using computational language assessments to diagnose mental health problems based on best-estimate assessments; J R Boehnke is as editor part of the International Society for Quality of Life Research and the Springer Nature Group; no other relationships or activities that could appear to have influenced the submitted work. ### Funding Statement V C Eijsbroek, K Kjell, and O Kjell received funding from FORTE (2022-01022); and H A Schwartz from the National Institutes of Health (Grant R01 AA028032-01). ### Author Declarations I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained. Yes The details of the IRB/oversight body that provided approval or exemption for the research described are given below: Swedish law (2003:460) and the Swedish Ethical Review Authority state that only research that includes i) collecting personal information, and/or ii) that involves 'obvious' (uppenbar) risk for physical or psychological harm, or iii) involves manipulating or deceiving individuals, should undergo an external ethical review. Considering that the current research involves asking participants to rate and comment on the reporting recommendations, this research does not fall within these criteria. The individual open-ended comments and closed-ended ratings in the open material are anonymized. The closed-ended ratings per reporting standard in the Supplementary Material are presented on group level. The study is deemed exempt from requiring ethical approval according to Swedish Law (see 3-4 of the Act [2003:460] on ethical review of research involving humans in Sweden). Hence the research should not be reviewed by the Swedish Ethical Review Authority. More information can be found at the Swedish Ethical Review Authority (<https://etikprovningsmyndigheten.se/>). I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals. Yes I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as [ClinicalTrials.gov][1]. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance). Yes I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable. Yes Data sharing statement: Open data (Delphi surveys 1 and 2), code (analyses), and material (surveys) can be found on the Open Science Framework: <https://osf.io/fkv4b/> <https://osf.io/fkv4b/> [1]: http://ClinicalTrials.gov

medRxiv

While I'm happy to have a new paper in #PsychologicalAssessment with my wonderful colleagues 🥳 , I'm done with #APA journals. No more!
We have reached a new low of #paywalls: #APA doesn't even provide authors with an authors' copy anymore. #ThankYouForNothing ...not even my own paper... 😔

https://psycnet.apa.org/doi/10.1037/pas0001295

#OpenScience vs #noaccess

Provisional principles for fostering cultural responsiveness in assessment:
https://onlinelibrary.wiley.com/doi/full/10.1002/ets2.12374
#ETSReport

It remains to be seen whether it is time to join the authors' enthusiasm who believe that
"...the application of these principles can address the criticisms of current assessments."

But some of the questions should be asked in any #HEI setting using assessments, e.g.,
"Why is this test needed?" 😉
"How will the test provide opportunity?"

#PsychologicalAssessment #Psychometrics

Did you know that the #AERA, #APA, #NCME Standards for Educational and Psychological Testing were available #OpenAccess in English and Spanish? https://www.testingstandards.net/open-access-files.html

#Psychometrics #PsychologicalTesting #PsychologicalAssessment #ValidityTheory ICYMI @TomJewell

Open Access Files

The Standards for Educational and Psychological Testing (Spanish and English) are now open access. Downloadable files are available below in multiple formats   2014 English edition PDF PDF eBook...

THE STANDARDS FOR EDUCATIONAL AND PSYCHOLOGICAL TESTING

Does anyone know whether there will be a new edition of the AERA APA NCME "Standards for Educational and Psychological Testing"?

https://en.wikipedia.org/wiki/Standards_for_Educational_and_Psychological_Testing

#Psychometrics #PsychologicalAssessment

Standards for Educational and Psychological Testing - Wikipedia

I got my psychological assessment report from my doctor yesterday.
Although the good news is that the chance that my condition is/will become schizophrenia is very low, to be honest, the experience of getting assessment is horrible.
I was asked to memorize a series of number and follow the rule to repeat those numbers again and again without having time to rest.
I cannot help but burst into tears and asking when will this assessment end.

#MCMI #Taiwan
#PsychologicalAssessment

/1