@handyschwartz @katarinakjell

#BeyondRatingScales

Language-based assessments also have the ability to be self-descriptive, moving beyond mere scores as the output.

For example, statistically significant descriptive words and key phrases can be visualized based on their underlying meaning along relevant dimensions.

The figure shows AI-generated summaries of the ten most negative and positive answers to: How are you feeling?

7/9

@handyschwartz @katarinakjell

Approaching a Theoretically Upper Limit
Language responses analyzed with #LargeLanguageModels can predict rating scales with a Pearson r of .85 🎯

This high accuracy approaches the rating scale's own reliability –i.e., a theoretical upper limit (r = .71–.84).

Rating scales are only a “proxy” and not a perfect true score of a psychological construct – so future research should move beyond only predicting rating scales.

6/9

@handyschwartz @katarinakjell

#LargeLanguageModels
#LargeLanguageModels have led to unprecedented accuracies over most computerized language processing tasks (https://gluebenchmark.com/leaderboard).

They largely owe their success to their ability to statistically model words in the context they are used.

Bringing such context to psychological text analysis can more precisely quantify the specific meaning of language and yield a truer understanding of the person behind the words.

5/9

@handyschwartz @katarinakjell
Favorable Measrument Characteristics
More information comes from having favorable measurement characteristics, including high:

range (e.g., absolutely loves and hates),
resolution (e.g., cherishes, loves, adores, likes), and
multi-dimensionality (e.g., love, excitement, joy, awe).

3/9

@handyschwartz @katarinakjell

Information-Rich Language

We observe that language-responses have many times more information than rating scale responses. 100 participants answered How are you feeling in an open response box and filled out a standard rating scale about their feelings (the PANAS):

The language-based responses included 4.8 times more information than the rating scale responses.

2/9