Mastodawn

Question for #SocialScience or #surveyresearch or #psychometrics or #statistics folks: How do I test the order of maybe-ordinal responses for some items in a survey?

I have done some scaling where the items are tested for ordered/monotonic properties (e.g., is item 1 really more of whatever than item 2?). However, this is a bit different, I think: I have been tasked with analyzing some items from a survey kind of like this:

I think pregnant women deserve the same rights as everyone else (choose one)
Absolutely
Hell yeah
Sure!
I don't know
Maybe
Nah
Nope

Is Absolutely actually more agreement than Hell yeah? Does a response of I don't know indicate more agreement than Maybe?

It's this kind of question. How do I test for the empirical order (if any) applied by about 800 participants?

#question #help #suggestion

Show thread

Jan R. Boehnke

@guyjantic If this question is the only thing you have, it is difficult to make an argument to support ordered categories.

In #SurveyResearch you could do it via previous qualitative work, e.g., cognitive interviews; or quantitative work (e.g., ask people to indicate the order of such terms). If you have specific rating scale anchors (I doubt it exists for the ones you gave as examples 😂 ), you could look for published work on them and at least argue that others have found these to be ordered.

Show thread

Jan R. Boehnke Jan 12

@guyjantic As you have data, you may be able to suggest order empirically w view to external reference data.

For example, using a multinomial regression model you could predict the answers to this question and then investigate whether increases in the predictors to together with increases in category choices. But the resulting order obviously only pertains with view to the data you use to predict your target question. But it might be relevant or at least enough for your particular application.

Show thread

Jan R. Boehnke Jan 12

@guyjantic If you have not only one, but multiple questions, several item response models extend this logic by assuming a latent continuous variable:
they evaluate whether the responses to the questions can be reconstructed as ordered responses - and ordered in this case meaning 'with reference to this latent variable' (e.g., does being more in favour of rights for pregnant people consistently align with choosing higher categories in the individual question responses).

#IRT #Psychometrics