@alesssia @albertcardona @timnitGebru
My bias, from being married to an assessment expert:
Most educators do not understand assessment at all and therefore fail to communicate about it to people using the results for hiring and so on. I have never met anyone who sets a university exam who even knows what construct validity means.
Assessments, broadly, fall into one of two categories, formative and summative. Formative assessments are intended to drive education. You do a formative assessment and it tells you and/or your teacher what areas you need to focus on. Used well, these let educators make far more efficient use of contact time. Summative assessments exist to measure what you have learned.
These need to be designed in different ways. A formative assessment doesn’t necessarily need to give consistent results between cohorts because you just want to use it to guide teaching of this cohort. A summative assessment does because people using them for things like hiring decisions expect the scores to convey information without knowing about the specific year group.
The people in the article are cheating on formative assessments, but the motivation to do this comes from the fact that they are being used as summative assessments .
In a well designed course, the outcome of summative assessments should not be a surprise. Students should know how well they’re doing and the summative assessments should reflect their level.
The most important aspect for these is understanding what they’re measuring and where the discrimination is. If you’re designing a deselection test, you want to have high discrimination in the low end. You want high confidence that someone is, say, in the bottom 40%, but the error margins for people the test puts in the top 20% may be very high. Conversely, a selection test needs high discrimination at the top: you want it to identify the best n people, you don’t care if it incorrectly orders the bottom.
If employers are going to use exam results in one of these ways, they need to know how the exam was designed and what these things actually mean. In my experience, most university exams are not designed considering any of this and so are likely to be inappropriate for most uses employers want to put them to.