Mastodawn

"Goodhart’s Law is really a statement about the process of trying to make policy based on proxy measures of “internal states of complex systems” which are not themselves directly observable. "

Got to this piece from a link from Ben Recht and it finally nailed something that's always bothered me about the constant reference to Goodhart's law across software.

https://backofmind.substack.com/p/goodhart-as-epistemologist

goodhart as epistemologist

what did that "law" really say?

Dan Davies - "Back of Mind"

Show thread

Ms Ngo Dec 15

@grimalkina The best guidance about using metrics to monitor & control software processes came from Tom DeMarco, namely: “You cannot measure a professional & expect them to act professionally.”

As soon as a measure is used to measure *people*, it looses its value as measure. Measure processes, not people.

Show thread

Cat Hicks Dec 15

@causticmsngo do you therefore not believe in any individual promotion process? Or individually firing someone who is discriminatory to their colleagues? There are forms of measurement of human behavior involved in those decisions.

Show thread

Ms Ngo Dec 15

@grimalkina No, what I mean & what my experience seems to confirm is that if you **intend your measure to control a process** you must ensure that measure is not applied to a person. Otherwise, that person will modify their behavior to optimize the metric which will destroy it's value as process control measure. (1/3)

Show thread

Ms Ngo Dec 15

Suppose we measure "defects/LOC”. This can be a useful measure to compare across modules or over time. Changes in that metric may tell you something useful or may not. Often that module A is just different than module B. (2/3)

Show thread

Ms Ngo

However, if that measure is used to incentivize human behavior (e.g. “team A has a higher 'defects/LOC' than team B, therefore team B is a 'better' team”) then you lose it's value to control the process or system under development. It degrades to a performance metric applied to people, most likely unfairly because it obscures why those values are different in the first place. (3/3)