Campbell’s Law: Why your metric will be gamed

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

–Donald Campbell, 1979

Campbell, originally an experimental psychologist and trained in experimental method as was customary in his field, soon realized that true experiments could not be done in any of the social sciences because no one would let social scientists treat human beings the way laboratory scientists treated rats and other experimental animals. You couldn’t manipulate people that way because they were free enough to reinterpret the conditions of any experiment and because the institutions where experiments were done were sensitive to the public relations, if not always the moral, issues involved.

[…]

An experimenter might choose a condition for the social program to be tested, but the subjects of the experiments–organizations and the people responsible for them–inevitably and quickly understood how the numbers their actions piled up could be used in ways that might help or hurt their interests. And so, just as routinely, did their best to make sure that the numbers came out the way that gave the best outcome from them, manipulating them in ways their organizational positions and knowledge made available to them. Who knew better how to to that? And that’s been a robust finding. It’s what people organizations do, if they can (and usually they can).

–Howard S. Becker, Writing for Social Scientists (find in a library)

Campbell’s Law: Why your metric will be gamed