Sunday, July 1, 2012

social preferences towards the experimenter

Periodically I run into the objection towards interpretations/designs of social preferences experiments that the experimenter is ignored as a player. That is, taking money from the experimenter is transferring it from taxpayers to the participants, and that might be something participants care about, in addition to transfers between participants.

Frankly I don't really care about that possibility. It certainly is something that warrants careful study, but I find it very unlikely and therefore not so interesting, personally. (And, it's only a substantial criticism for certain experiments, not usually the kind I think about.)

It occurs to me that that's probably because I view social preferences as largely driven by norms and image. We do things because we're expected to do things. We share money in laboratory dictator games because it's implied that we're expected to and sharing norms say we should and we don't want to look bad; we don't give money to random strangers on the subway because there is no such expectation or norm. If that's the case, it's pretty safe to ignore the experimenter as a player, because almost no experiment ever even hints at the experimenter's money as something the participants should care about. And, the mere fact that they're running the experiment tells participants that they want to use that money to see how people use it within a game among themselves. The expectation is, if anything, against having social preferences towards the experimenter.


Steven E Landsburg said...

So is it your expectation, then, that if the experimenter has a heart attack midway through the experiment, that the subjects will do nothing to assist her? After all, almost no experiment ever even hints at the experimenter's life as something the participants should care about.

Vera L. te Velde said...

Of course not; there are well established norms and expectations with regard to what to do when someone, anyone, is having a heart attack. That's not the case when it comes to how to handle the experimenter's budget. Subjects have been hired to play a game, so there's not much more of a norm of returning payments than there is when anyone is hired by anyone else to do something. How many times have you heard of someone returning payment, even if, for example, the employer is a non-profit charity, or if chance events made the payment seem unfairly generous in retrospect? Of course it's more complicated when the task itself is to share with others; nonetheless the point remains that there is not any kind of clear norm that suggests that returning money is the right thing to do and the fact that the experiment is being run at all suggests that the norm/expectation is in the opposite direction.

I'm sure some experimental design could incite subjects to transfer money from subjects back to experimenter, and the strength of this effect is ultimately an empirical question that can and should be studied; but I think there are good reasons to expect that it's not very important in most cases (and plenty of indirect evidence from experiments in which opportunities for such transfers are not taken.)

Steven E Landsburg said...

(Sorry, I attempted to post something like this earlier and am not sure it got through. Feel free to delete this if it's a duplicate.)

But when we do experiments, if we're going to make a priori assumptions about what the subjects do and don't care about, then according to your prescription, we have to rely on "well established norms and expectations". And how are we to know what those "well established norms and expectations" are? As far as I can tell, the only guidance you're offering is to "ask Vera". That doesn't seem like a very satisfactory basis for science.

My own expectation is that a) students would in fact intervene in the case of a heart attack by the experimenter; b) we can therefore infer that whatever the students are maximizing, the experimenter's utility is likely to enter as an argument; c) we can therefore infer that anything bad that happens to the experimenter will matter to the subjects, though perhaps only slightly, and d) the extent to which it matters is an empirical question, not one we should make up our minds about in advance.

Vera L. te Velde said...

Well, replace "ask Vera" with "use common sense and verify common sense as rigorously as possible" but basically yes :)

Seriously though, I very much doubt that we disagree about proper scientific methods. We just have different priors, interpret the different results we've seen to point towards slightly different models of the world, or at least have slightly different understanding of norms. We can certainly argue about those differences (maybe I falsely interpret or am not aware of crucial experiments that speak to this question, or maybe you're making a very large leap from "calling 9-11 during a heart attack" to "conserving the research budget of a stranger who is paying you to take part in his experiment"), but the only definitive answer will come from empirical evidence. I just don't expect it will play out in the way you claim, at least not in a magnitude of effect that is very often non-negligible.

In the meantime, it's still not completely pointless to argue, because no experiment is perfectly able to isolate single explanations as the cause of the observations, so if I can convince you that social prefs towards the experimenter is a relatively unlikely explanation, by appealing to a model that has broad and strong support from other sources (and asserting some things about actual norms that I derive from intuition... this is perhaps where the discussion breaks down, I admit), we can collectively move forward in our understanding of human behavior by placing a higher weight on the alternate explanations.

For example, if someone ran a dictator game using chocolate, and many dictators shared some chocolate with their partner, I'm very happy to ignore the possibility that those dictators just don't like chocolate. Sure, it may be true for a couple of them, but I don't need to run a separate experiment to verify that the size of that effect is dwarfed by other standard social preferences explanations. Likewise, if someone runs a dictator game in which for every dollar the dictator sacrifices, the recipient gets two dollars, if I see most dictators being selfish, I am happy to attribute that to selfishness rather than altruism towards the experimenter. Selfishness is overwhelmingly the likely explanation, and it would be very very hard to run an equivalent experiment that didn't have that particular confound.

In other experiments the alternate explanation might not be so much more likely, and in that case it's much more important to run separate identifying experiments to distinguish them (even if it's very hard to do so without introducing other problems; many independent imperfect tests that point in the same direction can be combined into a convincing argument). I would say that's the case in the paper you blogged about recently. But regardless, since experiments are never perfect, there's always an element of judgment that takes place. I don't think that invalidates experimental methods, and I think it's fair to argue about our different priors that cause those judgments to diverge on occasion.

Steven E Landsburg said...

I think you're probably right that our disagreements are mostly about priors, not about proper procedure, and that they can largely be resolved empirically.

That said, your prior still strikes me as exceptionally odd. Your expectation seems to be that subjects care more about other subjects who they've never met than about experimenters they've never met, which seems completely arbitrary --- like assuming that people will care much more about cab drivers they've never met than about mechanics they've never met. You're entitled to your priors, but when they're this arbitrary and (seemingly) unmotivated, I think it would be rash to put much weight on them.