Skip to content

FORUM: The myth of experience in refugee status determination

September 3, 2009


By Michael Kagan

I had a debate with a refugee status decision-maker recently. He told me, “I’ve been doing this for 20 years, so I have a good sense of when an applicant isn’t being credible.”


Credibility assessment is often the most critical part of deciding the fate of asylum-seekers. We know that some asylum-seekers lie in their applications, but we also know from research that honest refugees often appear incoherent, vague and inconsistent. Refugee credibility assessment is very tough, and if we get a case wrong someone could be thrown in prison, tortured or worse. But if cheaters get through too often the public loses confidence in the asylum system, and all refugees are threatened.

We’ve been taught to assume that we get better at most professional tasks as we get more experience. It sure would be nice if one could get better at credibility assessment over time. But in fact there may be little reason to think that we can. A person with 25 years of experience might be no better at judging the credibility of asylum-seekers than someone with 25 weeks of experience. The reason is that inductive learning, the means by which we get better from experience, doesn’t work in refugee status determination.

Here’s what I mean. I started playing tennis a few months ago, and I’m getting better with experience (though I’m still not very good). Experience helps in tennis because every time I hit the ball, I can see if it goes in, and then I can adjust my stroke. That’s inductive learning. From each individual case, I get better at a general skill.

But what if somehow I could not see what happens to the ball after I hit it? Would I be able to bet better at tennis? I would be able to get more comfortable hitting the ball with my racket. But I might actually be training myself to hit the ball too long or into the net without realizing it. Now, if I had a coach who told me “good” or “bad” each time, this would help – but if my coach gave me bad feedback because he also could not see where my balls landed, his advice might not make me better in the end. Instead, I might gradually grow more confident in my skills even if my actual performance was still poor.

Refugee status determination is like this. When an adjudicator decides, “I think that applicant was credible,” or not credible, she never gets to find out if the applicant was really being honest. We almost never have reliable independent evidence about why a person sought asylum. Over time the adjudicator will get more accustomed to interviewing asylum-seekers, will develop general impressions about how they usually talk, and may start to refine her assumptions about what a credible refugee sounds like. The adjudicator might also be told by a supervisor, a colleague or an appeals court “I did’t think that guy was believable,” or “you got that decision right.” The problem is that these professional peers also have no way of knowing for sure if an applicant is honest.

Experience can help people become better at interviewing asylum-seekers, and it can definitely make someone better at predicting how colleagues, supervisors and courts would react to similar case. But it cannot make people actually better at making the right decision. This means that experience is probably more helpful for lawyers than for judges. Lawyers need to know what will influence the judge, which comes with experience. But the judge is supposed to try to make the right decision, and that is much harder.

For the record, I think it was Geoffrey Care, the former president of the International Association of Refugee Law Judges, who said a few years ago that experience is not necessarily helpful in this field. But the problem goes beyond learning to make more accurate decisions. Experience can actually be harmful, if one factors in the impact of burnout and the common emotional defense mechanisms that people who work in traumatic fields often employ.

In refugee cases – with a risk of sending someone to persecution on one side and a danger of undermining the integrity of the asylum system on the other – it is natural that people will want to feel more confident in their decisions, and it may offend some to suggest that long experience does not necessarily make them better at their jobs. In the end, a good refugee adjudicator is someone very comfortable with doubt.

So, is it possible to learn to play tennis blind? Theoretically, yes – but you would have to use deduction instead of induction. Since you can’t see the result of hitting the ball in a particular way, you would have to learn to compute velocity, angles and so one to compute the best way to hit the ball. You have to trust abstract logic over concrete experience, which is probably counter to the way the human mind is wired.

To get better at credibility assessment in refugee cases, we have to resist our natural urge to trust anecdotal but ultimately subjective experience. We should strive to be more logical in our analysis, more rigorous about using empirical evidence, and less reliant on impressions and assumptions.

We also have to recognize that this is a balancing act. In tennis, we at least know the height of the net and the dimensions of the court. But in refugee status determination the lines are contested. How lenient should we be to make sure that genuine refugees get protection (even if they are not good at telling their stories)? How strict should we be to make sure non-refugees don’t cheat the asylum system?

These are abstract questions about how much doubt the refugee system can tolerate. But we have to accept that the doubt will not go away with time.

Comments are closed.

%d bloggers like this: