When A/B testing gets an F  Jonathan Grudin  A relationship is
like a shark, it has to constantly move forward or it dies. And
I think what we got on our hands is a dead shark. —Woody Allen,
Annie Hall  Like sharks in search of their next meal, living
websites constantly move forward. How do they decide where to
go? Many popular sites rely on A/B testing. Different versions
of a feature, layout design, or advertisement are presented to
thousands of users. What people try, how long they remain, and
whether they click through are logged. Although called A/B
testing, more than two alternatives can be compared.  This is a
modern variant of a familiar controlled usability experiment. In
the 1980s, my first usability studies required weeks to recruit
10 or 20 participants and get them into the lab. Today, it can
take just minutes to identify preferences with statistical
reliability. The website is the laboratory and the participants
are unpaid users. The winning design can be rolled out to
everyone. Refinement and