The direction of the kick is given by the polarization of the light, which is either left or right handed circular, giving different interactions with the different m f levels. The correct polarizations are used so that photons moving towards the center of the trap will be on resonance with the correct shifted atomic energy level, always driving the atom towards the center.