Minecraft wiki entry for Crossbow, associated text: Arrows in the off-hand, or the selected hotbar slot if the crossbow is in the off-hand, are prioritized first.
If there are no arrows in the off-hand or selected hotbar slot, the arrow closest to slot 0 is selected.