Leaving Rinnoji from the west side, you come to the broad Omote-sando avenue that leads uphill to the shrine itself.
After you leave the shrine on the north road, you get past Rinnoji.
After you leave the shrine on the north road, you don't get past Rinnoji.