Minecraft wiki entry for Multiplayer, associated text: If they select "Invite Only", only Xbox Live friends that are owner invites can join them.
If they select "Friends Only", then only their friends on Xbox Live can join them.
If they select "Friends of Friends", then the owner's Xbox Live friends and their Xbox Live friends can join. This is also the default option.