 Byrne had a brief relationship with Toni Basil in 1981 and he dated Twyla Tharp between 1981 and 1982.While visiting Japan in 1982, Byrne met costume designer Adelle Lutz, and they married in 1987.They have a daughter, Malu Abeni Valentine Byrne, born in 1989.