Haroche was born in Casablanca, Morocco, to Albert Haroche (1920–1998), from a Moroccan Jewish family, and Valentine Haroche, born Roubleva (1921–1998), a teacher who was born in Odessa to a Jewish family of physicians who relocated to Morocco in the early 1920s. His family had mixed Sephardic and Ashkenazi origins. His father, a lawyer trained in Rabat, was one of seven children born to a family of teachers, Isaac and Esther Haroche, who worked at the École de l’Alliance israélite (AIU).