 O'Callahan grew up in Brookline, Massachusetts, near the many hospitals of the Boston Longwood Medical Area, in a neighborhood which he has fictionalized in his stories as Pill Hill.He is the son of Edward J. and Helen Gately O'Callahan, two teachers who founded the Wyndham Secretarial and Finishing School on Marlborough Street in Boston's Back Bay, a women's secretarial college that sought to teach not only clerical skills but a broader liberal education.In the mid-1940s, his parents bought a 32-room mansion in Pill Hill built in 1883 for Charles Storrow and landscaped by Frederick Law Olmsted.