 After the Civil War, in which he served, Fitzgerald rejoined his family, who had moved to Hillsborough, North Carolina.There they set up a brickyard at their farm.He moved with a brother into Durham, building his business and becoming a major figure in the city's black community.