 Prior to the beginning of the Civil War, he established his own cigar-making business.Though his financial circumstances were modest, Cailloux became recognized as a leader within the community of free people of color in New Orleans.An avid sportsman, Cailloux was admired as one of the best boxers in the city.