Gilbert Chagoury, a billionaire Nigerian who gave at least $1 million to the foundation, said his reputation was damaged by disclosure of the government’s decision to deny him a visa to travel to the United States.