October 12, 1492 was the day that a sailor on the Pinta sighted land. It wasn't until 1792 that we have a record of the United States celebrating Columbus Day. The celebration was organized by the Society of St. Tammany, also known as the Columbian Order, to commemorate the 300th anniversary of Columbus' landing.
According to http://memory.loc.gov/ammem/today/oct12.html :
The 400th anniversary of the event inspired the first official Columbus Day holiday in the United States. President Benjamin Harrison issued a proclamation in 1892, “recommending to the people the observance in all their localities of the 400th anniversary of the discovery of America…” and describing Columbus as “the pioneer of progress and enlightenment.” Since then, school programs, plays, and community festivities have been organized across the country in celebration of Columbus Day.
Today, Columbus Day is a federal holiday observed on the second Monday of October.