 Fort Nassau was built on what is now called Westerlo Island and was formerly called Castle Island.The island was part of the town of Bethlehem until 1926 when it was annexed to the city of Albany.It has been part of the Port of Albany-Rensselaer since 1932.