Paul the Apostle Church, formerly known as St. Joseph's Catholic Church, was formed in 1889 by Slovenian immigrants who came to the Calumet area to work in the booming copper mines.The first church that they built burnt down in 1902, but the church was rebuilt in sandstone by 1908.