 The building and its Derby Square site maintain historical associations with Salem's prominent 18th and 19th century Derby family for whom Derby Square, Derby Wharf, Derby Street and the two Derby houses on the Salem waterfront were named.The building contains elements attributed to both Charles Bulfinch, the most influential Boston architect of the Federal period, and Samuel McIntire, Salem's renowned architect and woodcarver.The structure was saved from demolition by Salem preservation architect Philip Horton Smith in the 1930s, and underwent a partial restoration in the 1970s.