 He married four times and fathered thirteen children; Esther Borden Lippincott (daughter of Richard Lippincott) in 1806, Maria Taylor in 1828, Elizabeth Eleanor Caldwell in 1835 and Mary Priscilla Coates in 1850.In 1815 he built an estate Belle Vue, including an orchard and farm named on the south side of Russell Creek that followed the style of Georgian architecture.The property on which he built his new home, lot 17 and the eastern portion of lot 18 ran north from present day Queen Street West to Bloor Street and west from Lippincott Street to Augusta Avenue.