 Moot Hall continued to be used for court hearings until the establishment of the magistrates' court system in the 19th century, when a courthouse was built in Bedford.By 1773, the Hillersden family consisted of just two sisters who, with their finances dwindling, let Moot Hall, its equipment and fair tolls to Thomas Coleman.During the 1790s, the Hillersden sisters moved out of Elstow Place into the smaller Elstow Lodge, leaving the mansion to fall into ruin, and then gradually they sold, piece by piece, the rest of their Elstow estate to Samuel Whitbread.