 It is believed that the abandoned castle collapsed in the 17th century after the seat of the starost was moved to Panevėžys.The remnants of the castle survived into the 18th century.Upytė was a capital of the Upytė region () in the Grand Duchy of Lithuania.