 It was left vacant after this except for a year's rental to Captain John Piper in 1814.The original Vaucluse House was built by Sir Henry Brown Hayes, who had been transported to New South Wales in 1802 for kidnapping the daughter of a wealthy Irish banker.Governor King considered Hayes "a restless, troublesome character" and was keen to move him away from Sydney.