 From 3 June 1803 to March 1804 there were numerous votes in which Bruce did not line up with Pitt's ministry, and he continued this record into the brief Addington ministry.However Bruce did support the Tory Irish Volunteer bill on 16 April 1804.Thereafter he returned to the Pittite loyalties opposing Melville's Censure motion on 18 April 1805.