 He was appointed Surveyor-General of the Ordnance in early 1784, King George writing to Prime Minister William Pitt the Younger on 30 March 1784 that "Mr. James Luttrell I have always understood to be the best of that strange family, I have therefore signed the appointment of him as surveyor of the Ordnance.