 Like his older brother John, James embarked on a career in the navy, and after a period of service was promoted to the rank of lieutenant on 2 February 1770.While pursuing his naval career, Luttrell also took the opportunity to follow his father and older brothers into politics.His brother John had held the seat of Stockbridge with his father, but vacated it in 1775 for James.