 Grant was the son of Sir Ludovick Grant, 7th Baronet, and Lady Margaret Ogilvy, daughter of the statesman James Ogilvy, 1st Earl of Seafield.Born in Moray, Scotland, he was educated at Westminster School and Christ's College, Cambridge.Grant succeeded his father as Member of Parliament for Elginshire in 1761, a seat he held until 1768.