 He was born in Hunsdon, Hertfordshire, the eldest son of Thomas Howard, 3rd Duke of Norfolk by his second wife Elizabeth Stafford, a daughter of Edward Stafford, 3rd Duke of Buckingham.He was thus descended from King Edward I on his father's side and from King Edward III on his mother's side.He was brought-up at Windsor Castle with Henry FitzRoy, 1st Duke of Richmond and Somerset, the illegitimate son of King Henry VIII.