 Miller was born in Quincy, Norfolk County, Massachusetts, on May 23, 1933, son of Ellis Warren Appleton Miller (Concord, Middlesex County, Massachusetts, February 7, 1898 – May 1986), a bookkeeper and a salesman, and descendant of several Mayflower passengers, and wife (Cambridge, Middlesex County, Massachusetts, September 15, 1923) Sophia "Sophie" June Squarebriggs (Prince Edward Island, 10 June 1899 – Quincy, Norfolk County, Massachusetts, March 20, 1998), a Canadian-born former governess.Miller's ancestry is rooted in the British North American colonial elite, and descends over time from Henry I of England, Louis IV of France, and William I of Scotland.Miller attended Cornell University School of Hotel Administration and graduated in 1955, with a Bachelor of Science in Hotel Administration.