 Lee's children included Robert Lee, Lord Lee (1830-1890), Dr Thomas Masson Lee IMS and Alexander H. Lee (1828-1901).Lee was accomplished in almost every branch of knowledge, and in Scottish literary and ecclesiastical history had very detailed information.He collected a library of twenty thousand volumes.