After spending this past week with my family  the most amount of time I have been able to spend with them since March 2015  it is clear they need to be my top priority right now and this is not the right time to start a new job as demanding as White House communications director," Miller said, adding that he and his wife are expecting their second child in January.