A person with direct knowledge of Woolsey's decision said the former CIA chief had not been significantly involved in the Trump team's discussions on intelligence matters and became uncomfortable being labelled as an adviser.