wisely , charles made no secret about where jason was from , but he also let it be known that his brother was not a slaveholder and preferred to remain a neutral in the conflict that swirled around him .