many a man without wealth or the right family might ship himself to the colonies of jamaica or antigua , but plantations there were built on the backs of slaves , and i could not be a part of that vileness .