The argument that I often hear is “if your personal enjoyment comes at the price of other people’s hurt feelings, it’s oppressive and immoral”, but that only applies when I actually force people to consume the product of my imagination.