as AIs get more and more general, more like agents capable of autonomously solving new problems rather than unusually clever-looking ordinary computer programs, we should expect them to look more and more like expected utility maximizers, optimizing the universe with respect to some internal value criterion.