The Utopia of the Machines

What would a society and economy look like if it was comprised, not of flesh and blood humans, but of disembodied emulations of human minds, some occupying robots of all speeds, shapes and sizes, others completely disembodied, running in simulations of virtual reality in city-size cloud computing facilities? This is the premise of a sustained exercise in futurology by the economist Robin Hanson, in his recently published book “The Age of Em”.

This vision is underpinned by Hanson’s confidence that economic growth is destined to accelerate, driven by technological progress in computer power and nanotechnology, together with his transhumanist conviction that technology will bring about irreversible and far-reaching changes in the human condition.

But his vision, radical though it may seem, is tempered by conservatism in two respects. Unlike many transhumanists and singularitarians, he is deeply sceptical about the possibilities of creating artificial general intelligence. This is interesting, given that Hanson’s technical expertise, before becoming an academic economist, was in the field of AI. Secondly, he is remarkably confident about the applicability of his current understanding of social science to the dramatically changed circumstances of his vision of the future, which implies a degree of constancy of human nature even in the face of dramatic changes in its material circumstances.

While Hanson may be sceptical about the possibility of hand-coded artificial general intelligence, he is not sceptical enough about the idea of mind uploading. I’ve described at length why I think, with some confidence, that it will not be possible any time soon to simulate the operation of a human brain with enough fidelity to constitute a meaningful emulation of the mind (in my e-book Against Transhumanism, v1.0, PDF 650 kB – the most relevant chapter of which appeared on this blog as “Your mind will not be uploaded”). Rather than summarising a long argument I’ve made elsewhere, here I’ll just pick out a few key points.

The first is to stress that the basic unit of computation of the brain is not the neuron, or even the synapse, it is the molecule. This means that Ray Kurzweil style back-of-the-envelope comparisons of the numbers of neurons in brains with the future numbers of transistors in microprocessors, as extrapolated from Moore’s Law, are wrong by multiple orders of magnitude.

The second concerns the question of the correct level of coarse-graining at which it is sufficient to simulate the brain’s operation. To faithfully simulate the operation of a microprocessor, one doesn’t need to worry about what its individual atoms and electrons are doing, because there is a clean separation of the underlying solid state physics from the operation of the higher level components of the circuits, the transistors. It is this separation of levels that allows us to model the operations of the circuit at a level of digital abstraction, in terms of ones and zeros and the operation of logic gates. This doesn’t happen by accident; it is a product of how we design integrated circuits. The brain, however, is not the product of design, it is the product of evolution, and for this reason we can’t expect there to be such a digital abstraction layer.

A final point that is worth stressing arises from Hanson’s description of his “ems” – mind emulations – as fully formed individual consciousnesses capable of learning and changing. This means that the process of “uploading” a consciousness from a flesh and blood brain to a digital simulation needs to involve not just creating a snapshot of the brain in molecular detail at the time of “uploading”, difficult enough though that is to envisage. Because in the operations of the brain, there are no firm distinctions between hardware and software – the processes of learning and development involve physical changes at both the molecular and physiological levels. So constructing our our emulation would not just need a map of the connectivity of neurons and synapses and details of their molecular configurations at the moment of “upload”; it would need to incorporate a molecularly accurate model of brain development and plasticity, a task on an even greater scale.

The other strong claim of Hanson’s book concerns the predictive power of current social science. His argument is that our understanding of human nature and the operations of human societies – based largely on economics and evolutionary psychology – is now sufficiently robust that, even given the radical changes implied by human minds becoming unshackled from their fleshly bodies, meaningful predictions can be made about the character of the resulting post-human societies. I don’t find this enormously convincing.

One issue is that Hanson often is simply unable to make firm predictions; this is commendably even-handed, but somewhat undermines his broader argument. For example, he asks whether “ems” will be more or less religious than fleshly humans. It depends, it would seem, on how much importance em society attaches to innovation. “So if the innovation effect is important enough, ems will be less religious; otherwise, they’ll be more religious.” I imagine he’s not able to rule out the possibility that their degree of religiosity remains about the same, either.

One argument that Hanson makes considerable play of is a dichotomy in value systems associated with forager communities and farmer communities. He argues that modern societies have moved away from the communitarian values of farming societies back towards the more individualistic values that he believes characterised forager societies. On this basis, having argued that, for many ems, farmer-like values will once again be more favoured, he predicts that these ems will tend to prioritise self-sacrifice, patriotism and hard-work.

This general line of argument has a long pedigree, essentially following the Marxist principle that it is a society’s mode of production which determines the superstructure of its institutions and values, with a more recent gloss from evolutionary psychology. The specific farmer/forager dichotomy will seem problematic to many on empirical grounds, though. How do we know what forager values actually were? Very few forager societies survived in any form into historical times, that handful that did may have been influenced by surrounding farmer communities, and what we know about their values is mediated by the biases of the anthropologists and ethnographers that recorded them. Most of what we know about foragers and hunter-gatherers necessarily comes from archaeology, which unavoidably deals in the material remains of vanished cultures. The archaeological study of prehistoric mentalities is fascinating and current, but methodologically difficult. The early tendency was to argue on the basis of analogies with historical forager communities, now recognised to be problematic for the reasons we’ve just seen, while the nature of what remains to be studied naturally and inevitably biases archaeologists towards materialist explanations.

Even if one accepts a correlation between a society’s mode of production and the character of its predominant social institutions and values, it’s not at all clear in which direction causality runs. There’s a fashionable (and to me pretty convincing) line of argument from economists like Daron Acemoglu that the quality of a society’s institutions is a prime determinant of their economic success. Meanwhile a dominant strain of thinking about the origins of the historical transition to an industrial economy puts ideals and values ahead of materialist explanations such as the availability of fossil fuels. In the latter argument I’m personally much more in the materialist camp, but I find it difficult to reject the idea that the economic base of a society and its values and institutions must co-develop, rather than one simply being determined by the other.

If the empirical underpinnings of the forager/farmer polarity are dubious, its applicability to Hanson’s hypothetical future seems even more difficult to justify. The question that has to arise here is why one should believe that the opposition is strictly binary. There’ve been many different ways in which economies have been organised in the past – the slave economies of antiquity, feudal systems, nomadic pastoralism, capitalist industrial societies, state socialist economies, and so on – and it’s easy to argue that each has been accompanied by its own particular package of institutions and values. Given the massive scale of change Hanson is anticipating in his post-human economy, it’s difficult to see why we shouldn’t expect the emergence an entirely new package of values, which to us would probably seem very alien, rather than a reversion to a set of values supposed to be appropriate to some previous historical state.

So how should one read “The Age of Em” – what genre of writing should it be ascribed to? In my opinion it doesn’t succeed as a straight work of non-fiction; the technical underpinnings of its premise are not credible, and the social science bases of its speculations, interesting though they are, are not, to my mind, robust enough to sustain the weight of argument erected on them. On the other hand, it is clearly not by itself science fiction. It’s certainly an impressive exercise in world-building, which, with the addition of plot and character, would have the potential to make a spectacular series of novels.

But it occurs to me that the book might best be thought of as a Utopia, in the sense of Thomas More’s original. Stylistically, one can see the relationship, in the travelogue-like tone of the writing, dispassionate but not at all unsympathetic to the inhabitants of the strange world he’s describing. And there’s an ambiguity about what a reader might take to be the purpose of the exercise. What is described is a world which to some readers, perhaps, might seem admirable and enviable. It’s a world in which the vicissitudes and distractions of the flesh are absent, and as described by Hanson it’s a competitive world, meritocratic on the basis of pure intellect and character. Since the basic social unit consists of multiple emulations of a successful individual, readers who identify themselves with one of the “uploads” can imagine themselves surrounded by people just like them.

Or perhaps we should read it, as some have read More’s Utopia, as a satire on current society. What, we might ask, would a description of an economy completely decoupled from the needs and desires of flesh-and-blood human beings tell us about our world today?

3 thoughts on “The Utopia of the Machines”

  1. > To faithfully simulate the operation of a microprocessor,
    > one doesn’t need to worry about what its individual atoms
    > and electrons are doing, because there is a clean separation
    > of the underlying solid state physics from the operation of
    > the higher level components of the circuits, the transistors.
    > It is this separation of levels that allows us to model the
    > operations of the circuit at a level of digital abstraction,
    > in terms of ones and zeros and the operation of logic gates.
    > This doesn’t happen by accident; it is a product of how we
    > design integrated circuits.

    Indeed, and since the turn of the century, off-the-shelf microprocessors
    have been powerful enough to simulate older computers, at least at
    the instruction level, making it possible to “resurrect” older
    operating systems such as CTSS and Multics and giving retrocomputing
    enthusiasts plenty of entertainment.

    However, depending on the level of accuracy one wishes to achieve in
    emulation, it’s astonishing how much computer power can be soaked
    up replicating even simple devices at an extreme level of accuracy.

    To wit:
    How can it take 3GHz to emulate a Super Nintendo?

    . . .

    Honestly, even with all of the issues listed above, we’ve only
    the digital integrated circuit emulator. Here is an emulator that
    works at the transistor level for absolutely perfect recreation of
    full speed. It’s not that DICE is a slow program; indeed, it is
    very well optimized. It’s that there is enormous overhead to
    simulating every last transistor propagation delay. . .

  2. I like Robin’s book but disagree on one key assumption. As you say, Robin thinks that mind uploading is likely to be developed much before sentient AI. I think the two are likely to develop at comparable paces with strong feedback loops, with advances in one stimulating advances in the other (or roadblocks in one creating roadblocks in the other) and reach operational maturity at more or less the same time near the end of the century, give or take a couple of decades. Robin’s scenario is believable if uploading comes soon and sentient AI never follows, but I think his assumption is wrong. In my favorite scenario, human uploads and AIs co-evolve.

  3. Is exact replication really necessary to Hanson’s vision? Exact mind uploading seems very far away to me as well, but I am less convinced about good enough approximations. I am somewhat sceptical that there is such a large difference between designed and evolved systems: does current biology really justify such division? I would imagine that reliability is somewhat required for an evolved system to actually be successful. Isn’t all life described as replicators?
    Hanson’s vision really only requires working digital minds, not exact copies. Once they are virtualized, ems should be able to evolve themselves in the virtual space as well. Would they behave exactly as the original flesh and blood individual? I don’t think it really matters that much. When you have a model (even if coarse and simplified), what stops you tinkering with starting conditions until you get something that works good enough?
    Current AI research seems to follow a similar pattern to a certain extent (or at least some researchers do). I agree with the previous commenter that the division between AI and mind emulation seems wrong to me.
    It seems to me that that our brains are incredibly flexible: people with major trauma can still function even if their behaviour does change. That seems to suggest to me that there is an underlying strength in the design. If we can copy it, we might get working minds even if they are not exact copies. I understand this is beyond current manufacturing technology, but creating an artificial synapses seems at least possible to me. Some sort of 3-d printing might allow to copy an existing brain. Such a hardware solution could simplify the computational requirements massively. If a person can survive after brain trauma, why wouldn’t an imperfect copy not work at all? We just need one such copy to work, or even a small fraction.
    Would something be lost when moving from the current biological hardware to a different medium? Maybe, we cannot know but I personally doubt it.

Comments are closed.