Superintelligence: Paths, Dangers, Strategies

Here the author engages in what one might call prophetic analysis in order to assess the paths, dangers and strategies associated with the development of superintelligence. He begins from the assertion that the human brain has given us our dominant position on the planet, so that the fate of other species depends more on us than them. He projects the possibility some time over the next century that humans will build machine brains surpassing our own general intelligence and that these could pose a threat to their creators. In principle, as he observes, we could build the kind of superintelligence that would protect human values, but an unfriendly superintelligence might prevent us from replacing it or changing its preferences and motivations. Hence the need to understand the challenge presented by this prospect. There are various paths by which this might be achieved and indeed various forms of superintelligence, which one has to say is a left hemisphere vision gone wild, based as it is on principles of manipulation and control and defining intelligence in narrow analytical and computational terms. Given the history of the human race, the author contends that the prospect is a little like a child with an undetonated bomb in its hands where 'some little idiot is bound to press the ignite button just to see what happens.' He hopes that we can maintain our groundedness and common sense in these circumstances.