The paperclip maximizers

Or, the calls are coming from inside the house.

The paperclip maximizer

The paperclip maximizer, probably first described by Nick Bostrom, is

a superintelligence whose sole goal is something completely arbitrary, such as to manufacture as many paperclips as possible, and who would resist with all its might any attempt to alter this goal. […] with the consequence that it starts transforming first all of earth and then increasing portions of space into paperclip manufacturing facilities.

It is often used as a parable about the dangers of AI research, and particularly of creating AIs which are smarter than us.

But it’s obviously a fairly silly idea: how could anything which is really intelligent be dedicated to such a meaningless, useless goal, to a goal which, if pursued relentlessly and to the exclusion of all else, will certainly result in its own extinction?

A sufficiency of paperclips

The production of paperclips is not, in fact, a meaningless goal: paperclips are quite useful. The problem happens when the the production of paperclips in numbers greater than could ever be useful becomes the only goal.

But hold on. The production of wealth is not, in fact, a meaningless goal: wealth is quite useful. The problem happens when the production of wealth in amounts greater than could ever be useful becomes the only goal.

How rich do you have to be before ‘being richer’ becomes meaningless? I don’t know, but there is, quite clearly, a level at which this happens. And a large number of extremely wealthy people are far beyond this level. And yet they strive unceasingly to accumulate more paperclipsmoney, even though doing so has long lost any meaning for them or for anyone, and even though doing so is having catastrophic consequences for the future of us all.

Even more absurdly, we are all told by apparently well-educated and quite respectable people that endless economic growth is the cure for all our ills, even though endless economic growth means that resource requirements must grow exponentially with time, which is not physically possible. The pursuit of endless economic growth is merely the pursuit of paperclips in fancy dress: it will lead only to catastrophe.

Do you want to know what living in a world of uncontrolled paperclip maximizers looks like? Look around: the paperclip maximizers are us.