Wednesday, September 1, 2010

The Pathetic Golem

Robin Hanson, gamely considering the question of who should be brought into existence, outlines a model that's something like R.M. Hare's Golden Rule, plus economic efficiency:
Economically, creature X should exist if it wants to exist and it can pay for itself. That is, in a supply and demand world, if our only choice is whether X should exist, then an X that wants to exist should actually exist if its lifespan cost of resources used (including paying for any net externalities) is no more than the value it gives by working for others. [Similarly excerpted by Adam Ozimek on Modeled Behavior.]

In a comment on AO's post, RH says:

Surely there are many kinds of creatures where we could know with great confidence that they prefer to exist. Exact copies of other already existing creatures, for example. Can you accept that these creatures should exist?

I see a very serious problem with the move from "X creature is happy to exist" to "It is morally correct to make more creatures like X creature."

Say we make a golem out of clay, like in the old days. We bring it into existence to suffer a life of misery, as golems are want to have. But we endow it with a very special characteristic, along with life: the preference to exist. No matter what tortures we or the world inflict on our golem, it will keep on preferring to exist.

Is that moral? Can we create a Foxconn megafactory of such golems and keep them alive for miserable decade after miserable decade, with clean consciences?

The problem that I hope this raises is this: we expect preference to exist to be a function of quality of life, but it may actually be entirely independent from quality of life. People with every human advantage in the world (like me) often wish they had never been born; sick, suffering homeless people on the street often prefer to keep on living.

While I think we should respect an individual's decision as to whether it wishes to keep on living, this does not form a good guide as to whether to bring new people into existence.

The worst part: a pasted-on "preference for life" is exactly the sort of cruel trick we could expect evolution to play. What could be more beneficial? Except, perhaps, an unshakable preference to reproduce.

7 comments:

  1. I didn't mean to claim that all creatures that wish to continue to exist wish to exist. I just meant this determination could be a lot easier regarding real existing creatures.

    ReplyDelete
  2. That actually wasn't my objection, though I did smudge the distinction. The feeling of being glad to have been brought into existence is at least as potentially welfare-independent (paste-on-able) as the desire to go on living. And very fitness-promoting, though maybe not as much as clinging to life.

    ReplyDelete
  3. Even assuming that the preference to have come into existence matters (which it doesn't), preferences are not static. What if someone who now prefers to have come into existence changes her mind later? Then what? Should she have been brought into existence or not? Then we would have to wait until the person was dead to make that determination, which just doesn't strike me as very useful.

    The same problem applies to exact copies of existing creatures. We could make an exact copy of someone who is already dead and consistently preferred to have come into existence while he was alive. But unless we engineer the new person's environment and life events to be exactly the same as the old person's, we cannot be sure that the new person would retain that preference.

    ReplyDelete
  4. Our intuition about the relation between preference and utility comes largely from mundane problems. Sister Y's example explodes this relation in an extreme case. Preference is a good heuristic, usually, for mundane cases. Finding support for it beyond that is more challenging.

    ReplyDelete
  5. "The problem that I hope this raises is this: we expect preference to exist to be a function of quality of life, but it may actually be entirely independent from quality of life."

    If this is true, we're so completely and thoroughly fucked that we should spend our entire disposable time seriously thinking about how to eradicate all sentient life.

    ReplyDelete
  6. Consider, in contrast, the curious case of the Enthusiastic Golem. Like the Pathetic Golem, the Enthusiastic Golem always prefers to exist. However, unlike the Pathetic Golem, the Enthusiastic Golem is introspectively consistent, i.e. its preference to exist is rational from a welfarist perspective, and its affective mind processes conscious moment-to-moment existence as pleasurable.

    The Enthusiastic Golem is not a blissed-out apathetic. It orders the quality of experiences according to their goodness and acts accordingly. It prefers having sex over being injured, it likes pleasure better than pain. But it never feels miserable, as misery is in the mind, and the Enthusiastic Golem's mind is constructed to like every experience, however not to the same degree.

    Which is more plausible as a creature to populate the future - the Enthusiastic Golem or the Pathetic Golem? If it is the outcome of people's choices, and assuming that people maintain at least a baseline of benevolence bias, however weak, the Enthusiastic Golem will win more public favor than the Pathetic Golem. Furthermore, the Enthusiastic Golem is introspectively consistent, while the Pathetic Golem requires cognitive preferences that contradict its affective moment-to-moment emotions. Assuming that consistent minds are easier to build, to maintain, and/or more behaviorally functional, the Enthusiastic Golem is a more likely creature to find in the future than the Pathetic Golem.

    Unless the Pathetic Golem is intrinsically more adaptive. Does adaptive behavior strongly require subjective misery?

    ReplyDelete
  7. This post has very lucidly illustrated to me the problem of going from "creature X is happy to exist" to "It is morally correct to create more creatures like creature X" in a way that wasn't clear to me before.

    When I have made the move in the past, I assumed that the preference to exist was indicative of a good quality of life, for lack of a better term, at least for the person with the aforementioned preference. I see now that I may have been too rash in that assumption.

    I think the above commenter missed the intent of this post. The intent was to show that the preference to exist may not necessarily be indicative of a high quality of existence. It wasn't to compare or contrast imaginary scenarios.

    ReplyDelete

Tweets by @TheViewFromHell