Less Wrong: Ethical Injunctions

Ethical injunctions are rules not to do something even when it’s the right thing to do. (That is, you refrain “even when your brain has computed it’s the right thing to do”, but this will just seem like “the right thing to do”.)

For example, you shouldn’t rob banks even if you plan to give the money to a good cause.

This is to protect you from your own cleverness (especially taking bad black swan bets), and the corrupted hardware you’re running on.

This channel includes:

  • 8 articles written by Eliezer Yudkowsky
  • Including a list of replies by the author to questions in the comments of those articles
  • over 1.5 hours of audio read by George Thomas

Why Does Power Corrupt?

Play f114902fd0cfda50800be7fbb69752c4badad535050adaaeb6927f96d31eb7c8

  1. Why Does Power Corrupt?

    The circuitry that will respond to power by finding it pleasurable, is already wired into our young revolutionary's brain; but he does not know this. (It would not help him evolutionarily if he did know it, because then he would not be able to honestly proclaim his good intentions-though it is scarcely necessary for evolution to prevent hunter-gatherers from knowing about evolution, which is one reason we are able to know about it now.)

  2. Ends Don't Justify Means (Among Humans)

    I think you need to sit down and spell out what 'corrupt' means, and then Think Really Hard about whether those in power actually are more corrupt than those not in power;and if so, whether the mechanisms that lead to that result are a result of the peculiar evolutionary history of humans, or of general game-theoretic / evolutionary mechanisms that would apply equally to competing AIs.

  3. Entangled Truths, Contagious Lies

    Not all lies are uncovered, not all liars are punished; we don't live in that righteous a universe. But not all lies are as safe as their liars believe. How many sins would become known to a Bayesian superintelligence, I wonder, if it did a (non-destructive?) nanotechnological scan of the Earth?

  4. Protected From Myself

    Followup to: The Magnitude of His Own Folly, Entangled Truths, Contagious Lies Every now and then, another one comes before me with the brilliant idea: "Let's lie!" Lie about what?-oh, various things. The expected time to Singularity, say. Lie and say it's definitely going to be earlier, because that will get more public attention.

  5. Ethical Inhibitions

    But I suggest-if a bit more tentatively than usual-that by the time human beings were evolving the emotion associated with "ethical inhibition", we were already intelligent enough to observe the existence of such things as group sanctions. We were already smart enough (I suggest) to model what the group would punish, and to fear that punishment.

  6. Ethical Injunctions

    Someone who just looks at one or two reasons behind ethics, and says, "Okay, I've understood that, so now I'll take it into account consciously, and therefore I have no more need of ethical inhibitions"-this one is behaving more like a stereotype than a real rationalist.

  7. Prices Or Bindings?

    The trick would be establishing something of equivalent strength to a Catholic priest who believes God doesn't want him to break the seal, rather than the lesser strength of a psychiatrist who outsources their tape transcriptions to Pakistan. Otherwise serial killers will, quite sensibly, use the Catholic priests instead, and get less rational advice.

  8. Ethics Notes

    When the winners do something bad, it's never interpreted as bad after the fact. Firebombing a city to end a war more quickly, taxing a populace to give health care to the less fortunate, intervening in a foreign country's affairs to stop a genocide: they're all likely to be interpreted as evidence for "the ends don't justify the means" when they fail, but glossed over or treated as common sense interventions when they work.