The probability of these events may be very low, but the expected value of preventing them could be high, as it represents the value of all future lives. We review the challenges to studying human extinction risks and, by way of example, estimate the cost

2013-03-01: AI risk

‘The problem is you are building a very powerful, very intelligent system that is your enemy, and you are putting it in a cage,’

2014-05-29: Anders Sandberg is one of my heroes, for very good reason:

If humanity becomes extinct, at the very least the loss is equivalent to the loss of all living individuals and the frustration of their goals. But the loss would probably be far greater than that. Human extinction means the loss of meaning generated by past generations, the lives of all future generations (and there could be an astronomical number of future lives) and all the value they might have been able to create. If consciousness or intelligence are lost, it might mean that value itself becomes absent from the universe. This is a huge moral reason to work hard to prevent existential threats from becoming reality. And we must not fail even once in this pursuit.

• Uncategorized