We need to confront the simple but chilling idea: human nature will be transformed in the 21st century because intelligence is able to uncouple from consciousness.
Artificial Intelligence is already here and it will increase daily to an extent that AI will make own decisions and could help to integrate individual behaviour into the overall ecosystem of the planet and on climate control.
Superintelligence is already looming around the corner and it might not only serve the desires of individual humans or humanity in total.
Scientists including Stephen Hawking and Max Tegmark believe that superintelligent machines are quite feasible. And the consequences of creating them, they say, could be either the best or the worst thing ever to happen to humanity.
Certainly, achievements in computer science over the last 75 years have been astonishing. Most obviously, machines can now execute complex mathematical operations many orders of magnitude faster than humans. They can perform a range of tasks, from playing world-beating Go to flying a plane or a car, and their capabilities are rapidly growing. The consequences – from machines stealing your job to eliminating drudgery to unravelling the enigmas of cancer to remote killing – are and will continue to be striking.
But even the most sophisticated machines created so far are intelligent in only a limited sense. They enact capabilities that humans have envisaged and programmed into them. Creativity, the ability to generate new knowledge and generalised intelligence outside specific domains seem to be beyond them.
It will be very difficult – but perhaps not impossible – to engineer a superintelligence with preferences that make it friendly to humans or able to be controlled. As the fate of the gorillas now depends more on humans than on the species itself, so could the fate of humankind depend on the actions of the machine superintelligence. But we have one advantage: we get to make the first move.
We have to balance artificial intelligence with ART- Intelligence.