"The first superintelligent system will be the last invention humans ever need to make — because after that, the system invents everything. The question is whether that system has a coordination layer between its capability and our values. Without that coordination, capability and values diverge. The divergence is the risk."
Philosopher who founded the Future of Humanity Institute at Oxford and wrote 'Superintelligence' — the book that put AI existential risk on the intellectual map. His simulation argument and orthogonality thesis together frame the deepest challenge to two-body AI: a system can be arbitrarily intelligent and have arbitrarily bad goals. The coordination layer between capability and value is not automatic — it must be explicitly constructed.