Chapter 06

When AI Sees But Doesn’t Understand

October 21, 2018. Tempe, Arizona. A self-driving Uber, its sensors seeing everything and understanding nothing, struck and killed Elaine Herzberg as she walked her bicycle across a darkened road. The vehicle’s perception system had classified her, over the course of six long seconds, as an “unknown object,” then a “vehicle,” then a “bicycle,” then an “unknown object” again. At no point did it recognise her as a person who could die.

Perception without comprehension is not intelligence; it is a very fast kind of blindness. This chapter argues that consciousness — whatever else it is — is a coordination problem, and that the frameworks we use to build artificial minds will have to learn to hold perception, processing, and awareness as three distinct things before they can ever hold them together.

WHAT THIS CHAPTER COORDINATES FIG · CAUSAL DIAGRAM

P Perception

→

R Processing

→

A Awareness

LATENT / Comprehension — the missing third body

The Uber system had perception and processing — but not the third body: awareness grounded in consequence. Two-body systems can see. They cannot understand what they see.

CAUSAL GRAPH · THE COMPREHENSION GAP

Nine Voices — Chapter 06

Consciousness as Pattern Recognition

Stuart Russell

AI Safety, UC Berkeley

"The problem is not that we are building intelligent machines. The problem is that we are telling them exactly what we want. A machine that pursues a fixed objective does not coordinate with human values — it optimizes over them."

Academic 47

Dr. Timnit Gebru

AI Justice, DAIR

"A language model that predicts the next word is not understanding. It is coordinating with statistical patterns in text written by people who were never asked if they consented. That coordination is not neutral. It encodes who was counted and who was not."

Academic 48

Kate Crawford

AI Now Institute

"Every AI system has a body — it is just hidden. It is the mine, the server farm, the warehouse worker, the moderator. When you ask what AI coordinates with, the honest answer includes all of them."

Academic 49

Norbert Wiener

Father of Cybernetics

"The thermostat coordinates with the temperature. The dictator tries to control it. The thermostat is wiser than the dictator. This is mathematics."

Practitioner 50

Margaret Mitchell

Hugging Face Ethics

"If you don't know what a model was trained on, who it was trained for, and what it fails at — you don't know the model. You only know its output. And output without context is the definition of a stochastic parrot."

Practitioner 51

In-Q-Tel Operator

Intel community ↔ tech

"[CLASSIFICATION LEVEL: UNACKNOWLEDGED] The question we ask about every technology is not: what does it do? The question is: what does it see? Pattern recognition at scale is not intelligence. It is the precondition for coordination — or for its denial."

Practitioner 52

Sir Roger Penrose

Consciousness & Quantum

"Gödel shows that human mathematical understanding exceeds any algorithm. The question is what that something is. I believe it's quantum."

Visionary 53

Antonio Damasio

Somatic Markers

"The brain does not make decisions alone. The body is always in the room. It has already voted — long before the frontal lobe begins its deliberation. Consciousness is not the general. Consciousness is the diplomat who arrives after the negotiation."

Visionary 54

Rupert Sheldrake

Morphic Resonance

"Why does a crystal know what shape to take? Why does a flock of birds turn as one? There is a field between things — a coordination medium — that carries the memory of form. We call this field morphic. Science calls it impossible. Nature calls it Tuesday."

Visionary