The Other Quirks of Perception
Neuroscientist Kathleen McDermott of Washington University began by quoting famous memory researcher Endel Tulving, who called our ability to remember the past and to anticipate the future “mental time travel.” You don’t use the phrase “time travel” lightly in front of a group of physicists for whom the concept is not a convenient metaphor but a very real possibility. But when you hear about how our minds glide through time—and how our memory provides a link not only to the past but also to the future—you see Tulving’s point.
To investigate the perception of past and future in people without brain injuries, McDermott did fMRI brain scans of 21 college students, asking them to recall a specific incident in their past and then envision themselves in a specific future scenario. Subjectively, the two feel very different. Yet the scans showed the same patterns of activity. Areas scattered all over the brain lit up; our temporal perception is distributed. As a control, McDermott also asked the students to remember events involving Bill Clinton (presumably, ones they were not personally involved in), and the patterns were very different. In a follow-up study, McDermott asked 27 students to anticipate an event in both a familiar and an unfamiliar place. The brain scan for the familiar one resembled the one for the act of remembering; the unfamiliar one was the odd man out.
The bottom line is that memory is essential to constructing scenarios for ourselves in the future. Anecdotal evidence backs this up. Our ability to project forward and to recollect the past both develop around age 5, and people who are good at remembering also report having vivid thoughts about the future.
McDermott’s colleague Henry Roediger studies metacognition—thinking about thinking. Roediger pointed out that false negatives get less attention, but are equally bad. The worst eyewitnesses are full of passionate intensity, and the best lack all conviction. In both cases, innocent people can be sent to death row while the guilty walk.
Cognitive psychologists find that confidence sometimes correlates with accuracy, sometimes not. Roediger gave volunteers a memory word test. They had to study a list of words; afterwards, they were presented with a series of words and had to indicate whether each had been on the original list. They also had to say how confident they felt about their answer.
It’s not that our memory is a glitchy wetware version of computer flash memory; it’s that the computer metaphor just doesn’t apply. Roediger said we store only bits and pieces of what happened—a smattering of impressions we weave together into feels like a seamless narrative. When we retrieve a memory, we also rewrite it, so that the time next we go to remember it, we don’t retrieve the original memory but the last one we recollected. So, each time we tell a story, we embellish it, while remaining genuinely convinced of the veracity of our memories.
The fun thing about neuroscience is that you can do the experiments on yourself. David Eagleman of the Baylor College of Medicine proceeded to treat us as his test subjects. By means of several visual illusions, he demonstrated that we are all living in the past: Our consciousness lags 80 milliseconds behind actual events. “When you think an event occurs it has already happened,” Eagleman said.
In one of these illusions, the flash-lag effect, a light flashes when an object moves past it, but we don’t see the two as coincident; there appears to be a slight offset between them. By varying the parameters of the experiment, Eagleman showed that this occurs because the brain tries to reconstruct events retroactively and occasionally gets it wrong. The reason, he suggested, is that our brains seek to create a cohesive picture of the world from stimuli that arrive at a range of times. If you touch your toe and nose at the same time, you feel them at the same time, even though the signal from your nose reaches your brain first. You hear and see a hand clap at the same time, even though auditory processing is faster than visual processing. Our brains also paper over gaps in information, such as eyeblinks. “Your consciousness goes through all the trouble to synchronize things,” Eagleman said. But that means the slowest signal sets the pace.
The cost of hiding the logistical details of perception is that we are always a beat behind. The brain must strike a balance. Cognitive psychologist Alex Holcombe at Sydney has some clever demonstrations showing that certain forms of motion perception take a second or longer to register, and our brains clearly can’t wait that long. Our view of the world takes shape as we watch it.
The 80-millisecond rule plays all sorts of perceptual tricks on us. As long as a hand-clapper is less than 30 meters away, you hear and see the clap happen together. But beyond this distance, the sound arrives more than 80 milliseconds later than the light, and the brain no longer matches sight and sound. What is weird is that the transition is abrupt: by taking a single step away from you, the hand-clapper goes from in sync to out of sync. Similarly, as long as a TV or film soundtrack is synchronized within 80 milliseconds, you won’t notice any lag, but if the delay gets any longer, the two abruptly and maddeningly become disjointed. Events that take place faster than 80 milliseconds fly under the radar of consciousness. A batter swings at a ball before being aware that the pitcher has even throw it.
The cohesiveness of consciousness is essential to our judgments about cause and effect—and, therefore, to our sense of self. In one particularly sneaky experiment, Eagleman and his team asked volunteers to press a button to make a light blink—with a slight delay. After 10 or so presses, people cottoned onto the delay and began to see the blink happen as soon as they pressed the button. Then the experimenters reduced the delay, and people reported that the blink happened before they pressed the button.
Eagleman conjectured that such causal reversals would explain schizophrenia. All of us have an internal monologue, which we safely attribute to ourselves; if we didn’t, we might think of it as an external voice. So Eagleman has begun to run the same button-blink experiment on people diagnosed with schizophrenia. He reported that changing the delay time did not cause them to change their assessment of cause and effect. “They just don’t adjust,” Eagleman said. “They don’t see the illusion. They’re temporally inflexible.” He ventured: “Maybe schizophrenia is fundamentally a disorder of time perception.” If so, it suggests new therapies to cajole the brains of schizophrenic patients into recalibrating their sense of timing.
In the experiment for which Eagleman is best known, he sought to find out why time passes more slowly when we’re scared. Does something really happen in the brain—for instance, the time resolution of perception speeds up—or do we just think it does, in hindsight? After brainstorming scare tactics that probably wouldn’t have passed muster with a university ethics committee, he hit upon asking volunteers to take one of those Freefall or Demon Drop rides you find in amusement parks. They wore a special watch whose digits counted up too quickly for people to register them under normal conditions—thinking that, if perception really did speed up, people would be able to read the digits.
Alas, they couldn’t. Although they consistently reported that the ride took about a third longer than it really did, this must have been a trick of memory; their hyperacuity was a mirage.
Our memory becomes distorted because our brains react more strongly to novelty than to repetition. Eagleman investigated this effect by asking volunteers to estimate the duration of flashes of light; those flashes that were the first in a series, or broke an established pattern, seemed to last longer. This feature of consciousness, like the 80-millisecond rule, explain so much about our daily experience. When we’re sitting through a boring event, it seems to take forever. But when we look back on it, it went by in a flash. Conversely, when you’re doing something exciting, time seems to race by, but when you look back on it, it stretched out. In the first case, there was little to remember, so your brain collapsed the feeling of duration. In the second, there was so much to remember, so the event seemed to expand. Time flies when you’re having fun, but crawls when you recollect in tranquility.
You can probably tell from my lengthy description of Eagleman’s talk that it seemed to zip by at the time. The physicists in attendance found it one of the highlights of the conference. Not only was it engrossing in its own right, it had some professional interest for them. All theories of physics begin with sense-data. As Eagleman said, “We build our physics on top of our intuitions.”
We also build our physics on a recognition of the limits of perception. The whole point of theories such as relativity is to separate objective features of the world from artifacts of our perspective. One of the most important books of the past two decades on the physics and philosophy of time, Huw Price’s Time’s Arrow and Archimedes’ Point, argues that concepts of cause and effect derive from our experience as agents in the world and may not be a fundamental feature of reality.
Time plays a variety of roles in physics, from defining causal sequences to giving a direction to the unfolding of the universe. How many of these roles are rooted in the contingent ways our brains perceive time? How might an alien being, who perceives time in a radically different way, formulate physics?