Cognitive Miracles: When Are Fast Processes Unreliable?

Fast processes are unreliable when deployed to solve unfamiliar problems. But if (as I suppose) we do not know much about how the fast processes work in the case of ethics, we cannot know which problems are unfamiliar. Can we nevertheless make practical use of the principle that fast processes are unreliable when deployed to solve unfamiliar problems?

• stream
• youtube
• slides only

Notes

Cognitive Miracles

In what situations could fast processes yield correct responses?

‘genetic transmission, cultural transmission, and learning from personal experience […] are the only mechanisms known to endow [fast] processes with the information they need to function well’ (Greene, 2014, p. 715).

‘it would be a cognitive miracle if we had reliably good moral instincts about unfamiliar moral problems’ (Greene, 2014, p. 715).

‘The No Cognitive Miracles Principle: When we are dealing with unfamiliar* moral problems, we ought to rely less on […] automatic emotional responses and more on […] conscious, controlled reasoning, lest we bank on cognitive miracles’ (Greene, 2014, p. 715).

How to Apply the No Cognitive Miracles Principle?

It is tricky to apply this principle. For instance, is how to win a chess match an unfamiliar problem?

Although it may initially seem reasonable to speculate that how to win a chess match is an unfamiliar problem, expert chess players are supposed to rely on faster processes.

Are cartoons unfamiliar situations to someone who has never seen one? Although it may initially seem reasonable to speculate that they are (humans presumably encountered few 2D schematic animations in evolution), fast processes appear to have no problem with them. Why? Because the fast processes that underpin physical cognition are driven by principles, and these principles (although false) can be applied to new situations.

Because of the way we defined an unfamiliar problem, knowing whether a problem is unfamiliar typically depends on knowing something about the structure of the fast processes. Which, arguably, we do not in the case of ethics.

Does this mean the No Cognitive Miracles Principle is useless? Not at all. There are at least two ways we might apply it in practice even without knowing which situations are unfamiliar.

Wicked Learning Environments

Hogarth (among many researchers) has studied when fast processes can be reliably used even in the absence of knowing in detail how they work. (This is a practical problem in many areas of life.) He concludes:

‘When a person’s past experience is both representative of the situation relevant to the decision and supported by much valid feedback, trust the intuition; when it is not, be careful’ (Hogarth, 2010, p. 343; see Kahneman & Klein, 2009, p. 520 for a related view).

This suggests a practical way to avoid relying on cognitive miracles even without knowing exactly which situations and problems are unfamiliar.

But this is not the only way to avoid relying on cognitive miracles.

Disagreements

Greene argues that it is reasonable to suppose that where there is fully informed disagreement about what to do, we are likely to be in an unfamiliar situation:

‘we can use disagreement as a proxy for lack of familiarity*. If two parties have a practical moral disagreement–—a disagreement about what to do, not about why to do it—it’s probably because they have conflicting intuitions. This means that, from a moral perspective, if not from a biological perspective, at least one party’s automatic settings are going astray. (Assuming that both parties have adequate access to the relevant nonmoral facts.) Absent a reliable method for determining whose automatic settings are misfiring, both parties should distrust their intuitions’ (Greene, 2014, p. 716).

Greene (2017) provides further discussion relevant to the question of which situations are, or might reasonably be suspected of being, unfamiliar.

Which comparison: Linguistic or Physical?

The slides and recording use a comparison between ethical and physical cognition. This assists in arriving at the view that fast processes are sometimes unreliable.

How would things look if instead we compared ethical to linguistic cognition? Roughly speaking, the facts in linguistics are determined by how the fast processes operate together with some collective cultural decision-making. It is hard even to make sense of the idea that the fast processes are unreliable because the linguistic facts are overwhelmingly determined by how the fast processes operate. (The collective cultural decision-making is a relatively recent, and relatively superficial, phenomenon.)

To illustrate, Jackendoff (2003, p. 19) observes that a postulation about the syntactic structure of a sentence ’is to be treated as a model of something in the mind of a speaker of English who says or hears this sentence.’ In other words, the target is a fact about how the fast process operates. Claims of unreliability are therefore limited to cases where competence and performance come apart; it is not possible that a linguistic theory could discover that fast processes embody a systematically distorted view of the linguistic.

Is the comparison with the linguistic plausible? Relying on it would likely commit you to quite a dismal view of ethics (see Lecture 04, particularly Moral Pluralism: Beyond Harm).

Glossary

References