Link Search Menu Expand Document

Cognitive Miracles: When Are Fast Processes Unreliable?

Fast processes are unreliable when deployed to solve unfamiliar problems. But if (as I suppose) we do not know much about how the fast processes work in the case of ethics, we cannot know which problems are unfamiliar. Can we nevertheless make practical use of the principle that fast processes are unreliable when deployed to solve unfamiliar problems?

If the video isn’t working you could also watch it on youtube. Or you can view just the slides (no audio or video).

This recording is also available on stream (no ads; search enabled).

If the slides are not working, or you prefer them full screen, please try this link. The recording is available on stream and youtube.


Cognitive Miracles

In what situations could fast processes yield correct responses?

‘genetic transmission, cultural transmission, and learning from personal experience […] are the only mechanisms known to endow [fast] processes with the information they need to function well’ (Greene, 2014, p. 715).

‘it would be a cognitive miracle if we had reliably good moral instincts about unfamiliar moral problems(Greene, 2014, p. 715).

‘The No Cognitive Miracles Principle: When we are dealing with unfamiliar* moral problems, we ought to rely less on […] automatic emotional responses and more on […] conscious, controlled reasoning, lest we bank on cognitive miracles’ (Greene, 2014, p. 715).

How to Apply the No Cognitive Miracles Principle?

It is tricky to apply this principle. For instance, is how to win a chess match an unfamiliar problem?

Although it may initially seem reasonable to speculate that how to win a chess match is an unfamiliar problem, expert chess players are supposed to rely on faster processes.

Are cartoons unfamiliar situations to someone who has never seen one? Although it may initially seem reasonable to speculate that they are (humans presumably encountered few 2D schematic animations in evolution), fast processes appear to have no problem with them. Why? Because the fast processes that underpin physical cognition are driven by principles, and these principles (although false) can be applied to new situations.

Because of the way we defined an unfamiliar problem, knowing whether a problem is unfamiliar typically depends on knowing something about the structure of the fast processes. Which, arguably, we do not in the case of ethics.

Does this mean the No Cognitive Miracles Principle is useless? Not at all. There are at least two ways we might apply it in practice even without knowing which situations are unfamiliar.

Wicked Learning Environments

Hogarth (among many researchers) has studied when fast processes can be reliably used even in the absence of knowing in detail how they work. (This is a practical problem in many areas of life.) He concludes:

‘When a person’s past experience is both representative of the situation relevant to the decision and supported by much valid feedback, trust the intuition; when it is not, be careful’ (Hogarth, 2010, p. 343; see Kahneman & Klein, 2009, p. 520 for a related view).

This suggests a practical way to avoid relying on cognitive miracles even without knowing exactly which situations and problems are unfamiliar.

But this is not the only way to avoid relying on cognitive miracles.


Greene argues that it is reasonable to suppose that where there is fully informed disagreement about what to do, we are likely to be in an unfamiliar situation:

‘we can use disagreement as a proxy for lack of familiarity*. If two parties have a practical moral disagreement–—a disagreement about what to do, not about why to do it—it’s probably because they have conflicting intuitions. This means that, from a moral perspective, if not from a biological perspective, at least one party’s automatic settings are going astray. (Assuming that both parties have adequate access to the relevant nonmoral facts.) Absent a reliable method for determining whose automatic settings are misfiring, both parties should distrust their intuitions’ (Greene, 2014, p. 716).

Greene (2017) provides further discussion relevant to the question of which situations are, or might reasonably be suspected of being, unfamiliar.

Which comparison: Linguistic or Physical?

The slides and recording use a comparison between ethical and physical cognition. This assists in arriving at the view that fast processes are sometimes unreliable.

How would things look if instead we compared ethical to linguistic cognition? Roughly speaking, the facts in linguistics are determined by how the fast processes operate together with some collective cultural decision-making. It is hard even to make sense of the idea that the fast processes are unreliable because the linguistic facts are overwhelmingly determined by how the fast processes operate. (The collective cultural decision-making is a relatively recent, and relatively superficial, phenomenon.)

To illustrate, Jackendoff (2003, p. 19) observes that a postulation about the syntactic structure of a sentence ’is to be treated as a model of something in the mind of a speaker of English who says or hears this sentence.’ In other words, the target is a fact about how the fast process operates. Claims of unreliability are therefore limited to cases where competence and performance come apart; it is not possible that a linguistic theory could discover that fast processes embody a systematically distorted view of the linguistic.

Is the comparison with the linguistic plausible? Relying on it would likely commit you to quite a dismal view of ethics (see Lecture 04, particularly Moral Pluralism: Beyond Harm).


automatic : On this course, a process is _automatic_ just if whether or not it occurs is to a significant extent independent of your current task, motivations and intentions. To say that _mindreading is automatic_ is to say that it involves only automatic processes. The term `automatic' has been used in a variety of ways by other authors: see Moors (2014, p. 22) for a one-page overview, Moors & De Houwer (2006) for a detailed theoretical review, or Bargh (1992) for a classic and very readable introduction
cognitively efficient : A process is cognitively efficient to the degree that it does not consume working memory and other scarce cognitive resources.
fast : A fast process is one that is to some interesting degree automatic and to some interesting degree cognitively efficient. These processes are also sometimes characterised as able to yield rapid responses.
Since automaticity and cognitive efficiency are matters of degree, it is only strictly correct to identify some processes as faster than others.
The fast-slow distinction has been variously characterised in ways that do not entirely overlap (even individual author have offered differing characterisations at different times; e.g. Kahneman, 2013; Morewedge & Kahneman, 2010; Kahneman & Klein, 2009; Kahneman, 2002): as its advocates stress, it is a rough-and-ready tool, not the basis for a rigorous theory.
unfamiliar problem : An unfamiliar problem (or situation) is one ’with which we have inadequate evolutionary, cultural, or personal experience’ (Greene, 2014, p. 714).


Bargh, J. A. (1992). The Ecology of Automaticity: Toward Establishing the Conditions Needed to Produce Automatic Processing Effects. The American Journal of Psychology, 105(2), 181–199.
Greene, J. D. (2014). Beyond Point-and-Shoot Morality: Why Cognitive (Neuro)Science Matters for Ethics. Ethics, 124(4), 695–726.
Greene, J. D. (2017). The rat-a-gorical imperative: Moral intuition and the limits of affective learning. Cognition, 167, 66–77.
Hogarth, R. M. (2010). Intuition: A Challenge for Psychological Research on Decision Making. Psychological Inquiry, 21(4), 338–353.
Jackendoff, R. (2003). Foundations of Language: Brain, Meaning, Grammar, Evolution. Oxford, New York: Oxford University Press.
Kahneman, D. (2002). Maps of bounded rationality: A perspective on intuitive judgment and choice. In T. Frangsmyr (Ed.), Le prix nobel, ed. T. Frangsmyr, 416–499. (Vol. 8, pp. 351–401). Stockholm, Sweden: Nobel Foundation.
Kahneman, D. (2013). Thinking, fast and slow.
Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: A failure to disagree. American Psychologist, 64(6), 515–526.
Moors, A. (2014). Examining the mapping problem in dual process models. In Dual process theories of the social mind (pp. 20–34). Guilford.
Moors, A., & De Houwer, J. (2006). Automaticity: A Theoretical and Conceptual Analysis. Psychological Bulletin, 132(2), 297–326.
Morewedge, C. K., & Kahneman, D. (2010). Associative processes in intuitive judgment. Trends in Cognitive Sciences, 14(10), 435–440.
Thomson, J. J. (1976). Killing, Letting Die, and The Trolley Problem. The Monist, 59(2), 204–217.