Noise: A Flaw in Human Judgment is a book by Daniel Kahneman, Olivier Sibony, and Cass Sunstein that explores the concept of noise and how it can affect our judgments and decisions.
The book explores how noise can arise from various sources and how it can lead to mistakes in reasoning and decision-making. It also provides a framework for understanding how noise can be managed and minimized.
In addition, the book examines how noise can have a negative impact on fairness, accuracy, and trust. The book provides readers with a comprehensive understanding of noise and how it can be better managed and reduced in order to make better decisions.
Noise: The Premise
Everyone knows that humans are flawed by our very nature. Our inability to be absolutely perfect is a feature that is inextricable to our humanness. For the most part, this imperfection is neither positive or negative–it just is.
However, one of the stickier ways it shows up is in how we make decisions and judgments.
- Two different medical doctors can review the exact same case, described in the exact same chart, with identical language and history. Those two doctors can, and often do, reach a different diagnosis.
- A judge overseeing identical evidence and prior convictions of an accused person can level a completely different judgment than a different judge in a different court.
Similar examples are common in corporate customer service, human resources and personnel, forensic science, and more. To make matters worse, individuals’ judgments or behavior can be vastly different from day to day and even from morning till night.
Finding Noise
A noise audit can be implemented in any organization, public or private, to determine the extent to which noise is clouding the judgment of decision-makers. In the corporate world, this can expose opportunities to reduce costs and increase fairness.
To conduct a noise audit, decision-related data is gathered and compared to reveal unwanted variability. Multiple individuals judge the same problems, and the results are compared.
Whereas most of the decisions in a particular set should fall within a relatively small range, noise leads to a much larger than expected range. Once the existence of noise is clearly demonstrated, steps can be undertaken to reduce its impact.
The Three principles
Aggregation comes into play but in order to be valid, the evaluation of each candidate must be performed independently without any knowledge or exposure to the evaluations of others.
After evaluations are made independently the data can be aggregated to form a clearer and more accurate picture of the candidate’s viability. Taking it further though, the process should include three principles of structure:
- Decomposition “breaks down the decision into components” that aim the judgment to the most important cues, much like subjudgments do in guidelines.
- Independence “requires that each assessment be collected independently” to eliminate influence from one element to the other.
- Delayed holistic judgment allows the decision makers to rely on their intuition, as intuition is recognized as valid in decision making, but reliance on it should be delayed.
Decision Hygiene: Practical solutions for reducing noise
This approach includes four strategies, such as developing judgment guidelines, establishing shared scales based on outside views, structuring complex judgments, and using a mediating assessments protocol (MAP).
By conducting a noise audit, organizations can determine the extent of noise affecting their judgments and decide on solutions.
While replacing judgment with rules and algorithms may be effective, improving the judgment capabilities of decision makers is also essential.
Implementing the strategies of decision hygiene can lead to more reliable and accurate decision making in various sectors, including medicine, business, education, and government.
Mediating assessments protocol (MAP)
The mediating assessments protocol, or MAP, is an all-encompassing method designed to reduce noise in organizational decision making. It maximizes the value of information by keeping evaluations independent of each other and allows for divergent viewpoints to be heard.
The process ensures that all people and all decisions are made with dignity and respect, taking into account the full picture of our human experience. Noise-reduction strategies should not compromise flexibility or treat people unfairly.
MAP can be applied broadly and adapted to various organizations in multiple sectors.
The Guidelines that work
The shared scale grounded in an outside view is a strategy for reducing noise in performance evaluations. The problem with many rating scales is that they are inherently noisy, meaning that the personal biases, circumstances, and relationships of the rater can impact the end result.
The knowledge-based nature of many modern workplaces can make it difficult to objectively evaluate performance. This can lead to a lack of trust in the evaluation process and decreased morale among employees.
To combat this, organizations can use tactics such as aggregation and forced ranking, but these methods have their drawbacks.
A better strategy is to ensure a common frame of reference by using a shared scale grounded in an outside view. This involves using external benchmarks or standards to evaluate performance, rather than relying solely on subjective assessments.
Structuring complex judgments
The typical candidate interview for a position within an organization is a ripe opportunity for noise. Most interviews are unstructured in that the person conducting the interview asks the candidate a series of questions and allows the candidate to do the same.
The main goal of the interview is to determine whether or not the candidate will be successful in the position but unfortunately, unstructured interviews fail miserably in that index. Assessing a candidate’s viability in a position is a complex judgment call and therefore, it requires a more structured approach.
