Uncertain Judgements

Elicitation is the process of extracting expert knowledge about some unknown quantity or quantities, and formulating that information as a probability distribution. Elicitation is important in situations, such as modelling the safety of nuclear installations or assessing the risk of terrorist attacks, where expert knowledge is essentially the only source of good information. It also plays a major role in other contexts by augmenting scarce observational data, through the use of Bayesian statistical methods. However, elicitation is not a simple task, and practitioners need to be aware of a wide range of research findings in order to elicit expert judgements accurately and reliably. Uncertain Judgements introduces the area, before guiding the reader through the study of appropriate elicitation methods, illustrated by a variety of multi-disciplinary examples.This is achieved by:* Presenting a methodological framework for the elicitation of expert knowledge incorporating findings from both statistical and psychological research.* Detailing techniques for the elicitation of a wide range of standard distributions, appropriate to the most common types of quantities.* Providing a comprehensive review of the available literature and pointing to the best practice methods and future research needs.* Using examples from many disciplines, including statistics, psychology, engineering and health sciences.* Including an extensive glossary of statistical and psychological terms.An ideal source and guide for statisticians and psychologists with interests in expert judgement or practical applications of Bayesian analysis, Uncertain Judgements will also benefit decision-makers, risk analysts, engineers and researchers in the medical and social sciences.

Preface.1. Fundamentals of Probability and Judgement.1.1 Introduction.1.2 Probability and elicitation.1.3 Uncertainty and the interpretation of probability.1.4 Elicitation and the psychology of judgement.1.5 What use are such judgements?1.6 Conclusions.2. The Elicitation Context.2.1 How and who?2.2 What is an expert?2.3 The elicitation process.2.4 Conventions in Chapters 3 to 9.2.5 Conclusions.3. The Psychology of Judgement Under Uncertainty.3.1 Introduction.3.2 Understanding the task and the expert.3.3 Understanding research on human judgement.3.4 The heuristic and biases research programme.3.5 Experts and expertise.3.6 Three meta theories of judgement.3.7 Conclusions.4. The Elicitation of Probabilities.4.1 Introduction.4.2 The Calibration of Subjective Probabilities.4.3 The calibration in subjective probabilities: theories and explanations.4.4 Representations and methods.4.5 Debiasing.4.6 Conclusions.5. Eliciting Distributions - General.5.1 From probabilities to distributions.5.2 Eliciting univariate distributions.5.3 Eliciting multivariate distributions.5.4 Uncertainty and imprecision.5.5 Conclusions.6. Eliciting and Fitting a Parametric Distribution.6.1 Introduction.6.2 Outline of this chapter.6.3 Eliciting opinion about a proportion.6.4 Eliciting opinion about a general scalar quantity.6.5 Eliciting opinion about a set of proportions.6.6 Eliciting opinion about the parameters of a multivariate normal distribution.6.7 Eliciting opinion about the parameters of a linear regression model.6.8 Eliciting opinion about the parameters of a generalized linear model.6.9 Elicitation methods for other problems.6.10 Deficiencies in existing research.6.11 Conclusions.7. Eliciting Distributions - Uncertainty and Imprecision.7.1 Introduction.7.2 Imprecise probabilities.7.3 Incomplete information.7.4 Summary.7.5 Conclusions.8. Evaluating Elicitation.8.1 Introduction.8.2 Scoring rules.8.3 Coherence, feedback and overfitting.8.4 Conclusions.9. Multiple Experts.9.1 Introduction.9.2 Mathematical aggregation.9.3 Behavioural aggregation.9.4 Discussion.9.5 Elicitation practice.9.6 Research questions.10. Published Examples of the Formal Elicitation of Expert Opinion.10.1 Some applications.10.2 An example of an elicitation interview - eliciting engine sales.10.3 Medicine.10.4 The nuclear industry.10.5 Veterinary science.10.6 Agriculture.10.7 Meteorology.10.8 Business studies, economics and finance.10.9 Other professions.10.10 Other examples of the elicitation of subjective probabilities.11. Guidance on Best Practice.12. Areas for Research.Glossary.Bibliography.Author Index.Index.