Table of Contents
Acknowledgement vii
Chapter 1 Introduction and problem statement
1.1 Background 1
1.1.1 High stake policy problems with large uncertainties 1
1.1.2 Successes and failures in dealing with uncertainty 2
1.1.3 Increased realization for the importance of dealing
with uncertainty 4
1.2 Classes of policy problems 4
1.2.1 The changing nature of policy problems 4
1.2.2 The development of policy approaches and analytical tools 5
1.3 Overview of some prevailing policy approaches to deal with uncertainty 7
1.3.1 Do-nothing policy approach 8
1.3.2 Delay policy approach 8
1.3.3 ‘Optimal’ policy approach 9
1.3.4 Static robust policy approach 9
1.3.5 Adaptive policy approach 9
1.4 Quantitative analytical methods for dealing with uncertainty 9
1.4.1 Analytical methods for dealing with uncertainty 10
1.4.2 Information gained from methods for dealing with uncertainty 10
1.4.3 The progress of the development of analytical methods for
dealing with uncertainty 11
1.5 Exploratory modeling and analysis 12
1.5.1 A brief account of exploratory modeling and analysis 12
1.5.2 The status of EMA 12
1.5.3 EMA is worth further developing 13
1.6 Research Questions 13
1.7 Outline of the dissertation 14
References 15
Chapter 2 Research methodology and selection of application cases
2.1 Introduction 19
2.2 Defining the scope and specifying a research methodology 19
2.3 Research approach 20
2.3.1 Research philosophy 20
2.3.2 Research strategy 21
2.3.3 Choice of research instruments 22
2.3.4 Consequences of research instrument choice 23
2.4 Selection of application cases 23
2.4.1 Typology of policy problem to guide selection 24
2.4.2 Selection criteria and positioning of the application case 26
Table of contents x
2.4.3 Selected application cases 26
2.4.4 The choice of application cases satisfy criteria
for methodological development 28
2.5 Evaluation framework for EMA 28
2.5.1 Specification of added values 28
2.5.2 Approaches to assess the added values 29
2.6 The structure of the application cases 29
References 30
Chapter 3 A conceptual basis for dealing with uncertainty
3.1 Introduction 33
3.2 Philosophical views underlying the notion of uncertainty 33
3.2.1 Positivism vs. social constructivism 34
3.2.3 On the validity of scientific knowledge 34
3.3 The notion of uncertainty from a system perspective 35
3.3.1 The nature of uncertainty 35
3.3.2 The location and level of uncertainty 36
3.3.3 Additional characterizations of uncertainty 38
3.3.4 Deep uncertainty 39
3.4 Conceptual basis for dealing with uncertainty 39
3.4.1 Probability concept and theory 40
3.4.2 Ways of reasoning under uncertainty 41
3.4.3 Normative theories of decisionmaking 44
3.4.4 Descriptive theories of decisionmaking 46
3.5 Conclusions 48
References 48
Chapter 4 State of the art of exploratory modeling and analysis
4.1 Introduction 53
4.2 The fundamental elements of exploratory modeling 54
4.2.1 The core ideas of exploratory modeling 54
4.2.2 Exploratory modeling challenges prevailing
concepts and practices 54
4.2.3 Methodological challenges to exploratory modeling 56
4.3 The policy analysis framework 59
4.4 Major procedural elements of exploratory modeling and analysis 61
4.4.1 Step 1: conceptualize the policy problem 61
4.4.2 Step 2: specify the uncertainties relevant for policy analysis 62
4.4.3 Step 3: develop a computer model 67
4.4.4 Step 4: perform computational experiments 67
4.4.5 Step 5: specify a criterion for choosing a policy 70
4.4.6 Step 6: explore and display the outcomes of computational
experiments to reveal useful patterns of system behavior 71
4.4.7 Step 7: making policy recommendations 77
4.5 Summary 78
References 78
Exploratory modeling and analysis to deal with deep uncertainty xi
Chapter 5 Electricity Power Plant Investment
5.1 Introduction 81
5.2 Real options analysis 81
5.2.1 Real options 82
5.2.2 Real options and uncertainty 83
5.3 A simple electricity power plant investment model 84
5.3.1 The real option investment 84
5.3.2 The system model 85
5.3.3 Investment alternatives and characteristics 87
5.3.4 Uncertainty category and treatments 87
5.4 Applying exploratory modeling to support real options analysis 89
5.4.1 Plausible future scenarios 89
5.4.2 Computational experiments 90
5.4.3 Decision criteria 90
5.5. Insights from EMA 91
5.5.1 Determination of option value 93
5.5.2 Determination of regret value of the real option 94
5.5.3 The robustness of investment performance 95
5.5.4 Seeking further future conditions that might turn
a robust decision into a failure 99
5.5.5 Summary on the insights obtained from EMA 100
5.5.6 Final decision and corrective responses to improve decision
performance 100
5.6 Insights from existing methods (Monte Carlo simulation) 101
5.6.1 Specify probability density function 102
5.6.2 Specify correlations between input variables 103
5.6.3 Sampling the joint probability density functions 104
5.6.4 Perform sensitivity analysis 104
5.6.5 Perform robustness analysis 105
5.7 Comparing added value of EMA and traditional methods 106
5.7.1 Limitation of the real option set up 106
5.7.2 EMA comparison with existing methods 107
5.7.3 Added value measures 108
References 111
Chapter 6 The Implementation of Intelligent Speed Adaptation
6.1 Introduction 113
6.2 Problem specification 114
6.2.1 Problem statement 114
6.2.2 A simple model of the road safety system 115
6.2.3 Specifying the system model and the uncertainty ranges 118
6.2.4 Definition of success under uncertainty 125
6.2.5 Complete policy system space for
computational experiments 125
6.2.6 Basic results 126
6.3 Application of EMA to support an adaptive policy design 128
6.3.1 The design of an adaptive policy 129
6.3.2 Step 1: Specifying the policy problem 130
Table of contents xii
6.3.3 Step 2: Assembling a basic policy 131
6.3.4 Step 3: Specifying the rest of the policy 132
6.3.5 Step 4: Learning from real world experience 134
6.3.6 Step 5: Adapting the policy 136
6.3.7 Summary 137
6.4 Multi-criteria analysis for ISA implementation 138
6.4.1 Multi-criteria analysis for ISA implementation 138
6.4.2 Step 1: Conceptualization 139
6.4.3 Step 2: Specification of the uncertainties 140
6.4.4 Step 3: Integration of Analytic Hierarchy Process (AHP) 144
6.4.5 Step 4: Performing computational experiment 147
6.4.6 Step 5: Specification of a robustness criterion
for choosing ISA policy 147
6.4.7 Step 6: display and analysis of insights from EMA 148
6.5 Evaluation of EMA added value 154
6.5.1 Prior work on dealing with uncertainty in representing
the road safety system 154
6.5.2 Existing approaches in dealing with uncertainty in MCA 155
6.5.3 Assessment of EMA added values 157
References 159
Chapter 7 Policy design to reduce carbon emissions in the Dutch
household sector
7.1 Introduction 163
7.2 The policy issue in the Dutch household sector 164
7.3 A conceptual framework to deal with uncertainty and complexity 165
7.3.1 System-of systems perspective 166
7.3.2 System-of-Systems lexicon 167
7.3.3 Conventional representation of the energy sector 168
7.3.4 Representation of the energy sector from a SoS perspective 169
7.3.5 Transformation from a system-of-systems (SoS) into
a system of policy systems (SoPS) 171
7.3.6 Specification of the system of policy systems 173
7.3.7 Summary 182
7.4 EMA to support adaptive policy design 184
7.4.1 Computer model 184
7.4.2 Computational experiments 187
7.4.3 Trajectories of carbon emission reduction 188
7.4.4 Circumstances required to achieve the 2025 target 190
7.4.5 Conditions and guidance for policy adaptation 191
7.4.6 Analysis of Case1 191
7.4.7 Analysis of Case2 194
7.4.8 Implications for policy design 195
7.5 Evaluation of EMA added values 197
7.5.1 Comparison with safe landing approach 197
7.5.2 Value added assessment 199
References 201
Exploratory modeling and analysis to deal with deep uncertainty xiii
Chapter 8 Sampling and visualization within exploratory modeling
and analysis
8.1 Introduction 205
8.2 Establishing an appropriate number of samples 206
8.2.1 First Analysis 206
8.2.2 Second analysis 211
8.3 Generating and presenting EMA results 213
8.3.1 Explorative scorecard approach 215
8.3.2 ‘If-then’ rules by the rough set approach 221
8.3.3 ‘If-then’ rules using CART 224
8.3.4 ‘If-then’ rules using CART’s box plot 225
8.4 Comparison of EMA visualizations with those of existing methods 227
8.4.1 Analysis of variance (ANOVA) 227
8.4.2 Factor interaction analysis 228
8.4.3 Value added assessment on visualization 229
References 231
Chapter 9 Added values, reflections, and further study
9.1 Introduction 233
9.2 Insights from EMA 234
9.3 EMA insights to support policy design 236
9.4 Comparison with traditional uncertainty analysis methods 238
9.4.1 Added value in generating insights 239
9.4.2 Added value in using insights to support policy design 243
9.4.3 EMA added values and additional cost of resources 244
9.5 Contributions to and lessons from EMA development 245
9.5.1 Model building 246
9.5.2 Performing computational experiments 247
9.5.3 Making inferences 249
9.5.4 Policy design support 251
9.5.5 Summary of Contributions in furthering EMA development 252
9.6 Reflections 253
9.6.1 On the risk attitude in decisionmaking 253
9.6.2 On the scenario versus the probabilistic approach 254
9.6.3 On the concept of plausibility 256
9.6.4 On overall versus relevant insight 257
9.6.5 On the system of system perspective 259
9.7 Some personal accounts on EMA 260
9.7.1 Application cases as vehicles for learning 260
9.7.2 Human process aspects of EMA development and application 261
9.7.3 Acceptance of EMA 262
9.8 Further study 262
References 264
Appendices
Appendix 1: Formulation of the investment model in
electricity power plant 267
Table of contents xiv
Appendix 2: An interpretation of translational and transformational
relationships for ISA 267
Appendix 3: Results from the application of rough set approach to
the household heating case 272
Executive summary 275
Samenvatting 279
Curriculum vitae 283
Abstract
Faced with policy problems with high stakes, decisionmakers have increasingly recognized the importance of appropriately handling uncertainties. The nature of policy problems, however, is changing. Of particular concern are policy problems involving deep uncertainty when analysts do not know or the parties to a decision cannot agree upon (1) the appropriate conceptual models to describe interactions among a system’s variables, (2) the probability distributions to represent uncertainty about key parameters in the models, and/or (3) how to value the desirability of alternative outcomes. Exploratory Modeling and Analysis (EMA) is an analytical, model-based method for dealing with deep uncertainty. One of the foundations of EMA is the idea of exploring multiple hypotheses about the system of interest by broadening the assumptions underlying the system model. EMA explores multiple hypotheses about the system by means of computational experiments. A computational experiment is a single computer run of the system model using one set of assumptions. Each run is treated as a deterministic hypothesis about the system of interest. One can explore the system’s behavior by asking for each run, what if the hypothesis was correct. Broadening the assumptions of a system model and exploring the resulting behavior poses some major challenges. These challenges include, among others, how to sample from an (almost) infinite uncertainty space, and how to digest and present the information from the many computer runs (thousands to hundreds of thousands) in a way that is useful for policy design. This dissertation addresses these challenges. It uses three policy analysis cases as a testing ground for the application, development, and evaluation of EMA: (1) a real options analysis of a power plant investment decision, (2) the implementation of Intelligent Speed Adaptation (a technological solution for improving road safety), and (3) the analysis of policies to mitigate carbon emissions in the Dutch household sector. These three applications demonstrate a number of insights that can be obtained from EMA that complement those that can be obtained from traditional policy analysis methods. First, EMA can provide insights into the boundaries between the success and failure of a policy, which can help to identify “landmines” for the policy. Second, EMA can help identify the different sets of exogenous, system and policy assumptions that can lead to the achievement of a given policy goal, which can support different parties in negotiating a common policy. Third, EMA can provide insights into the robustness of a policy across the uncertainty space, which may enable policy implementation to begin despite the uncertainties. Finally, EMA can provide a policy menu that shows which policy performs best in which circumstances, which can support policy adaptation over time. In addition to these insights into the added value of EMA, the dissertation makes a number of original contributions to the EMA methodology, in particular regarding the sampling method, the analysis of the data generated, and the presentation of the insights obtained.