ITE4

ITS 832 CHAPTER 3 THE QUALITY OF SOCIAL SIMULATION: AN EXAMPLE FROM RESEARCH POLICY

MODELLING

INFORMATION TECHNOLOGY IN A GLOBAL ECONOMY

DR. JORDON SHAW

INTRODUCTION

• The Quality of Social Simulation: • An Example from Research Policy Modelling

• A simulation is good • “… when we get from it what we originally would have liked to get from the target”

• Different views • Standard

• Constructionist

• User community

• Chapter focus • Different approaches to assessing the quality of a simulation

SIMULATION COMPARISON

STANDARD VIEW

• Verification • Does the code do what it is supposed to do?

• Validation • Do the outputs resemble observations of the target?

• Relies on the observability of reality • Must be able to compare simulation output to reality

• Standard view may suffer from under-determination • Multiple incompatible theories may result from the same data

CONSTRUCTIONIST VIEW

• Compares

• What you observe in the real world with,

• What you observe as simulation output

• Seems similar to Standard view, right? • Constructionists view all observations as constructions

• Evaluation is not possible • Even observations of reality lack the ability to pass validation

USER COMMUNITY VIEW

• Evaluation is carried out

• Using the observations of the affected user community

• Not just based on prior knowledge

• Closer to “real” results

• Often, results are influenced by multiple related factors

POLICY MODELLING FOR EX-ANTE EVALUATION OF EU FUNDING PROGRAMS

HORIZON 2020 STUDY WORKFLOW

SUMMARY

• Simulation quality depends on simulation process

• Three different simulation views • Standard

• Constructionist

• User community

• User community view • Most promising

• Most work-intensive