paper

ITS 832 Chapter 3

The Quality of Social Simulation: An Example from Research Policy Modelling

Information Technology in a Global Economy

Professor Michael Solomon

Introduction

• The Quality of Social Simulation: • An Example from Research Policy Modelling

• A simulation is good • “… when we get from it what we originally would have liked to get

from the target”

• Different views • Standard • Constructionist • User community

• Chapter focus • Different approaches to assessing the quality of a simulation

Simulation comparison

Standard View

• Verification • Does the code do what it is supposed to do?

• Validation • Do the outputs resemble observations of the target?

• Relies on the observability of reality • Must be able to compare simulation output to reality

• Standard view may suffer from under-determination • Multiple incompatible theories may result from the same data

Constructionist View

• Compares • What you observe in the real world with,

• What you observe as simulation output

• Seems similar to Standard view, right? • Constructionists view all observations as constructions

• Evaluation is not possible • Even observations of reality lack the ability to pass validation

User Community View

• Evaluation is carried out • Using the observations of the affected user community

• Not just based on prior knowledge • Closer to “real” results • Often, results are influenced by multiple related factors

Policy Modelling for Ex-ante Evaluation of EU Funding Programs

Horizon 2020 Study Workflow

Summary

• Simulation quality depends on simulation process • Three different simulation views

• Standard

• Constructionist

• User community

• User community view • Most promising

• Most work-intensive