Discussion question 1 DQ 1

Towards an alternative framework for the

evaluation of translational research initiatives

Jordi Molas-Gallart1,*, Pablo D’Este1, Oscar Llopis1,2 and

Ismael Rafols1,3

1INGENIO (CSIC-UPV), Edificio 8E, Camino de Vera s/n, Universitat Politècnica de València, 46022, València

Spain, 2Université Bordeaux, GREThA - CNRS, Avenue Leon Duguit, F-33608 Pessac Cedex, France and 3SPRU -

Science Policy Research Unit, Jubilee Building, University of Sussex, Brighton, BN1 9SL, England

*Corresponding author. Email: [email protected].

Abstract

The perception that many promising results from basic biomedicine have not systematically con-

tributed to medical treatments and, ultimately, health care improvements, has led to a wide range

of publicly funded initiatives aiming at facilitating the ‘translation’ of scientific discoveries into

beneficial applications and practices. Many of these initiatives have been branded as

‘Translational Research’ (TR), a term widely applied to large research programmes, research

activities, and even academic journals. With the popularity of the term, a debate has emerged

about the models of research that are to be considered ‘translational’. Consequently, the ways in

which TR should be analysed and, more specifically, the approaches to the evaluation of TR pro-

grammes are also the subject of debate. Given the substantial investments in TR programmes,

the definition of TR evaluation strategies and approaches has become an important element of

the policy process. In a context of ambiguity about the type of activities to be considered as TR,

evaluation approaches and practices can play an important role in determining what actions and

outcomes are conceived, in practice, to be relevant and significant, and in doing so, shaping the

future nature of TR initiatives. This article discusses the dominant approaches to TR evaluation

and proposes an alternative evaluation framework, which would have implications both for TR

evaluation processes and for the future shaping of TR programmes.

Key words: translational research; proximities framework; biomedical research; research application.

Introduction

The pathways between basic science, clinical practice, and health

outcomes are multifaceted and complex. Analysis of these path-

ways has attracted the interest of the biomedical research commu-

nity and public health agencies. Researchers and funding agencies

are concerned with the ways in which scientific breakthroughs and

evidence-based clinical findings are converted into practices with

beneficial health impacts, including, but not limited to, therapies

and medical guidelines. This interest is driven largely by the per-

ception that many promising results from basic biomedicine have

not contributed systematically to medical treatments and, ultim-

ately, health care improvements.1 In response, a wide range of

publicly funded initiatives has been set up to address this problem.

As the main aim of these initiatives is to facilitate the ‘translation’

of scientific discoveries into beneficial applications and practices,

many of these initiatives have been branded ‘Translational

Research’ (TR).

TR has become a very popular term and has been applied to large

research programmes, research activities, and even academic jour-

nals, and has attracted the interest of biomedical scholars and institu-

tions (Marincola 2003; Zerhouni 2007; Woolf 2008). The origins of

the concept can be traced back to the 1990s, when the US National

Cancer Institute developed the Specialized Programs of Research

Excellence (SPORE) (Lander and Atkinson-Grosjean 2011). Starting

in 1992, SPORE provided support for efforts to facilitate the ‘trans-

lation’ of basic discoveries generated at academic centres, into new

interventions aimed at preventing and treating various types of can-

cer. Since then, several policy initiatives have focused on the trans-

formation of basic knowledge into health benefits. In the USA, the

National Institutes of Health (NIH) launched the Roadmap Initiative

VC The Author 2015. Published by Oxford University Press. 235

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits

unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.

Research Evaluation, 25(3), 2016, 235–243

doi: 10.1093/reseval/rvv027

Advance Access Publication Date: 24 November 2015

Article

(Zerhouni 2003), the Clinical and Translational Science Awards

(Heller and de Melo-Martı́n 2009), and, in December 2011, a $575

million National Center for Advancing Translational Sciences.2 TR

initiatives have also been launched in the European Union (EU) and

its Member States. Some are explicitly labelled TR programmes,

others have similar objectives and rationales, although they are

labelled differently. For example, in 2006, the Spanish Ministry of

Health launched the Networked Centres of Biomedical Research

(CIBER),3 together with other research initiatives to facilitate the re-

lationship between basic scientists and healthcare practitioners (Rey-

Rocha and Martı́n-Sempere 2012).

Often the more popular the policy concept, the more ambiguous

it becomes. This clearly applies to TR, and there is emerging debate

on the models of research which are considered to be ‘translational’

and the nature and characteristics of a putative TR discipline

(Littman et al. 2004). Consequently, the ways in which TR should

be analysed, and, more specifically, the approaches to the evaluation

of TR programmes are similarly open to debate. Given the substan-

tial investment in TR programmes, the definition of TR evaluation

strategies has become important for policy definition and implemen-

tation. Given the ambiguity about the type of activities that can be

considered TR, evaluation approaches and practices can play an im-

portant role in determining these activities, thus shaping the nature

of TR initiatives. This article discusses the dominant approaches to

TR evaluation and proposes an alternative evaluation framework,

which may have implications for the way in which TR programmes

may be defined in the future.

First, we provide an overview of the different ways in which TR

is conceptualized. Many approaches consider TR as activities that

bridge gaps in a continuum stretching from basic research to health

outcomes (Morris et al. 2011). Other views emphasize the research

process, how different groups interact, and how their roles may be

redefined by a TR initiative (Currie and White 2012).

Second, we discuss different evaluation approaches associated

with these different views of TR. One of the dominant approaches

focuses on the outputs generated at different points of the ‘transla-

tional research continuum’ and estimates the time it takes to pro-

duce them. A focus on ‘what’ and ‘when’ implies a TR evaluation

approach that attempts to identify results and how they differ from

what would have been achieved in the absence of the initiatives

being assessed. It should be emphasized that this focus on outputs

may derive from an explicit view of TR as addressing ‘translational

gaps’ along a ‘TR continuum’, or may emerge without any explicit

‘theory’ of the processes and objectives of TR. TR then is measured

against success criteria based on the generation of outputs that are

no different from the outputs generated by traditional research.

Note that, in the absence of a ‘programme theory’, the objectives of

TR will, de facto, be defined by the evaluation strategy chosen.

Our proposal is to focus, instead, on ‘how’ research is carried

out, on the processes of collaboration and exchange that can be

attributed to TR initiatives. To this end, we develop an alternative

TR evaluation framework that focuses on understanding the proc-

esses of change across the divides that hinder the application of the

capabilities and knowledge generated by basic biomedicine to health

care. The extant literature attributes the low level of practical appli-

cation of biomedical research to a variety of causes, including:

(1) the divide between the interests and skills of the different profes-

sional communities and disciplines, such as basic scientists, clinical

scientists, and clinical practitioners (Currie and White 2012); (2) the

problem of communication among fields that are becoming more

complex and specialized (Littman et al. 2004); and (3) the existence

of institutional barriers and occupational boundaries that hinder the

effective flow of knowledge and interests (Lander and Atkinson-

Grosjean 2011).

Third, following the conceptual framework proposed by Ron

Boschma (2005), we suggest that TR initiatives can operate by gen-

erating ‘proximities’ along five different analytical dimensions: cog-

nitive, social, organizational, institutional, and geographical. We

define these dimensions and argue that a TR support activity can be

defined by the type of proximity it targets. We illustrate this by

referring to two Spanish biomedical research programmes, and con-

clude by exploring the implications of using this alternative way of

framing TR evaluation.

TR: a variety of approaches

The development of TR initiatives reflects the perceived need to in-

crease the chances that scientific discoveries will lead to benefits for

patients, and to improve the alignment between biomedical research

in biomedicine and health needs. TR emerged at a time when other

initiatives aimed at facilitating the ‘valorization’ or uptake of research

by socio-economic actors were flourishing (Bozeman and Boardman

2004). TR has become part of the policy discourse, justifying many

current research-funding programmes. The inclusion of the TR dis-

course in the policy agenda has been accompanied by intense aca-

demic discussion among biomedical scholars. What policy or

management measures are needed to speed up the process of applica-

tion of biomedical research advances to clinical practice? How should

we characterize TR and evaluate TR initiatives? The development of a

conceptual framework to describe the TR process and evaluate TR

initiatives has become a theme in the academic research literature (for

a review of the discussion see Drolet and Lorenzi, 2011).

The linear model of TR The most popular representations of TR assume a linear model of

innovation (Rogers 2003), prioritizing basic research as the primary

source of new discoveries, which subsequently are developed into

therapeutic solutions and finally are diffused to patients and the

wider society. In the medical field, the adoption of this approach

sees basic scientists at the origin of the innovation process, produc-

ing a large amount of fundamental knowledge at the molecular or

cell level, some of which will be relevant for the development of new

drugs or therapies. The fundamental knowledge generated by basic

scientists moves forward through the stages of a ‘translational con-

tinuum’, until eventually, it is translated into specific benefits for pa-

tients or the general population, in the form of new drugs, devices,

and new prevention and treatment options. Every step in this linear

progression addresses a specific problem and is undertaken by a spe-

cialized group of researchers. In this view, the successful application

of new knowledge is dependent on the successful completion of

every one of the stages along the ‘translational continuum’ (Van der

Laan and Boenink 2012) (Fig. 1).

Indeed, the idea of moving forwards through stages, which has

been described as going from ‘bench to bedside’, is thoroughly in-

grained in most of the existing conceptualizations of TR—in aca-

demia and among practitioners and policymakers (Sung et al. 2003;

Khoury et al. 2007). This stage approach to TR does not differ sub-

stantially from the classic linear stage process, which characterizes all

clinical research. What makes TR approaches different is the explicit

236 Research Evaluation, 2016, Vol. 25, No. 3

identification of which steps in these processes are more problematic

and slow the progression towards application and health benefits.

Thus, TR models identify a series of translational chasms, gaps, or

blocks that need to be bridged (Woolf 2008). These chasms are

viewed as obstacles and typically are described using what has come

to be known as a ‘T-terminology’ (Dougherty and Heller 1994),

which consists of a structured list of (T)ranslation gaps to be bridged.

According to these models, the main objective of TR is to bridge

these gaps so as to facilitate more rapid movement of knowledge

through the successive steps from basic research to application.

One of the first models adopting a T-terminology was developed

by the US Institute of Medicine’s Clinical Research Roundtable,

which identified two main gaps: T1 and T2. The first chasm (T1) is

related to the transfer of basic discoveries into human clinical test-

ing; the second, T2, refers to the dissemination and adoption of suc-

cessful clinical discoveries into daily clinical practice. As TR

research has developed, more detailed models have been proposed,

which include more T-phases and more chasms to be bridged.

Westfall et al. (2007) proposed a TR model beginning at T1, where

knowledge coming from basic science moves to human clinical re-

search through the development of Phase I and Phase II clinical tri-

als. According to this model of TR, the process starts at the ‘bench’,

with fundamental discoveries in molecular biology, genetics, and

other basic sciences which may be of interest for understanding

human health. The T2 chasm comprises the activities related to the

translation of initial human testing results into clinical practice.

Activities such as Phase III and Phase IV clinical trials, and observa-

tional studies and survey research, are considered to occur at this

stage. The final gap (T3) deals with the translation into practice,

and the dissemination of the new clinical treatment (e.g. through the

development of guidelines for clinical practice, patients and the gen-

eral population). Other models break down the translational con-

tinuum even further by proposing an additional final gap (T4),

which emerges in the effort to advance towards real-world health

outcomes by promoting the adoption of evidence-based recommen-

dations by health practitioners (Khoury et al. 2007).

Some of the proponents of these linear TR models acknowledge

that knowledge can flow also from ‘bedside to bench’. For instance,

Marincola (2011) stresses that hypotheses tested in basic research

experiments can be based on observational evidence from practi-

tioners. Research based on clinical evidence is particularly important

because it provides factual knowledge collected through practitioner

observation (e.g. through direct contact with patients), which can be

translated into specific hypotheses to be tested in the lab, or lead to

ideas that open up new research avenues. However, although the bi-

directional nature of TR is acknowledged in most TR models, the

majority of TR policy initiatives pursue or are based implicitly on a

unidirectional ‘bench to bedside’ understanding of knowledge gener-

ation and application, as reflected in the terminology referring to

consecutive gaps (T1, T2, T3, . . . ) which need to be bridged.

This focus on the identification of T-gaps has posed a series of

challenges that have framed much research on TR analysis and

evaluation and some problems. In analysing TR to find solutions to

different translational gaps, the identification of these gaps and the

different views of stakeholders about how to address them, can lead

to different understandings of what TR is about (Van der Laan and

Boenink 2012) and what specific skills scientists should develop to

support TR (Rubio et al. 2010). Littman et al. (2007) point out that,

for academics, TR represents (1) a channel to test whether novel

ideas generated by basic science have the potential to translate into

practical applications, (2) an opportunity to gain observational in-

sights and develop novel scientific hypotheses to be tested in the lab,

and (3) a means to gain legitimacy and improved access to research

funding. However, for clinical practitioners such as physicians or

clinical staff, TR is viewed primarily as responding to the need to

shorten the path between scientific evidence and actual practice

(Davis et al. 2003). Business organizations view TR as a process to

accelerate the development of a new drug or therapy and as an op-

portunity to make go/no-go decisions at an early stage in the bio-

medical innovation process—potentially resulting in major savings

by avoiding unproductive investments. Also, the fact that public or-

ganizations conduct TR is seen by industry as an opportunity to

save on research whose returns are very uncertain (Littman et al.

2007).

The interactive-process model of TR Although different stakeholders may hold different views on the ob-

jectives of TR, there is a consensus on specific ‘gaps’ among the suc-

cession of translational gaps. This ‘gap-centred’ perspective,

implicitly or explicitly sees knowledge accumulating through differ-

ent stages from fundamental to applied research. However, some

Basic Research Clinical Trials

• Early Transla�on • Phase I Trials • Phase II Trials

• Late Transla�on • Phase III Trials • Guidelines

Development • Dissemina�on &

Implementa�on Research

• Phase IV Trials

Adop�on and Evalua�on

• Educa�onal Materials & Training

• Payment mechanisms to support adop�on

• Measurement Health Impacts

Figure 1. A model of Translational Research Continuum.

Research Evaluation, 2016, Vol. 25, No. 3 237

scholars have expressed concern about the adequacy of a linear TR

model to frame analysis and develop policy strategies (Graham et al.

2006; Littman et al. 2007; Marincola 2011). A linear TR model

builds implicitly on the theoretical separation between basic and

applied research. Although this separation is widely used when talk-

ing about science and technology policy, numerous studies show

that it is problematic. In particular, fundamental knowledge can be

sought in order to solve an applied problem, à la Pasteur. This argu-

ably is a different form of research, ‘use-inspired basic research’

(Stokes 1997), which fits well with the TR goal of generating know-

ledge with an explicit focus on patient applications and public

health benefits. However, a TR model based on a linear progression

from basic science to health applications cannot account for the ex-

istence of ‘user-inspired basic research’. Also, a linear conceptualiza-

tion of TR may clash with the evidence on how medical innovation

processes emerge and develop in the health sector. The medical re-

search process does not always proceed linearly. Rather, innovation

scholars conceptualize it as an iterative process in which insights

provided by one group of stakeholders spurs advancement in sur-

rounding professional and epistemic groups (Consoli and Mina

2009; Hobin et al. 2012).

The idea that knowledge moves forwards and backwards along

the unidimensional line of the TR continuum may partially capture

the view of a dynamic relation between basic and applied research

and the application of its results, but cannot account for the exist-

ence of a different type of research, involving scientists who carry

out fundamental research while are also systematically considering

potential health applications. In fact, the participants in TR can

interact, playing different roles simultaneously. It has been argued

that the progress of biomedical research depends increasingly on

close collaboration between researchers, practitioners, medical insti-

tutions, patient communities, and research sponsors (Meslin et al.

2013). Here, the role of ‘boundary spanners’, that is, actors who fa-

cilitate communication across different communities, is particularly

important (Swann et al. 2007; Lander and Atkinson-Grosjean

2011). Boundary spanners are individuals who engage in significant

transactions with members of other communities, facilitate know-

ledge exchange between groups, and manage intergroup conflicts

(Richter et al. 2006).

From this perspective, clinical scientists working at the interface

between basic scientists and health practitioners could play a crucial

‘boundary spanning’ role, intermediating between the needs and ob-

jectives of the different actors, and conveying knowledge in a fast

and timely manner (Kelley et al. 2012). To be effective, clinical sci-

entists need management and coordination skills and knowledge of

the different ‘languages’ used by the diverse ‘epistemic cultures’ of

basic scientists and clinicians (Roberts et al. 2012). This is more eas-

ily achieved if individual researchers engage in both basic and clin-

ical work. It has been suggested that clinicians working at the

interface between basic scientists and the final beneficiaries of the re-

search (e.g. patients) can contribute to the establishment of new

partnering mechanisms with patients, with the objective of assessing

therapies and performing observational studies (Kelley et al. 2012).

However, the links among the diverse groups of actors involved in

the development of new drugs and therapies can be problematic.

From this perspective, the main objective of TR can be redefined.

Rather than focusing on bridging the gaps between successive stages

in the TR process, the emphasis is placed on the roles and inter-

actions of different actors who traditionally have fulfilled very speci-

alized and compartmentalized roles. For instance, basic scientists

may become involved in TR when conducting ‘user-inspired re-

search’ in close contact with, among others, clinicians and patients.

Another form of TR can emerge when patient organizations collab-

orate with researchers to identify research problems. Clinicians can

act as ‘boundary spanners’ helping others make connections. There

are many different ways to establish much closer working inter-

actions among the different actors in the research process. Instead of

seeing TR as addressing the problems that appear at specific points

in a traditional, staged, linear research system, in our approach, TR

addresses the separation between different groups of researchers and

stakeholders throughout the process, linking research to the devel-

opment and application of solutions to health problems. To do so,

TR focuses on processes—on how the sharing, exchange, and acqui-

sition of knowledge are articulated and how different actors get

involved in this process.

TR evaluation: gaps and lags versus proximities

Different ways of understanding the notion of TR open the way to a

variety of translational policies and initiatives with different object-

ives and logics. These various notions are associated also with differ-

ent ways of evaluating TR initiatives. In the preceding section, we

defined two main, contrasting, views of TR. They lead to different

approaches to the evaluation of TR.

Gaps and lags: evaluating the translational continuum If TR is seen as the attempt to bridge a series of sequential gaps that

hamper the translation of research results into socially beneficial ap-

plications, its evaluation should focus on the specific gaps that TR is

supposed to address rather than the whole R&D process. The suc-

cess of a TR programme can then be defined by the extent to which

it has reduced or bridged these gaps. The diversity of evaluation

techniques that have been proposed reflects the different definitions

of these gaps and the different indicators used to measure how well

they have been bridged. Morris et al. reviewed 23 TR evaluation

papers and concluded that ‘different studies use different measures,

of different things, at different time points’. The authors argue fur-

ther that ‘understanding lags first requires agreeing models, defin-

itions and measures, which can be applied in practice’ (Morris et al.

2011).

This perspective assumes that the key indicator to assess TR ini-

tiatives is the time it takes for the different translational gaps to be

bridged and, therefore, for the research to be translated into treat-

ments and other health-improving measures. Time lags are used also

by Trochim et al. (2011). They develop a generic approach to TR

evaluation and propose a flexible solution focusing on what they

consider to be the final objective of TR: the reduction in the time

needed to develop new clinical practices and drugs patients. They

adopt a generic linear TR model to identify ‘markers’ in the transla-

tion process and to assess the time that it takes for outputs to move

across these markers. The identification of these markers is flexible

and does not require a choice between one and another model of

TR. There is flexibility also in the direction of the activity across

markers, allowing for both ‘bench to bedside’, and ‘bedside to

bench’ movements. The approach suggested by Trochim et al.

focuses on the outputs of TR, and on the time it takes for the output

of a specific activity to be translated into a different type of output

identified by another marker. In other words, this form of

238 Research Evaluation, 2016, Vol. 25, No. 3

evaluation is concerned with TR outputs rather than how these out-

puts are achieved.

Several TR evaluations have adopted this procedure. Even when

the TR programmes themselves are unclear in the definition of their

goals, time lag studies have become an increasingly dominant ap-

proach to their evaluation. In these evaluations, the gap between the

different research phases and the outputs they generate ceases to be

an indicator of success and is converted into the objective of TR:

shortening the time between different research stages becomes what

TR is about. It could be argued that there is a risk of the tail of

evaluation wagging the dog of TR. In other words, how success is

measured may define the nature of the research objectives being pur-

sued. Further, TR evaluations that focus on measuring the time lags

between different research stages pay little attention to the standard-

ization of such lag measurements and to understanding the factors

underlying the measured lags (Morris et al. 2011). This presents

problems for evaluation practice; for instance, time lags may be

poorly assessed if the differences between scientific domains are not

considered in the estimation (Contopoulos-Ioannidis et al. 2003).

A focus on outputs and time lags has some advantages. For in-

stance, it allows the use of traditional indicators, such as scientific

publications, by linking them to a ‘marker’. This may be convenient

since traditional indicators are easily available, but their application

may reinforce some of the processes and practices that TR is sup-

posed to combat. For instance, basic scientists can be assessed ac-

cording to their published output, particularly the time it takes for

publication to take place; however, this takes no account of how the

outputs were generated, how the different research steps were

defined, and the roles of the different actors involved in the process.

An understanding of TR that takes account of how the research and

innovation processes are organized calls for a different evaluation

approach. In the next section, we formulate an evaluation frame-

work, which we argue is better suited to addressing the ways in

which TR initiatives affect the research and application processes.

Proximities: evaluating interaction processes Our proposal is to focus on how TR programmes affect the ways in

which research objectives are defined, research is conducted, and its

results applied in practice. We posit that TR initiatives attempt to

address problems in the organization and management of biomed-

ical research by bridging the divide between different actors involved

in the development of new drugs, therapies, diagnostics, or public

health practices. These actors include the doctors and patients

involved in the identification and definition of therapeutic and

health problems, the researchers defining and addressing the rele-

vant fundamental research challenges, and the clinicians and doctors

developing and testing solutions. These different groups belong to

different organizations, follow different implicit and explicit rules,

and respond to different sets of incentives and performance criteria.

These conflicting logics (Sauermann and Stephan 2013) and differ-

ent epistemic cultures (Knorr-Cetina 1999) can make it difficult to

align objectives and establish information flows. Stakeholders sepa-

rated by institutional and organizational boundaries find it difficult

to communicate needs and results. For instance, Ferlie et al. (2005)

show that maintaining institutional separation among different med-

ical professions is a major barrier to the development of medical

innovations.

Thus, TR initiatives can seek to reduce some of the divides

among biomedical researchers, clinical doctors, general

practitioners, regulators, etc. It is important to emphasize the net-

worked and non-linear nature of these social interactions; for in-

stance, basic research can be influenced by insights from general

practitioners and regulators, without the mediation of clinical doc-

tors. We would suggest that these interactions may be difficult to

operationalize due to the various types of distance between these dif-

ferent groups. Following Boschma (2005), learning processes and

knowledge exchange interactions are facilitated and strengthened by

five forms of proximity: cognitive, social, organizational, institu-

tional, and spatial.

Cognitive proximity reflects similarities in the way people per-

ceive, interpret, understand, and evaluate the world (Wuyts et al.

2005). A certain degree of cognitive proximity, that is, the extent to

which the actors share a similar knowledge base, is a prerequisite

for interactive learning, as it facilitates effective communication and

a common reference space to process and transfer complex informa-

tion and knowledge. However, as Nooteboom (2000) and Boschma

(2005) point out, both too much or too little cognitive proximity

can be detrimental to innovation and learning processes. A high level

of cognitive proximity can result in the exchange of redundant

knowledge due to the similarity in the knowledge sources. Too little

cognitive proximity can lead to the exchange of information that is

not fully understood by the actors, rendering the communication in-

effective. Thus, research on cognitive proximity claims that there is

an optimal level of cognitive proximity for interactions between

actors to be productive (Boschma and Frenken 2011).

Social proximity refers to relations built on common experience,

friendship, and kinship, which can facilitate interaction and commu-

nication based on mutual trust and reciprocity. As social proximity

between the actors increases, they know one another better commu-

nication becomes easier.

Organizational proximity refers to the governance structure

shaping the interactions between actors. High organizational prox-

imity is often associated with a hierarchical structure governing the

actors’ interactions, while low organizational proximity is generally

associated with a flat governance structure or arms’ length inter-

actions among the actors.

Institutional proximity refers to the norms, rules, and values that

influence how actors behave. Large institutional distance, promoting

behaviour that responds to different, potentially conflicting, sets of

incentives or values, may impose serious impediments to fruitful

learning interactions among the actors. For example, the institu-

tional distance between universities and firms is considerable be-

cause their incentives and norms differ significantly. Institutional

proximity contributes to reducing uncertainty among the actors,

even in the absence of previous social interaction (Lagendijk and

Lorentzen 2007). In the biomedical field, the institutional distance

between different occupational and professional boundaries repre-

sents a major barrier to the exchange of knowledge and the spread

of innovations (Ferlie et al. 2005; Currie and White 2012).

Finally, geographical proximity refers to the spatial or physical

distance between actors. This matters in knowledge dynamics be-

cause spatial co-location favours the exchange of knowledge that is

complex or difficult to transfer (i.e. tacit knowledge) (Ponds et al.

2007; Frenken et al. 2012; D’Este et al. 2013).

All these types of proximity are interrelated. Some may be com-

plementary, others may act as substitutes. For instance, Harrison

(1992) and Howells (2002) argue that geographical proximity facili-

tates face-to-face interactions, favouring trust-based relationships

and knowledge exchange, suggesting a reinforcing effect of spatial

Research Evaluation, 2016, Vol. 25, No. 3 239

proximity on social proximity. In contrast, some proximity dimen-

sions may be substitutes for each other: barriers to knowledge ex-

change posed by geographic distance might be overcome if the

interacting partners share a well-defined division of labour (i.e. or-

ganizational proximity) (Rallet and Torre 1999).

The main challenge for TR programmes is developing interven-

tions that generate a configuration of distances between actors that

is appropriate for the specific goals of the programmes. Distance

problems can be addressed along one or more of the dimensions re-

viewed above. For instance, initiatives can be designed to improve

communication and understanding between patients, clinicians, and

researchers (addressing cognitive distance), to introduce coordin-

ation mechanisms across different organizations involved in the

research and application processes (addressing organizational dis-

tance), to align their incentives and norms (addressing institutional

distance), or to improve trust and cohesion among actors (address-

ing social distance). In other words, different TR initiatives may be

aimed at reducing the different distances among the actors involved

in biomedical research and the application of its results.

TR evaluation: increasing proximities and ‘programme theories’

The first step of an alternative TR evaluation strategy is to deter-

mine the, implicit or explicit, ‘programme theory’ underpinning the

TR initiative under study. In other words, we need to define the pol-

icy goals in terms of the proximities that are to be improved and the

process by which such improvements are expected to occur. As

argued above, while the ‘gaps and lags’ approach to evaluation

focuses on outputs and the time it takes to progress from one output

to another, in a proximities approach, the attention is on the re-

search and knowledge utilization processes. This is in line with cur-

rent proposals to focus research impact evaluations on the processes

by which impacts are generated (Kok and Schuit 2012).

How TR programmes pursue their goals will vary across initia-

tives. For example, a TR project may be aimed at facilitating TR by

merging two laboratories. In this case, the TR programme logic is

focused on geographical and organizational proximity. In other

cases, laboratories (e.g. in a university and a hospital) are expected

to interact, but to remain separate, with joint activities organized to

foster knowledge exchange. In this case, the logic focuses on over-

coming institutional distance through social and cognitive proxim-

ities. Drawing on two contrasting Spanish examples, the August Pi i

Sunyer Biomedical Research Institute (IDIBAPS) centre and the

CIBER networks initiative, we show how the dimensions of proxim-

ity can be used to describe the different ‘programme theories’ of TR

initiatives.

Collaborative centres seeking to enhance interaction via

spatial proximity: the case of IDIBAPS IDIBAPS is a TR centre which is located opposite to the Clinic

Hospital of Barcelona. It houses some 460 researchers with diverse

institutional affiliations: the Clinic Hospital, the University of

Barcelona, Spanish and Catalan research establishments Spanish

National Research Council (CSIC) and Catalan Institution for

Research and Advanced Studies (ICREA), and its own staff. It pro-

vides a space for researchers with different affiliations and expertise

to work together on different themes (e.g. biomarkers for oncology),

using common facilities (e.g. bioinformatics, biobanks, imaging

instrumentation). IDIBAPS can be described as a TR initiative pri-

marily acting mainly upon spatial proximity through the co-location

of laboratories and shared access to research facilities. This spatial

proximity is expected to generate other forms of proximity; for in-

stance, IDIBAPS work practices are expected to enhance trust and

collaboration among the actors (thus increasing social proximity).

Ultimately, these proximities should facilitate knowledge flows

(increasing cognitive proximity) among different cognitive areas.

Therefore, the evaluation of an organization like IDIBAPS should

seek to determine whether and how expectations about interactions

and cross-fertilization are being met.

Collaboration networks seeking to enhance interaction

via organizational proximity: the CIBERs The CIBERs were established in 2006 by the Spanish Government

to promote excellence in biomedical research through the establish-

ment of stable cooperative research arrangements, which could be

defined as loosely coupled networks.4 The CIBERs were selected

based on an open call to biomedical research groups. Applicants had

to propose broad networks of research groups including universities,

public research organizations, hospitals, clinics, and research foun-

dations. Nine CIBER research networks5 were founded between

2006 and 2007, each focused on a specific pathology or disease, in

line with the strategic goals of the Instituto Carlos III (equivalent to

the US NIH), and with the explicit task of conducting TR.

Since members of the CIBERs continued to work in their own or-

ganizations, increasing spatial proximity was clearly not the object-

ive of the initiative. It was aimed at coordinating, through the

common legal and economic framework provided by the CIBER

platform, diverse biomedical groups within universities, hospitals,

and public research organizations. The CIBER platform provides an

articulated governance structure to catalyze coordinated actions

among the actors involved in the TR process. By connecting these

research groups through mechanisms and common practices and

decision-making processes agreed among all partners, the CIBER

platform aims to increase organizational proximity among a group

of heterogeneous research actors.

Although the groups belonging to a CIBER are only ‘loosely

coupled’, we can expect that, by setting some basic conditions for

the creation of common rules and shared expectations, the CIBERs

may provide the means to generate social proximity and through it

increase cognitive proximity. In other words, the CIBER networks

can be understood as an organizational arrangement, aimed primar-

ily at increasing the organizational proximity among the actors

(basic researchers, doctors, patient groups), who tend to be distant

in all dimensions. This greater organizational proximity should en-

hance social proximity, which should facilitate cognitive flows.

Analysing proximities and the implications for evaluating TR

The need for a programme theory TR addresses a problem that has organizational, social, and cogni-

tive roots: different communities with different practices and values

are involved in a process that is complex and difficult (Currie and

White 2012). The above two cases illustrate how two different ini-

tiatives, both labelled translational, have taken different approaches

and implicitly draw on different programme theories. The forms of

collaboration and interaction that we can expect from these

240 Research Evaluation, 2016, Vol. 25, No. 3

initiatives are very different, and we suggest that when evaluating

them we should aim at understanding these differences. We are

interested in investigating how research objectives and projects are

designed, how the research is conducted, and how the application of

research results is and carried out. In other words, we need to under-

stand the variety of the processes involved in TR initiatives.

We argue that a process-based approach to evaluation has ad-

vantages over an evaluation focusing on outputs. First, focusing on

outputs provides no information on why an output has or has not

been generated according to the initial, implicit or explicit, expect-

ations of stakeholders. Second, existing practices could be relabelled

‘translational’ if TR policies and their evaluation were concerned

only with the generation of outputs and their identification. In the

case of TR, where there is ambiguity about what differentiates it

from other research forms, understanding how interventions operate

in practice and what processes they trigger is particularly important.

If TR initiatives are to be transformative, they must implement

changes in the way research and the development of clinical prac-

tices and therapies are conducted. This calls for an approach that

goes beyond ‘linear evaluation’ of TR. It requires an evaluation ap-

proach that focuses on these processes. Third, evaluation frame-

works are not neutral in relation to the objectives of an initiative.

The way in which a project is evaluated will affect how it is con-

ducted and how its performers conceive their objectives. Focusing

on specific outputs can implicitly suggest an intervention rationale

that is not concerned with the organization of the research or how

specific ‘translational gaps’ are addressed.

The proximities framework we have proposed help to focus at-

tention on the way the research is conducted and the specific aspects

that the initiative is intended to address. These aspects may some-

times be made explicit in the definition of the intervention, but they

may also only be implied in relation to how the initiative is imple-

mented. In this latter case, the framework could be used to explore

and develop a ‘programme theory’ for a TR initiative; in other

words, help policymakers to reflect on and explore the initiative’s

rationale. The cases described show how the framework can be used

to describe both the goals of a TR initiative and how these goals can

be achieved.

In adopting this approach, we are proposing that the immediate

goal of a TR initiative is to address the problem of the distances sep-

arating the different groups involved in medical research. The ‘trans-

lational gaps’ are due to excessive distance in one or more

significant dimensions. The groups involved in the translational pro-

cess have cognitive differences, are institutionally separate, and,

therefore, follow different rules, are faced with different types of in-

centives, and often are geographically dispersed. The definition of

an initiative and its evaluation must be flexible and allow for the

fact that too much proximity is not always desirable. For instance,

too large a cognitive distance can be problematic, but so can too

much cognitive overlap; cognitive proximity is positive only up to a

certain level. A specific programme theory must reflect this problem

of balance, and the interpretation of evaluation results must be sen-

sitive to this potential problem if there is a possibility that it may be-

come relevant.

The programme theory of a TR initiative should define expect-

ations about whether and how changes in proximity in one or more

dimensions caused by the intervention, will trigger shifts in the other

dimensions, and the effects of these changes on the development and

application of beneficial goods and services. These effects will be

mediated by changes to the way research is carried out. Increased

proximity can result in increased collaboration among the groups

involved in the various tasks that constitute the TR process (the def-

inition of fundamental and clinical research objectives, research, and

the application of its results). We can expect changes in proximity to

generate new interactions across groups, for instance, between re-

search performers and the diverse users and beneficiaries of the re-

search results, where knowledge moves back and forth along

various channels, and within networks, rather than along a linear

bedside to bench continuum.

We can identify other building blocks of a TR programme the-

ory. An intermediate outcome of increased proximities may be the

generation of complex interactions among the different groups

involved in the TR process. Although this may vary across initia-

tives, it is important to take into account the broad variety of poten-

tial stakeholders: basic researchers, clinical researchers,

technologists, practitioners (doctors, nurses, etc.) public health and

private industry managers, and patients. The ways in which stake-

holder groups interact can be traced and analysed using instruments

developed for the evaluation of the socio-economic impact of re-

search, such as those developed by the EU-funded Social Impact

Assessment Methods through the study of Productive Interactions

(SIAMPI) project (Molas-Gallart and Tang 2011; Spaapen and van

Drooge 2011), which focuses on the processes of collaboration that

can be linked to an initiative.

Moving forward: the operationalization of a

proximities approach to TR evaluation This article has argued that a different approach to the evaluation of

TR is both possible and desirable and has shown that it will rely on

programme theories which will need to be developed for each differ-

ent TR initiative. The next step will be to operationalize the ap-

proach. While an output-based gaps and lags evaluation provides

easily measurable and potentially comparable results, a proximities

approach to evaluation will yield a description of the contextual

conditions in which the initiative was implemented and its effects on

the relevant processes. Narratives built on qualitative research tech-

niques could describe how the different proximities evolve through-

out an intervention. The narratives could be accompanied by

quantitative indicators; there are numerous examples from the in-

novation studies, economic geography, and management literatures

of attempts to measure different proximity dimensions. The main

approaches to developing proximity measures draw heavily on two

alternative sources: primary data mainly from surveys and fieldwork

research, and secondary data relying mainly on bibliographic and

patent information. Cognitive proximity has been measured as the

degree of overlap among the knowledge bases of potentially inter-

acting actors. This overlap can be captured at the individual level by

analysing the similarity between scientists’ publications or patents

profiles (Kotha et al. 2013). The cognitive distance between actors

can be captured by focusing on the patent subcategories in which

their discoveries are classified (Tzabbar 2009). Finally, science maps

offer a visualization of the overlap between actors across scientific

fields (Rafols et al. 2010) or patent categories (Kay et al. 2014).

Geographical proximity can be operationalized by measuring the

physical distance between the interacting partners, based on their

physical addresses (Laursen et al. 2011). Geographical information

based on postcodes allows the calculation of the ‘great circle’ dis-

tances between any two partners, while relative measures of dis-

tance, such as travel times between any two location points, have

Research Evaluation, 2016, Vol. 25, No. 3 241

also been used (Ejermo and Karlsson 2006). Institutional proximity

refers to whether actors belong to similar or distinct types of institu-

tions (e.g. universities, hospitals, public research organizations,

etc.). Bibliometric analysis may be useful to explore actors affiliated

to more than one institution (e.g. a hospital and a university).

Organizational proximity is often measured by whether actors be-

long to the same organizational structure and, thus, answer to the

same authority. A more detailed approximation of organizational

proximity can be obtained by measuring the degree to which the or-

ganization is characterized by a flatter or more vertical chain of

command (Ponds et al. 2007), and the relative position of actors in

these ‘hierarchies’. Social proximity can be proxied by the extent to

which actors have interacted in the past, for instance, via co-

authorship or co-inventorship, and the duration of such linkages

(McFayden et al. 2009). Softer measures based on social network re-

search can be captured through surveys enquiring about the extent

to which actors have developed strong ties with each other, reflect-

ing a relationship of kinship, trust, and reciprocity (McFayden and

Cannella 2004).

Thus, there is a variety of techniques that can be adapted for

use in evaluation research. Use of a proximities framework does

not determine the research techniques to be employed; these need

to fit the specific circumstances of the individual initiative being

assessed. The activities supported by a TR initiative will be differ-

ent, be implemented in different contexts, and will have different

targets and objectives. For instance, the research techniques

applied to an initiative that focuses mainly on cognitive issues, will

be different from those applied to one that addresses institutional

differences.

Since the adequacy of a specific research technique will depend

on the specific TR evaluation problem and its context, it follows

that the outputs of TR evaluations will not and should not be dir-

ectly comparable. Calls for an approach based on a single set of re-

search techniques, yielding measurable and comparable indicators

of TR ‘output’, from our perspective, are out of place. An evaluation

approach that focuses on processes will seek to provide detailed in-

formation on the effects of an initiative starting at the level of those

groups directly involved in it. However, the way that this informa-

tion is shaped, and the indicators on which it is based will depend

on the type of initiative, its objectives, and the types of proximities

the programme is designed to address.

Acknowledgements

The authors wish to acknowledge the support of the Generalitat Valenciana

Prometeo programme, grant number PROMETEO/2012/008, and of the UK

Medical Research Council, research grant MR/M00838X/1.

Conflict of interest statement. None declared.

Notes 1. Some analysts estimate that less than 10% of the most promising

biomedical discoveries resulted in any benefit to clinical practice

two decades later (Contopoulos-Ioannidis et al. 2003 Ioannidis

2004

2. www.ncats.nih.gov

3. CIBER is the Spanish acronym for ‘Centro de Investigación

Biomédica en Red’. http://www.isciii.es/ISCIII/es/contenidos/fd-

investigacion/fd-ejecucion/fd-centros-participados/fd-consor

cios2/cibers.shtml

4. Loosely coupled networks are organizational structures that

may help coordinate transactions among highly heterogeneous

partners, providing a balance between the mechanisms of con-

trol and flexibility. Loosely coupled networks lie somewhere be-

tween highly hierarchical organizational structures that impose

a strong degree of control and bureaucracy on learning-related

activities, and weakly articulated governance structures that

provide a fragile setting for building trust-based and sustainable

relationships.

5. Bioengineering, Biomaterials, and Nanomedicine (CIBER-BBN),

Epidemiology and Public Health (CIBER-ESP), Obesity and

Nutrition (CIBER-OBN), Hepatic and Digestive Diseases

(CIBER-EHD), Neurodegenerative Diseases (CIBER-NED),

Respiratory Diseases (CIBER-ES), Rare Diseases (CIBER-ER),

Mental Health (CIBER-SAM), and Diabetes and Metabolic

Associated Diseases (CIBER-DEM).

References

Boschma, R. (2005), ‘Proximity and Innovation: A Critical Assessment’,

Regional Studies, 39/1: 61–74.

Boschma, R. and Frenken K. (2011), ‘The Emerging Empirics of

Evolutionary Economic Geography’, Journal of Economic Geography,

11/2: 295–307.

Bozeman, B. and Boardman C. (2004), ‘The NSF Engineering Research

Centers and the University-Industry Research Revolution: A Brief History

Featuring an Interview with Erich Block’, The Journal of Technology

Transfer, 29/3–4: 365–75.

Consoli, D. and Mina A. (2009), ‘An Evolutionary Perspective on Health Innovation

Systems’, Journal of Evoluationary Economics, 19/2: 297–319.

Contopoulos-Ioannidis, D. G., Ntzani E. E. and Ioannidis J. P. A. (2003),

‘Translation of Highly Promising Basic Science Research into Clinical

Applications’, The American Journal of Medicine, 114/6: 477–84.

Currie, G. and White L. (2012), ‘Inter-Professional Barriers and Knowledge

Brokering in an Organizational Context: The Case of Healthcare’,

Organization Studies, 33/10: 1333–61.

D’Este, P., Guy F. and Iammarino S. (2013), ‘Shaping the Formation of

University-Industry Research Collaborations: What Type of Proximity Does

Really Matter?’, Journal of Economic Geography, 13/4: 537–58.

Davis, D. et al. (2003), ‘The Case for Knowledge Translation: Shortening the

Journey from Evidence to Effect’, British Medical Journal, 327: 33–5.

Dougherty, D. and Heller T. (1994), ‘The Illegitimacy of Successful Product

Innovation in Established Firms’, Organization Science, 5/2: 200–18.

Drolet, B. C. and Lorenzi N. M. (2011), ‘Translational Research:

Understanding the Continuum from Bench to Bedside’, Translational

Research, 157/1: 1–5.

Ejermo, O. and Karlsson C. (2006), ‘Interregional Inventor Networks as

Studied by Patent Coinventorships’, Research Policy, 35/3: 412–30.

Ferlie, E. et al. (2005), ‘The Nonspread of Innovations: The Mediating Role of

Professionals’, Academy of Management Journal, 48/1: 117–34.

Graham, I. D., et al. (2006), ‘Lost in Knowledge Translation: Time for

a Map?’, The Journal of Continuing Education in the Health Professions,

26/1: 13–24.

Harrison, B. (1992), ‘Industrial Districts: Old Wine in New Bottles?’,

Regional Studies, 26/5: 469–83.

Heller, C. and de Melo-Martı́n I. (2009), ‘Clinical and Translational Science

Awards: Can the Increase the Efficiency and Speed of Clinical Translational

Research?’, Academic Medicine, 84/4: 424–32.

Hobin, J. et al. (2012), ‘Engaging Basic Scientists in Translational Research:

Identifying Opportunities, Overcoming Obstacles’, Journal of Translational

Medicine, 10/1: 72.

Howells, J. R. (2002), ‘Tacit Knowledge, Innovation and Economic

Geography’, Urban Studies, 39/5–6: 871–84.

242 Research Evaluation, 2016, Vol. 25, No. 3

Ioannidis, J. (2004), ‘Materializing Research Promises: Opportunities,

Priorities and Conflicts in Translational Medicine’, Journal of Translational

Medicine, 2/1: 5.

Kay, L. et al. (2014), ‘Patent Overlay Mapping: Visualizing Technological

Distance’, Journal of the Association for Information Science and

Technology, 65/12: 2432–43.

Kelley, M. et al. (2012), ‘Values in Translation: How Asking the Right

Questions can Move Translational Science Toward Greater Health Impact’,

Clinical and Translational Science, 5/6: 445–51.

Khoury, M. J. et al. (2007), ‘The Continuum of Translation Research in

Genomic Medicine: How can we Accelerate the Appropriate Integration of

Human Genome Discoveries into Health Care and Disease Prevention?’,

Genetics in Medicine, 9/10: 665–74.

Knorr-Cetina, K. (1999), Epistemic Cultures: How the Sciences Make

Knowledge. Harvard University Press, Cambridge, Massachusetts.

Kok, M. O. and Schuit A. J. (2012), ‘Contribution Mapping: A Method for

Mapping the Contribution of Research to Enhance its Impact’, Health

Research Policy and Systems, 10/21: 1–16.

Kotha, R., George G. and Srikanth K. (2013), ‘Bridging the Mutual

Knowledge Gap: Coordination and the Commercialization of University

Science’, Academy of Management Journal, 56/2: 498–524.

Lagendijk, A. and Lorentzen A. (2007), ‘Proximity, Knowledge and

Innovation in Peripheral Regions. On the Intersection Between

Geographical and Organizational Proximity’, European Planning Studies,

15/4: 457–66.

Lander, B. and Atkinson-Grosjean J. (2011), ‘Translational Science and the

Hidden Research System in Universities and Academic Hospitals: A Case

Study’, Social Science & Medicine, 72/4: 537–44.

Laursen, K., Reichstein T. and Salter A. (2011), ‘Exploring the Effect of

Geographical Proximity and University Quality on University-Industry

Collaboration in the United Kingdom’, Regional Studies, 45/4: 507–23.

Littman, B. et al. (2004), ‘Lost in Translation: Obstacles to Translational

Medicine’, Journal of Translational Medicine, 2/1: 14.

Littman, B. et al. (2007), ‘What’s Next in Translational Medicine?’, Clinical

Science, 112: 217–27.

Marincola, F. M. (2003), ‘Translational Medicine: A Two-Way Road’,

Journal of Translational Medicine, 1/1: 1.

Marincola, F. M. (2011), ‘The Trouble with Translational Medicine’, Journal

of Internal Medicine, 270/2: 123–27.

McFayden, M. A. and Cannella A. A. (2004), ‘Social Capital and Knowledge

Creation: Diminishing Returns of the Number and Strength of Exchange’,

Academy of Management Journal, 47/5: 735–46.

McFayden, M. A., Semadeni M. and Cannella A. A. (2009), ‘Value of Strong

Ties to Disconnected Others: Examining Knowledge Creation in

Biomedicine’, Organization Science, 20/3: 552–64.

Meslin, E. M., Blasimme A. and Cambon-Thomsen A. (2013), ‘Mapping the

Translational Science Policy “Valley of Death"’, Clinical and Translational

Medicine, 2/1: 14.

Molas-Gallart, J. and Tang P. (2011), ‘Tracing “Productive Interactions” to

Identify Social Impacts: An Example for the Social Sciences’, Research

Evaluation, 20/3: 219–26.

Morris, Z. S., Wooding S. and Grant J. (2011), ‘The Answer is 17 Years, What

is the Question: Understanding Time Lags in Translational Research’,

Journal of the Royal Society of Medicine, 104/12: 510–20.

Nooteboom, B. (2000), Learning and Innovation in Organizations And

Economies. Oxford: Oxford University Press.

Ponds, R., Oort F. and Frenken K. (2007), ‘The Geographical and Institutional

Proximity of Research Collaboration’, Papers in Regional Science, 86/3:

423–43.

Rafols, I., Porter A. L. and Leydesdorff L. (2010), ‘Science Overlay Maps:

A New Tool for Research Policy and Library Management’, Journal of

the American Society for Information Science and Technology, 61/9:

1871–87.

Rallet, A. and Torre A. (1999), ‘Is Geographical Proximity Necessary in the

Innovation Networks in the Era of Global Economy?’, GeoJournal, 49/4:

373–80.

Rey-Rocha, J. and Martı́n-Sempere M. J. (2012), ‘Generating Favourable

Contexts for Translational Research Through the Incorporation of Basic

Researchers into Hospitals: The FIS/Miguel Servet Research Contract

Programme’, Science and Public Policy, 39/6: 787–801.

Richter, A. W. et al. (2006), ‘Boundary Spanners’ Identification, Intergroup

Contact, and Effective Intergroup Relations’, Academy of Management

Journal, 49/6: 1252–69.

Roberts, S. F. et al. (2012), ‘Perspective: Transforming Science into Medicine:

How Clinician-Scientists can Build Bridges Across Research’s “Valley of

Death"’, Academic Medicine, 87/3: 266–70.

Rogers, E. M. (2003), Diffusion of Innovations. New York: Free Press.

Rubio, D. M. et al. (2010), ‘Defining Translational Research: Implications for

Training’, Academic medicine: journal of the Association of American

Medical Colleges, 85/3: 470–75.

Sauermann, H. and Stephan P. (2013), ‘Conflicting Logics? A

Multidimensional View of Industrial and Academic Science’, Organization

Science, 24/3: 889–909.

Spaapen, J. and van Drooge L. (2011), ‘Productive Interactions in

the Assessment of Social Impact of Research’, Research Evaluation, 20/3:

211–18.

Stokes, D. E. (1997), Pasteur’s Quadrant: Basic Science and Technological

Innovation. Washington, DC: Brookings Institution Press.

Sung, N. S. et al. (2003), ‘Central Challenges Facing the National Clinical

Research Enterprise’, JAMA: The Journal of the American Medical

Association, 289/10: 1278–87.

Swann, J. et al. (2007), ‘Modes of Organizing Biomedical Innovation in the

UK and US and the Role of Integrative and Relational Capabilities’,

Research Policy, 36/4: 529–47.

Trochim, W. et al. (2011), ‘Evaluating Translational Research: A Process

Marker Model’, Clinical and Translational Science, 4/3: 153–62.

Tzabbar, D. (2009), ‘When does Scientist Recruitment Affect Technological

Repositioning?’, Academy of Management Journal, 52/5: 873–96.

Van der Laan, A. L. and Boenink M. (2012), ‘ Beyond Bench and Bedside:

Disentangling the Concept of Translational Research’, Health Care

Analysis, 1–18.

Westfall, J. M., Mold J. and Fagnan L. (2007), ‘Practice-based Research –

“Blue Highways” on the NIH roadmap’, JAMA: The Journal of the

American Medical Association, 297/4: 403–6.

Woolf, S. H. (2008), ‘The Meaning of Translational Research and Why it

Matters’, JAMA: The Journal of the American Medical Association, 229/2:

211–13.

Wuyts, S. et al. (2005), ‘Empirical Tests of Optimal Cognitive Distance’,

Journal of Economic Behavior and Organization, 58/2: 277–302.

Zerhouni, E. (2003), ‘The NIH Roadmap’, Science, 302, 63–72.

Zerhouni, E. (2007), ‘Translational Research: Moving Discovery to Practice’,

Clinical Pharmacology and Therapeutics, 81/1: 126–28.

Research Evaluation, 2016, Vol. 25, No. 3 243

Copyright of Research Evaluation is the property of Oxford University Press / USA and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.