Annotated bibliography first batch

tiggerbusterdngh

milberg-2014-applied_vegetation_science.pdf

Home >Business & Finance homework help >Economics homework help >Annotated bibliography first batch

Applied Vegetation Science 17 (2014) 604–608

FORUM Evidence-based vegetation management: prospects and challenges

Per Milberg

Keywords

Applied research; Evidence based; Knowledge

transfer; Manager; Metaanalysis

Received 29 October 2013

Accepted 28 January 2014

Co-ordinating Editor: Michael Palmer

Milberg, P. ([email protected]): IFM Biology,

Conservation Ecology Group, Link€oping

University, Link€oping, SE-581 83, Sweden

Abstract

The effect of applied vegetation science on society has the potential to increase

by adopting an evidence-based approach. However, this would require a shift in

focus towards effect size and results suitable for meta-analyses, a focus on practi-

tioners as potential readers, more emphasis on practical problems rather than

mechanism, and an acceptance of all well-executed experimental studies, even

if confirmatory. Thus, the prevailing editorial policies need to be reconsidered,

as well as the methods of analysing, reporting and evaluating research, for our

research efforts to be of better use within society.

Background

Some of us work in applied research, but what does

‘applied’ actually mean? I prefer to think of ‘applied

research’ as being of more direct interest to society and that

there are – beyond fellow researchers – two potential

groups targeted by such research (Cook et al. 2013) 1 . The

first group is policy-makers, a group that consists of people

who prepare and make new laws, as well as government

or company officials who set up rules for activities in

society or within their organization. The second group is

managers, which consists of people who make operational

decisions and their advisors (e.g. medical doctors, teach-

ers, foresters, farmers, extension officers). It is our hope

that the managers within our field regularly read Applied

Vegetation Science and similar journals. However, the pro-

cess of knowledge transfer from applied research to prac-

tice is often disappointing (e.g. Nutley et al. 2007; Braun &

Hadwiger 2011; Dagenais et al. 2012; Rojek et al. 2012).

Poor or slow knowledge transfer indicates missed opportu-

nities and a waste of resources, both in science and society.

But there are ways in which we researcher, the ‘donors’ of

knowledge, can facilitate this process.

In our field, managers who are potentially interested in

our work are often highly educated. However, do we have

them in mind when we write our papers? Or is it that our

focus has gradually shifted from the potential end-user of

our findings to a concentration on the continuously tight-

ening requirements by the scientific community to pass

the editorial and review processes?

A reform in the way we think about, analyse and

present our applied research is welcome, and it would

affect authors, referees and editorial policies. With such

a reform, our research might have a greater impact in

society, and this is what applied research should strive to

achieve.

An example of the failure to communicate

I recently experienced how my own research and that

of others was ignored (or not known) by policy-makers

and practitioners: a notion that I believe not to be uncom-

mon among applied researchers (Cook et al. 2013). More-

over, I realized that the fault was not entirely theirs:

nice ordination analyses do not communicate well with

busy managers. Furthermore, the results that I had pub-

lished did not provide any estimate of effect size, which

is often the main focus of a manager. Or put another

way, a significant P-value is of much less interest than

a number showing how much two treatments differ (Di

Stefano et al. 2005; Cumming 2012). It is only with an

effect size that a manager can properly weigh the costs

against benefits. I should have considered practitioners

when publishing, but seem to have forgotten that anyone

outside academia might be interested. As a minimum, I

could have formulated clear and explicit recommendations

for practitioners (Memmott et al. 2010; Simonetti 2011).

So, I have no reason to blame the policy-makers and

practitioners that failed to find my study in the vast ocean

of published research. 1 Research also has an educational role towards the general public, but this

role is a joint role of both applied and basic research.

Learning from reforms in other fields of applied

research

Other fields of applied research have embraced evidence-

based management (Hansen & Rieper 2009). In short,

management (or policy) should be based on the best

available knowledge. This might appear as a rather trivial

statement, but new findings often take considerable time

before translating into action, thereby wasting the

resources society has invested in research. For example,

Gilbert et al. (2005) estimated that if a recommendation

(sleeping position of infants) had been changed when the

evidence was available, rather than after 25 yrs, 10,000

lives could have been saved in the UK alone. In another

example, a systematic review showed that costly methods

used (to increase salmonid fish abundance by in-stream

structures) for 80 yrs were of rather doubtful value

(Stewart et al. 2009). Within medicine, all parties includ-

ing taxpayers, insurance companies and patient’s next-of-

kin, expect doctors to make a well-informed decision. The

medical field is also where the evidence-based movement

has experienced its greatest achievements: evidence-based

medicine was voted among one of the ten most important

medical advances during the last centuries (Ferriman

2007). Aided by meta-analyses, systematic reviews are the

foundation of evidence-based medicine. In such a review,

which focuses on a specific question rather than a conven-

tional review, a literature search is performed systemati-

cally, and studies are selected for inclusion according to

predefined criteria and, most often, the published numbers

are entered into meta-analyses. Thus, all relevant informa-

tion can be quantitatively summarized, and doctors can

base their treatment alternatives on such reviews.

Not all fields have such simple outcome variables as

medicine, and might involve more complex decisions by

managers. Furthermore, meta-analysis and systematic

reviews has created new challenges (Kueffer et al. 2011;

Lindenmayer & Likens 2013). However, this does not

preclude benefits to society by adopting an evidence-based

approach within an area (e.g. Hattie 2009). Furthermore,

within the last several years, evidence-based move-

ments within environmental management have emerged

(www.environmentalevidence.org, www.eviem.se, www.

cebc.bangor.ac.uk). In fact, there are already a number of

published systematic reviews focused on vegetation man-

agement (e.g. Newton et al. 2009; Kettenring & Adams

2011; Humbert et al. 2012).

The new statistics

Importantly, a ‘statistical reform’ is currently underway,

where a shift away from P-values to a focus on effect sizes

can be observed (Fidler et al. 2004; McCloskey & Ziliak

2009; Cumming 2012). This is a cornerstone in evidence-

based management and goes hand-in-hand with meta-

analyses. As an indication of how far this reform has come

in some fields, Epidemiology, a major journal in its field,

already in 1998 stated in the instructions for authors that,

“When writing for Epidemiology, you can also enhance your

prospects if you omit tests of statistical significance. . . In

Epidemiology, we do not publish them at all” (Rothman

1998). In contrast, ecologists seem to continue to think that

null hypotheses and P-values are essential for publication.

Neither meta-analyses nor ‘statistical reform’ is new to

ecology (e.g. Fidler et al. 2004; Koricheva et al. 2013;

Vetter et al. 2013). The use of meta-analyses among ecolo-

gists, however, has not been within the context of evi-

dence-based management, but rather to support reviews

primarily aimed for other researchers. So time should be

ripe for its usage also in management-related questions,

with a focus on (i) ‘what works best’ (rather than on pro-

cesses and mechanisms) and on (ii) practitioners as poten-

tial readers (rather than on researchers only). Another

new consideration, which has bearing on all research pub-

lished, is to ensure that your data presentation allows

inclusion in future meta-analyses (i.e. a focus on effect

sizes rather than P-values). Imagine the horror when you

realize that all of your research effort is nullified by being

excluded from the next systematic review. Thus, the

presence of systematic reviews has resulted in greater con-

formity in how the medical researcher decides to design

and particularly analyse and present data from clinical

trials (and more recently, also in placing data in data

repositories). In contrast, ecologists seem to strive for

diversity in analysis and presentation.

New tools needed to enable meta-analysis of

vegetation data

Vegetation is complex, often species-rich, and thus com-

plex to analyse, and our field has a long history of relying

on various multivariate methods of analysis (e.g. Kent &

Ballard 1988; Masing 1994). However, because the

vegetation composition varies over sites and situations, it

is not always easy to analytically compare results of

experiments from different studies with multivariate

methods. Of course, there are other methods to simplify

data that might be more appropriate for meta-analysis

(e.g. Diekmann 2003; Milberg et al. 2014), and which can

be used to supplement more conventional multivariate

analyses. A substantial challenge for evidence-based

vegetation management is the development of a ‘common

currency’ suitable for meta-analysis. It might involve

increased effort in classification of species into desired/

undesired for particular management goals (Milberg et al.

2014).

P. Milberg Evidence-based vegetation management

A reform has formidable enemies

It is somewhat paradoxical that despite the existence of

more journals and excellent literature search facilities, the

standards for publication have considerably tightened

and rejection rates have increased (Hochberg et al. 2009;

Jackson 2009). A few years ago, 75% of papers were

rejected by Applied Vegetation Science (Chiarucci et al.

2010), and this would be a typical value of an ecological

journal (Pautasso & Sch€afer 2010). The increasing rejec-

tion rates, noted by many (e.g. Jackson 2009; Statzner &

Resh 2010) indicate that much more effort has to go into a

study than once required (Campos-Arceiz et al. 2013).

There is also a risk that sound trials that have been con-

ducted remain unpublished (Scherer et al. 2007).

When editors have many manuscripts to choose from,

the key for a successful manuscript in most journals is no

longer whether a study is well conducted and with justified

conclusions, but the somewhat subjective and elusive

‘novelty factor’. Thus, if a study is merely confirmatory, as

judged by referees, or with non-significant results, then it

is less likely to be published (Dwan et al. 2008; Hopewell

et al. 2009). In the history of scientific publishing, this can

be seen as a shift in focus from documenting research

(‘anything that is well described and with conclusions

that are justified is OK’), to publishers and editors trying

to maximize profit and bibliometric outcomes, respec-

tively (e.g. Wellcome Trust 2003; Falagas & Alexiou 2008;

Statzner & Resh 2010). Considering the enormous volume

of published literature per year, we as readers might wel-

come this filtering of findings. However, knowledge is not

well built if only based on novelty and the extraordinary

or when there is a strong bias towards ‘significant’ results

(e.g. Ioannidis 2005; Knight 2006; Moonesinghe et al.

2007; Ridley et al. 2007; Fang & Casadevall 2011; Fanelli

2012; Giner-Sorolla 2012; Brodeur et al. 2013; Schoenfeld

& Ioannidis 2013).

Evidence-based management works best when all

well-executed studies are published and without delay.

In reality, there is a bias favouring exceptional results

while punishing the ‘uninteresting’. Thus, a reform would

need a resurrection of the value of confirmatory studies

(Asendorpf et al. 2013). In addition, we need to find a way

to document non-significant results (e.g. Kotze et al.

2004) that are of interest, at least if a study appears to have

been properly replicated.

Reform for both applied vegetation science and

Applied Vegetation Science?

Taken together, to enable the emergence of evidence-

based vegetation management, researchers need the fol-

lowing: (i) a shift in focus towards effect size and results

that are suitable for meta-analysis; (ii) to consider practi-

tioners as potential readers; (iii) more focus on practical

problems rather than mechanism; and (iv) acceptance of

well-executed confirmatory studies.

These above points have implications for editorial poli-

cies of all journals claiming to be a source for applied

research. So if Applied Vegetation Science wants to fully live

up to its name, why not an editorial demand for papers

with practical relevance, to present results in a manner

that enables future meta-analysis and to allow space for

confirmatory studies? Both Journal of Applied Ecology and

Ecological Applications claim practitioners as a target audi-

ence, why should Applied Vegetation Science be different?

And it would be excellent if Applied Vegetation Science aimed

to be an avenue for publication of systematic reviews, as

do the journals Biological Conservation and Environmental

Evidence.

Acknowledgements

I would like to thank referees and Lars Westerberg for

comments on an earlier version of the manuscript, and

extend the thanks also to other colleagues with whom I

have discussed ‘knowledge transfer’, statistical reform and

publication policy.

References

Asendorpf, J.B., Conner, M., De Fruyt, F., De Houwer, J., Denis-

sen, J.J.A., Fiedler, K., Fiedler, S., Funder, D.C., Kliegl, R.,

(. . .) &Wicherts, J.M. 2013. Recommendations for increasing

replicability in psychology. European Journal of Personality 27:

108–119.

Braun, S. & Hadwiger, K. 2011. Knowledge transfer from

research to industry (SMEs): an example from the food

sector. Trends in Food Science & Technology 22: S90–S96.

Brodeur, A., L�e, M., Sangnier, M. & Zylberberg, Y. 2013. Star

Wars: The empirics strike back. Forschungsinstitut zur Zukunft

der Arbeit, Discussion Paper Series, No 7268.

Campos-Arceiz, A., Koh, L.P. & Primack, R.B. 2013. Are conser-

vation biologists working too hard? Biological Conservation

166: 186–190.

Chiarucci, A., P€artel, M., D�ıaz, S. & Wilson, J.B. 2010. Applied

Vegetation Science in 2010: new opportunities for the

vegetation scientists. Applied Vegetation Science 13: 1–4.

Cook, C.N., Possingham, H.P. & Fuller, R.A. 2013. Contribution

of systematic reviews to management decisions. Conservation

Biology 27: 902–915.

Cumming, G. 2012. Understanding the New Statistics: effect sizes, con-

fidence intervals, and meta-Analysis. Routledge, New York, NY,

USA.

Dagenais, C., Lysenko, L., Abrami, P.C., Bernard, R.M., Ramde,

J. & Janosz, M. 2012. Use of research-based information by

school practitioners and determinants of use: a review of

Evidence-based vegetation management P. Milberg

empirical research. Evidence & Policy: A Journal of Research,

Debate and Practice 8: 285–309.

Di Stefano, J., Fidler, F. & Cumming, G. 2005. Effect size esti-

mates and confidence intervals: an alternative focus for the

presentation and interpretation of ecological data. In: Burke,

A.G. (ed.) New trends in ecology research, pp. 71–102. NOVA

Publishers, New York, NY, US.

Diekmann, M. 2003. Species indicator values as an important

tool in applied plant ecology: a review. Basic and Applied Ecol-

ogy 4: 493–506.

Dwan, K., Altman, D.G., Arnaiz, J.A., Bloom, J., Chan, A.-W.,

Cronin, E., Decullier, E., Easterbrook, P.J., Von Elm, E., (. . .)

&Williamson, P.R. 2008. Systematic review of the empirical

evidence of study publication bias and outcome reporting

bias. PLoS ONE 3: e3081.

Falagas, M.E. & Alexiou, V.G. 2008. The top-ten in journal

impact factor manipulation. Archivum Immunologiae et Thera-

piae Experimentalis 56: 223–226.

Fanelli, D. 2012. Negative results are disappearing from most dis-

ciplines and countries. Scientometrics 90: 891–904.

Fang, F.C. & Casadevall, A. 2011. Retracted science and the

Retraction Index. Infection & Immunity 79: 3855–3859.

Ferriman, A. 2007. BMJ readers choose the “sanitary revolution”

as greatest medical advance since 1840. British Medical Jour-

nal 334: 111.2.

Fidler, F., Cumming, G., Burgman, M. & Thomason, N. 2004.

Statistical reform in medicine, psychology and ecology.

Journal of Socio-Economics 33: 615–630.

Gilbert, R., Salanti, G., Harden, M. & See, S. 2005. Infant sleep-

ing position and the sudden infant death syndrome: system-

atic review of observational studies and historical review of

recommendations from 1940 to 2002. International Journal of

Epidemiology 34: 874–887.

Giner-Sorolla, R. 2012. Science or art? How aesthetic standards

grease the way through the publication bottleneck but under-

mine science. Perspectives on Psychological Science 7: 562–571.

Hansen, H.F. & Rieper, O. 2009. The evidence movement: the

development and consequences of methodologies in review

practices. Evaluation 15: 141–163.

Hattie, J. 2009. Visible learning: a synthesis of over 800 meta-analyses

relating to achievement. Routledge, New York, NY, USA.

Hochberg, M.E., Chase, J.M., Gotelli, N.J., Hastings, A. & Naeem,

S. 2009. The tragedy of the reviewer commons. Ecology Letters

12: 2–4.

Hopewell, S., Loudon, K., Clarke, M.J., Oxman, A.D. & Dicker-

sin, K. 2009. Publication bias in clinical trials due to statistical

significance or direction of trial results. Cochrane Database of

Systematic Reviews 2009, Issue 1. Art. No.: MR000006.

Humbert, J.-Y., Pellet, J., Buri, P. & Arlettaz, R. 2012. Does

delaying the first mowing date benefit biodiversity in mead-

owland? Environmental Evidence 1: 9.

Ioannidis, J.P.A. 2005. Why most published research findings

are false. PLoS Medicine 2: e124.

Jackson, M.B. 2009. AoB PLANTS: origins and features. AoB

PLANTS 1: 2.

Kent, M. & Ballard, J. 1988. Trends and problems in the applica-

tion of classification and ordination methods in plant ecol-

ogy. Vegetatio 78: 109–124.

Kettenring, K.M. & Adams, C.R. 2011. Lessons learned from

invasive plant control experiments: a systematic review and

meta-analysis. Journal of Applied Ecology 48: 970–979.

Knight, A.T. 2006. Failing but learning: writing the wrongs after

Redford and Taber. Conservation Biology 20: 1312–1314.

Koricheva, J., Gurevitch, J. & Mengersen, K. 2013. Handbook of

meta-analysis in ecology and evolution. Princeton University

Press, Princeton, NJ, US.

Kotze, D.J., Johnson, C.A., O’Hara, R.B., Veps€al€ainen, K. & Fow-

ler, M.S. 2004. Editorial: the journal of negative results in

ecology and evolutionary biology. Journal of Negative Results:

Ecology & Evolutionary Biology 1: 1–5.

Kueffer, C., Niinemets, €U., Drenovsky, R.E., Kattge, J., Milberg,

P., Poorter, H., Reich, P.B., Werner, C., Westoby, M. &

Wright, I.J. 2011. Fame, glory and neglect in meta-analyses.

Trends in Ecology & Evolution 26: 493–494.

Lindenmayer, G. & Likens, G.E. 2013. Benchmarking open

access science against good science. Bulletin of the Ecological

Society of America 94: 338–340.

Masing, V. 1994. Approaches, levels and elements of vegetation

research. Folia Geobotanica et Phytotaxonomica 29: 531–541.

McCloskey, D.N. & Ziliak, S.T. 2009. The unreasonable ineffec-

tiveness of Fisherian “tests” in biology, and especially in

medicine. Biological Theory 4: 44–53.

Memmott, J., Cadotte, M., Hulme, P.E., Kerby, G., Milner-

Gulland, E.J. & Whittingham, M.J. 2010. Putting applied

ecology into practice. Journal of Applied Ecology 47: 1–4.

Milberg, P., Akoto, B., Bergman, K.-O., Fogelfors, H., Paltto, H. &

T€alle, M. 2014. Is spring burning a viable management tool for

species-rich grasslands? Applied Vegetation Science 17: 429–441.

Moonesinghe, R., Khoury, M.J. & Janssens, A.C.J.W. 2007.

Most published research findings are false: but a little replica-

tion goes a long way. PLoS Medicine 4: e28.

Newton, A.C., Stewart, G.B., Myers, G., Diaz, A., Lake, S.,

Bullock, J.M. & Pullin, A.S. 2009. Impacts of grazing on low-

land heathland in north-west Europe. Biological Conservation

142: 935–947.

Nutley, S.M., Walter, I. & Davies, H.T.O. 2007. Using evidence: how

research can inform public services. The Policy Press, Bristol, UK.

Pautasso, M. & Sch€afer, H. 2010. Peer review delay and selectiv-

ity in ecology journals. Scientometrics 84: 307–315.

Ridley, J., Kolm, N., Freckelton, R.P. & Gage, M.J.G. 2007. An

unexpected influence of widely used significance thresholds

on the distribution of reported P-values. Journal of Evolution-

ary Biology 20: 1082–1089.

Rojek, J., Alpert, G. & Smith, H. 2012. The utilization of research

by the police. Police Practice and Research 13: 329–341.

Rothman, K.J. 1998. Writing for epidemiology. Epidemiology 9:

333–337.

Scherer, R.W., Langenberg, P. & von Elm, E. 2007. Full publica-

tion of results initially presented in abstracts. Cochrane Data-

base of Systematic Reviews , Issue 2. Art. No.: MR000005.

P. Milberg Evidence-based vegetation management

Schoenfeld, J.D. & Ioannidis, J.P.A. 2013. Is everything we eat

associated with cancer? A systematic cookbook review.

American Journal of Clinical Nutrition 97: 127–134.

Simonetti, J.A. 2011. Conservation biology in Chile: Are we

fulfilling our social contract? Revista Chilena de Historia

Natural 84: 161–170.

Statzner, B. & Resh, V.H. 2010. Negative changes in the scientific

publication process in ecology: potential causes and conse-

quences. Freshwater Biology 55: 2639–2653.

Stewart, G.B., Bayliss, H.R., Showler, D.A., Sutherland, W.J. &

Pullin, A.S. 2009. Effectiveness of engineered in-stream struc-

ture mitigation measures to increase salmonid abundance: a

systematic review. Ecological Applications 19: 931–941.

Vetter, D., Rucker, G. & Storch, I. 2013. Meta-analysis: a need

for well-defined usage in ecology and conservation biology.

Ecosphere 4: 74.

Wellcome Trust. 2003. Economic analysis of scientific research

publishing. A report commissioned by the Wellcome Trust. Avail-

able at www.wellcome.ac.uk/About-us/Publications/Publi

cationsA-Z/ Accessed 28 October 2013.

Evidence-based vegetation management P. Milberg