Week 2 - Assignment: Apply Critical Reading and Techniques for Notes

Journal of University Teaching & Learning Practice Volume 15 Issue 4 Research Skill Development spanning Higher Education: Curricula, critiques and connections

Article 1

2018

Research skill development spanning higher education: Critiques, curricula and connections John W. Willison Dr University of Adelaide, [email protected]

Follow this and additional works at: https://ro.uow.edu.au/jutlp

Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library: [email protected]

Recommended Citation Willison, John W. Dr, Research skill development spanning higher education: Critiques, curricula and connections, Journal of University Teaching & Learning Practice, 15(4), 2018. Available at:https://ro.uow.edu.au/jutlp/vol15/iss4/1

Research skill development spanning higher education: Critiques, curricula and connections

Abstract The Research Skill Development (RSD) framework was devised in 2006 to articulate what educators do when they facilitate student learning through active exploration in ways that enable their skills to grow in sophistication and rigour. This special issue of JUTLP comprises eight articles that focus on the critique, adaptation and application of the RSD. This article introduces the special issue and presents the 2018 version of the RSD, in response to 12 years of feedback on the framework. Changes in the RSD include improvements in articulation of the facets of research, better delineation of autonomy, and the addition of the affective domain and guiding questions.

Of the other seven articles, five are set in the undergraduate years and two in master’s, with contexts including biology, education, engineering, humanities and interdisciplinary studies. The articles are based across diverse settings, including laboratory, online, language and workplace learning. Of these seven, one critiques the RSD, three focus on curriculum design and three connect the RSD to enduring issues of current concern by adapting the framework for academic literacy, work skills and problem solving. These three adaptations are examples of numerous emerging models that modify RSD terminology and shape, within its broad parameters, and are introduced in this issue as the Models of Engaged Learning and Teaching (MELT).

Keywords Research Skill Development, student autonomy, teacher autonomy, sophisticated thinking, Models of Engaged Learning and Teaching, collaborative partnerships, connecting curricula, cognitive and affective

Cover Page Footnote On behalf of all the authors, I thank the peer reviewers of the papers in this Special issue of JUTLP: Said Al- Sarawi, Ryan Banow, Jack Bowers, Cally Guerin, Michele Jacobsen, Hanno van Keulen, Kara Loy, Diane Mayorga, Ursula McGowan, Julia Miller, Dorothy Missingham, Nayana Parange, Jenny Shanahan, Shinta Sari, Colin Sharp, Tara Shepperson, Sylvia Tiala, Ray Tolhurst, Lyn Torres, Lalitha Velautham, Gerda Visser, Helen Walkington, Georgina Willets, Mike Wilmore, David Wilson

This journal article is available in Journal of University Teaching & Learning Practice: https://ro.uow.edu.au/jutlp/vol15/iss4/1

Introduction

The explicit and coherent development of student research skills has been an enduring educational concern at least since Dewey’s (1908) call for student discovery learning and Vygotsky’s work on the Zone of Proximal Development (ZPD: Vygotsky 1978). Discovery learning requires a raft of skills, and the ZPD provokes thought about the level of guidance that students may need from educators in the discovery process. One model that brings together these vital aspects of discovery learning is the Research Skill Development (RSD: Willison & O’Regan 2006) framework, which elaborates the skills associated with research and discovery into a continuum that describes the extent of student autonomy. Research skill development not only remains current for educators and researchers, but recent educational trends towards increased research in coursework have amplified the need for an explicit, developmental learning process. The publication of the article that explicated the RSD framework in 2007 (Willison & O’Regan 2007) prompted educators to consider their role in modelling, scaffolding and withdrawing for students’ own research, problem solving and project-based learning, and to what extent to make research processes explicit to students. The RSD is a conceptual framework, not a set of rules or a rubric, and is designed for educator engagement that enriches their pedagogical content knowledge (Gudmundsdottir & Shulman 1987) so that they know how to teach students sophisticated thinking skills within (inter)disciplinary contexts.

The original concepts that explicitly informed the 2006 RSD framework are present in the updated 2018 version (see Table 1: Willison & O’Regan 2006/2018). The six ‘standards’ of the Australian and New Zealand Institute of Information Literacy (ANZIIL: Bundy 2004) provided the starting point for the left-hand column of Table 1 and these were combined with Blooms et al.’s (1956) Taxonomy to create the six facets of research. These six facets were elaborated then, as now, into five levels of student autonomy, and the elaboration of one of the facets, analyse & synthesise was informed by the SOLO taxonomy (Biggs & Collis 1982). Trialling of the pre-published versions of the RSD with first-year human biology students was carried out by Eleanor Pearce and Mari Ricci, and many iterations, evaluations and changes in terminology were devised before a workable version was published in 2006.

Table 1: The 2018 version of the Research Skill Development framework (Willison & O’Regan, 2006/2018).

The RSD framework was designed from the outset to inform the coherent, incremental, explicit and cyclic development of the skills associated with research, primary school to PhD for any discipline or context (Willison & O’Regan 2005; 2006; 2007). The original journal article introducing the RSD demonstrated a clear need for explicit and coherent research skill development in school and university education (Willison & O’Regan, 2007). Subsequently, the usefulness of the RSD to address this need was empirically demonstrated for a broad range of university undergraduate courses (Willison 2012), across entire degrees (Willison & Buisman- Pijlman 2016; Wilmore & Willison 2016; Ain, Sabir & Willison 2018), for master’s programs (Willison, Schapper & Teo 2009; Willison, Sabir & Thomas 2017) and for PhD supervision (Velautham & Picard 2009). Use in primary and secondary schools is only now starting to take off (e.g., Heck 2017; Home 2017; Sari 2017) with much work to be done in terms of evaluating effectiveness of RSD implementations in a variety of contexts.

1

Willison: Research skill development spanning higher education

D isce

rn in

g

H a rm

o n isin

g C

u rio

u s

D e te

rm in

e d

D isce

rn in

g C

re a tive

C o n stru

ctive

Table 1: Research Skill Development Framework Fo r educators to facilitate the explicit, coherent, incremental and cyclic development of the skills associated with researchin g, proble m solv in g, crit ic al thin kin g and clin ic al reasonin g.

Pr escribed Research

Hi ghly structured directions and modelling fr om e ducato r p rompt r esearching, i n wh ich…

Bo unded Re search

Bo undaries set by and limited directions fr om e ducato r c hannel re searc hing, in wh ich…

Sc affolded Re search Sc affolds placed by educator shape in dependent researchin g, in wh ich…

Op en-en ded Resear ch St udents initiate research and this is gui ded by the educ at or .

Un bounded Re search

St udents determine guidelines for re searc hing that are in accord with di scipl ine or cont ex t.

Em bark & Clarify Wh at is our purpose? St udents respond to or initiate direction, cl arify and co nsi der ethical, cu ltural, so ci al and team (E CST) issues.

St udents respond to questions/tasks that are di rec ted. Use a pr ov ided, struc tur ed appr oac h to c larify q uesti ons, te rms, r equirements , ex pec tat ions and et hi cal, cul tur al , soc ial and te am is sues.

St udents respond to questions/ ta sks w ith l imite d o pti ons. C hoos e from se ve ral provided st ruct ures to cl arify ques tions , req ui rem ent s, ter ms, ex pec tat ions and et hi cal, cul tur al , soc ial and team issues .

St udents respond to broad tasks/ ques tions given. Cho os e from a range of pr ov ide d appr oac hes or struc tur es to cl arify requirements, quest ions, ex pec tat ions and et hi cal, cul tur al , soc ial and team issues .

*S tudents generate questions /a ims/h ypoth eses/p urpose fr amed w ith in st ruct ured guidelines* . An ticipate and prepare for et hi cal , cul tur al , so ci al and team is sues.

*S tudents generate questions/aims/ hy pot hes es /pur pos e bas ed on ex per ienc e, ex per tise an d liter at ur e. De lve into and prepare for et hi cal , cul tur al, so ci al and team is sues.

Fi nd & Generate Wh at do we need? St udents find information and generate dat a/ ideas us ing appr opr iat e met hod ol ogy .

St udents collect and record required in formatio n/data usin g a prescrib ed me thodology from a prescribed source in wh ich the information/data is evident.

St udents collect and re cord appropri ate in formatio n/data usin g giv en methodolo gy fr om p re-det er mine d sour ce/ s wher e in formatio n/data is not obvio us.

St udents collect and record appropriate in formatio n/data from self -se lect ed so urce s usi ng one of se ve ral provi ded me thodologies.

St udents collect and record self-det ermined in formation/data choosing an appropria te me thodology based on parame ters set.

St udents collect and record in formatio n/data from self -se lect ed so urce s, ch oosi ng or devi si ng an appr opr iat e met hod ol ogy with se lf- st ruct ured guidelines.

Ev aluate & Reflect Wh at do we trust? St udents determine the credibility of sour ces , inf or mat ion, dat a and ideas , and mak e thei r ow n res ear ch pr oc es ses vi si ble.

St udents evaluate so urce s/ in formatio n/data usin g sim ple prescribed cr iteria to sp eci fy cr edibility and to reflect on and impr ov e the pr oc es s us ed.

St udents evaluate sources/ in formatio n/data usin g a choic e of pr ov ide d criter ia to spec ify credi bi lity and to r efl ect o n a nd i mp rove processes used.

St udents evaluate sources/ in formatio n/data and the processes to fi nd/g enerate , u sing c rite ria r elate d to th e ai ms of the inqui ry to ref lec t on and im prove processes used.

St udents evaluate information/data and th e i nquiry p rocess u sin g self -det er mined cr iteria deve loped within parameters given. Re flect to refine own and others’ pr oc es ses .

St udents evaluate information/data and in quir y process rig orously using self - gener at ed criter ia bas ed on ex per ienc e, ex per tise an d the liter at ur e. Re flect to re new own and others ’ pro cesses.

Or ganise & Manage Ho w do we arrange? St udents organise information & data to re veal pattern s/themes, managing teams and pr oc es ses .

St udents organise information/data using pr es cribe d struc tur e. Manage linear pr oc es s pr ov ided (with pr e-sp ecified team ro les).

St udents organise information/data using a ch oice of give n st ruct ures. Manage a pr oc es s whi ch has al ter nat ive pos sibl e pat hw ay s (an d spec ify team rol es ).

St udents organise information/data using pr ov ide d gui del ines to choos e struc tur es . Ma nage processes (and teams ) with mu ltiple possible pathways.

St udents organise information/data using se lf-det er mined or gr ou p-det er mined st ruct ures, and manage the processe s (i ncluding team function) within the par am et er s se t.

St udents organise information/data using se lf-det er mined or gr ou p-det er mined st ruct ures and management proce sse s (i ncluding team function).

An alyse & Syn thesi se Wh at does it mean? St udents analyse information/data cr itica lly and syn thesi se new kn owledge to produce co herent in div id ual/ team understandin gs.

St udents interpret given information/data, det er mine pat ter ns and synt hes ise kn owledge into prescr ibed formats. *As k emergent questions of cl arifica tion/cu riosi ty *.

St udents analyse trends or themes in se ve ral so urces of information/ data and syn thesi se to integrate kn owledge into pr ov ide d standar d for mat s. *As k emergent, re levant and re searc hable questions.*

St udents analyse trends or themes in in formation/data and synthesis e to fully integrate co mponent parts in st ruct ures that are app rop riat e to tas k. *As k rigorous, researchable questions based on new un der stan di ngs *.

St udents analyse information/data and syn thesi se to fully integrate co mponents, co nsi st ent with se lf-det er mined par am et er s. Fi ll knowledge gaps that are stated by ot her s.

St udents analyse and synthesise in formatio n/data to generalis e or abstract kn owledge that addresse s se lf-id entif ie d or gr oup -id entifie d gaps in understandin g.

Co mmunicate & Ap ply Ho wdo we relate? St udents apply their understanding and di scus s, listen, write, per for m, res pon d to fe edback a nd p resent p rocesses, kn owled ge an d implication s of resear ch, hee di ng et hical , cul tur al, soc ial an d team is sues an d au di enc e ne ed s.

St udents di scus s wi th each other, listen, re ad and write to re late their pri or and new kn owledge to se t tasks. Use prescr ibed la nguage and genre to develo p under- st anding and then demonst rate this to a sp eci fied audience . Apply to a si milar co ntext the kn owledge developed. Follow pr om pt s on et hi cal l, cul tur al , soc ial is sues.

St udents use some discipline-sp ecific la nguage and genre to rela te their prior and new ly de ve loped kn owledge to tasks and then to a spec ified aud ienc e. App ly th e k nowledge d eveloped to s everal si milar co ntext s and st ay within boundaries se t for et hi cal , cul tur al , soc ial an d team is sues.

St udents use discipline-sp eci fic or other appr opr iat e langu ag e and select genres to dev el op und er standi ng an d rel at e thi s to an audi enc e chos en from gi ven opt ions . Ap ply the knowledge developed to di ffer ent cont ex ts and spec ify the et hi cal , cu ltural, so ci al and team is sues that em er ge.

St udents choose appropriate language, genr e an d per for manc e to ex tend the kn owledge of an aud ience they have se lect ed. Apply the knowledge deve loped to di ver se cont exts an d spec ify et hical , cul tur al , so ci al and team is sues in init iating, co nduct ing and co mmu nicating.

St udents choose appropriate language, genr e an d per for manc e to ex tend the kn owledge of a range of audience s. Apply in novativ ely the knowle dge develo ped to mu ltiple contexts. Probe and specify et hi cal , cul tur al, soc ial an d team is sues th at e merge br oad ly.

Sc ope for St ude nt Aut onomy www.rsd.edu.au [email protected]

F a c e t s o f R e s e a r c h

Students develop research mindedness when they…

Creative Commons 4.0

Creative Commons 4.0

T here has been broad recognition that many disciplines’ conceptions of research ‘… align with the six facets of Willison & O’Regan’s (2007) Research Skill Development framework’ (Walkington, Griffin, Keys-Mathews, Metoyer, Miller, Baker & France 2011, p. 317). This alignment mirrors the disciplinary use of the RSD to inform curriculum and assessment design, and scaffold student learning in a variety of disciplines from accounting (Wilkin 2014) to zoology (Hazel, Heberle, McEwen & Adams 2013). This A to Z use of the RSD spans first year to PhD. Moreover, the RSD has been used to enable rich curriculum conversations and collaboration between academics, sessional teachers, library staff and academic language and learning staff in diverse disciplinary areas (Torres et al. 2012; Torres & Jansen 2016) and interdisciplinary studies (Venning & Buisman-Pijlman 2011; 2013). Initial use of the RSD was in Australia, and there has been an escalating uptake internationally, including in countries as diverse as Cambodia (Serey & Sok 2017), Indonesia (Mataniari 2017), nations of the South Pacific (Janif 2017), Malaysia (Wong & Yahya 2017; Kananatu 2017) and the USA (Maurer 2017; Tiala 2017; Shanhan 2017).

With broad-ranging use has come diverse feedback, enabling improvement of the framework. This paper serves two purposes: the first purpose is to present the 2018 articulation of the RSD (Willison & O’Regan 2006/2018; Table 1) in response to the community of users and commentators. The second purpose of the paper is to introduce the seven other articles in this special issue of the Journal of University Teaching and Learning Practice (JUTLP). The contributing authors have provided papers that critique the RSD, use the RSD to evaluate and redesign curricula, and show the connections between research skills, problem solving, work integrated learning and academic literacies.

2

Journal of University Teaching & Learning Practice, Vol. 15 [2018], Iss. 4, Art. 1

https://ro.uow.edu.au/jutlp/vol15/iss4/1

Since the first publication of the RSD, there have been at least three major changes that have increased the need for an articulation like the RSD. One change is the acceleration of student learning through research in numerous nations over the past decade (Jenkins & Healey 2012). The Council on Undergraduate Research in the USA has shifted from almost exclusively mentored research models to include in-curricula models of undergraduate research (Auchincloss et al. 2014). Many Australian universities have instituted research-based learning in the undergraduate years, with research capstones becoming common (Brew 2013), while coursework master’s in Australia have a federally-mandated obligation since 2015 to enable students to work highly autonomously on research projects, scholarship or capstone experiences (Australian Qualifications Framework Council 2013). Developing countries like Cambodia (Om 2011; Serey & Sok 2017), Fiji (University of the South Pacific 2015; Janif 2017) and Indonesia (Mataniari 2017) are looking to develop research capacity commencing from the undergraduate years through in-curricula models for all students.

Closely connected to this increased requirement for student involvement in research is the second major change, that student need for coherent skill development has escalated, especially due to the variable student preparedness for engaging in research-based learning. In research that probed student perspectives of research-based learning (RBL) ‘[the] main concern from the student perspective was the need to slowly build up their skills and confidence to be able to effectively engage with research’ (Gresty, Heffernan, Pan & Edwards-Jones 2015, p. 48). Here, slowly build up suggests explicit and coherent development over time. When students engage in open discovery modes throughout their years of study, they risk applying the ‘same level of sophistication’ in their final year as in their first year (Chaplin 2003, p. 231). Moreover, gains of research skills made in the timeframe of a semester are at risk of atrophying if the development is not explicit in subsequent classes (Willison 2012; Willison & Buisman-Pijlman 2016). While it is easy to understand students’ expectations of slowly building up their skills in anticipation of large research projects towards the end of a degree, it is harder to realise the coherence needed to achieve this at a program level without an overarching conceptual framework.

The third change is also a consequence of increased RBL use and requirements, where there has been a corresponding proliferation of studies on the effectiveness of research-based learning. However, a meta-analysis of RBL outcomes found it difficult to connect the results of different studies and determine the type of guidance that is appropriate for students in RBL ‘due to the fact that guidance is often classified ad hoc’ (Lazonder & Harmsen 2016, p. 684). Lazonder & Harmsen (2016, p. 84) suggest that an a priori classification based on a conceptual framework might be ‘more fruitful and ease interpretation of the findings’. The RSD and its classification of levels of guidance can enable a clearer conceptual connection between otherwise separate studies, including studies using action research, ethnographic studies and quantitative studies, if it is used as an a priori framework for constructs and for interpretation of findings for those studies.

Because of the accelerated need to articulate research skill development for students and educators, there is a connected need to ensure an evidence-basis for decisions on whether to use the RSD for this purpose. A suitable framework requires a sound basis, should communicate effectively within and across disciplines and institution-wide, be adaptable and be subject to ongoing improvement through research and feedback. This requires research that determines whether the framework is well-fitted to a range of contexts, where it needs to be adapted and how, and makes provision for modifications, especially as each context changes over time. Each implementation of the RSD brings its own set of challenges, and each implementation can range in effectiveness from superficial to sensational; however, it is the sum of implementation evaluations that give the sense of effectiveness of the RSD.

3

Willison: Research skill development spanning higher education

As the need for a conceptualisation of the development of student research skills has increased, so too has a substantial body of critiques of the RSD, which are discussed below.

Critiques and Improvements

Modifications to the RSD framework in response to criticism comprise three changes in articulation and two additions to the framework. The changes in articulation involve rephrasing of the facets of research, the levels of autonomy and the resulting cell details. The new additions match the cognitive facets and are the inclusion of the affective domain and the addition of questions that capture the key purpose of each facet. Moreover, in this special issue, the article by Wisker further critiques aspects of the RSD.

1. Facets of the RSD

The biggest change to the articulation of the cognitive facets of research was in response to the critique by Hughes, Tucker & Knaggs (2011, p. A57) who said that in the RSD’s facet descriptions:

… the critical thinking skills so important for academic writing are not… clearly delineated and tend to be grouped together in the final steps.

The clumping together of synthesis, analysis and application in the 2006 version of the RSD was in part a legacy of our interpretation of the ANZIIL ‘standards’ (Bundy 2004), and the explicit fusion of these with Bloom et al.’s (1956) taxonomy. This critique helped set in motion the biggest reshuffle of the facet descriptions. Moreover, in the course of more than 120 RSD workshops run during two Office for Learning and Teaching (OLT) projects, there was also feedback that the framework was lacking necessary components of the research process: ‘reflection’ and ‘management’. Given that two existing facets already had verbs coupled together: embark & clarify and find & generate, it made sense to maintain a consistent form; ‘reflect’ was coupled with ‘evaluate’, ‘manage’ with ‘organise’ and ‘communicate’ with ‘apply’, which left ‘analyse’ and ‘synthesise’ already together. The current form of facets is shown in the 2018 version of the RSD, Table 1’s left hand column. The verb couplets were used from 2011 as headings for each facet, which were previously sentence-long descriptions labelled with A, B, C, etc. Using titles of active processes like embark & clarify or evaluate & reflect is more communicative than the original titles of Facet A or Facet C (for empirically-based understanding of the facets, see Wilmore & Willison 2016).

2. Articulations of Levels of Autonomy

The most common critique of the slippery concept of autonomy was not of the continuum per se but of use of the term ‘levels’. Autonomy is the fundamental concept of the RSD that allows it to recycle throughout education, from primary schooling to Ph.D. Facets of the RSD reveal the ‘what’ of research processes and the levels of autonomy show ‘how to facilitate’ the facets, by providing a sense of the level of structure and guidance required by students. In the 2006 version of the RSD, the problem was that autonomy level tended to be conflated with education level, where the latter was a kind of shorthand for a more linear increase in sophistication. Therefore, from 2009, autonomy was articulated using descriptive words, and these came to be prescribed, bounded, scaffolded, open-ended and unbounded, to represent the spectrum of autonomy. The

4

Journal of University Teaching & Learning Practice, Vol. 15 [2018], Iss. 4, Art. 1

https://ro.uow.edu.au/jutlp/vol15/iss4/1

main idea was for people to use the actual description of autonomy, such as prescribed research, because this is a better starting point for communication than something as ambiguous as Level 1.

Not all people have been enamoured with the continuum of autonomy, however. Allin (2014, p. 98) regrets that, through the RSD:

… students tend to be viewed as beginning researchers… requiring facilitation by lecturers to develop along a continuum of research skill development (Willison & O’Regan 2007). Hence traditional power boundaries between lecturer and student remain.

It may be that delineating and making explicit student autonomy in research could set power boundaries in concrete or at least maintain them. However, an alternative perspective is that if educators and students collaborate to demystify these boundaries, it may enable deeper levels of student understanding of the educative relationships in learning how to engage in research. Such a process could serve to empower students, especially keeping in mind findings that many students want to see their research skills slowly build up. Moreover, the RSD represents the continuum of autonomy as cyclic rather than a linear progression. Educators could give, or students could take, a high degree of autonomy in student research at the outset, if there were reasons this was appropriate (Maurer 2017) and then later need greater structure and therefore act with less autonomy. This depends on many factors including the degree of familiarity with and difficulty of the context, and the rigour required. (See Willison, Sabir & Thomas (2017) for an in-depth treatment of autonomy from a student perspective).

The RSD guides educators’ intentions to provide appropriate levels of autonomy for students, and so the levels describe what educators do (prescribe, scaffold, etc), and the autonomy continuum is labelled as ‘scope for autonomy’ in the 2018 RSD. The use of the term autonomy also suggests that not only should the students be provided with ways to increase in learning autonomy, but that educators should also have scope to increase teaching autonomy. Teaching autonomy, if well informed by sound pedagogical conceptualisations, could aid educators’ efforts towards improving curricula, and help them to develop a level of professional judgment which is valued by their students and peers, ultimately empowering educators. Whether autonomy is explicitly articulated to students or not, it is a core feature of MELT, with its corresponding question being: ‘how much guidance do these students need?’

3. Changes to the Articulations of Cells

Clearer top-level descriptions for facets and the spectrum of autonomy led to numerous small changes to cell contents. In addition, authors pointed out the problems with specific descriptions. For example, Spronken-Smith et al. (2013, p. 107) found:

… with an increased level of learner autonomy, there is a shift from the use of lay language to the language of the discipline. We contend that, as the level of exposure increases, the reverse is often the case… Learning to communicate complex ideas without recourse to discipline-centred terminology and jargon requires significant skill development.

This perspective resulted in the articulation of the 2018 RSD facet communicate & apply (see Table 1) in terms of using language appropriate to an audience. In addition, there was also an evident need to emphasise communication and application in the RSD primarily as processes that

5

Willison: Research skill development spanning higher education

ultimately result in a product later, and so the facet description now commences with ‘talk about’. Similarly, ‘listen’ and ‘respond to feedback’, were added to provide the previously missing explication of communicating being a two-way process. These words are intended to provoke educators to think more about not merely providing feedback, but also what students do with the feedback (Boud 2013) and how students themselves may provide peer feedback and reflect for self-improvement.

Notwithstanding all the above changes, the original use of information literacy standards (Bundy 2004) as the starting points for the six research facets (Willison & O’Regan 2005; 2007) provided a solid structure from which to build. Likewise, a continuum of autonomy has remained educationally useful as elaborated in five delineations, and the RSD framework over the past 12 years has shown some hallmarks of a viable and stable representation of the development of research skills. This is supported by research of RSD implementation in 29 courses in five universities (Willison 2012) as well as degree-program evaluations (Wilmore & Willison 2016; Willison & Buisman-Pijlman 2016; Wilmore & Willison 2016; Willlison, Sabir & Thomas 2017; Ain, Sabir & Willison 2018). The above changes, then, involved fine-tuning existing structures to gain clearer articulations that were better-suited to teachers and students. However, the changes described in the next section were more dramatic and required entirely new representations.

4. Affective Descriptors

The first major addition to the RSD since 2007 is the description of the affective domain (values, motivators and drivers: Krathwohl, Bloom & Masa 1964) that parallel the cognitive domain of the original RSD. Cumming (2010, p. 412) asserted that the original RSD was complicit with the ‘skills agenda in higher education as part of broader schema concerned with increased efficiency and productivity’. Managerial ‘efficiency and productivity’ is a legitimate perspective of the contemporary skills agenda, with positive and negative connotations. Just as appropriately, the term ‘skill’ may be thought of as an empowerment agenda, for students to develop a raft of cognitive and affective skills, placing them as graduates who are equipped for life as well as employment. The old Norse ‘skil’, from which English ‘skill’ derives, means ‘power of discernment’ (Online Etymology Dictionary 2018). Discernment is not just about the cognitive process of making distinctions, sorting wheat from chaff; it also implies having more affective- oriented insight, a very empowering feature which is central to higher education’s purpose.

Work on the motivations and drivers for the RSD was precipitated in one workshop - for researchers at Monash University Peninsula campus in 2009 - where one participant asked this question: ‘I like the framework… but what about passion?’ One concern about representing the affective domain, however, was the risk of creating an artificial separation of cognitive and affective domains. For example, in addition to the Taxonomy of the Cognitive Domain, Bloom and co-authors developed a second, complex framework, the Taxonomy of the Affective Domain (Krathwohl, Bloom & Masa 1964), hoping that two separate frameworks would be educationally useful.

The desire to emphasise the RSD’s affective domain (without creating a second framework) resulted in the choice of six single-word adjectives. Each of these adjectives described an affective state corresponding to one of the cognitive facets. The original list was: curious, determined, critical, organising, creative and persuasive. In light of feedback over the past eight years, including at conferences (Willison 2010; Bandaranaike, Snelling, Karanicolas & Willison 2012) and four recent workshops (Willison 2016), the improved affective adjectives are listed in Table 2, and in the 2018 RSD (Table 1), they run perpendicular to the more cognitive facets.

6

Journal of University Teaching & Learning Practice, Vol. 15 [2018], Iss. 4, Art. 1

https://ro.uow.edu.au/jutlp/vol15/iss4/1

Table 2: Affective facets and key questions that relate to each cognitive facet from the 2018 RSD.

Cognitive facet Affective facet Key Question Embark & Clarify Curious What is our purpose? Find & Generate Determined What do we need? Evaluate & Reflect Discerning What do we trust? Organise & Manage Harmonising How do we arrange? Analyse & Synthesise Creative What does it mean? Communicate & Apply Constructive How do we relate?

These affective words are context-sensitive. For example, in health science, empathy is frequently a more important driver than curiosity, and meticulous may be more important than determined (Willison & Parange 2016). The affective words are to be evocative of educational purpose and so of what needs to be prioritised. For example, having students learn rigorous research techniques that bores them may be counterproductive to motivational elements.

The current RSD version addresses the question about passion through the elucidation of the affective domain and connecting it with the cognitive research skills. These two domains complement and reinforce each other in the RSD.

5. Questions that Relate to the Six Facets

The second major addition to the RSD was of questions that provide a sense of each facet’s core intention (Table 2), and have been helpful for educators to understand the parameters of each facet. These key questions have been invaluable to make connections between research skill development and the development of a variety of other skill sets, such as problem solving and critical thinking (see the MELT section below).

6. Critique in this Special Issue

Contributing to the literature that provides an impetus for improving the RSD, the second article in this issue, by Wisker (2018), is titled ‘Frameworks and freedoms: Supervising research learning and the undergraduate dissertation’. Wisker asks:

Will the use of the Research Skill Development … and other frameworks at every step of the undergraduate research journey form a constraint, or an essential scaffold?

This is a vital question for the RSD and any conceptual or theoretical framework. Wisker unpacks the tensions between supportive scaffold and straitjacket and this provides insights into the risk of over-structuring.

Curriculum

Higher education must promote the developing learning autonomy of all students, the passionate dedication of all to the learning enterprise within the constraints of funding and society. The RSD has been used in large courses, including one course of 3000-plus students across 14 campuses (University of the South Pacific 2015), and the amount of autonomy provided has been necessarily limited. However, it has been used on the continuum of slowly building up skills as called for by

7

Willison: Research skill development spanning higher education

students in their learning about the complexities of engaging in research. Empowerment of students is not always about making them the architects of their learning immediately, but can involve realistic and ‘slow improvement’. It can be perilous for students to engage in the journey of an open discovery if they are not familiar with what is required or equipped with the necessary cognitive and affective skills (Willison & O’Regan 2007). Frequently, students desire and need guidance as they peer into a place they have never been, with some desire and trepidation. Articles three to five address various aspects of curriculum in this special issue.

In the third article in this special issue, Torres (2018) uses the RSD facets and extent of autonomy to critique an existing curriculum in inquiry-oriented biology labs, exploring the tensions between design intention and actual implementation. Her article, Research skills in the first-year biology practical - Are they there? asks about the skills that are used, developed and assessed in the curriculum. Torres provides a fresh perspective on all of university learning by pondering how, in a laboratory context:

… collaborations with the RSD would enable new understanding of how librarians could contribute to the development of students’ research skills in the practical experience.

In the fourth article, Jacobsen et al. (2018) consider Graduate students' research-based learning experiences in an online Master of Education program. They explore:

… research-based teaching and learning designs that support graduate students in learning how to conduct educational research on problems of practice, as well as the collaborative research-based teaching that supports students in carrying out their capstone research projects.

The collaborative aspect is again a key part of the design, but its fully online learning environments add a different dimension to Torres’ (2018) laboratory focus. Jacobsen et al.’s (2018) small-scale study raises issues of scalability of RSD use in online courses, including Massive Open Online Courses (see also Parange et al. 2017).

Gyuris, in the fifth article, writes on Evaluating the effectiveness of postgraduate research skills training and its alignment with the Research Skill Development framework, concluding:

Many students, while achieving much improvement in their proposal score, did not demonstrate in their portfolios an awareness of the processes that allowed them to improve. We need to equip students with the skills to be competent, self-regulated learners, so they can understand and practice the metacognitive skills that allow them to perform at their highest level.

This has been an enduring issue in research into RSD use, where the process of making learning intentions explicit seems very overt to the teachers, but often un-noticed by students (Willison 2012).

Connections

8

Journal of University Teaching & Learning Practice, Vol. 15 [2018], Iss. 4, Art. 1

https://ro.uow.edu.au/jutlp/vol15/iss4/1

There is a need to provide coherent ways of understanding and connecting engaged teaching and learning approaches that require similar modes of student activity, such as research-based learning, inquiry-based learning, problem-based learning, project-based learning, undergraduate research, Process-Oriented Guided Inquiry Learning and discovery learning. As noted earlier, there have been calls for an a priori framework that may connect otherwise disparate studies of RBL. In many ways, this is realisable if there are shared, flexible parameters that allow for adaptations of language that suit each context studied. The last three articles in this special issue demonstrate the connections between the skills associated with research and those associated with academic literacy, Work Integrated Learning and problem solving.

In the sixth article, McGowan (2018) looks at Integrated academic literacy development: Learner- teacher autonomy for MELTing the barriers. She considers a:

… collaborative approach to genre pedagogy that has the potential for overcoming the content-language dichotomy and also the cost barrier. It provides a method for the discipline lecturer’s progress from initial dependence on the literacy specialist’s expertise towards learner and teacher autonomy.

This article then deepens the collaborative theme which is presented across articles in this special issue; however, this time, the language of the RSD is shifted and presented in McGowan’s Accelerating Academic Literacy Development framework . The first representational shift was to harness the six research skill facets explicitly as learning facets for an academic literacy development tool, while the second shift was to demonstrate a path to autonomy in academic literacy learning and teaching within the context of a STEM (Sciences Technology Engineering & Mathematics) discipline. Mirroring a theme from Wisker’s (2018) Frameworks and freedoms article, McGowan highlights the fluidity needed in the move towards greater autonomy and ownership by the discipline-based teachers involved.

In the seventh article, From Research Skill Development to Work Skill Development (WSD), Bandaranaike (2018) demonstrates the strong and intentional alignment of:

…the skills needed for employability with the research skills articulated in the RSD framework.

This use of the WSD is an important segue from the world of university to the world of Work Integrated Learning, and raises questions about how to better connect the academic curriculum with the workplace. Bandaranaike shifts language from ‘research’ to ‘work’ while maintaining the same conceptual structure as the RSD: much needs to be done to determine whether the RSD can be used in concert with the WSD to make meaningful university-work connections.

In the final paper, Student engineers optimising problem solving and research skills, Missingham et al. present a student-developed adaptation of the RSD, the Optimising Problem Solving (OPS) pentagon. Their article shows how closely the skills associated with problem solving correspond with the skills associated with researching, and:

If the connection between problem-solving skills and research skills is made explicit, there is opportunity for otherwise separate conceptualisations to work together as thinking routines for students over time, enabling them to become increasingly metacognitive.

9

Willison: Research skill development spanning higher education

The emergence of the WSD in 2009, followed by other frameworks that used the parameters of the RSD, marked a major organic shift towards considering the six facets as key aspects of context- sensitive sophisticated thinking. Guided in particular by the questions for each facet mentioned earlier, terminology may be adapted to fit the context of use, as shown in the above three articles. Student-tutors reconfigured the RSD into OPS to provide a visual, communicative, engineering- oriented version for first-year students.

In all cases, the parameters of the RSD - six facets and the consideration of student autonomy - set the boundaries to devise models that are conceptually interlinked but contain shifts in language appropriate to each context. The whole organic family was named the Models of Engaged Learning and Teaching (MELT). MELT (www.melt.edu.au) legitimise and require educator modifications to fit each context and show a capacity to help teachers and students to make conceptual connections between otherwise disparate initiatives and skill sets. MELT were the subject of an international conference in 2017 (www.i-melt.edu.au), with numerous models and applications across many contexts available in the proceedings.

Conclusion

Numerous critiques have successfully evoked changes in the RSD framework, including improvements in the articulations of facets of research and the spectrum of autonomy, the connection with discipline content and research processes, as well as specification of the affective domain, the internal motivations, values and drivers of researching. You could say that the conceptual jewel of researching now has facets that sparkle or glow with a warm light, reminding all involved in the education process that facilitating student cognitive engagement needs to take into consideration appropriate motivations and drivers of learning. Critiques of the RSD have been very helpful to improve the articulations in it, and further critiques from a variety of perspectives are essential. The first 12 years of RSD use suggest that it has potential to guide educators in higher education around the development of skills associated with discipline-specific research skills, and interdisciplinary and transdisciplinary projects. Numerous studies of RSD implementation are needed to:

 Determine the long-term efficacy of RSD implementations for students and graduates in a greater variety of university contexts, including at course level, program level and institution level, and also in primary and secondary school contexts.

 Determine the efficacy of the processes described in the MELT as an underlying concept to unite researching, critical thinking, problem solving, clinical reasoning, evidence- based decision making and other forms of learning that provoke sophisticated thinking (Willison 2015).

The authors in this special issue highlight the collaborative aspects of university teaching - including collaborations with students - and the absolute need for flexibility within common parameters. The emerging frameworks that share the parameters of the RSD, known as the Models of Engaged Learning and Teaching (MELT), portrayed in this special issue and elsewhere, are a recognition of the need to be the flexible, collaborative and conceptually connected. Higher education’s purpose can be clarified by the use of MELT as a tool for developing graduates who are metacognitive and knowledge-base savvy, with the cognitive skills to continue their own learning with high levels of autonomy because they are increasingly curious, determined, discerning, harmonising, creative and constructive.

10

Journal of University Teaching & Learning Practice, Vol. 15 [2018], Iss. 4, Art. 1

https://ro.uow.edu.au/jutlp/vol15/iss4/1

References

Ain, CT, Sabir, F & Willison, J 2018, ‘Research skills that men and women developed at university and then used in workplaces’, Studies in Higher Education, viewed 27 August 2018, <https://www.tandfonline.com/doi/pdf/10.1080/03075079.2018.1496412?needAccess=true>.

Allin, L 2014, ‘Teaching and learning inquiry’, The ISSOTL Journal, vol. 2, no. 1, pp. 95-102.

Auchincloss, LC, Laursen, SL, Branchaw, JL, Eagan, K, Graham, M, Hanauer, DI, Lawrie, G, McLinn, CM, Pelaez, N, Rowland, S, Towns, M, Trautmann, NM, Varma-Nelson, P, Weston, TJ & Dolan, EL 2014, ‘Assessment of course-based undergraduate research experiences: A meeting report’, CBE—Life Sciences Education, vol. 13, no. 1, pp. 29-40.

Australian Qualifications Framework Council 2013, Australian Qualifications Framework, viewed 12 January 2014, <http://www.aqf.edu.au/>.

Biggs, JB & Collis, KF 1982, Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome), Academic Press, New York.

Bloom, BS 1956, Taxonomy of Educational Objectives. Vol. 1: Cognitive Domain, McKay, New York.

Boud, D & Molloy, E 2013, ‘Rethinking models of feedback for learning: The challenge of design’, Assessment & Evaluation in Higher Education, vol. 38, no 6, pp. 698-712.

Brew, A 2013, ‘Understanding the scope of undergraduate research: A framework for curricular and pedagogical decision-making’, Higher Education, vol. 66, no. 5, pp. 603-618.

Bundy, A (ed.) 2004, Australian and New Zealand information literacy framework, Principles, Standards and Practice, 2nd Edition, Australian and New Zealand Institute for Information Literacy, Adelaide.

Chaplin, S 2003, ‘Guided development of independent inquiry in an anatomy/physiology laboratory’, Advances in Physiology Education, vol. 27, no. 4, pp. 230-40.

Cumming, J 2010, ‘Contextualised performance: reframing the skills debate in research education ’, Studies in Higher Education, vol. 35, no. 4, pp. 405-419.

Dewey, J 1908, ‘What does pragmatism mean by practical?’ The Journal of Philosophy, Psychology and Scientific Methods, vol. 5, no. 4, pp. 85–99.

Gresty, K, Heffernan, T, Pan, W & Edwards-Jones, A 2015, ‘Minimising risks in research informed teaching’, Higher Education Review, vol. 48, no. 1, pp. 40-67.

Gudmundsdottir, S & Shulman, L 1987, ‘Pedagogical content knowledge in social studies’, Scandinavian Journal of Educational Research, vol. 31, no. 2, pp. 59-70.

Hazel, SJ, Heberle, N, McEwen, MM & Adams, K 2013, ‘Team-based learning increases active engagement and enhances development of teamwork and communication skills in a first-year course for veterinary and animal science undergraduates’, Journal of Veterinary Medical Education, vol. 40, no. 4, pp. 333-341.

11

Willison: Research skill development spanning higher education

Heck, D 2017, ‘Reframing the Research and Work Skill Development framework facets to support becoming a teacher’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 8 April 2018, <https://www.adelaide.edu.au/rsd/i-melt/program/IMELT_booklet_181217.pdf>.

Home, J 2017, ‘Multidisciplinary approach to MELT use from Grade 5 to Year 12’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 12 April 2018, <https://www.adelaide.edu.au/rsd/i- melt/program/IMELT_booklet_181217.pdf>.

Hughes, P, Tucker, K & Knaggs, C 2011, ‘Sitting on the same bench: Complementing law learning outcomes’, Journal of Academic Language and Learning, vol. 5, no. 2, A50-A59.

Janif, S 2017, ‘Research Skills Development at the University of the South Pacific’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 2 April 2018, <https://www.adelaide.edu.au/rsd/i- melt/program/IMELT_booklet_181217.pdf>.

Jenkins, M & Healey, M 2012, ‘Developing and embedding inquiry‐guided learning across an institution’, New Directions for Teaching and Learning, vol. 2012, no. 129, pp. 27-37.

Kananatu, T 2017, ‘Using the Research Skill Development framework to construct marking rubrics for law assessments’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 15 May 2018, <https://www.adelaide.edu.au/rsd/i-melt/program/IMELT_booklet_181217.pdf>.

Krathwohl, DR, Bloom, BS & Masia, BB 1964, Taxonomy of Educational Objectives. Volume 2: Affective Domain, McKay, New York.

Krathwohl, DR 2002, ‘A revision of Bloom's taxonomy: An overview’, Theory into Practice, vol. 41, no. 4, pp. 212-218.

Lazonder, AW & Harmsen, R 2016, ‘Meta-analysis of inquiry-based learning: Effects of guidance’, Review of Educational Research, vol. 86, no. 3, pp. 681-718.

Maurer, TW 2017, ‘Guidelines for authorship credit, order, and co-inquirer learning in collaborative faculty-student SoTL projects’, Teaching & Learning Inquiry, vol. 5, no. 1, pp. 1-17.

Mataniari, R 2017, ‘Research Skill Development (RSD)-integrated online report for critical thinking skills’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 3 April 2018, <https://www.adelaide.edu.au/rsd/i-melt/program/IMELT_booklet_181217.pdf>.

Om, S 2011, ‘Cambodian post-graduate students’ perceptions of their development as beginning researchers’, Master’s thesis, Royal University of Phnom Penh, Cambodia.

Parange, N, Osborne, B, Maranna, S, Childs, J, Lamb, K & Thoirs, K 2017, ‘Teaching evidence- based practice online using a programmatic approach in a postgraduate program’, in Proceedings

12

Journal of University Teaching & Learning Practice, Vol. 15 [2018], Iss. 4, Art. 1

https://ro.uow.edu.au/jutlp/vol15/iss4/1

of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 18 May 2018, <https://www.adelaide.edu.au/rsd/i-melt/program/IMELT_booklet_181217.pdf>.

Sari, S 2017, ‘Teachers’ and tutors’ perceptions of the Optimising Problem Solving (OPS) framework for solving math’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 28 April 2018, <https://www.adelaide.edu.au/rsd/i-melt/program/IMELT_booklet_181217.pdf>.

Sok, S & Sok, V 2017, ‘Integrating the Research Skill Development (RSD) framework into master’s curriculum at the Royal University of Phnom Penh’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 5 May 2018, <https://www.adelaide.edu.au/rsd/i-melt/program/IMELT_booklet_181217.pdf>.

Spronken-Smith, R, Brodeur, J, Kajaks, T, Luck, M, Myatt, P, Verburgh, A, Walkington, H & Wuetherick, B 2013, ‘Completing the research cycle: A framework for promoting dissemination of undergraduate research and inquiry’, Teaching and Learning Inquiry: The ISSOTL Journal, vol. 1, no. 2, pp. 105-111.

Tiala, S 2017, ‘Stages of concern: A facilitator’s reaction to implementing the RSD in a community of practice’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 3 May 2018, <https://www.adelaide.edu.au/rsd/i-melt/program/IMELT_booklet_181217.pdf>.

Torres, L & Jansen, S 2016, ‘Working from the same page: Collaboratively developing students' research skills across the university’, Council on Undergraduate Research Quarterly, vol. 37, no. 1, pp. 26-33.

Torres, L, McCann, L, Croy, G & Mayson, S 2012, ‘Bridging the divide: Generating dynamic university wide library-faculty connections to enable the explicit development of students’ research skills ’, paper presented at the Higher Education Research and Development Society of Australasia (HERDSA) International Conference, Hobart, 3-7 July.

University of the South Pacific 2015, Research Skill Development site, University of the South Pacific, viewed 23 January 2018, <http://research.usp.ac.fj/?page_id=135>.

Velautham, L & Picard, MY 2009, ‘Collaborating equals: Engaging faculties through teaching-led research’, Journal of Academic Language and Learning, vol. 3, no. 2, A130-A141.

Venning, J & Buisman-Pijlman, F 2011, ‘The development of an assessment matrix to promote student learning in postgraduate multidisciplinary research projects’, Ergo, vol. 2, no. 2, pp. 37-44.

Venning, J & Buisman-Pijlman, F 2013, ‘Integrating assessment matrices in feedback loops to promote research skill development in postgraduate research projects’, Assessment & Evaluation in Higher Education, vol 38, no. 5, pp. 567-579.

Vygotsky, LS 1978, Mind in Society, Harvard University Press, Cambridge, Massachusetts.

Walkington, H, Griffin, AL, Keys-Mathews, L, Metoyer, SK, Miller, WE, Baker, R & France, D 2011, ‘Embedding research-based learning early in the undergraduate geography curriculum’, Journal of Geography in Higher Education, vol. 35, no. 3, pp. 315-330.

13

Willison: Research skill development spanning higher education

Willison, J 2012, ‘When academics integrate research skill development in the curriculum’, Higher Education Research & Development, vol. 31, no. 6, pp. 905-919.

Willison, J 2013, ‘Inquiring ape?’ Higher Education Research and Development, vol. 32, no. 5, pp. 861- 865.

Willison, J 2014, Outcomes and uptake of explicit research skill development across degree programs , Office for Learning and Teaching, Sydney, viewed 12 May 2018, <http://www.adelaide.edu.au/rsd/docs/pdf/RSD_degree_program_2014.pdf>.

Willison, J & Buisman-Pijlman, F 2016, ‘Ph.D. prepared: Research skill development across the undergraduate years’, International Journal of Researcher Development, vol. 7, no. 1, pp. 63-83.

Willison, J & O’Regan, K 2005, ‘2020 Vision: An information literacy continuum for students primary school to post graduation’, in Proceedings from the HERDSA Conference, University of Sydney, Sydney, 3-6 July, viewed 4 May 2018, <http://www.herdsa.org.au/publications/conference-proceedings/research- and-development-higher-education-higher-education-121>.

Willison, J & O’Regan, K 2006/2018, Research Skill Development framework, viewed 10 September 2018, <www.adelaide.edu.au/rsd>.

Willison, J & O’Regan, K 2007 ‘Commonly known, commonly not known, totally unknown: A framework for students becoming researchers’, Higher Education Research and Development, vol. 26, no. 4, pp. 393–409.

Willison, J, Sabir, F & Thomas, J 2017, ‘Shifting dimensions of autonomy in students’ research and employment’, Higher Education Research & Development, vol. 36, no. 2, pp. 430-443.

Willison, J, Schapper, J & Teo, E 2009, ‘Multiple measures of improvement of research skills in business ethics and business law’, in The Quantitative Analysis of Teaching and Learning in Higher Education in Business, Economics and Commerce: Forum Proceedings, University of Melbourne, Melbourne, 9 February, pp. 79-95.

Wilkin, CL 2014, ‘Enhancing the AIS curriculum: Integration of a research-led, problem-based learning task’, Journal of Accounting Education, vol. 32, no. 2, pp. 185-199.

Wilmore, M & Willison, J 2016, ‘Graduates’ attitudes to research skill development in undergraduate media education’, Asia Pacific Media Educator, vol. 26, no. 1, pp. 113-128.

Wong, T & Yahya, E 2017, ‘Towards a more cogent curriculum for experimental writing: A case study’, in Proceedings of the International Conference on Models of Engaged Learning and Teaching, Adelaide, 11-13 December, viewed 23 May 2018, <https://www.adelaide.edu.au/rsd/i- melt/program/IMELT_booklet_181217.pdf>.

14

Journal of University Teaching & Learning Practice, Vol. 15 [2018], Iss. 4, Art. 1

https://ro.uow.edu.au/jutlp/vol15/iss4/1

Copyright of Journal of University Teaching & Learning Practice is the property of University of Wollongong and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.