due tonight at 9pm
Babee_girl
document48.pdf
At Sea in a Deluge of Data
By Alison J. Head and John Wihbey July 07, 2014
This spring, more college students than ever received baccalaureate degrees, and their career prospects
are brighter than they were for last year’s graduates.
Employers responding to this year’s National Association of Colleges and Employers’ "Job Outlook 2014
Survey" said they planned to increase entry-level hiring by almost 8 percent. But what they may not
realize is that these seemingly techno-savvy new hires could be missing some basic yet vital research
skills.
It’s a problem that we found after interviewing 23 people in charge of hiring at leading employers like
Microsoft, KPMG, Nationwide Insurance, the Smithsonian, and the FBI. This research was part of a
federally funded study for Project Information Literacy, a national study about how today’s college
students find and use information.
Nearly all of the employers said they expected candidates, whatever their field, to be able to search
online, a given for a generation born into the Internet world. But they also expected job candidates to
be patient and persistent researchers and to be able to retrieve information in a variety of formats,
identify patterns within an array of sources, and dive deeply into source material.
Most important, though, employers said they need workers who can collaborate with colleagues to
solve problems and who can engage in thoughtful analysis and integrate contextual organizational
details rarely found online.
Many employers said their fresh-from-college hires frequently lack deeper and more traditional skills in
research and analysis. Instead, the new workers default to quick answers plucked from the Internet.
That method might be fine for looking up a definition or updating a fact, but for many tasks, it proved
superficial and incomplete.
It turns out that students are poorly trained in college to effectively navigate the Internet’s
indiscriminate glut of information.
Another Project Information Literacy study, involving more than 8,300 undergraduates at 25 American
colleges, found that most make do with a very small compass. They rely on tried and true resources such
as course readings, library databases, Google, and Wikipedia.
Only 20 percent of the students said they ever sought help from librarians, the mavens of searching and
finding in the digital age, especially when it comes to learning how to "ping pong" effectively and
strategically among offline sources, experts, and online information, blending the full range of
knowledge sources in all their forms.
The skills that students cultivate through traditional assignments—writing essays based on library
research—are far different from those required to perform efficient, high-level, accurate research in the
digital world. All of those types of research skills take practice under the eye of experts.
Sharon Weiner, a professor of library science at Purdue University, has argued that those essential core
competencies belong not to any single discipline, but to all of them.
This skills gap is not necessarily the fault of faculty members, administrators, librarians, or even students
themselves. Part of the blame lies with the crowded information landscape that students inhabit today.
Knowledge work—processing information and thinking for a living—is faster in the 21st century, and
meaning and credibility are more fluid and tougher to ascertain than they were in the 20th century.
Frequently now, subjects are, as the information-technology thinker David Weinberger put it, "too big to
know." One undergraduate we interviewed told us that while traditional research skills still matter, "the
hardest part of research is figuring out the question to ask."
While students will always need to think critically and ask the right questions, emerging in this new
world is the need for a skill set we call "knowledge in action," a kind of athletics of the mind aided by
Internet-enabled devices, search engines, and pools of data from a wide variety of outlets.
We recognize knowledge in action when we see it done effectively, but too many professors don’t teach
this fundamental skill systematically and progressively as part of an academic program. Sure, students
know how to use keywords and how to refine searches. But imagine that you’re Googling "vaccine
autism" or "violent video game" or any other hot topic. You’ll soon be flooded with a torrent of
conflicting findings, many not credible. Knowledge in action means being able to sort through that
growing thicket of information. This is a lifelong learning skill, crucial to health, wealth, social equality,
and well-being. In an era of partisan fog and the polarization of many subjects, it is a skill vital for
effective citizenship.
This goes well beyond search techniques. Engaging knowledge at the speed of the web takes three
additional things, which tend to be separate in our curricula rather than integrated: a basic
understanding of statistics and inference; a sense of the major research institutions—a basic
understanding of what it means when you see results attached to URL’s such as "cdc.gov," "imf.org" or
"pewresearch.org" and how those institutions produce knowledge; and a sense of how the scientific
method works and what it means to test a hypothesis with data.
Further, because our web experience will increasingly be personalized through algorithms that key off of
everything from geolocation to our prior digital traces, students must learn to recognize the limits of
their online environment and to seek information creatively outside of channels that serve up results
skewed by Internet companies and other paternalistic, biased, or profit-driven gatekeepers.
Yet from community colleges to the Ivy League, a significant learning gap is widening. Librarians, trained
in both digital and print research techniques, are in the best position to step into the breach. But that
will require more support for library services at a time when budgets are under siege. And it will take an
administrative commitment to ensure that training is incorporated comprehensively throughout the
curricula.
This is not to say that everyone must develop the hybrid expertise of an investigative journalist, high-
level consultant, or front-line infectious-disease analyst. But that blend of speed, smarts, and problem
solving will prove essential in the 21st-century workplace for effective and informed decision-making,
creative solutions to problems in both science and public policy, and breakthrough discoveries and
innovations.
Alison J. Head is director of Project Information Literacy, a research scientist at the University of
Washington’s Information School, and a fellow at Harvard’s Berkman Center for Internet and Society.
John Wihbey is managing editor of JournalistsResource.org, a project of the Kennedy School’s
Shorenstein Center at Harvard University, and a lecturer in journalism at Boston University.
How college students find and use information [Infographic] Posted by reyjunco on November 26, 2012 in Infographic | ∞
My friends over at Project Information Literacy have just released this infographic to summarize their
recent research on how college students find and use information. Data in this infographic come from
PIL’s publications Balancing Act: How College Students Manage Technology While in the Library during
Crunch Time and Truth Be Told: How College Students Evaluate and Use Information in the Digital Age. If
you are interested in college students and information literacy, check out PIL’s other reports here.
This infographic is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0
license so feel free to share on your own website by linking back to Project Information Literacy.
oject Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
1
T R U T H B E
T O L D
:
How College Students
Evaluate and
Use Information
in the
Digital Age
Abstract:
A report
about
college students and their information-
seeking
strategies and research
difficulties
, including findings from 8,353
survey
respondents
from
college students on
25 campuses
dis
tributed across the U.S. in
spring of 2010
, as part of Project Information Literacy
. Respondents
reported taking little at face value and were frequent evaluators of Web and library sources used for
course work, and to a
lesser extent, of Web content
for p
ersonal use. Most respondents turned to
friends and family
when asking for help with
evaluating information for personal use and instructors
when evaluating information for course research. Respondents
reported using a repertoire of
research techniques
—mostly for writing papers
—for completing one research assignment to the next,
though f
ew respondents reported using Web 2.0 applications for collaborating on assignments.
Even
though most respondents considered themselves adept at finding
and evaluating
information,
especially when it was
retrieved
from the Web,
students
reported difficulties
getting started with
research assignments and
determining the nature and scope
of what
was required
of them.
Overall,
the
findings suggest students use an
informati
on
-seeking and research strategy driven by efficiency
and predictability for managing and controlling all of the information available to them on college
campuses, though c
onducting
comprehensive research
and learning something new is important
to
most
, al
ong with
passing the course and the
grade
received
. Recommendations
are included
for how
campus
-wide stakeholders
—faculty, librarians
, and higher
education
administrators
—can work
together
to help inform pedagogies for a new century.
A 32
-year
-old librarian relates what now seems like a quaint memory from a simpler time. Not
that many years ago, while conducting a literature review for her own humanities dissertation,
she was able to
search and
exhaust every information source her camp
us library
had to offer.
But for
many of today
ʼ
s undergraduates, the id
ea of being able to conduct an
exhaustive search
is inconceivable.
Information
seems to be
as limitless
as the universe. And research is one of
the most difficult challenges
facing stu
dents in the digital age.
Introduction
P
roject Information Literacy (PIL) is a national research study based i
n the University of
Washington
Information School.
1
In our ongoing research, we have
studied
how college students
conduct research and find information
—their needs, strategies, and
workarounds
. So far,
we
have focused our research efforts on
how
students find information and their
preferred use of
information
sources
for
course work and
in their daily lives.
In this report, we
continue our investigation by asking
how
students evaluate
information
and
use
information
once they have found it
. What
difficulties
do
students encounter
with
course
-rel
ated
and everyday life research from start to finish?
We collected data to answer these questions by administering a student survey in the spring of
2010 to
112,844
undergraduates
. Our findings are based on a collective sample of 8,353
students
enrolled at 25 U.S. colleges and universi
ties.
2
Major Findings
The beginning of the course-
related research process is rife with challenges for
most
college
students, according to our survey results.
Even though
many students may consider themselves adept at
evaluating
information
and
applying
techniques for
tackling
one course
-related rese
arch assignment to the next
, the
sheer
act
of
just getting started
on research assignments
and defining a research inquiry
was
overwhelming for students
—more
so
than any of the
subsequent steps in the research process
.
1
Project Information Literacy (PIL) is co
-directed by Alison J. Head, Ph.D., R
esearch Scientist in the University of
Washington Information School
and Michael B. Eisenberg, Ph.D., Dean Emerit
us and Professor in the University of
Washington Information School. PIL is
supported with contributing funds from the John D. and Catherine T.
MacArthur
Foundation. Communication about this progress report should be sent to Dr. Alison Head at
or Dr.
Michael Eisenberg at
. Visit the PIL project Web site
for an overview of PIL’s
ongoing research
.
2
We administered a 22
-item online survey to sophomores, juniors, and seniors at 25 institutions across the
U.S.
during
March through May 2010.
See Appendix C in this report for the survey instrument we administered.
All but
the State
College of Florida
Manatee
-Sarasota were
four-
year institution
s. Given the sample size (n=8,353), our study is the
largest scholarly survey analysis of information l
iteracy, to date. For a full list
of institutions participating in the study and
a discussion of methods, see the Appendix A: Methods.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
3
Moreover, half
of the students in our sample
reported nagging uncertainties with concluding and
assessing the quality of their own research efforts.
3
Have I done a good job? How do I sort
through all that I
ʼ
ve
found to find what I need? How do I know when to stop looking?
All in
all,
the fi
ndings suggest students in both
large universities
and small
colleges
use a
risk
-
averse
strategy
based on efficiency and predictability in order to manage and control the
information available to them on campuses. Still, most students struggle with the same
frustrating open
-endedness
when trying to find information and conduct research
for college
courses
and to a far lesser extent, for solving an information
problem in the
ir personal lives.
Major findings are as follows:
1.
Students
in the sample took little at
face value
and reported they were frequent
evaluators of information
culled from
the
Web and
to a lesser extent,
the campus library.
More often than anything else,
respondents
considered
whether information
was up
-to-
date
and current
when evaluating Web content (77%) and library materials
(67%) for
course work.
2.
Evaluating
information
was often
a collaborative
process
—almost two
-thirds of the
respondents (61%)
reportedly
turned to
friends and/or family members
when they
needed help and advice with
sorting through
and evaluatin
g
information for personal use. Nearly
half
of the students in the
sample
(49%)
frequently
asked instructors for assistance with
assessing the qua
lity of sources for course work
—far fewer
asked
librarians
(11%) for assistance.
3.
The majority of the sample
used
routines for completing one
research assignment to the next, including writing a thesis
statement
(58%), adding personal perspective to papers (55
%),
and developi
ng a working outline (51%). Many techniques were
learned in
high school
and ported to col
lege, according to
students we interviewed.
4.
Despite
their reputation of being avid
computer
users who are fluent with new
technologies, few students in our sample
had
used a growing number of Web 2.0
applications
within
the past six months
for collaborating on course research assignments
and/or
managing research tasks.
5.
For over
three
-fourths (84%)
of the students surveyed, the
most difficult step
of
the
course
-related research process
was getting
started
. Defining
a topic (66%), narrow
ing
it
down (62
%), and filter
ing
through irrelevant results (61%)
frequently
hampered students
in the sample, too.
Follow
-up interviews suggest students lacked the research acumen
for framing an inquiry in
the digital age
where information aboun
ds and
intellectual
discovery was
paradoxically
overwhelming
for them
.
6.
Comparatively, students
reported having far fewer problems finding information for
personal use, though sorting through results for
solving an information problem
in their
daily lives hamstrung more than a third of t
he students in the sample (41%).
3
On average, 48% of the sample reported having difficulties with these three steps during their course
-related research
process.
For over three
-
fourths (84%) of
the students
surveyed, the most
difficult step of the
course
-
related
research process
was getting
started.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
4
7.
Unsurprisingly, what mattered most to students while they were working on course
-
related research assignments was passing the course (99%)
, finishing the assignment
(97%), and getting a good grade (97
%). Yet, three
-quarters of the sample also reported
they considered carrying o
ut comprehensive research of a
topic (78
%) and learning
something new (78%) of importance to them, too.
Our analysis shows
robust relationships
and similarities
among
variables from our sample of
students at 25
educa
tional institutions in the U.S.
However, t
hese findings should
not be viewed
as comprehensive,
but as another part of our ongoing research.
While additional research is warranted in
order to confirm whether or not our conclusions may be
generalized to the nationwide college and university population, the size of our sample and
consistent patterns of responses do lend credibility to our findings.
In the following pages, we present deta
iled fin
dings from our analysis in three
parts:
Part One:
A comparative analysis of how students find information and prioritize their
use of information sources, based on survey data from last year (2009) and this year
ʼ
s
survey (2010).
Part Two:
Findings
about how students evaluate information
they find on the Web and
through the library for course work and personal use.
In addition
, findings about how
students use
routine techniques
for
completing for course
-related research assignments,
including their
use of Web 2.0
applications.
Part Three:
Findings about the difficulties, challenges, and obstacles students frequently
encounter during the entire research process
—from start to finish
—for course work and
for personal use.
Approach
O
ur ongoing study is grounded in information
-seeki
ng behavior research. We study
how college
students conceptualize and operationalize course
-related and everyday
life research. We
investigate
these research processes through students
ʼ
accou
nts, reports,
experiences
, and
processes
.
We
define
course
-related research
in broad terms
—from the moment students receive a
research assignment in a college course through collecting materials until
turning in the final
assignment
to an instructor
.
According to this year
ʼ
s survey res
ults,
the majority of
students
in the sample
reported frequently
conducting research for argument papers (74%), oral presentations (60%),
and
interpretative
readings of texts (i.e., close readings) (57%).
We
define
everyday life research
as the research students conduct for
personal reasons and for
use
in their daily lives.
When it came to conducting research for persona
l use, the majority of s
tudents in this year
ʼ
s
sample most frequently searched for information about news and current events (79%),
pur
chasing a product or a service (74%), health and wellness (74
%), or work and/or career
(67%), travel and trip
-planning (61%), and social contacts (51%)
The Survey
We
collected data
from a large voluntary sample of sophomores, j
uniors, and seniors enrolled at
U.S.
colleges and universities
during
the spring
of 2010.
4
Our survey instrument was constructed
to collect data about how
students
fin d, use
, and appl
y information for course work and personal
use
. We also collected data about the difficulties they have with
research steps that occur throughout.
To frame our research questions in this study, we
have
draw
n on our
own ongoing research and
existing literature about information literacy
and
research
studies
about digital natives.
In particular, we asked:
1. How
do
undergraduates
find, evaluate, and select the
information they need for course
-
rel
ated
and everyday life research?
2. What techniques and routines
do
students
use
for information
and
fulfilling cour
se-
related research assignments
, including their use of Web 2.0 applications
?
3. What difficulties arise for students during the
different steps and
stages of
course
-
related an
d everyday life research?
4. What recommenda
tions can we make as researchers,
in light of this study’s findings, for
teaching and working with today’s students?
We
analyzed our data using frequencies, cross tabulations, scales, and statistical
tests
about
comparative groups within the sample
.
5
We also conducted a small set of follow
-up telephone
interviews with students from the sample to provide qualitative texture
to the survey data.
6
Detailed Findings
Part One
: Finding Information Revisited
S
ince 2008,
we
have conducted
ongoing research about
how
students
find information,
including
how students
in the digital age conceptualize and operationalize research
.
So
far
, we
have
discovered
students
begin their
course
-related research activities
in search of
research contexts
,
in one form or another. Specifically, finding
a research
context entails getting
4
See Appendix A for more details about the study
ʼ
s research methods
, for descriptive data about the sample
, and for
who participated in the study
. In addition
, see page
45 of this report (Appendix A: Methods) for a discussion of the
sampling meth
ods we used and the limitations of voluntary survey samples.
5
We used t
-tests and analysis of variance (ANOVA) to
compare whether the differences among
groups (i.e., disciplinary
areas of study and enrollment level and dif
ficulties with research stages) w
ere statistically significant.
6
We conducted 25
follow
-up
telephone interviews (15
– 30 minutes in length) during August and September 2010 with
students in the sample
who had volunteered their time.
The purpose of the interviews was to add qualitative texture in
the form of supplementary details to the survey results.
The script appears at the end of the
Appendix A: Methods
.
How do
undergraduates
find, evaluate, and
select the
information they
need for
course
-
related and
everyday life
research?
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
6
information for interpreting an
d defining information need. It
is often a laborious and/or frustrating
step in the student
s
ʼ
research process.
We
have defined
a working typology of
four research context
s undergraduates often seek
during
their research processes, whether they are searching for information for course
work or personal
use
(Figure 1)
.
The contexts we have defined are for finding: (1) big picture context or background about a topic,
(2) the meaning of language, (3) situational factors, including another person
ʼ
s (i.e., instructor
ʼ
s)
expectations, and (4
) relevant information sources from all the sources that may be available.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
8
Our
comparative
analysis
is revealing
and informative,
especially giv
en the collective sample sizes of
over
10,000
respondents
surveyed from 31
U.S. colleges and universities.
The results
are
very similar
from one
year to the next.
We summarize the findings as follows:
1.
The
students in the 2010 sample used the same set of information resources
for course-
related research
in the same order of frequency
as
did students in
the 2009 sample
(Figure
2).
9
In other words, most students
studied in either year
ʼ
s sample relied on t
he same few
sources
of
information
and
turned to the same sources first, second, third, and so forth
to
fulfill course
-related research assignments.
2.
Notably
, almost all
students
reported tu
rning to course readings first
—not
to search engines
such as
, as assumed
by some librarians and
educators
.
10
In addition
, students
consulted
Wikipedia
to a lesser extent than
they used
instructors, scholarly research
databases, search engines, and course readings when completing research for courses.
3.
Students
in the 2010 sample and students in the 2009 sample
exhibited the same
preferences for information sources used in
everyday life research
—with only
a few
exceptions (Figure 2).
4.
In large part,
almost all
students we have
studied turn to search
engines
—such as Google
—and Wikipedia, and friends most often
when looking for information for use in the
ir daily lives.
5.
Students in the 2010 sample used social networking sites and
friends more frequently than did the 2009 sample. Also, a third
fewer students in this year
ʼ
s sample (40%) used scholarly research
databases for everyday life research
than did students in last
year
ʼ
s sample (66%)
.
6.
Students
relied on
librarians infrequently, if ever, whether they were
conducting research for course work or for personal use.
Moreover, students in this year
ʼ
s
sample reported
using librarians
less often than they reported
in the 2009 survey results.
The
comparative
findings from this analysis
are
a milestone
in our research about how students
conduc
t resea
rch in the digital age. T
his year
ʼ
s survey findings
validate last year
ʼ
s findings.
Moreover, the
data provides
strong evidence that
students
are
driven by familiarity and habit and
that
they
use
the same set of information resources in a
very
similar
order of preference
for course-
related
and everyday life research
.
11,12
9
For the 2009
percentage results of
resource prioritization
for course
-related and everyday life research, we averaged
students
ʼ
use of sources for finding big picture, language, situational, and information
-gathering context.
See findings
and the discussion in
A. J. Head and M. B. Eisenberg (2009)
. Lessons Learned: How College Student Seek Information
in the Digital Age.
In all cases, the percenta
ge has been calculated to represent “use” and is based on conflated
responses of “almost always,” “often,” and “sometimes.”
10
For discussions of students
ʼ
predominant use of the Web, including search engines such as Google, see S. Kolowich
(2009).
Searching for Better Research Habits
.
Inside HigherEd
and
C. Thompson (2003). Information Illiterate or Lazy:
How College Students Use Web Resources.
Libraries and the Academy
,
3
(2), 259
-268.
11
In particular, the comparative sets of findings
from 2009 and 2010 student surveys
add
construct validity
to our
findings
about how students prioritize their use of information resources, as the measures ask precisely about
the very
phenomena and underlying
construct
that we purport to be measuring.
12
To evaluate agreement between the rankings for each year of survey results in Figure 1 and Figure 2, we
calculated
Kendall's W,
also known as the coefficient of concordance, Generally, Kendall's W ranges from
0 (no agreement) to 1
(100% agreement). In our results, the result for
course
-related research was 1.0 (2009 vs. 2010) and the result for
Students in this
year’s sample
reported using
librarians less
often than they
reported in the
2009 survey
results.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
9
Part Two
: Evaluating and Using
Information
None of the old
--timers—
the old professors
—can really give us m
uch advice on sorting
through and evaluating resources.
I think we're kind of one of the first generations to have too
much informati
on, as opposed to too little. We’ve
never had instruction really on navigating
the Internet a
nd picking out good resources.
We've kind of been tossed into this and we've just
learned through e
xperience we have to go on a Web
site and
just raid it for information.
So I
would say that despite all that
’s out there, it certainly is harder to find the right source and
evaluate whether it's good
, or not
, because there's so much
—you only have a little bit of time
to spend on each source
you find
.
-- Engineering
student
in a follow
--up interview
A
n essential step in the research process
is evaluating the quality of information
once it is
found
. Is the
source credible?
Is the information up
-to-date? Is the information accurate? Is the
source useful
for the solving t
he information problem
at hand
?
Wheth
er it occurs with a cursory glance or an exhaustive review,
evaluation
requires
critical
thinking and dec
ision
-making about the validity
of a source
based on
a diverse set of
available
criteria. Evaluation involves
assessment about the
poten
tial usefulness of information
within a
set of circumstances
and
information
needs.
In this se
ction, we investigate how college students evaluate
information
they have found
for
course work and
that they
use in everyday life. Further, we investigate
students
ʼ
routines
and
techniques for applying sources they have selected to course research assignments.
How Students Evaluate Web Content
How do students evaluate content they have found on the
World Wide Web
—the omnipotent,
go
-to public source
?
13
The need
to vet the Web
ʼ
s staggering amount
of
unfiltered and collaborative content is a topic
frequently addressed
by librarians and educators
.
14
Much of the discussion
centers
on
effective
methods
of
determining credibility
and reliability
of
content
, coupled with the
potential complexity
of the Web evaluation process
—depending
, of
course, on how thoroug
h an assessment is.
15
,16
everyday life research (2009 vs. 2010) was .98,
indicating
that the
rankings
from the 2009 and 2010 surveys
, per type of
researc
h, were very high in agreement.
13
Definitions for public Internet sources can be
ambiguous
. We
define public Web sources as a subset of the Internet
that has a URL ending in .com,
gov. or .org and further, that tends, for the very large part, to be
“no fee” vs. “for fee.”
14
See
ACRL
ʼ
s Information Literacy
Competency
Standards for Higher Education
(2000) and a discussion about the
“escalating complexity” of the information landscape
that raises questions about
“authenticity, validity, and reliability
” of
“unfiltered” and “multiple media” (accessed from the Association of College and Research Libraries
(ACRL) site on
August 18, 2010).
There have
also been practical guides for evaluating
Web content, such as J. Kapoun (1998).
Teaching Undergrads Web Evaluation: A G
uide for Library Instruction.
C&RL News
, July/August 1998, 522-
523.
15
For further reading, see S. S. Sundar (2007).
The MAIN Model: A
Heuristic Approach to Understanding Technology
Effects on Credibility. In
M. J. M
etzger & A. J. Flanagin (Eds),
Digital Media, Youth, and Credibility
. Cambridge, MA: MIT
Press,
2008,
73-
100 and M. J. Metzger (2007). Making Sense of Credibility on the Web:
Models for Evaluating Online
Information and Recommendations for Future Research.
Journal of the American Society of Information Science and
Technology
,
53
(13), 2078.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
10
We
collected data from students about three types of
evaluation criteria
that
may be
used for
evaluating the quality of information:
(1)
timelines
s and authority,
(2)
dom
ain-
specific
standards,
and
(3)
self
-taught methods
.
17
In Figure 4, we outline the evaluation criteria used in the analysis.
Figure 4: Evaluation Criteria for Web Content
Types of Evaluation Standards for
Web Content
1.
Traditional standards of timeliness and authority: Timeliness is the basis for determining
the currency of research material (e.g., publication date). Authority is the basis for
determining whether a source has reliable authorship (e.g., who created the so
urce;
author
ʼ
s credentials). These criteria are derived from the scholarly print world and
librarianship
.
18
2.
Domain
-
specific standards: Domain
-
specific standards have been ushered in by the
Internet and are often used for judging the reliability, authority, and credibility of Web
content. In our analysis, we included criteria for deciphering the origins of a URL,
analyzing the presence of external links to other sites on the Internet, and
the presence
of footer details (e.g., when a site was
last updated
)
.
19
3.
Self
-
taught methods: Self
-
taught methods are the basis for judging content by applying
intuition and/or te
chniques acquired from friends, classmates, or other informal
contacts.
20
In our analysis, we include familiarity with a site and/or gauging a site
ʼ
s visual
presentation for judging authority and credibility. These criteria tend to be personal and
highly qu
alitative formal standards.
16
For purposes of our discussion, we use Tseng
ʼ
s and Fogg
ʼ
s definition of Web credibility as being a perceived quality of
the believability
as determined from the
perspective of the observer
ʼ
s individual perception. See S. Tseng and B. J.
Fogg (1999).
Credi
bility and Computing Technology.
Communications of the ACM
,
42
(5), 39-
44.
17
Our use of the three types of evaluation methods in our analysis are not meant to be an exhaustive
collection of
methods
and criteria
that
could b
e applied for determining the qua
lity of information
presented. Rather, our methods are
a representation of what students in our ongoing research have reported frequently using, as well as
standards notably
communicated by
library Web sites about evaluation methods. For comparative purpos
es and given the scope of our
research inquiry, we have also included a category for librarian referrals as criteria for determining the
quality of Web
content. For a discussion of the checklist vs. contextual approach to evaluating resources, see C. Meola
(2004).
Chucking the Checklist: A Contextual Approach to Teaching Undergraduates Web
-Site Evaluation
.
Libraries and the
Academy
, 4,
(
3), 331-
344.
18
This
a general discussion
of two librarian standards for assessing the information quality of research sources and
suggest the following sites for more background information: U.C. Berkeley
ʼ
s Library and
How t
o Critically
Evaluate
Information Resources
from Cornell University
ʼ
s Libra
ry (accessed on July 30, 2010).
19
In developing our survey questions about the use of Web evaluation standards, we consulted
several library online
evaluation guides, including tho
se from
Blue Ridge Community College
, Colgate University
, New M
exico State
University
, Purdue University
, and U.C. Berkeley
. (Access
ed on January 5, 2010).
20
During our student discussion groups
in 2009 at six institutions
, participants described using
various methods for
determining the information quality of sources. Some methods students discussed were described as
being self
-taught,
such as relying on brand loyalty of a site (i.e., Google) or gauging the design of a site. For related
scholarly resear
ch
about how students assess Web content, especially the credibility of sites, see E.
Ha
rgittai, L. Fullerton, E. Menchen
-
Trevino, and K. Thomas (2010).
Trust Online: Young Adults
ʼ
Evaluatio
n of Web Content.
International Journal of
Communication
,
4
, 468
-494 and C. N. Walthen and J. Burkell. (2002).
Believe It or Not: Factors Influencing Credibility on
the Web
.
Jour
nal of the American Society of Information Science and Technology
,
53
(2), 134
-144. (Accessed on August
3, 2010.)
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
11
We present a comparative chart
in Figure
5 with results about the most frequently applied
evaluation criterion.
Figure 5
: Criteria for Evaluating Web Content
Results
are
ranked from most frequent to least frequent
evaluation techniques
respondents used for course work (blue bars)
. Evaluation
techniques for personal use
(green bars) do not correspond to the same order as those for
course work.
Responses of “almost always”
and “often” have been conflated into a new category of “frequent use.”
See Appendix B
, Figures 5A and Figure 5B
for complete data sets.
In Figure 6
(on the following
page)
we present
a side
-by-side comparison of the criteria students
used for evaluating Web
content for course work and
personal u
se.
Taken together, the findings suggest students are frequent evaluators of the information they
find on the Web
. The majority of students applied a blended approach to critically evaluate both
Web content for course work or personal use, drawing on
procedural and formal standards for
judging information quality and on self-
taught methods.
When evaluating Web content for course work, students in the sample most often used formal
standards for sizing up the quality of what they had found. They assessed whether Web content
was up
-to-date (77%) more than they used any other proxy for informatio
n quality.
Yet, many respondents also used self
-taught methods nearly as frequently as formal standards,
especially assessing the design of a site (71%). In follow
-up interviews, students mentioned
checking a site for typos, misspellings and a poor use o
f graphics and navigational systems as a
basis of evaluating design and the credibility of content.
77%
73%
71%
71%
66%
62%
62%
61%
59%
59%
54%
25%
54%
48%
49%
56%
43%
54%
44%
39%
32%
42%
23%
11%
0%
25%
50%
75%
100%
Currency (e.g., publication date)
Author’s credentials
URL (e.g., Web domain)
Interface design
External linkage (if links exist)
Familiarity from previous use
Heard about site before
Chart quality (if they exist)
Author credits others for ideas
Different viewpoints acknowledged
Bibliography included
Mentioned by librarian
Course work
Personal use
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
12
Figure 6
: Comparative Details about
Evaluating
Web Content
Web Content for Course Work
Web Content for Personal Use
1.
Most often, students in the
sample used formal
standards
—
authority and timeliness
—
for
evaluating Web content. Respondents considered
the currency of information presented (77%) and
to a lesser extent, an author
ʼ
s credentials (73%)
and/or a site
ʼ
s URL (71%).
2.
Two
-
thirds of the sample
also used self
-
taught
methods when evaluating Web content for course
work. Most often, respondents considered the
design of a site (71%), and to a lesser extent,
familiarity from prev
ious use (62%),
or having
heard
about a site (62%).
3.
About half the sampl
e used domain
-
specific
standards for judging Web content for course
work. Respondents checked for external links
(66%), whether an author gave credit for sources
used (59%), and/or if a bibliography exists (54%).
1.
More students in the sample relied on sel
f
-
taught
methods for evaluating Web content for personal
use. Respondents considered a site
ʼ
s design
(56%) most often and to a lesser extent whether
they were familiar with a site from prior usage
(54%).
2.
About half the respondents used formal standards
of
timeliness (54%) and authority when evaluating
a site for personal use. Standards included judging
a site
ʼ
s URL (49%) and/or an author
ʼ
s credentials
(48%).
21
3.
Less than half of the respondents used domain
-
specific standards for determining the accuracy
of
Web content for personal use. Respondents
checked for external links (43%), whether an
author credited sources used (32%), and/or
whether a bibliography existed (23%).
By comparison,
in this case, students in the sample evaluated Web content less
often when
information was for personal use. Still, the majority of students surveyed relied on self
-taught
methods to suss out quality, including a site
ʼ
s design (56%) and whether they had used the site
be
fore (54%) along with formal standards such as how
current information presented was (54%).
A referral from a librarian had little impact on how students weighed the information
they found
on a
Web site.
Only one in four of the respondents (25%) frequently considered whether a
librarian had referred them
to using a site when they evaluated the quality of Web content for
course
-related research or even less so
, for everyday life research (11
%).
Asking Others for H
elp
and Advice
In our prior research, we found
that
students, in general, relied on several key people for
guidance during their research process.
22
21
In a 2010 study of first-
year college students and their online credibility assessment practices, researchers found
branding (e.g., trust in certain search engines and the results they returned) and routine (i.e., familiarity
with a site from
previous use)
were “essential signifiers” for assessing the credibility of Web sites. Our findings about
familiarity
(i.e.,
evaluating a site based on previous use and/or hearing about a site) corroborate this study
ʼ
s findings. See E. Harigittai et
al,
Trust Online; Yo
ung Adults
ʼ
Evaluation
of Web Content, op cit.
22
In our 2009 student survey, we found, on the average, respondents reported using the following people
as sources for
course
-related research: Instructors (88%), friends (58%), and librarians (47%). For ever
yday life research, respondents
reported using the following people as sources
: Friends (85%), instructors (73%), and librarians (33%).
See findin
gs and
the discussion about
“Resource Prioritization”
when the need for certain research contexts arises
on pa
ges 14
- 18
in A.
J. Head and M. B. Eisenberg, (2009), Lessons Learned: How College Student Seek Information in the
Digital Age.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
13
In this study, we investigated this finding in greate
r depth. We asked students
whom
they turned
to when asking for
help
with evaluating both online and print sources for course work and
personal use.
In Figure 7
, we present a comparative chart. We ranked the people respondents turned to for
evaluation help when conducting research for course work and personal use.
Figur
e 7
: Asking for Help with Evaluation
Results are r
anked from most frequent to least frequent
used people that students turn to for evaluation guidance and help. Responses of
“almost always” and “often” have been conflated into a new category of
“frequen
t use.” See Appendix B, Figure 7A and 7
B for complete
data.
We summarize the findings as follows:
1.
Almost two
-thirds of the students sampled (61%)
asked friends and/or family when they
needed
help with evaluating sources for personal use
—more than
they asked
classmates (43%) or instructors (18%)
—or anyone else.
2.
For course
-related research, more
students in the sample turned to instructors for help
(49%)
with evaluating course
-related research materials found on the Web or through
the library.
23
3.
Few students in the sample asked librarians (11%) or writing center staff (7%) for help
when judging the information quality of sources for course work, and even fewer used
turned to librarians for help evaluating information for personal use (5%).
4.
A smal
l percentage of the sample
turned to licensed professionals (e.g., physicians,
attorneys, therapists) for help with evaluating information for use in their everyday lives
(12%)
or course work (
5%).
23
See findings and the discussion about the r
ole of instructors as resea
rch coaches on pages 28
- 30 in Head, A. J. and
Eisenberg, M. B. (2009),
Lessons Learned: How College Student Seek Information in the Digital Age.
49%
32%
18%
11%
7%
5%
18%
43%
61%
5%
12%
0%
25%
50%
75%
100%
Instructors
Classmates
Friends/Family
Librarians
Writing Center
Licensed Professionals
Course work
Personal use
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
14
Taken together,
the
findings suggest students frequently turn to certain
go
-to people when they ask for
help with evaluating the information they
have found.
At the same time, the findings suggest evaluat
ion is far
from being a solitary task for most students.
When conducting research for course work, students most frequently
turned to instructors (49%) for help. During research for personal use,
students relied on support from friends and/or family (61%)
.
These findings suggest evaluation requires collaborative support and
input from a friend, family member, or instructor.
According to these findings, few students in our
sample let W
eb content
stand on it
s own without some discussion
with someone else.
Focus on
Course
-Related Research
As
an additional part of our inquiry we focused on course
-related research and how students in
the sample evaluated and applied information.
In this section, we
present results
about how students evaluated Web and library sources
and
services
for course work.
We also
present findings about the routines they used
for completing
one assignment to the next.
Evaluating Library Materials
vs. Web Content
In our first analysis
, we
investigated how students
evaluated library
materials
—books, print
journals, and scholarly research database materials (e.g., EBSCO, JSTOR, and/or ProQues
t).
When conducting
research for course work
, do
students evaluate
library materials as frequently
and using the same criteria as they do
Web content? The results appear in Figure 8
(on the
following
page)
.
Overall, the results indicate students in the sample evaluated Web content using more available
criteria than they did library materials (i.e., pr
int and online sources).
At the same time, the majority of students surveyed reported using four or fewer standards
(57%) for evaluating library materials.
24
In general, students evaluate Web content for course
-
related research using more standards of scr
utiny than they do for everyday life research.
24
We calculated an evaluation score based on the frequencies per respondent with which specific
responses occurred
about criteria for evaluating Web content and library sources. We then summarized the
results of the scores by using
percentages.
Few students in
our sample let Web
content stand on
it
s
own
without
some discussion
with someone else.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
15
Figure 8
: Criteria for Evaluating Library Sources
vs. Web Content
Ranked from most frequent to least
frequent evaluation techniques
. Responses of “almost always” and “often” have been conflated into a
new category of “frequent use.”
See Appendix B
, Figure
8A and Figure 8B,
for complete data sets
Moreover, when we calculated individual “evaluation scores,” for each respondent, we found the
majori
ty of students in the sample (55
%) reported using
7 or more standards (e.g., currency,
URL, and interface des
ign) for evaluating Web content and about half as many, 4 or fewer
standards, for judging library materials.
The results appear in Figure 9.
Figure 9: Evaluation Scores
Source of Information
Evaluation Score
Public Web sites
7
or more standards
used
Library materials (print and online)
4 or fewer
standards
used
There is a logical explana
tion for why
students evaluated
library materials less often
. Librarians
use collection development policies for building
their
collection, editorial boards of peer
-reviewed
scholarly journals, and aggregators of scholarly research databases.
Given these assurances, it is fair to assume that students think librar
y sources require less
evaluation than information posted by anyone on the open
-source Web. Students may figure a
less exhaustive evaluation is needed when they are conducting course
-related research.
Still,
currency matters. S
tudents in the sample weigh
ed the currency of information they found
for course work more often than anything else, whether they found the
content on the Web
(77%) and/or to a lesser extent, in
the library (67
%).
67%
57%
50%
46%
45%
40%
39%
29%
22%
18%
77%
61%
59%
59%
54%
73%
63%
62%
72%
25%
0%
25%
50%
75%
100%
Currency (e.g., publication date)
Chart quality (if they exist)
Different viewpoints acknowledged
Author credits others for ideas
Bibliography included
Author’s credentials
Familiarity from previous use
Heard of source before
Publisher of source
Mentioned by librarian
Library materials
Web Content
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
18
For many students
we inter
viewed
, evaluating sourc
es was
time
-consuming and arduous,
though it was
perceived as
essential
. Not only was t
he evaluation process
inextricably tied to
scaffolding
a sound argument,
but also
selecting valid and reliable sources
for an assignment
was
important to their credibility as researchers
and a
s student
s.
A student in the social sciences explained:
Evaluation is the hardest
part of research
for me, honestly because it's the part that involves
the most critical thinking. So, that tends to be harder than just picking a topic or whatever.
But I think the main thing is trying to sort through all of the i
nformation that you've gathered
and you have different points of view and you have different authors
—different people that
have already written about a topic
—so, then it becomes your job to sift through all that
information and make sense of all that you h
ave found for yourself. I think that's the most
challengin
g part of a research assignment
—I always spend time on evaluation because it
hurts my own credibility as a researcher if I don’t
scrutinize
the resources I’m relying on.
As a whole
, our
results show
that the
students surveyed
were
frequent evaluators of information
when condu
cting research for course work.
The
evaluation scorecard indicates
about
twice a
s
many standards were used to
evaluate Web content, than library
materials.
However, the underlying question about
how much time and effort
students in the sample actually spent on
evaluation
resulted in some
revealing qualitative analysis from the interviews. Few
students we
interviewed thought of
evaluation
as a
sep
arate, disembodied step in
their
research process.
Rather, a
greater number of the students interviewed
—two
-thirds
—
considered evaluation a
necessary winnowing process for
scaffolding
arguments and information
they presented in their assignments
, which
was tied to their credibility as researcher
s.
27
Research Routines for Course Work
Most of college I've used just about the same techniq
ues from one assignment to the next: I do
the research,
then I figure out what I'm going to write, and then I
sit down and write the
paper.
I kind of took the best parts of what they had us do in high school and got rid of all the
tedious parts, like writing 50
0 note cards and la
beling them.
I've condensed it to just taking
notes on pieces of paper
and different things like that. So, my research style is somewhere
between high school
—where they fo
rced you to do their techniques
—and then college
—
where they don't
really care where you get your information—
as long as it's right and you
write a good paper.
-- Student in the sciences
in a follow
--up interview
In this part of our analysis, we turn our attention to
research styles
—the practices, routines,
techniques, and workarounds students use
for completing course
-related research assignments
from one class to the next.
27
We acknowled
ge the findings from our interviews about how much time and effort students actually
spend
on
evaluating Web and library materials are limited by the size
of the sample and the findings and warrant further research.
Few students we
interview
ed
thought of
evaluation a
s a
separate,
disembodied step
in their research
process.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
19
In our prior research, we have stated that today
ʼ
s students are not naïve about sources,
systems, and information services available to them and apply
their problem
-solving strategies
that help them meet their school needs, as they arise.
28
In this year
ʼ
s survey, we examine this finding
in greater depth by asking what research styles
and routine techniques do students
often
have in their repertoire for c
ompleting course
-related
research assignments
. How do students apply and use information for assignments?
In Figure 10
(on the
following page)
, we rank the techniques, practices, and approaches
that
students in the sample
most frequently
used for course
-related research.
We summarize the findings, as follows:
1.
Over half the students in the sample used routines
for carrying out the writing part of
course
-related research assignments, including crafting a thesis statement (58%),
including their own pe
rspec
tive in what they wrote (55
%), and developing and using an
outline for executing what they turned in to the instructor (51%).
2.
Students had fewer techniques for conducting research and finding information than
for
writing papers. The most frequently used routines included using some organizational
system for
research materials they
had found (43
%), creating
search terms early on (
36%), and developing a research plan
about where to look for resources (33%).
3.
Routines designed to
save time
and effort were practiced by a
small amount of those in the sample,
such as ending a search
once the required number of citations for an assignmen
t had
been found (26%), allotting an amount of time to complet
e one
assignment to the next (21
%), and start over with an entirely
new topic if they could not find materials in their
first couple of
attempts (12%).
4.
Few students used routines bent on recyc
ling their previous research efforts
, such as
using the same journal articles and/or books
from
one assignment to the next (28%
) and
using the same to
pic for different assignments (6
%).
28
See “What Makes
Today
ʼ
s Students Different” on p. 33 in A. J. Head and M. B. Eisenberg (2009).
Lessons Learned:
How College Student Seek Information in the Digital Age.
Students had
fewer techniques
for conducting
research and
finding
information than
for
writing papers.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
20
Figure 10
: Students
ʼ
Research Styles and Techniques
Results are r
anked from most frequent to least frequent
for research
techniques
students reported using during course
-related research.
Responses of “almost always” and “often” have been conflated into a new category of “frequent use.”
For the complete data set see
Appendix B.
Taken together, these findings indicate half of the students in the sample reported using routine
practices from one research assignment to the next. For the
most
part, the course
-related
research workarounds students used were geared
more toward
writing papers
than to
conducting research and finding information.
In the follow
-up s
tudent interviews, we asked
students about techniques they routinely used and
how they had come to learn them.
Student Interviews: Adapt and Survive
How do students
routinely apply and use information for completing assignments once they have
found it
, and where do their techniques originate?
One of the most frequent routin
es students we interviewed used
was going to
Wikipedia as a
starting point. Nearly half the s
tudents interviewed used Wikipedia for the hypertext
-linked
bibliography at the bottom of most entries
for finding sources
. Most
described
Wikipedia
as
a
source to get them started and they definitely continued with their research, often using library
databases.
58%
55%
51%
43%
36%
33%
28%
26%
26%
23%
21%
12%
6%
0%
25%
50%
75%
100%
Create a thesis statement early on
Add in my own perspective (e.g., for a paper)
Develop an outline for guidance (e.g., writing paper)
Use system for organizing sources found
Figure out search terms early on
Develop an overall research plan for guidance
Use the same set of sources for different assignments
End research after finding citations required
Sit down and write, with no plan at all
Use interlibrary loan for getting sources
Spend same amount of time on different assignments
Start over with new topic, after couple of searches
Use same topic for different assignments
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
21
An engineering student said:
Once I got used to the Internet,
then
a lot of times I'll
just Google a topic. I
don't remember
when I started this but I'll start on Wikipedia, read about whatever I'm trying to figure out
and then go down to the bottom of that page
and look at their references.
I can't really tell you
how it has
even
evolved.
For students such as these their routines were considered to be self
-
taught. Most of the students we interviewed were not sure where their
routines originated, other than from family, friends, or fellow classmates.
Most of these students also discussed the
impact of the Internet on their
research styles for course work.
However,
the vast
majority of
students we
interviewed
reported their
research style
originated
in high school
and was learned from teachers
and librarians and later modified. If the research technique worked for
them and stood the test of time, students ported the technique to
college. S
tudents we interviewed talked about weaving in the basics
from
high school
into their
research
repertoire
for college
—and
often
made
few changes.
A h
umanities student offered the following account:
I studied in Mexico and high
school there is pretty different.
Teachers would give you a
problem and you would have to do it by yourself.
So that type of independence brought me to
where I am today. Even th
ough
high school
was simpler and you didn't h
ave to do as much
research, the high school teachers
showed me the basics, they showed me I needed a deadline
to turn in my stuff, I needed to use encyclopedias,
the I nternet
, books to get my information.
That's
pretty much what I'm doing now. I look at the possibilities and see which one I can do
faster and better.
I start doing research first at my house.
So if I'm doing research on pandas
I'll just to look at Wikipedia and when I go to the library I have stuff
I've found at my house
and I tr
y to look at it in more detail.
For
other students,
high school
gave them the
foundation for writing papers, but
they continued
to adapt and modify their routines as they were in college.
In a few cases
—about 1 in 10
—
stude
nts we intervi
ewed mentioned
having learned
how to conduct research and find
information
from campus librarians.
A business administration student explained:
Thinking back to high school
, I had a couple of really good teachers that were pretty
fundamental in giving
me my base for writing papers. B
ut I don't think they quite emphasized
the research portion of it
all enough.
In college
, I've gotten a lot better job at scouring lots of
differ
ent sources and finding more information, more thorough information, and
counterbalancing arguments
, and things like that.
The honors res
earch project was a really
good. W
e got to sit down with librarians and go
through a
search methodology and how you
loo k for sources and how sources link to other sources and how you can use that, which is
something that I ne
ver even thought about before. Things like j
ust going through a
bibliography for a paper that you think will be useful and looking at the sources they
've used
and so
that's sort of what I look at now.
In the follow
-up interviews, we found the majority of
students discussed their research styles as
a collection of
techniques
they had learned in
high school and
adapted and sometimes
re-
fashioned for use in college, as needed.
Students we
interviewed talked
about weaving in
the basics from
high school into
their research
repertoire for
college
—
and often
made
few changes.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
22
In a few cases, the students we interviewed did have a fuller grasp of research as an iterative
process. Where did the
se students learn the process?
They learned
it from
campus librarians.
Productivity Tools
for Research
As another step in our analysis about course
-related research routines
, we investigated how
often students in the
sample used productivity tools
for supporting various tasks of their course
-
related research process.
Some of the data we colle
cted
pertained to
the use of Web 2.0 programs, including social
bookmarking sites, blogs, and wikis within the last
six
months. For purposes of the survey, we
defined We
b 2.0 programs as applications
that were
interactive
, running on a Web platform, and
used for
information shar
ing and building collective knowledge
.
29
In Figure 11
, we rank the tools students frequently used while working on course
-related
research assignments within the last six months.
Figure 11
: Productivity Tools Used for
Course
-Related Research
Results are r
anked from most frequent to least frequent
for research
techniques
students reported using during course
-related research.
Responses of “almost always” and “often” have been conflated into a new category of “frequent
use” in the last 6 months. For the
complete data set see Appendix B.
29
We are grateful to
colleagues
at
the British Library and JISC, who
granted us
permission to use and modify thei
r
survey question (#12), which was used in their
2010 “Researchers of Tomorrow Survey.” The survey was
administered
to British doctoral students
born between 1982
and 1994, in October 2009.
The researchers found that “only a small
proportion of respondents in any age group say they use
ʻ
emergent technology (e.g., Web 2.0 applications) in their
research, although most those that do find it valuable,” in
Researchers of Tomorrow. Interim Report 1
(2009). Summary
Report, p. 1. (Accessed September 10, 2010)
62%
55%
48%
29%
26%
24%
21%
18%
14%
14%
10%
10%
9%
8%
0%
25%
50%
75%
100%
Highlighting feature in software package
Citation-making programs (e.g., EndNote)
Document sharing applications (e.g., Google Docs)
Digital “Sticky Notes”
Online forum
Photo sharing sites (e.g., Flickr)
VOIP (e.g., Skype)
Wikis (other than Wikipedia)
Virtual Research Environments (VREs)
Blogging (e.g.,LiveJournal)
Social bookmarking (e.g., digg, delicious)
Web alerts (e.g., Google Alerts)
Online time management (Google Notebook)
Microblogs (i.e., Twitter)
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
23
We summa
rize the findings as follows:
1.
More students
used
software solutions, including
highlighting text on the screen (62%)
and
formatting
citations
(55%) for the solitary management
of course
-related research
tasks as opposed to Web applications for collaborating and working with other students.
2.
When it came to Web 2.0 applications, more students used an application for
document
creation and shar
ing, such as Google Docs (48%), than they did
online forums (26%)
photo
-sharing sites, such as Flickr (24%), wikis (other than Wikipedia) (18%), and
blogging (14%) for supporting course
-related research tasks.
3.
Few students used
virtual research environmen
ts (14%), where online tools, processes,
and systems exist
for managing research projects.
4.
Only 1 in 10 students used social bookmarking
(10%), such as
delicious,
for organiz
ing and sharing Web content with others or
alerting programs for automated content monitoring.
5.
Only 8% of the sample of students reported frequently using
microblogs, such as
Twitter, for managing various tasks
associated with their course work.
As a who
le, students in the sample were not avid users of Web 2.0 applications
for
supporting
course
-related research
tasks
. The most frequently used Web application
s were document
sharing Web
-based applications, such as Google Docs, available since 2006.
By comp
arison, r
elatively few students used popular emergent technologies (e.g., Flickr, Twitter,
Delicious) sharing information and collaboratively working on projects together during course
-
related research.
Instead, students in the sample were frequent users
of software productivity tools, such as word
processing programs with an on
-screen highlighting feature or packages for managing
bibliographic citations.
The
se findings are
revealing. The data counter
the argument that
most college students are
digita
l natives and that the majority of them are
comfortable
with technology to the point of
embracing
participatory
engaging
learning environments.
The finding suggests even though students may
be
heavy users of social networking sites, such
as Facebook, Web 2.
0 applications for course research have not
yet
found
their way into
students
ʼ
research repertoire
—yet. Recent research suggests that this trend is very likely to
change within the next few years as students demand for digital course work and time
-shifted
instruction inevitably increase.
30
30
See
the section on “Online Learners” in The Chronicle of Higher Education
ʼ
s research report, The College of 2020:
Students.
(Accessed October 20, 2010).
The data counter
the argument that
most
colle
ge
students are
digital natives...
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
24
Part Three
: Difficulties with the Research Process
What makes course research so difficult is the fact that
I can't give
up on it
—it’s something I
need, something I have to do. But in my
personal life, if I don't find
somethin
g, well, I can
forget about it. I don’t really need it. If it's something I can't find or if it's something that I'm
having trouble finding information on then I can generally just blow it off. But if it's for school
,
if research is something I need to do
—that makes it ten times harder because you have that
pressure.
-- A student in the sciences in a
follow
--up interview
I
n our
last
section
, we
turn
our attention
to the
barriers, obstacles, and challenges students
en
counter as they go through their course
-related and everyday life research process.
In our prior research,
students in our focus sessions reported that research and finding
information was
far m
ore difficult to conduct in today
ʼ
s digital age than it was in
the past
.
31
In this section of the study, we explore this earlier finding in greater depth.
Which parts of the
research process are particularly difficult?
How do difficulties with research differ for students
when they are condu
cting research for course work
as opposed
to
in their everyday lives?
We use two related analyses to
answer
these questions. First, we provide a
microanalysis
of
the
individual steps students reported having difficulty with when conducting
course
-related and
everyday life research.
We also provide a
macroanalysis
of
the stages students reported having difficulty with when
conducting research for course work and for personal use.
In this analysis we present the results
of a “
difficulties
scale” about rese
arch stages.
Microanalysis of Research Difficulties
We began our microanalysis by analyzing data about what students found difficult during the
individual steps associated with research.
32
For course
-related research, we collected data from 20 survey
questions about
student
ʼ
s
difficulties with
research. The difficulties were likely to occur at some point during the research
process
—from the moment a course
-related research assignment is received to when the final
results are delivered (e.g., a research
paper, a multimedia or oral presentation).
33
31
See
conclusions from PIL
ʼ
s report
A. J. Head and M. B. Eisenberg (2009)
. Finding Context: What Today
ʼ
s College
Students Say about Conducting Research in the Digital Age.
The report pres
ents findings from focus groups
with 86
undergraduates
on seven
U.S. college and university campuses in Fall 2008.
32
The questions appeared as #12 (course
-related research) and #17 (everyday life research) in the
online
survey
instrument
we administered
. Data was collected from a series of statements about difficulties and respondents were
asked whether they strongly agreed, somewhat agreed, neither agreed or disagreed, somewhat
disagreed, or strongly
disagreed with each statement. Responses for “strongly
agree” and “somewhat agree” were conflated into a new
category for “agree” in our analysis.
33
In the survey, we used statements about research difficulties students in our studies had reported
having in our prior
focus groups and interviews for both course
-related and everyday life research. We asked respondents how much they
agreed or disagreed with statements (e.g., “Getting started on an assignment is difficult”). We used a
seven
-point scale
for response categories
(i.e.
strongly agree, somewhat agree, n
either agree or disagree, strongly disagree, don
ʼ
t know,
and no experience with this situation).
We recoded the scale responses (interval variables) into dichotomous (nominal)
variables (1: agree and 2: disagree) for reporting the results in Figure 15.
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
25
For everyday life research, we collected data from 15 survey questions about research
difficulties.
The majority of questions we asked about everyday life research difficulties were
similar to
those
asked about course-
related research (e.g., narrowing down a topic, determining credibility,
filtering search results). In a few cases, we excluded certain difficulties in this scale, since the
previous questions asked were associated only with cours
e work (e.g., taking notes, writing, and
citing sources).
What did we learn about students
ʼ
difficulties with research processes? What did the majority of
students in the sample struggle with the most when conducting research for coursework and for
use in
their personal lives?
We provide two comparative charts where we have ranked the difficulties from most to least
frequent
for course work (Figure 12) and personal use (Figure 13
). First, we provide the analysis
of difficulties with course
-related researc
h.
Figure 12
: Difficulties with
Steps during
the
Course
-Related Research
Process
Results are ranked from most to least agreed statements about student difficulties with research
. Responses for “strongly agreed” and
“somewhat agreed” have been
conflated into a new category of “agreed.”
See Appendix B for complete data sets.
84%
66%
62%
61%
46%
42%
41%
41%
40%
38%
37%
37%
35%
31%
31%
30%
29%
29%
26%
21%
0%
25%
50%
75%
100%
Getting started is difficult
Defining a topic
Narrowing down a topic
Filtering irrelevant results
Knowing if good job done
Finding articles in library databases
Knowing how to cite sources
Determining credibility
Reading materials
Writing about what is found
Deciding if done
Finding up-to-date sources
Figuring if use constitutes plagiarism
Creating search terms
Finding Web sources
Integrating information from different sources
Know when to cite
Figuring out where to find sources
Evaluating sources
Taking notes
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
26
We summarize the findings from our analysis, as follows:
1.
Getting started
in the
course
-related research process
stymied
a large majority of
students in the sample (84%) m
ore than any other step
in their research process
. Other
significant challenges occurred toward the beginning of the research process, too,
including defining a topic (66%), narrowing it down (62%), and sorting through search
results
to find relevant mater
ials
(61%).
2.
Students were perplexed with completing the research process with
almost half of the
students
finding it
difficult
to decide whether they had done a “good job,” or not (46%)
.
Over a third of the respondents
finding it
difficult to decide when they were finished with
the research process (37%).
3.
Steps associated with searching for materials were
difficult for
respondents,
but less for
finding Web content (31%), creating search terms (31%), and figuring out where to find
sources on campus (29%).
The most difficult parts of searching were
searching library
databases, such as JSTOR and ProQuest (42%),
and
finding current materials (37%),
4.
Determining the credibility (i.e., believability)
of Web content
was more difficult (41%) than
generally evaluating print and
Web content
(26%) for course work
, according to respondents.
5.
Over a third of the students in the sample reported difficulties
with knowing
how to cite
(41%) and writing about research
results (38%) were difficult steps in the course
-related research
process.
And a
bout a third of the respondents (35%) reported it
was
difficult to figure out if their use of a source constituted
plagiarism, or not, when completing course-
related research
assignments.
6.
About a third (30%) found integrating information from different sources and reading
through research materials difficul
t (40%) during the course
-related research process.
On the foll
owing page, we present Figure 13
, which provide
s the analysis of difficulties with
everyday life research.
...
almost half of
the students
finding it difficult
to decide whether
they had done a
“good job,” or not
(46%).
Project Information Literacy Progress Report: “Truth Be Told” | November 1, 2010 | Head and Eisenberg
27
Figure 13
: Difficulties with Steps during
the
Everyday Life Research
Process
Results are ranked from most to least agreed statements about student difficulties with research
. Responses for “strongly agreed” and
“somewhat agreed” have been conflated into a new category of “agreed.”
See Appendix B for complete data sets.
We summariz
e the findings from our analysis as follows:
1.
Fewer students surveyed reported having difficulties with finding information for personal
use
than with course
-related research
. Respondents reported having the most trouble
with sorting through what they had
found during a search
—separating
relevant from
irrelevant
sources
(41%)
and
deciding when they were
finished
(23%), even if they were
unable to locate the information they knew existed somewhere online (33%).
2.
Few
er respondents had difficulties with searching library databases (20%), finding
current sources (19%), locating sources (18%), creating search terms (17%), and finding
Web content (11%) for everyday life research.
3.
Few
er respondents (19%) found it difficult
to integrate research materials from different
sources difficult when conducting everyday life research and reading through research
materials was difficult for relatively few (19%) respondents.
41%
33%
26%
24%
23%
20%
19%
19%
19%
19%
18%
17%
15%
15%
11%
0%
25%
50%
75%
100%
Filtering irrelevant results
Know answer is somewhere online
Determining credibility
Evaluating sources
Deciding when finished
Finding articles in library databases
Integrating different sources
Defining the topic
Finding up-to-date materials
Reading materials found online
Finding sources
Creating search terms
Narrowing down the topic
Getting started
Finding Web content
- At Sea in a Deluge of Data
- How college students find and use information [Infographic]
