Importance/Performance Evaluation Report

Finance & Econ

3003292_lecture_notes_2.rar

Home >Human Resource Management homework help >Importance/Performance Evaluation Report

lecture notes/1002HSL Lecture 8 1202.ppt

Lecture 8
The Basis of Quantitative Research

Surveys, Questionnaires and Sampling

TWO MAIN RESEARCH APPROACHES – a revision

- Quantitative positivistic

- Qualitative interpretive

Two main approaches to social science research are:

QUANTITATIVE POSITIVISTIC RESEARCH:

The basis of what most of us think of when we think ‘research’
Based on trying to find an ‘answer’ to a given ‘issue’
Embedded in the scientific tradition of experimentation
Assumes that generalisations can be made from answers derived from a ‘representative’ sample.

QUANTITATIVE POSITIVISTIC RESEARCH:

“Physical science” (positivistic in nature)
Seeks abstract explanation (knowledge - “laws”) with a view to prediction
Believes in physical and social reality
Deterministic - the exact questions are formulated before the study by researcher
Strict logic applied to develop tests (statistics)
Aspires to experimental (interventions) tests of the proposition
Seeks replication of the research

QUANTITATIVE POSITIVISTIC RESEARCH:

Based on precise numerical quantification (scores) (reductive)
Representative sampling - random allocation to groups
Uses statistical analyses
Reductive - splits issues into smaller component parts
Researcher tends to own the study
Low personal researcher involvement - seeks to be objective - value free - neutral

ADVANTAGES/ DISADVANTAGES OF QUANTITATIVE POSITIVISTIC STUDIES:

ADVANTAGES

Cheaper than detailed qualitative studies
Can be sharply focussed
Can come closer to establishing “cause-effect” relationships
Representative
Quick - speedy computer analyses

DISADVANTAGES

Inflexible - difficult to redirect
Not probing - cannot follow up
- cannot seek detailed explanations
Summary information - basic conclusions with no rich details
Mechanical view of people
Reduces people to numbers! - not relevant (empowering) to people

QUANTITATIVE POSITIVISTIC RESEARCH:

To conduct quantitative research
need data in numerical form
also need instruments that will provide numerical data
Easy in physical sciences
e.g., temperature, length, time etc
In social sciences use surveys/questionnaires
facilitate the generation of numerical data
e.g., averages

Definitions

Questionnaire or ‘interview schedule’:
A printed list of questions.
Survey
Whole process of conducting an investigation, which involves a number of ‘subjects’.
Questionnaire survey
A survey involving the use of a questionnaire.

SURVEYS USING QUESTIONNAIRES

Ask a large number of questions (items)
to test a number of variables
Data from a large number of respondents
All respondents respond to same items
Can be self-administered or interviewer administered
Accompanied by representative sampling

Roles, Limitations, Merits

Typically questionnaire surveys involve just a sample of the population being studied
Rely on information supplied by respondents
Therefore dependent on:
accuracy of recall
honesty
Problems of exaggeration and underestimation.

Types of questionnaire survey
Interviewer-completion vs Respondent-completion

Interviewer-completion: interviewer conducts interview based on questionnaire, and records answers on the questionnaire.
Respondent-completion: respondents fill out the questionnaire themselves.

Types of questionnaire
survey – characteristics

Interview in street/ mall

Interview at respondent’s home

Method/location

High

Long

Whole population

Expensive

Either

1. Household

Medium

Response rate

Short

Possible length of questionnaire

Most of population

Sample

Medium

Cost

Interviewer

Respondent or Interviewer-completed

2. Street

Type

Types of questionnaire
survey

Questionnaire mailed

Interview by telephone (land-line)

Method/location

High

Short

People with telephone (land-line)

Medium

Interviewer

3. Telephone

Low

Response rate

Varies

Possible length of questionnaire

General or Special

Sample

Cheap

Cost

Respondent

Respondent or Interviewer-completed

4. Mail

Type

Types of questionnaire survey

Organised group (eg. School class)

At leisure/ tourism site

Via email/ Internet

Method/location

Varies

Address list

Cheap

Respondent

5. E-survey

High

Medium

Site-users only

Medium

Either

6. On-site

High

Response rate

Medium

Possible length of questionnaire

Group only

Sample

Cheap

Cost

Respondent

Respondent or Interviewer-completed

7. Captive group

Type

Household survey

Nature
Can cover all adult age-groups
Representative of whole community
Common for government, market research
Conduct
Typically interviewer-completed, but ‘drop-off and collect’ respondent-completion sometimes used – or combination
Omnibus surveys
One questionnaire includes questions on a number of topics for multiple clients.
Time-budget studies
Respondents complete a 1 or 2-day diary of activities.

Street survey

Nature
Conducted in:
shopping street/mall
tourism areas (but may be seen as ‘site survey’)
transport nodes (bus-stations, airports) - tourism
Conduct
Interviewers typically given quotas related to known demographics of the community
Still problems of representativeness regarding people who
are housebound
do not visit shopping streets
do not visit particular tourist locations (eg. VFR, business tourists).

Telephone survey

Nature
Common for political polling.
Problem of excluding those without land-line telephones.
Emerging problem of resistance/non-response in some countries.
Conduct
Fast, using CATI – computer-aided telephone interviewing
Numbers selected/dialed automatically
Data keyed directly into computer.
Not possible to show lists to respondents.
Anonymity may help honesty of response.

Mail survey

Nature
Ideal for surveying widely dispersed lists – eg. members of organisations.
Conduct
Low response rates – eg. 30% often quoted as ‘acceptable’ …. but is it?
Factors affecting response rates – see next slide.

Factors affecting mail survey response rates

The interest of the respondent in the survey topic

The length of the questionnaire

Questionnaire design/presentation/complexity

The style, content and authorship of the accompanying letter

The provision of a postage-paid reply envelope

Rewards for responding

The number and timing of reminders/follow-ups.

Veal, Research Methods for Leisure and Tourism, 3rd edition © Pearson Education Limited 2006

Mail survey response pattern – example

Source: Based on data from Robertson and Veal (1987).

Veal, Research Methods for Leisure and Tourism, 3rd edition © Pearson Education Limited 2006

E-surveys

Nature
Conducted via email/Internet
Dependent on availability of email list or other means of inviting sample to participate
Conduct
Partially electronic
Questionnaire downloaded, printed and completed manually
Fully electronic
Respondents log-on and key responses into web-based questionnaire
Problem of response rate due to growth of ‘junk mail’.

On-site surveys

Nature
Alternative terms:
On-site, site, user, visitor or audience surveys
Common among managers
Conduct
Respondent-completion, if not closely supervised, can result in:
Low response rates
Poor quality responses (incomplete etc.)

Captive group surveys

Nature
Respondents in organised group
May have little choice but to participate
Ethically, must be given the option
Conduct
Typically respondent-completed under supervision
Quick and cheap

SAMPLING
PROCEDURES

POPULATION OF SCORES

A population is the theoretical aggregation of study elements (units of analysis)
i.e. Australians (how defined?)

Populations are usually defined in geographical or other terms
e.g. level of participation of people in Brisbane Metropolitan Area
Populations are not restricted to people
can be associated with other units
e.g. organisations, sports halls, equipment

Samples and Populations

Population:
Total category of subjects that is the focus of attention in a particular research project (can be non-human)
Sample:
A number of subjects drawn from the population
Two key issues:

What procedures must be followed to ensure that the sample is representative of the population?

How large should the sample be?

POPULATIONS AND SAMPLES

A sample is a subset of a population
Samples in positivistic research should be representative
The sample is sampled in order to talk about the population
Samples can be studied
Know precisely what the nature of the sample is!
We estimate and infer nature of populations from samples

POPULATIONS AND SAMPLES

POPULATION

OBTAIN INFORMATION ABOUT SAMPLE

SAMPLE

POPULATION

GENERALISE

SELECT A

REPRESENTATIVE

SAMPLE

Sampling Procedure

Define the population

e.g., geographical (e.g., households in suburb)
month of arrival (e.g., tourists for May)
activity participation (e.g., hours of activity)
organisation (e.g., budgets of tourist hotels)

2. Obtain a representative sampling frame of population units

e.g., electoral role, membership list,
map of households, list of hotels

3. Agree on an unbiased unit selection procedure

4. Apply selection procedure rigorously

Types of Sampling

Convenience	Use of conveniently located persons or organisations – for example, friends, colleagues, students, organisations in the neighbourhood, tourists visiting a local popular attraction.
Criterion	Individuals selected on the basis of a key criterion – for example, age-group, membership of an organisation, purchasers of souvenirs.
Homogeneous	Deliberately selecting a relatively homogeneous sub-set of the population – for example, university-educated male cyclists aged 20–30.
Opportunistic	Similar to 'convenience' but involves taking advantages of opportunities as they arise – for example, studying major sporting event taking place locally, or a holiday resort the researcher is holidaying at.

Types of Sampling

Maximum variation	Deliberately studying contrasting cases. Opposite of 'homogeneous'.
Purposeful	Similar to 'criterion' but may involve other considerations, such as 'maximum variation'
Stratified purposeful	Selection of a range of cases based on set criteria, for example, representatives of a range of age-groups or nationalities.

lecture notes/1002HSL_Lecture_9_1103.ppt

Lecture 9

MEASUREMENT QUESTIONNAIRES AND ITEM CONSTRUCTION

VARIABLES

Variables can take on differing values (attributes) (at least 2) e.g.,:
boys, girls
safe V's unsafe!
kg’s weight
very boring, boring, neither, exciting, very exciting
children, teenagers, young adults, middle aged, new retirees, elderly

VARIABLES

Associated with units of analysis
e.g.:
tourist's leisure experience
depressive patient’s relaxation
equipment’s safety
organisation’s responsiveness
employee friendliness
adventure hikers group size

OPERATIONALISATION OF A (VARIABLE) CONCEPT

Theoretical abstract definitions
Concepts may have several slightly different broad definitions
e.g., health is the absence of illness
health is a heightened state of physical well-being

Select a particular abstract definition that suits and justify based on YOUR research focus, background literature etc.
Operational definition
Select from a variety of different indicators for concepts
Need to be specific - make the variable observable and measurable

e.g., health is the level of severity (as rated by medical
experts) of current and recent (last 3 months) illnesses

Variables

Cases and variables
Case: single example of phenomenon for which data have been collected – e.g a leisure participant, a tourist
Variable: item of information about a case – e.g. age, income
Types of Variables
Independent – variables that are free to change or be manipulated.
Dependent – the main variable of interest. The variable that is measured to determine the effect of the independent variable.

LEVELS OF MEASUREMENT

Nominal - categories
e.g., sex (male, female)
Ordinal - ranked
e.g., ranked importance of service features (first, second, third)
Interval - equal intervals
e.g., 5 point scale (termed a Likert scale) (SD, D, N, A, SA)
Ratio - equal intervals AND absolute zero
e.g., community participation rates

Nominal

The first level of measurement is nominal measurement.
In this level of measurement, the numbers are used to classify the data.
Also, in this level of measurement, words and letters can be used.
Suppose there are data about people belonging to two different genders.
In this case, the person belonging to the female gender could be classified as F, and the person belonging to the male gender could be classified as M.
This type of assigning classification is nothing but the nominal level of measurement.

Ordinal

The second level of measurement is the ordinal level of measurement.
This level of measurement depicts some ordered relationship between the number of items.
Suppose a student scores the maximum marks in the class.
In this case, he would be assigned the first rank.
Then, the person scoring the second highest marks would be assigned the second rank, and so on.
This level of measurement signifies some specific reason behind the assignment.
The ordinal level of measurement indicates an approximate ordering of the measurements.
The researcher should note that in this type of measurement, the difference or the ratio between any two types of rankings is not the same along the scale.

Interval

The third level of measurement is the interval level of measurement.
The interval level of measurement not only classifies and orders the measurements, but it also specifies that the distances between each interval on the scale are equivalent along the scale from low interval to high interval.
For example, an interval level of measurement could be the measurement of anxiety in a student between the score of 10 and 11, if this interval is the same as that of a student who is in between the score of 40 and 41.
A popular example of this level of measurement is temperature in centigrade, where, for example, the distance between 940C and 960C is the same as the distance between 1000C and 1020C.

Ratio

The fourth level of measurement is the ratio level of measurement.
In this level of measurement, the measurements can have a value of zero as well, which makes this type of measurement unlike the other types of measurement, although the properties are similar to that of the interval level of measurement.
In the ratio level of measurement, the divisions between the points on the scale have an equivalent distance between them, and the rankings assigned to the items are according to their size.
http://courses.csusm.edu/soc201kb/exercises.htm

QUALITIES OF A GOOD MEASURE

Validity

Truthfulness of the score - the scale truly measures the characteristic and does not measure anything else

Reliability

the consistency of the measure - the scale gives consistent scores - measure the same unit twice and get the same score

Applicability

ease in application - it is useable - respondents understand it

STEPS IN PLANNING SURVEY RESEARCH USING QUESTIONNAIRES

decide type of survey (mail, structured interview, phone)
decide who completes
arrange sampling
write items
based on variables with foresight of analysis
organise into questionnaire sections
add instructions
layout questionnaire
pilot test questionnaire - refine

1: DESIGN QUESTIONNAIRE:

CONTENT OF ITEMS

Behaviours (what you did)
Attitudes/ opinions (what you believe)
about yourself, others, events things
Current status (e..g. age, fitness, years service)
Intentions (what you plan to do)
Knowledge ( the budget, board goals)

Should be questions on things the respondent knows about themselves (or their organisation etc)

TYPES OF ITEMS

Each item fill in (open)

What is your gender? __________

Response Modes:

TYPES OF ITEMS

Each item response set

Age (tick one ONLY): 12 - 15 [ ]

16 - 20 [ ]

21 - 25 [ ]

Over 25 [ ]

Response Modes:

TYPES OF ITEMS

How many times have you exercised in the gymnasium this week? (tick one ONLY):

0 [ ]

1 [ ]

2 [ ]

3 [ ]

4+ [ ]

TYPES OF ITEMS

Each item response set

To what extent do you agree that you should pay extra for aerobic classes with your gym visit? (TICK ONE ONLY)

strongly disagree [ ] disagree [ ] neither [ ] agree [ ] strongly agree [ ]

Response Modes:

TYPES OF ITEMS

Each item response set

Rate your satisfaction with the following (circle one ONLY):

NS = not at all, SS = slightly, MS = moderately, VS = very, ES = extremely

friendliness of instructor NS SS MS VS ES

instructor helpfulness NS SS MS VS ES

frequency of exercise NS SS MS VS ES

location of gym NS SS MS VS ES

Response Modes:

TYPES OF ITEMS

Many item check list

Which of the following activities do you enjoy the most in order during your gym visit? (number 1 representing the most and – 5 representing the least)

aerobics [ ]

bicycle [ ]

dumbells [ ]

bench press [ ]

running machine [ ]

Response Modes:

CONSTRUCTING GOOD ITEMS

Write using simple (grade 5) structure and vocabulary
Explain (e.g., use examples)
Use neutral (unbiased, unemotional) language
Ask only what respondent can reasonably know
Avoid providing unnecessary information
Each item to seek single piece of information
Give opportunities for wide response
Response sets of discrete (not overlapping) categories
Take care with socially sensitive items

PARTS OF A QUESTIONNAIRE

Attractive front cover
“About this study” section (brief explanation - e.g., purpose)
Introduction to researcher with contact information
General instructions
Informed consent
Sections
in logical order
with titles and special instructions
including section for general or specific other comments
Thank you
Return instructions (if applicable)

EXPLANATION OF PURPOSE OF THE STUDY

ABOUT THIS STUDY

This study seeks to explain some of the relationships between stress, leisure and

health. In doing so it is hoped that useful information will be provided that will

assist teachers, educational authorities and others foresee problems associated

with stress and assist teachers in preventing stress related illnesses.

Your school is among those selected from the schools in this education region for

the study. All teachers in this school have been asked to participate in this

study. Participation will require each teacher to complete three, half hour

questionnaires; two identical questionnaires so that changes in stress and health

can be monitored over a teaching semester and one about your lifestyle.

The success of this study depends on the cooperation of teachers and their

administrators in providing information about themselves. For this substantial

contribution the researcher is very grateful.

Signed, Researcher

ABOUT THIS STUDY

This study seeks to explain some of the relationships between stress, leisure and

health. In doing so it is hoped that useful information will be provided that will

assist teachers, educational authorities and others forsee problems associated