Research Skills Essay
johnathan1993
lecture_4_research_design.pptxLECTURE 4: RESEARCH DESIGN By: Dr. Tao Zhang Email: [email protected]
RESEARCH SKILLS (SINGAPORE PROGRAMME)
Key knowledge Points
Research
Positivism
Interpretivism
Realism
Problem definition
Research questions
Research objectives
Qualitative research
Quantitative research
Learning Outcomes
At the end of this lecture you are expected to be able to:
Identify the various sources of data
Understand and distinguish various research designs
Understand the criteria for assessing the quality of business research.
Be familiar with the terminologies, processes and techniques of sampling
Apply these concepts to specific scenarios
Figure 1.2 The research process
Source: © Mark Saunders, Philip Lewis and Adrian Thornhill 2011
Data Sources
Information relevant to the research problem
Adopted from: © 2000 McGraw-Hill Ryerson Limited
Research Designs and Methods
A Research Design provides a framework for the collection and analysis of data.
Choice of research design reflects decisions about priorities given to the dimensions of the research process.
Experimental
Cross- sectional
Longitudinal
Case Study
Comparative
Experimental design elements
Random assignment of subjects to experimental and control groups,
Pre-testing of both groups,
Independent variable manipulated; all other variables held constant,
Post-testing of both groups,
Computation and analysis of group differences
Manipulation: in order to conduct a true experiment, it is necessary to manipulate the independent variable in order to determine whether it does in fact have an influence on the dependent variable.
Exercise – The Marshmallow Experiment
What is the aim of this study?
Determine the dependent and independent variable(s)
Implications/extensions of the study?
https://www.youtube.com/watch?v= Yo4WF3cSd9Q
Cross-sectional Design
Entails the collection of data on more than one case (usually quite a lot more than one) and at a single point in time in order to collect a body of quantitative or quantifiable data.
Usually in connection with two or more variables, which are then examined to detect patterns of association.
Mostly associated with survey method
Survey research comprises a cross-sectional design in relation to which data are collected predominantly by questionnaire or by structured interview
Longitudinal design
Survey of the same sample on more than one occasion
Typically used to map change in business and management research
in a panel study (e.g. WERS – Research in focus 2.14- same managers surveyed in 2011)
or a cohort study (e.g– all graduates from a business studies course in the same year)
Case study design
Detailed and intensive analysis of one case
Examples:
A single organization – e.g. studying the organizational culture of a specific company;
A single location – such as a factory production site etc
A person – e.g. study of women managers, where each woman constitutes a separate case, using a biographical approach
A single event – e.g. a major accident
Often involves qualitative research
Case is the focus of interest in its own right - location/setting just provides a background
Comparative Design
Using the same methods to compare two or more meaningfully contrasting cases
Can be qualitative or quantitative
Often cross-cultural comparisons
Hofstede’s (1984) study of IBM managers in different countries
Includes multiple case studies
Problem of translating research instruments and finding comparable samples
Examples
Survey/Interviews
http://www.slideshare.net/jenniferdoris/chap2-latest
http://www.slideshare.net/jenniferdoris/chap2-latest
Assessing the Quality of Research
Criteria
Reliability
Replicability
Validity
Reliability
Concerned with the question of whether or not the measures devised for concepts in business are consistent
For example measures used to determine ‘motivation’ should be consistent when used across different studies.
If measures used appear to be inconsistent then their reliability would be questioned.
Replicability
Sometimes researchers replicate others’ studies
All procedures in the study must therefore be clearly spelt out
The measures used must also be replicable and hence consistent
P/s: Replication of studies is seen to compromise originality
Validity
Concerned with the integrity of the conclusions that are generated from a piece of research
There are different types of validity:
Measurement validity
Internal validity
External validity
Ecological Validity
Types of Validity
Measurement (or construct) validity
Does the measure used to measure a concept really reflect the concept it is denoting
Related to reliability as measures must be consistent in representing the concepts they denote. E.g. If IQ tests do not really measure variations in intelligence then they are not valid.
Internal validity
Are causal relations between variables real?
If suggested that x causes y can we be sure that y is not caused by other factors?
Types of Validity Cont’d
External validity
Can results be generalized beyond the research setting?
Issues around how research participants are suggested become pertinent here
Quantitative researchers aim to generate representative samples for this reason
Ecological validity
Are findings applicable to everyday life, natural settings?
Research findings may be technically valid but have little to do with real life situations, e.g. using experiments
Alternative Criteria in Qualitative Research
Trustworthiness (Lincoln and Guba (1985)
Credibility, parallels internal validity - i.e. how believable are the findings?
Transferability, parallels external validity - i.e. do the findings apply to other contexts?
Dependability, parallels reliability - i.e. are the findings likely to apply at other times?
Confirmability, parallels objectivity - i.e. has the investigator allowed his or her values to intrude to a high degree?
Relevance (Hammersley 1992), e.g. Importance of topic in its field
Threats to internal and external validity
What might compromise internal and external validity?
Discuss for 10 minutes
Other (non-experimental) events may have caused the changes observed (‘history’)
Subjects may become sensitized to ‘testing’
People change over time in any event (‘maturation’)
External: uniqueness of context, pre-testing, uniqueness of subjects etc
Research Methods
A Research Method is simply a technique for collecting data.
Choice of research method reflects decisions about the type of instruments or techniques to be used.
Figure 5.2 Methodological choice
Source: © Mark Saunders, Philip Lewis and Adrian Thornhill 2011
Bringing research strategy and research design together
Both quantitative and qualitative strategies can be executed through any of the research designs covered in this chapter – although experimentation is rarely used in qualitative research.
Use of comparative design (for example):
quantitative: Brengman et al’s (2005) study of Internet shoppers in the United States and Belgium
qualitative: Hyde et al’s (2006) evaluation study of role design in the NHS
There is a distinction between a research method and a research design.
There are three key technical terms for evaluating research: reliability, validity, and replicability
There are five key research designs: experimental, cross-sectional, longitudinal, case study, and comparative
There are various potential threats to validity in non-experimental research.
Key Points
SAMPLING
Sampling Definition
Sampling is the procedure that draws conclusions based on measurements of a portion of the population
Zikmund et al 2013
Figure 7.1 Population, sample and individual cases
Sampling Terminology
Sampling Terminology Definitions
29
Population (universe)
Employees of a company
Census
A study of all the employees of the company
Sample
Sampling Unit
A subset or some part of the employees
One employee in the sample, e.g. Manager
Sampling Frame
List/record of the employees by HR
The Sampling Design Process
Define the target population
Determine the sampling frame
Select sampling technique
Determine the sample size
Execute the sampling process
Malhotra and Birks 2007
Define the Target Population
Target population is the people or objects which possess the information sought by the researchers and about which inferences are to be made.
If you wanted to determine factors affecting choice of universities what would your target population be?
Determine the Sampling Frame
Adequacy
Completeness
No duplication
Accuracy
Convenience
Access
Selecting a Sampling Technique
Sampling
Techniques
Non-Probability
Sampling
Techniques
Convenience sampling
Judgmental sampling
Quota sampling
Snowball sampling
Probability
Sampling
Techniques
Simple random sampling
Systematic sampling
Stratified sampling
Cluster sampling
Other
Probability Sampling
Probability (or random) sampling results in every sampling unit in a population having a known and non-zero probability of selection.
E.g. if a sample of 500 people is to be chosen from a population of 50,000 each member of the population has what chance of being chosen?
Each member has a 1 in a hundred chance of being chosen
Probability Sampling Simple Random Sampling
Simple random sampling is a probability sampling technique in which each element has a known and equal probability of selection.
Example of simple random sampling
Research in a company with 9000 full-time employees
Define population (N)
Select/Devise sampling frame
Decide your sample size (n)
List your sampling units and assign them numbers between 1 and N
Select n different random numbers between 1 and N with a computer program (e.g. Excel)
The employees with the n random number denote the sample
N=9000
List of employees from HR
n=450 (pre-determined sample size)
Probability Sampling Simple Random Sampling
Advantages –
representative of the population.
minimises interviewer bias
Disadvantages –
difficult to obtain or construct a sampling frame.
time consuming.
may not be an adequate approach if representation of specific groups is required.
Probability Sampling Systematic (Quasi) Random Sampling
The sample is chosen by selecting a random starting point and then picking every ith element in succession from the sampling frame.
E.g. Require a sample of 250 from a population
of 10,000.
Sample interval 10,000/250 = 40
Select number from 1 to 40 e.g. 6
Select from sampling frame 6th ,then 46th,
then 86th etc.
Probability Sampling – Systematic (Quasi) Random Sampling
Advantages –
- Sample should be representative of the population.
Minimises interviewer bias.
Quick and easy to use.
Disadvantages –
- Does not overcome the problems of minority representation.
Probability Sampling Techniques Stratified Sampling
- Stratified sampling uses a two step process.
Partition the population into sub-populations or strata.
Select elements from each stratum using a random procedure.
Probability Sampling Technique Stratified Sampling - Proportionate
E.g. Clinique want to examine their customers’ responses to a number of new adverts. They know their customers are between ages of 25- 50, with
10% between 25 - 29
25% between 30 - 34
35% between 35 - 39
20% between 40 – 44
10% between 45 - 50.
In a sample of 200, how many people do Clinique want in each age group if the sample numbers are to be proportionate to the number in the population?
25 - 29 =
30 - 34 =
35 - 39 =
40 - 44 =
45 - 50 =
Probability Sampling Techniques Stratified Sampling - Disproportionate
E.g. A survey of chemists in the UK is being carried out to investigate the selling prices of a range of toiletries or drugs. A sample of 1,000 chemists might be considered.
40% Boots (limited variation)
60% Independent chemists
In a sample of 1,000 chemists
200 Boots
800 Independent chemists
Probability Sampling Techniques Cluster Sampling
Cluster sampling is a two step technique.
The target population is first divided into mutually exclusive and collectively exhaustive sub-populations called clusters.
Select a sample of clusters using a probability based technique such as simple random sampling.
For each cluster either all the elements are included in the sample or a sample of elements is drawn probabilistically.
https://www.youtube.com/watch?v=QOxXy- I6ogs
Probability Sampling Cluster Random Sampling
Advantages –
- Useful when the list of sample population in not available.
Lower cost and time
Disadvantages –
- Vulnerable if clusters are too similar
- Potential for over/under representation
NON-PROBABILITY SAMPLING TECHNIQUES
Non probability sampling relies on the personal judgement of the researcher rather than on chance to select sample elements.
Non-Probability Sampling Techniques Convenience Sampling
Convenience sampling involves obtaining a sample of convenient elements. The selection of the elements is left primarily to the researcher.
Non-Probability Sampling Techniques Convenience Sampling
Advantages
Respondents are accessible and cooperative
The least time consuming
The least expensive
Disadvantages
Biased – respondent self selection
Not representative of a specific population
Not suitable for causal research
Non-Probability Sampling Techniques Judgemental Sampling
Judgemental sampling is a form of convenience sampling in which the respondents are selected based on the judgement of the researcher.
Non-Probability Sampling Techniques Judgemental Sampling
Advantages
Inexpensive, convenient and quick
Tentative generalisability from the sample to the population
Useful for questionnaire development
Disadvantages
Judgements of typicality may be very subjective
Non-Probability Sampling Techniques Quota Sampling
A process that produces a sample that reflects a population in terms of the relative proportions of people in different categories, e.g. gender, race etc.
However final selection of the sample is done judgmentally
Non-Probability Sampling Techniques Quota Sampling
Advantages
Ensures representation of minority groups
Disadvantages
Need considerable information about the target population
Possibility of overlooking important characteristics
Vulnerable to interviewer selection bias
Non-Probability Sampling Techniques Snowball Sampling
Snowball sampling involves selecting an initial group of respondents randomly. Subsequent respondents are selected based on referrals or information provided by the initial respondents. By obtaining referrals from referrals this process is carried out in waves.
Non-Probability Sampling Techniques Snowball Sampling
Advantages
Useful for obtaining respondents that are rare in the wider population
Disadvantages
Biased
Might not be representative of a specific population
*Better fit for qualitative research
Determining the Sample Size
Importance of decision
The nature of the research/analysis
The number of variables being investigated
Heterogeneity of the population
Non-response
Resource constraints (e.g. time and cost)
Sample sizes in similar studies
Determining the Sample Size
| Type of Study | Minimum Size | Typical range |
| Problem identification research (e.g. market potential) | 500 | 1000-2500 |
| Problem solving research (e.g. pricing) | 200 | 300-500 |
| Product tests | 200 | 300-500 |
| Test marketing studies | 200 | 300-500 |
| TV, radio or print advertising (per commercial tested) | 150 | 200-300 |
| Test-market audits | 10 stores | 10-20 stores |
| Focus groups | 2 groups | 4-12 groups |
Malhotra and Birks 2007
Sampling Exercise
You wish to carry out a study to determine the kind of leisure activities undertaken by foreigners working in your country. You hope that this study would inform your decisions to develop products that suit such foreigners.
What sampling technique would you ideally recommend for this task?
What would be the limitations and nearest alternative to the approach you have recommended? Detail the stages involved in making this work.
QUESTIONS?
