HPI Week 5 Discussion

profileDrgraham27
hpi_501_lesson_5.ppt
Home>Business & Finance homework help>Management homework help>HPI Week 5 Discussion

HPI 501- LESSON 5

  • Objective – Examine and articulate effective performance measurement techniques

Required reading: Articles 31, 32 and 33 of the Handbook of Human Performance Technology

*

*

*

QUANTITATIVE VS. QUALITATIVE RESEARCH METHODS

*

*

*

QUALITATIVE AND QUANTITATIVE RESEARCH – 4 MYTHS

  • In her article, Pershing takes on 4 common myths on these two forms of research and disqualifies them
  • Let’s look at each one in turn:

*

*

*

  • Quantitative position is rooted in the notion that the world is made up of observable and measurable facts
  • Objectivity is central
  • Data is gathered in a decontextualized way
  • Qualitative position is that reality is socially constructed and always changing
  • It recognizes that there are multiple realities
  • It is contextual and acknowledges that there is an interactive link between researcher and participants

MYTH 1 - PHILOSOPHICAL POSITIONS OF QUALITATIVE AND QUANTITATIVE RESEARCH ARE INCOMPATIBLE

  • The two most common accusations made against Qualitative research methods center around:
  • Population and Samples
  • Standards of Rigor

MYTH 2 - QUANTITATIVE RESEARCH IS MORE RIGOROUS THAN QUALITATIVE RESEARCH

  • Quantitative research usually involves selecting a representative, probability sample of the population under study. This means that each case in the population has an equal chance of being included in the sample
  • Qualitative research involves selecting a nonprobability or purposeful sample. Expert judgment may be used to select a representative sample of the population
  • See pg.748 for a more detailed explanation

POPULATION AND SAMPLES

  • The question here, according to the author is really whether the standards of rigor for each approach is used correctly
  • Both approaches have their corresponding standards that rely on the validity and reliability of the data gathered
  • Qualitative studies also stress the credibility and transferability of the data obtained
  • See page 750 for more detail

STANDARDS OF RIGOR

  • On this point, the author argues for a “methodological continuum” that goes from highly quantitative methods to highly qualitative with many in between
  • Therefore, it’s difficult, if not impossible to differentiate method as belonging to one approach or the other
  • Rather, the investigator needs to use the appropriate tool for the appropriate problem
  • See figure 31.1 on page 752 for details

MYTH 3 - QUANTITATIVE RESEARCH USES MANY METHODS WHILE QUALITATIVE RESEARCH USES ONLY ONE

  • It is important here to distinguish the difference between methods and data
  • It is entirely possible for a quantitative study to produce qualitative data if, for example, a questionnaire contains open-ended questions
  • Conversely, a qualitative study could contain multiple-choice questions that yield quantitative data
  • Please see pgs. 754-758 for a more detailed explanation of this

MYTH 4 - QUANTITATIVE METHODS YIELD QUANTITATIVE DATA WHEREAS QUALITATIVE METHODS YIELD QUALITATIVE DATA

CONSTRUCTING EFFECTIVE QUESTIONNAIRES

*

*

*

  • Written questionnaires are an important tool for gathering data
  • As a diagnostic tool, they are often used for needs assessment
  • They are an indirect way of gathering data when you are not able to meet face-to-face with respondents
  • They are relatively inexpensive to administer and can furnish a large amount of data

QUESTIONNAIRES ARE AN IMPORTANT DATA GATHERING TOOL

  • Pederson (2000) states that there are seven distinct steps in the construction of a questionnaire:

Review the information requirements necessitating a questionnaire

Develop and prioritize a list of potential questions that will satisfy the information requirements

Address each potential question carefully

Determine the types of questions to be asked

Decide on the specific wording of each question to be asked

Determine the structure of the questionnaire

Evaluate the questionnaire

QUESTIONNAIRE CONSTRUCTION

  • You must understand the information requirements – what information is needed and how that information will be used before starting construction of a questionnaire
  • It is important to review and understand the information requirements of the problem, opportunities or decisions that led to the need for a questionnaire
  • This information will pertain to the nature of the project goals and how they connect with human performance

REVIEWING THE INFORMATION REQUIREMENTS FOR THE QUESTIONNAIRE

  • Once the decision has been made to gather specific information through a questionnaire, it is important to translate that information into specific questions
  • Questions should be as specific as possible as this will make it easier to evaluate the results of the questionnaire later
  • Each potential question should be screened in terms of (1) how the answers to it will be analyzed, (2) the anticipated information it will provide and (3) how the ensuing information will be used

DEVELOP AND PRIORITIZE A SET OF POTENTIAL QUESTIONS

  • Each question should be assessed by asking three sequential questions:

Can participants understand the question?

Can participants answer the question?

Will participants answer the question?

  • For a question to be administratively viable, the answer to the above three questions must be “yes”

ASSESS EACH POTENTIAL QUESTION CAREFULLY

  • Introduction, Instructions, and Closing Statement
  • Introduction – An introductory statement may include information on the general purpose of the questionnaire, a request for cooperation and a statement on the anonymity policy
  • Instructions – Every self-administered questionnaire should include clear, easy-to-understand instruction on how to complete it. If there are open-ended questions, participants should be told what kind of length their responses should be
  • Closing statement – This typically thanks participants for taking the questionnaire and also how instruction on how to submit it

ESSENTIAL ELEMENTS OF A GOOD QUESTIONNAIRE

  • Starting on page 765, the author enumerates many valuable tips and suggestions for writing effective survey questions
  • A summary of these ten tips follows

QUESTION CONSTRUCTION

  • To avoid confusion and ambiguity, questions should be simple, clear and as short as possible
  • Shorter questions tend to produce higher response rates

1. WRITE SIMPLE, CLEAR AND SHORT QUESTIONS

  • Specific questions are better than more general questions because of their accuracy and the fact that they will be interpreted in the same way by participants
  • The more general the question, the wider its interpretation will be

2. MAKE SPECIFIC AND PRECISE QUESTIONS

  • Use appropriate wording in the questions that aligns with the audience taking the questionnaire
  • Stay away from jargon, abbreviations and acronyms unless your audience is thoroughly familiar with them

3.USE APPROPRIATE LANGUAGE

  • Respondents must be competent to answer the questions
  • Make sure the respondents know about the subject matter being asked about in the questionnaire
  • If the respondents are not familiar enough with the subject matter or, if only a few are, the data gathered will not be useful

4. ENSURE RESPONDENTS’ ABILITY TO ANSWER

  • Each question should be related to one topic or idea
  • As a general rule, if the word “and” appears in a question or statement, question developers should check to see if there is more than one idea or topic embedded in the question

5. INCLUDE ONLY ONE TOPIC OR IDEA PER ITEM

  • Use emphasis tools such as italics, underlining and boldface letters to highlight and clarify key items in a given question
  • Inconsistent or careless use of emphasis will only confuse survey participants

6. USE APPROPRIATE EMPHASIS FOR KEY WORDS IN THE QUESTION

  • Special care should be taken when asking about salary, race, ethnicity and other like items
  • It’s also important to avoid questions that use words or phrases of regional terminology or that reflect occupational or social class differences

7. TAKE CARE WITH SENSITIVE QUESTIONS

  • Using negative statements in a questionnaire can easily create confusion with respondents
  • Double negatives are even more confusing and should not be used as they are grammatically incorrect

8. AVOID NEGATIVE QUESTIONS OR DOUBLE NEGATIVES

  • The way in which questions are worded, or the inclusion of certain terms or phrases may encourage some respondents to respond more than others. These are called “loaded” or “biased” questions and they can create an inaccurate dataset

9. AVOID BIASED OR LOADED QUESTIONS AND TERMS

  • Answers to questions with a hypothetical situation tend to be inaccurate and unreliable
  • In general, if the respondent disagrees with the premise of a question, he/she will likely not answer it or they will answer it in a less than forthcoming way

10. AVOID QUESTIONS WITH FALSE PREMISES OR FUTURE INTENTIONS

  • Each item in the questionnaire must differentiate those respondents with a favorable attitude from those with an unfavorable attitude
  • In addition these attitudes can range in intensity and that can also be measured by a scale that can go from a simple 3 point scale (like/neutral/ dislike) to one with more points
  • See page 769 for a more detailed explanation

RATING SCALES

  • Questionnaire layout is also important
  • The appearance and arrangement of the questionnaire should be clear, neat and easy to follow
  • Clear appealing format with use of graphics if necessary, will encourage respondents to answer the questions
  • A good layout will also facilitate subsequent data analysis once the questionnaires are complete

QUESTIONNAIRE LAYOUT OR FORMAT

  • Close-ended questions can be grouped numerically and displayed in many ways to look at trends and overall scores
  • Open-ended answers can be coded and summarized into broad categories that will help interpret the results of the questionnaire (see pg. 775)

DATA ANALYSIS

INTERVIEWS AS A TOOL TO ANALYZE AND EVALUATE HUMAN PERFORMANCE

*

*

*

  • Often used by HPT specialists as the first step in collecting information on an improvement initiative
  • Interactive, real-time exchange with a participant
  • Sharing ideas
  • Problem-solving can sometimes occur
  • Helpful, but seldom used as the sole source of data

WHY CONDUCT INTERVIEWS?

  • Interviewer can seek clarification when a respondent gives unclear or ambiguous answers
  • Effective for building rapport
  • Flexibility, a wide range of subjects can be covered

ADVANTAGES

  • Interviewers can be biased
  • Note-taking by the interviewer can be distracting
  • Expensive and time-consuming
  • Results can sometime be hard to summarize
  • Fewer respondents can be interviewed than with questionnaires and other tools

DISADVANTAGES

  • Structured
  • Semistructured
  • Unstructured

TYPES OF INTERVIEWS

  • Fixed questions
  • Similar to questionnaires
  • Effective when the goals of the Performance improvement initiative are clear
  • Questions are concise and singularly address the issue at hand
  • Typically last no more than 20 minutes

STRUCTURED INTERVIEWS

  • Include questions that provide a fixed set of response choices in addition to one or more unrestricted response categories
  • Idea is that the structured responses provide highly specific information while allowing additional information to come in via open comments from participants
  • Similar to written survey questionnaires that employ open and closed-ended questions
  • Typically last 30 to 60 minutes

SEMISTRUCTURED INTERVIEWS

  • - Used when the information required is broad and non-specific
  • Similar to a questionnaire with open-ended questions
  • Similar to a brainstorming session, used in the early stages of a performance diagnosis
  • Can be useful to tease out sensitive or personal information
  • Data collection can be challenging due to the wide variety of possible data collected (see pgs. 783-784)

UNSTRUCTURED INTERVIEWS

  • Telephone interviews
  • Face-to-face interviews

METHODS OF INTERVIEWING

  • Save time and money
  • Helpful when several people need to be interviewed
  • Tend to last less time and can gather information quickly
  • Interviewer cannot assess body language
  • Respondents may be difficult to reach in some cases

TELEPHONE INTERVIEWS

  • Effective for understanding deeper, more complex issues
  • Rapport is more easily established
  • More expensive, fewer respondents can be interviewed
  • Non-verbal cues can be observed
  • Interviewer note-taking can be distracting

FACE-TO-FACE INTERVIEWS

  • Step 1: Determine the objective of the interviews
  • Step 2: Prepare for the interviews
  • Step 3: Carry out the interview
  • Step 4: Conclude the interview process
  • Step 5: Compile and analyze results

INTERVIEW PROCESS

  • Determine the information being sought
  • Make sure standard terms are used
  • Typical questions range from asking about performance gaps, desired outcomes and issues to feelings about a particular aspect of the problem at hand

STEP 1: DETERMINE THE OBJECTIVE OF THE INTERVIEWS

  • Select respondents
  • Decision on sampling must be made
  • Develop the interview protocol
  • This is the script and questions that will be asked
  • Prepare a timetable for the interviews
  • Duration, question length
  • Conduct a pilot test
  • Train interviewers
  • Schedule the interviews
  • See pgs. 787-790 for details

STEP 2: PREPARE FOR THE INTERVIEWS

  • Open the interview
  • Frames and sets expectations
  • Conduct the interview
  • See pg. 791 for a detailed description
  • Close the interview
  • Record the interview

STEP 3: CARRY OUT THE INTERVIEW

  • When information gathered begins to repeat itself, it’s time to close the interview process
  • Reassure interviewees on anonymity and any other matters covered in the opening

STEP 4: CONCLUDE THE INTERVIEW PROCESS

  • Select the appropriate data analysis tools
  • Graphs and tables for quantitative data
  • Pictures and quotes for qualitative data
  • Combinations of both
  • Know your audience

STEP 5: COMPILE AND ANALYZE RESULTS

ASSIGNMENT

  • Provide an example of a situation when you would use a quantitative data analysis technique
  • Provide an example of a situation when you would provide a qualitative data analysis technique
  • Who do you admire as an interviewer, why?

*

*