technology and information management

First Cycle Coding

Content drawn from Johnny Saldana’s The Coding Manual for Qualitative Researchers.

David Lee — TIM 158, Spring 2019

Credit: YCombinator : How to Start a Startup

Recall

generating hypotheses

Needfinding is about Recall

Who / What How / WhyHello! Thanks!

Recall

Summary • Go from what to how and why, why, why

• Develop a model of an individual

• setting, actions → thoughts, feelings → values, motivations

• Then reflect on needs

• also consider: what is top of mind? hacks and workarounds?

Recall

Deep, rich understanding of individuals

Hypotheses about narrow user segments

Hypotheses about solution concepts

Recall

dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation 7

Generate Evaluate Generate Evaluate

Recall

dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation 8

Questions Prototype Questions Prototype

Recall

dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation2018/10/08 9

Recall

Follow-up with a 30-min interview.

We’re still in hypothesis generating mode!

Recall

The next two weeks • Revisiting unpacking

• Communicating your concept

The prototyping process Generate questions !Untested design thesis !Risky design decisions !Unobserved user behaviors

Rank questions Which is most critical?

Build and test a prototype Answer only the most critical question

Recall

Today • Overview of Qualitative Analysis and Coding

• First Cycle Coding and Analytic Memos for HW #3

What is qualitative research?

So first…

Qualitative research • When you’re trying to

• develop a rich understanding of a complex phenomena and the complex interactions between factors

• communicate a holistic interpretation or narrative that helps readers experience “being there”

• Ask → Collect → Organize → Analyze → Theory • In our case: a model of the user segments and their context or experiences in

relation to our product (segment, setting, sequence, satisfaction)

How do you go from qualitative data to patterns, concepts, and theories?

Unpacking so far…

dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation2018/10/01

Recall

dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation

KEEP A LIST OF

TENSIONS, CONTRADICTIONS, SURPRISES

say

do

think

feel 2018/10/01

USE TO FIND NEEDS & INSIGHTS

Empathy Map to Help Synthesize

dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation

INSIGHTS

I wonder if this means . . .

think

feel

TENSIONS, CONTRADICTIONS, SURPRISES

2018/10/08 20

USERS & NEEDS

dt+UX: Design Thinking for User Experience Design, Prototyping & Evaluation

combine to create a point of view need insight

2018/10/01 … SU

RP ISE

D TO

D ISC

OV ER

...

… GA

ME -C

HA NG

ING TO

…

user attribs.

WE MET . . . (extreme user you are inspired by)

WE WERE AMAZED TO REALIZE . . . (what did you learn that’s new? What is their need?)

IT WOULD BE GAME-CHANGING TO . . . (frame up an inspired challenge for yourself – the insight.)

(don’t dictate the solution.)

There are many techniques • Flow models

• Critical incident analysis

• Conversation analysis

• Fixed coding

• Open coding / grounded theory (our focus)

• The Coding Manual for Qualitative Researchers, Johnny Saldana

Grounded theory is just a more systematic version of what you’ve done

“essence-capturing and essential elements of the research story that, when clustered together according to similarity and regularity - a pattern - facilitate the development of categories and analysis of their connections.”

Qualitative coding is about

The Coding Manual for Qualitative Researchers, Johnny Saldana

DATA → CODES → CATEGORIES → THEMES → THEORY

General process • Start from raw data such as interview transcripts, images, etc.

• First cycle coding

• Create essence-capturing codes (labels of and links between data)

• Second cycle coding

• Build on codes to develop categories (the shape of the explicit data)

• Build on categories to define concepts and themes (get to higher-level abstract concepts)

• Show how themes and concepts lead to theory (how they systematically interrelate)

The Coding Manual for Qualitative Researchers (2009), Johnny Saldana (p. 12)

First cycle focused on who/what

Second cycle moving from who/what to

how/why

General process • As you go, write analytic memos

• Partial reflections and syntheses written as you go

• Happens in a cyclical manner. You’ve collected enough data when

• You can provide deep slices of understanding to others

• You are starting to encounter the same things (not learning)

• You can use 6-10 as a rough heuristic

The Coding Manual for Qualitative Researchers (2009), Johnny Saldana (p. 43)

First and second cycle coding intermixed with

data collection and memo writing

Gradual development from codes to categories to theory, facilitated by

memo writing

General process • End with annotated data, hierarchical coding scheme, codebook, analytic

memos, developed theory

• For us, the “theory” we’re aiming for is a hypothesized model that is:

• grounded in user data and quotes

• a cohesive narrative and logical interpretation

• communicated richly to help the reader in “being there”

• Remember: we are generating not validating hypotheses, so choose diverse perspectives

SEGMENT → SETTING → SEQUENCE → SATISFACTION

Recall: the model in our case

We’re just going to get a taste of this process

Today • Overview of Qualitative Analysis and Coding

• First Cycle Coding and Analytic Memos for HW #3

Coding methods

Lots of different coding methods!

The Coding Manual for Qualitative Researchers (2013), Johnny Saldana (p. 59)

representing raw data with discrete “codes” that define “summative, salient, essence-capturing, evocative” aspects of the data

First cycle coding

The Coding Manual for Qualitative Researchers, Johnny Saldana

transitioning from managing, focusing, highlighting, filtering data to generate categories, themes, and theory

Second cycle coding

The Coding Manual for Qualitative Researchers, Johnny Saldana

The Coding Manual for Qualitative Researchers (2013), Johnny Saldana (p. 59)

• You can think of these as just fancy names for different approaches or focuses to creating labels,

• At what level of granularity?

• What are we labeling?

• How are we labeling it?

• Different coding methods support different theoretical emphases and have different ways of analyzing them

• Not mutually exclusive (you could be doing multiple at a time)

What level of granularity? • Attribute coding: labeling the dataset as a whole

• fieldwork setting and time, participant characteristics, variables of interest for analysis

• Holistic / structural coding: labeling sections of data

• skim through and chunk text into topic areas (emergent or study based)

• Line-by-line coding: create a finer discrete representation of data

• summative, salient, essence-capturing, evocative labels

How are the codes chosen? • In Vivo coding: grounding codes in words of the user

• codes labeled in “quotes” drawn from user responses

• Hypothesis coding: predetermined codes

• record instances conforming to a coding scheme to assess a researcher- generated hypothesis

The Coding Manual for Qualitative Researchers (2013), Johnny Saldana (p. 189)

Attribute codes

Holistic/structural code

Line-by-line/sentence- by-sentence codes

What is the coding focus? • Descriptive coding: the topics

• the things being talked about (often nouns)

• Process coding: the action in the data

• what is happening? has happened? would like to happen? (often gerunds, -ing words)

• observable (e.g. reading) and conceptual (e.g. struggling)

Saldana, pg 89

Saldana, pg 97

What is the coding focus? • Emotion coding: the feelings a participant

experiences

• a feeling and its distinctive thoughts, psychological and biological states, and range of propensities to act

• Evaluation coding: judgments of merit of program/product

• describe attributes and details assessing quality

• compare how the program measures up to a standard or ideal.

• predict recommendations for change and how they might be implemented.

Saldana, pg 107

Saldana, pg 121

What is the coding focus? • Values coding: a participants’ integrated

value, attitude, and belief systems

• A value is the importance we attribute to oneself, another person, thing, or idea.

• An attitude is the way we think and feel about ourselves, another person, thing, or idea.

• A belief is part of a system including our values, attitudes, personal knowledge, experiences, opinions, prejudices, morals, and other interpretive perceptions. Can be considered rules for action.

• Versus coding: the tensions and conflicts

• the individuals, groups, social systems, organizations, phenomena, processes, concepts, etc., in direct conflict with each other

Saldana, pg 112

Saldana, pg 116

Who / What How / WhyHello! Thanks!

Recall

The Coding Manual for Qualitative Researchers (2013), Johnny Saldana (p. 189)

Can mix codes: 
 Descriptive, Emotion, In Vivo, Versus Codes

representing raw data with discrete “codes” that define “summative, salient, essence-capturing, evocative” aspects of the data

First cycle coding

The Coding Manual for Qualitative Researchers, Johnny Saldana

Recall

A typical first cycle coding process • Decide what set of coding methods are appropriate for your goals

• Make 1-2 passes through the data to create those codes

• While you are doing this:

• Mark any particularly interesting quotes (evocative, illustrative, etc.)

• Write down reflections you have as short analytic memos

• Remember your goals are to develop grounded theory and communicate a rich sense of “being there”

Analytic memos

Reflections on “your coding process and choices; how the process of inquiry is taking place; and the emergent patterns, categories, and subcategories, themes and concepts in your data — all possibly leading toward theory”

Analytic memos are essential personal reflections

Help facilitate codes to theory • Data collection, coding, and analytic memo writing are intermixed from the

beginning to the end


• From codes: “Think of a code not as a label, but as a prompt or trigger for written reflection on the deeper meanings it evokes”

• To categories: “Analytic memo writing serves as an additional code and category-generating method”

• To final write-up: “Your reflections generate potential material for formulating a set of core ideas for presentation”

• “Whenever anything related to or significant about the coding or analysis of the data comes to mind, stop whatever you’re doing and write a memo about it immediately”

How to write memos • Just write your reflections! Then categorize and label

• 1) date, 2) title describing type of memo, 3) evocative subtitle describing essence of content

• Many types of memos such as reflections on

• Your study’s research questions

• Possible networks (links, connections, overlaps, flows) among codes, categories, concepts

• Emergent categories, concepts, themes

• Emergent or related existing theory

• Future directions for the study

Example: passage being coded

Saldana, pg 43

Memo type: research study

Saldana, pg 44

Write the predetermined research questions, purposes, or goals and then elaborate on answers in progress

Memo type: networks

Saldana, pg 45

Integrate your codes into a narrative (codeweaving) to interpret how individual components weave together in hierarchies, chronological flows, influences and affects, etc.

Memo type: emergent categories/concepts

Saldana, pg 45

Begin to create a sense of order to your analysis through tentative categories, themes, higher-level concepts, assertions, etc.

Example: related theory memo

Saldana, pg 46

Speculate how your observations generalize to larger groups or even the universal. Predict or explain actions. Explore metaphors suggesting transferability. Integrate/compare existing theory.

Example: future directions memo

Saldana, pg 47

Note down new ideas, research questions, need for additional data. Could even reconceptualize entire initial approach.

HW #3 is just a small taste of this systematic process
 


HW #3: Code your interview • Preparation: Annotate it with attribute and holistic/structural codes

• Who/What: use descriptive and process codes to describe the characters, setting, and other context

• How/Why: Use emotion, evaluation, values, versus coding to develop a richer view of the participant context and experiences. Bias towards using in vivo codes to ground these in the user’s language.

• Memos: Write short analytic memos that use codeweaving to write short narratives of the participant context and experiences

• (On Wednesday) We’ll talk about organizing codes into categories and defining models of our participant and of hypothesized user segments