615 forum

Journal of Library Administration, 56:41–51, 2016 Published with license by Taylor & Francis ISSN: 0193-0826 print / 1540-3564 online DOI: 10.1080/01930826.2015.1105035

Decoding via Coding: Analyzing Qualitative Text Data Through Thematic Coding and

Survey Methodologies

PORCIA VAUGHN Biology, Biochemistry, & Nursing Librarian, University of Houston Libraries, Houston, TX, USA

CHERIE TURNER Chemical Sciences Librarian, University of Houston Libraries, Houston, TX, USA

ASBTRACT. In order to effectively analyze qualitative data one must use a systematic process to organize and highlight meaning. This article suggests a practical method of navigating the chal- lenges of organizing and classifying qualitative data through the use of thematic coding and the reorganization of text data into a SurveyMonkey survey.

KEYWORDS qualitative research, analysis or data handling, sur- vey, thematic coding

Due to tight budgets, academic libraries are facing increasing pressure to justify services, which leads to expanding, and more exploratory, library assessment (Association of College & Research Libraries [ACRL], 2015). Ad- ministrators are more dependent on data-driven decision-making practices for strategic planning and organizational growth. In an effort to prioritize li- brary services, many academic libraries have gathered a tremendous amount of data from their users with hopes of improving services and increasing impact within the academic community.

As a part of these efforts, many libraries have gathered qualitative data in the form of free-text responses in surveys, comment cards, emails, interviews, or focus groups; however, many libraries struggle with what to do with this information. Many have used the content from their qualitative data as a

© Porcia Vaughn and Cherie Turner Address correspondence to Porcia Vaughn, Biology, Biochemistry, & Nursing Librar-

ian, University of Houston Libraries, 4333 University Drive, Houston, TX 77204, USA. E-mail:[email protected]

Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/wjla.

41

42 P. Vaughn and C. Turner

means of anecdotal support, but systematic analysis can provide a wealth of information that may inform library administrators and stakeholders (Dennis & Bower, 2008).

In exploring the abundance of data from open-ended surveys, focus groups, interviews, and other free-text responses, we have felt unsure about how to use the data meaningfully. When sharing the basis of this article as a poster at the ACRL 2015 Conference, many colleagues commented that they have had similar struggles with conducting qualitative analysis. When identifying qualitative assessment strategies in the library science professional literature we found analysis of comments from the LibQUAL+ R© survey to be the most applicable to our analysis.

In one review article discussing coding practices for comments from LibQUAL+ R©, researchers found that a variety of software tools were used to organize text data, including ATLAS.ti, Excel, NVivo, and Access (Neurohr, Ackermann, O’Mahony, & White, 2013). In Neurohr and colleagues’ assess- ment of coding practices, the research team found incomplete methods, and methods too specific for other institutions to adapt. No one methodology was exclusive or consistent with another, leaving the reader questioning best practices for coding qualitative responses within LibQUAL+ R©. Although cod- ing practices varied among those discussed by Neurohr et al. (2013), many of the reporting institutions highlighted the value of the information they received from analyzing the qualitative data in their own unique way.

The Value of Qualitative Research

Qualitative research methods provide a means of capturing the complexity of our users. Because our users are complex individuals, we can capture descriptive data which will allow us to gain a deeper understanding of users’ needs and expectations for library services and performance (Given, 2006; Habich, 2008). Throughout the literature, four important benefits of qualitative research are highlighted:

• Gaining a deeper understanding of user needs with specific details (Begay, Lee, Martin, & Ray, 2004)

• Identifying specific context which can lead to creation of new services (Knapp, 2004)

• Complimenting quantitative findings with user perception (Friesen, 2008; Guidry, 2002)

• Building closer relationships with users (Gorman & Clayton, 2005)

The Challenges of Qualitative Research

While qualitative methods can strengthen and enrich the decision-making process, they tend to present challenges for analysis. There have been

Decoding via Coding 43

attempts to create measurable meaning from qualitative text, but it has been communicated “informally, sporadically, or not comparable with other data” (Knapp, 2004; Neurohr et al., 2013). Some of the challenges include issues with managing, organizing, and analyzing the data:

• Organizing large amounts of qualitative data systematically • Selecting a tool to manage the data • Storing data accessibly after analysis • Maintaining consistency in coding • Using interpretation appropriately

In order to identify meaningful themes in large amounts of text data, it is helpful to organize the data question by question. Software like those mentioned as options for coding of LibQUAL+ R© comments can help to sort by question. But when such software is available, there may still be limita- tions that create challenges for a research team. Training will be necessary if the research team is unfamiliar with the software. Additional, and potentially costly, software may be needed to interpret the data beyond basic statistical analysis. Furthermore, many of these tools may not allow for collaborative coding, leaving a complex and time-consuming coding process to one per- son.

The project discussed here focuses on analysis of data from semi- structured interviews. Semi-structured interviews are a great way to build an understanding of our complex users, including their perspectives, expec- tations, and assumptions, while building a rapport (Rubin & Rubin, 1995). These conversations, typically valued as anecdotal evidence to support quan- titative findings, can also provide meaningful information that can drive the decision-making process. We will introduce the method of using a survey to quantify and organize large amounts of text data from interviews.

BACKGROUND

The University of Houston is a public research university, with more than 40,000 students in over 300 undergraduate and graduate programs. The uni- versity has recently been shifting focus towards research in an effort to attain and maintain the Carnegie Foundation’s Tier-One designation. Because of the changing emphasis, more research is occurring on campus. This has promoted the establishment of centers and institutes, both for-profit and non-profit, existing within departments and as separate entities. Tradition- ally, the library has not specifically aimed to support centers and institutes, instead assuming that the majority are aware of these resources and services through other avenues. We have become more aware of the potential for unique library needs for centers.

44 P. Vaughn and C. Turner

Our increased awareness of centers and institutes led us to begin to in- vestigate their library-related needs. Our project had three primary goals: to learn more about centers and institutes, to identify library-related needs, and to create opportunities for outreach. These combined goals led the group to focus on qualitative methodologies, which allowed us to focus on under- standing user needs while building a rapport with centers and institutes. In particular, semi-structured interviews were chosen because this methodol- ogy allowed us to set goals for our interview but remain flexible enough to let conversations develop naturally and change direction as needed.

We recognized 50 active centers based on prior research and the activity of each center’s Web site. We solicited interviews with representatives of each of these 50 centers and conducted semi-structured interviews with 21 of the centers. Interviews focused on understanding each center’s goals, role within the university, and view of the library.

Interviews were recorded via handwritten notes rather than through au- dio or video recordings because of the additional challenges of managing the data in those forms. Each interview was attended by two researchers whose notes were combined and entered into a text document. In many cases, excerpts or comments from the center’s Web site were added to sup- plement interview notes. Our interview notes consisted of 21 separate text documents. Individually these interviews provide a rich source of informa- tion, but collectively they were difficult to analyze in this form due to the depth and variety of responses.

METHODS

In order to pull the data from each individual interview’s notes together for analysis, a more complex structure was needed. Software options were considered, but many of the challenges of using software encountered by others were seen with this project as well. In particular, software options were ruled out because of lack of availability for a large group, the difficulties of doing collaborative coding, and the time needed to train members of the group who were unfamiliar with software. Instead, we decided to try organizing the data into a single location by creating a survey. The library’s current survey tool, SurveyMonkey’s Professional Gold Plan, was used to create a survey that would allow the research team to organize the data from each notes document. This tool offered some very advantageous features in that it allowed for some embedded text analysis and cross-tabulation, both of which were helpful to analysis. Most importantly, SurveyMonkey was readily available, and the project team was already familiar with most features.

The entire research team collaborated to create survey questions that correlated to each interview question and allowed the survey to high- light some of the complexity in the answers. When long text responses

Decoding via Coding 45

TABLE 1 Strategies for qualitative analysis.

Strategies Use to Ask

Identify categories Move from a qualitative methodology to a quantitative methodology where possible.

Are there distinct or inherent categories in the responses that can be used for analysis?

Map relationships Conceptualize themes and relationships between responses and existing structures.

What other internal structures can we turn to map relationships? A library or university strategic document?

Set exclusion criteria Identify usefulness to the current project.

Are responses generalizable? What might we gain by doing analysis of this question?

provided valuable information, open-ended questions were used to main- tain that complexity as well as to ensure that all information was included. When responses had some inherent categories, it was frequently possible to use closed-ended questions, including multiple-choice and check-box ques- tions to show those categories. This resulted in a sort of coding built in to the survey itself, which saved time during analysis.

In many cases a single interview question needed to be broken down into multiple survey questions to most effectively illustrate the responses. Frequently this would include one or more closed-ended questions along with an open-ended question to allow for the full-text response to be in- cluded and analyzed. Once the survey was complete, each member of the research team helped to ensure all of the relevant information from each interview was included through the survey.

Once each interview’s notes were input into the survey, we were able to begin to review responses question by question and to identify themes. By looking for themes, we were able to develop some strategies for identifying the information that can be gathered from the data. Some trends were seen in the ways that data challenges could be managed. Three strategies, shown in Table 1, were developed to guide the group in both survey-question development and the process of identifying themes. These strategies revolve around questions we could ask to help us identify goals and set priorities for analysis.

Once themes for analysis were identified using these strategies, a code dictionary was created, including codes and defined meanings. A draft of the coding dictionary was then shared with the rest of the project team to gain consensus. The group then used the coding dictionary and SurveyMonkey’s MyCategory feature to code each response based on the codes for that question. After coding, the data was exported to be maintained in Excel. Figures 1 and 2 show coding in SurveyMonkey using MyCategory and the

46 P. Vaughn and C. Turner

FIGURE 1 Thematic coding using SurveyMonkey’s MyCategory in Analyze view.

FIGURE 2 Thematic coding in an Excel spreadsheet for long-term storage.

Decoding via Coding 47

coding as seen in the stored spreadsheet, where a 1 indicates that the code was applied.

STRATEGIES FOR ANALYSIS

Each of the strategies summarized in Table 1 highlight important aspects of our survey-question development and coding processes. When we apply the identify-categories strategy to a question, we are able to find straight- forward ways to gather quantitative data from the responses. The simplicity of analyzing our data along existing categories made this strategy essential, so it was applied to each interview question. When we do not find distinct or inherent categories in the responses, we would then turn to the remain- ing strategies to provide focus. In some cases it was necessary to apply the map-relationships strategy to look for themes and relationships that were not readily apparent. In other cases, doing further analysis for this project was not feasible or useful, and we needed to use the set-exclusion criteria strategy. Five interview questions are highlighted below to illustrate how each of these strategies was applied for our analysis.

Identify Categories

Because our interview questions frequently involved demographic informa- tion, we asked many questions with specific, defined responses. In these cases, our greatest challenge was structuring survey questions to highlight the inherent meaning, generally resulting in a closed-ended question. These quantitatively focused questions allow for quick frequency analysis. In one example, when asked “What departments are represented in this center,” interviewees provided us with verbal lists which we could supplement with information from the centers’ Web sites. In order to move from the list format to something that provides a better basis for demographic information, we applied the identify-categories strategy to create a checkbox question, shown as Figure 3. Our team selected every department and college affiliated with each center, which allows us to use statistical frequencies by department or by college.

In other instances, a more complex approach to this strategy was needed. We asked “Are you currently satisfied with the library resources and services? Why?” To find what interviewees perceived as our strengths and weaknesses, we used a multiple-choice question to capture the “yes” or “no” response, and an open-ended question was included to show the more detailed, actionable feedback we were given. We coded responses based on whether interviewees provided compliments, areas for improvement, or both, and the area of their comments. Most interviewees responded “yes”

48 P. Vaughn and C. Turner

FIGURE 3 Close-ended checkbox question created from inherent qualitative responses.

but still provided areas for improvement. This kind of analysis simply made feedback more visible, regardless of satisfaction.

Map Relationships

For questions that do not have inherent qualitative categories, as shown by the previous examples, the map-relationships strategy provides essential focus. Responses to some interview questions were anecdotally useful but frequently were not easily linked to library resources and services. To help make these responses more broadly meaningful and actionable for the li- brary, we needed to change the way we viewed the responses. By asking questions about how responses connect to library and university strategic documents or to the structure of the university, we could develop coding that makes connections between the work of a center and library or univer- sity priorities.

One example of this was in the question “Do you have a strategic plan, goals, vision, etc. that you can share with us?” Mission statements were greatly varied from one center to another and across disciplines. It was determined that coding for the focus of each center was far too specific to be meaningful for library support. Instead, the final codes, shown in Table 2, focus on broad goals which can be related to the library and university strategic goals.

TABLE 2 Categorical coding dictionary mapped to campus strategic goals.

Code Definition

Education Including training Support Support of researchers, students, others Outreach Community related, etc. Collaboration Focus on supporting collaboration

Decoding via Coding 49

TABLE 3 Hierarchical coding dictionary using library-created categories of collaboration.

Code Definition 2nd Tier Code Definition

Support Supporting units IT Departmental/college or campus IT

DOR Division of Research Library Library

Centers Other centers and institutes Colleges Specific colleges or

departments Government Government organizations Academic Academic institutions Corporate Companies or industry NonProfit Non-profit organizations

Another set of questions that benefited from a shift in focus was a group of three questions about collaboration. Interviewees were asked three separate questions: “Do the researchers affiliated with this center collabo- rate with other groups on campus? Other groups off-campus? Support units on campus?” Most researchers did not seem to differentiate between these questions.

In conducting analysis, we relied on the structure of our university for focus. By merging the responses to all three questions, we were able to code based on the type of unit or institution with which the center collaborated. We retained our differentiation of support units through hierarchical coding, because this is meaningful to us, if not to researchers at centers and institutes. This resulted in a relatively general analysis that shows the types of collab- orations that are occurring, and open-ended responses can be accessed for more specific, generally anecdotal, information. The coding used for these three questions is included as Table 3.

Set Exclusion Criteria

Finally, because of time constraints and limitations of qualitative analysis, we found that some questions couldn’t be effectively analyzed. By applying our set-exclusion criteria strategy, we were able to make practical decisions and limit analysis to data that is helpful to this project. One question we excluded was “Have any researchers affiliated with this center been awarded a major award(s) for their research?” When asking this question, our group anticipated finding out about instances where researchers were recognized for their research, but the interviewees interpreted this as grant funding. In many fields, receiving funding may be considered recognition, but we found that this question had high potential to skew based on discipline or center. Because of this skew, our group opted to leave this question un-coded. Individual responses remain valuable to this project as anecdotal evidence.

50 P. Vaughn and C. Turner

These responses may eventually lead to additional research and outreach opportunities.

CONCLUSION

One of the greatest challenges of conducting qualitative research is deter- mining what is worth analyzing. Coding along themes and topics can help to highlight priorities and provide focus to the process of analyzing qual- itative data. We were able to identify and apply three basic strategies to our text-interview data from 21 centers, which allowed us to look deeper and provide statistically meaningful information to library administrators and stakeholders.

The identify-categories strategy helped us to shift as seamlessly as pos- sible from a long text form to quantitative responses. The mapping relation- ships strategy helped us identify ways that the library could contribute to the work of the center, even when it wasn’t immediately obvious, by linking to existing strategic structures in the library or the university. Setting exclusion criteria allowed the project team to set realistic expectations for our analysis and ensure that we didn’t spend time on questions that were not possible or helpful to analyze.

While there are many tools available to libraries conducting analysis of text responses, we found creating and coding a survey in SurveyMonkey to be very effective. This nontraditional approach allowed us to highlight important aspects of our data without significant software or training costs. The survey method helped us to incorporate a large group of people into the coding process, ensuring that coding was truly meaningful and allowing for more flexibility in the process.

REFERENCES

Association of College & Research Libraries [ACRL]. (2015). Assessment in action: Academic libraries and student success. Retrieved from http://www. ala.org/acrl/aia

Begay, W., Lee, D. R., Martin, J., & Ray, M. (2004). Quantifying qualitative data. Journal of Library Administration, 40(3/4), 111. doi:10.1300/J111v40n03_09

Dennis, B. W., & Bower, T. (2008). Using content analysis software to analyze survey comments. Libraries and the Academy, 8(4), 423.

Friesen, M. (2008). Applying ATLAS.ti and Nesstar WebView to the LibQUAL+ R© results at UBC library: Getting started. Library Assessment Conference, 449.

Given, L. (2006). Qualitative research in evidence-based practice: A valuable part- nership. Library Hi Tech, 24(3), 376. doi:10.1108/07378830610692145

Gorman, G. E., & Clayton, P. (2005). Qualitative research for the information profes- sional (2nd ed.). London: Facet Publishing.

Decoding via Coding 51

Guidry, J. A. (2002). LibQUAL+TM spring 2001 comments: A qualitative anal- ysis using ATLAS.ti. Performance Measurement and Metrics, 3(2), 100. doi:10.1108/14678040210429008

Habich, E. C. (2008). Analyzing LibQUAL+ R© comments using Excel: An accessible tool for engaging discussion and action. Library Assessment Conference, 417.

Knapp, A. E. (2004). We asked them what they thought, now what do we do?. Journal of Library Administration, 30(3/4), 157. doi:10.1300/J111v40n03_12

Neurohr, K., Ackermann, E., O’Mahony, D. P., & White, L. S. (2013). Coding practices for LibQUAL+ R© open-ended comments. Evidence Based Library and Informa- tion Practice, 8(2), 96.

Rubin, H. J., & Rubin, I. S. (1995). In S. McElroy, Qualitative interviewing the art of hearing data. Thousand Oaks, CA: Sage Publications.

Copyright of Journal of Library Administration is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.