Exploratory Project #3: Interview and Field Observation Report

Researching Online Communities

Richard Colby, University of Denver The internet has extended and connected communities throughout the world, and some communities exist almost entirely online. The amount of data in these online communities and sites, as you can imagine, is immense. There are 3 million blog posts per day on Wordpress sites (and Wordpress is the technology behind 36% of all websites). There are 500 hours of content uploaded to YouTube every minute. Reddit, the fifth most visited website on the Internet, has 430 million monthly users, and they host 2.8 million comments per day. There are an estimated 100 million posts per day on Instagram, 500 tweets per day on Twitter, and the 2.5 billion worldwide facebook users, on average, like 10 posts, make 4 comments, and click on 8 ads per month. There are countless websites that invite comments, from news sources (The New York Times moderates 12,000 comments per day), to companies, to organizations. There are online communities for gaming, knitting, and politics, as well as groups that help those with cancer, grief, and depression. While collecting data from online communities can offer insight into those communities, there are ethical considerations that you should be mindful of. This handout will provide practical advice on how to conduct research of online communities (often called netnography, and you can use that term in your writing and research). The handout will also give you a checklist of online research ethics. But first, I want to emphasize the most important point. Online communities are just that— communities: these are real people behind the images and words. Yes, there are trolls, agitators, and role-players, and maybe a bot or a Russian social media hacker or two, but the majority of users in an online community are normal, earnest people with jobs, mothers, and golden retrievers named Ginger.

Research Ethics

What are the ethical considerations of online research? It’s easy to imagine that everything on the Internet is public. If it appears in a Google search, why can’t it become data in your study? There are three problems with this thinking. The first problem is that while many people are concerned about online privacy, not everybody understands their own online privacy settings. A person might not understand that their post was public. The second problem is the nature of communities. Conceptually, it’s easy for us to forget that even when we are in an online community, it is connected to the web-at-large. Imagine you stay up too late playing Fortnite with your friends instead of studying for a test. The next day, you get to class a bit late because of it. When you enter the classroom, your friends promptly say aloud for all to hear, “you are late because you played Fortnite all night instead of studying.” Your friends collapsed the community of Fortnite with the community of school and violated your trust. This can happen a lot in online communities, in what danah boyd calls “context collapse.” We might think our complaint about our boss stays in the context of

the CNN news article we were commenting on, but if we suddenly become online data (or even if the same article is read by our boss), these contexts collapse. There is a third problem that is often ignored in research, and that is the license agreements that users have agreed to (even if they haven’t read them). On the big four (facebook, Twitter, Instagram, and Youtube), the license agreements say that posters own their own content but give permission for the companies to use that content. Note, it says that the user owns the content and not “random internet researcher owns that data to use in their study.” Besides these big four sites, there are many websites that have stricter rules and license agreements that prevent researchers and outsiders using content on their websites as data. You can get banned and there can be legal risks if you don’t read license agreements and forum rules before conducting research. It’s important to remember that these are all ethical considerations and not roadblocks or rules preventing you from doing research. The point in raising these issues is for you to be aware of these concerns before conducting research. How can I ethically use online data? While informed consent is one way to be extra sure that any information that you get from persons on the internet is used with their permission, in many cases it would be logistically impossible to get and it would often create undue problems. Institutional Review Boards (IRB) currently consider most online forum content on the internet as “public,” which means you can safely use it in your study. They suggest that any site that requires approval to enter or participate should be considered private, and you should not use that data without informed consent. Otherwise, you should be fine. However, to avoid collapsed contexts, you should also ask yourself whether the information you are looking at was posted with a reasonable expectation that it was public, and whether it is not sensitive. Heidi McKee and James Porter describe the continuum of public/private, sensitive/not sensitive with the following figure (above the dotted line, you are fine; below the dotted lines, you should seek informed consent):

As long as you are at a public website or forum, and the information shared doesn’t appear to be sensitive, you are safe to use that information as data (as long as the rules of the website don’t prevent you from doing so). You can also use posts from a forum or website without ending up in legal trouble. Despite the rules and license agreements, most content on the internet is considered published and subject to copyright rules. Fair use rules allow you to copy and paste and to change the nature of the work, including remixes, commentary, and parody without seeking permission, either from the poster or the company (subject to the rules of the website, however). There is no rule about the amount of data that you can capture from a website, but ethically, you should limit your data gathering to avoid trouble. Collecting and analyzing 1,000 posts is fine. However, collecting 30,000 posts could create problems for the site and may flag your activity as suspicious. You should also consider your level of involvement in the research. Importantly, if you are using the internet to recruit participants for a survey, this is within the purview of IRB, and you should follow those guidelines from your professor. Be aware, many forums will consider surveys posted as spam, and you can be reported for spam activity or banned from the site. It also changes the community if you post on the forum that you are a researcher at a university conducting a study. Imagine you are in Starbucks, and somebody enters and says, “I’m a researcher from the University of Denver observing your behavior. Just act naturally.” Online communities will react in the same way. There are subtle ways to be a participant observer (covered in “How Do I Collect Data” later), but you should avoid deception and anything that would increase risk or disrupt the community. What ethical questions should I ask? Here’s how to translate the previous discussion into practical advice. Ask yourself the following questions:

1. How public is the information? Did it appear to be posted with a reasonable expectation that it would be just for those within the site? Does the site require a password to enter and is it moderated? If it is public, it is safe to use as data.

2. How sensitive is the information? If it is not sensitive, it is safe to use as data. 3. What are the rules and licensing agreements of the website? Do they forbid you from

using posts from the website for research? If there is nothing forbidding you from using the posts as data, it is safe to use.

4. How much data are you collecting? Are your collecting a reasonable amount of data that would not disrupt the functioning of the site? If it is below 1,000 posts, it is safe to use.

Research Techniques

What types of research can I conduct? The great thing about online communities for research is that you can conduct any sort of research with them. Let’s take a simple forum activity, the posting of memes (and the discussions that often follow). You could do the following with it:

• Rhetorical analysis of the comments

• Quantitative analysis of the length of posts and their relationship to some other variable, such as gender, age, occupation, or if demographics are difficult to determine, length of posts based on the level of agreement or disagreement with the original poster.

• Qualitative coding of the engagement—in other words, how are people reacting to each other compared to reacting to the world/situation at large, compared to reacting to the meme itself.

Online communities lend themselves to mixed method research, allowing you to collect quantitative data, qualitative data, and do some humanities-based analysis. You can also use the online communities to collect inquiry data. In other words, you can ask a question of the community and see how they respond. While you can ask new questions of a community, most communities have had questions asked of them already, and you can search for those questions. In fact, it’s important that before you ask anything of a community, you should search whether that question has been asked before. For example, let’s say I wanted to know what television shows Electric Unicycle enthusiasts liked. I go to the Electric Unicycle Forum and search to find that somebody has already asked that question. I don’t have to ask it again (and in fact, participants might be annoyed if I did since they already answered that). What sort of data do you want? You want to begin by asking the question, what do you want from the community? Is this an observation of natural community activity? In other words, are you coding what they normally talk about? Natural observations allow you to ask research questions about normal practice without your interference. Or, are you conducting an inquiry into the community’s attitudes or behaviors? Asking questions (or inquiry) changes the scope of your research by collecting responses to a question, and it is less about the typical interactions of that community. How do I collect data? There are many ways to collect data from online communities, and there are too many iterations to cover them all here; however, the most common way is called scraping. This is the process of copying and pasting into a new document online posts. It can be as simple as doing just that, copying and pasting into a Word or Excel document, but most serious researchers use services or powerful tools to scrape data from the internet, and these can cost a lot of money. Whether you are low tech or not, you can still use the term in your method section, describing that you “scraped 100 posts from the League of Legends forum.” Why do you scrape data? Because the internet is a dynamic place, and people remove, edit, and change posts all the time. You might find an important discussion that you want to study only to have it get deleted the next day by moderators. Advanced scraping can also help you organize data, allowing you to do word counts, stylistic analyses, and sort poster names or dates more easily. This is NOT meant to be an easy process (which is why services charge hundreds of dollars to do it for you).

However, you will find it easier to analyze data that you scrape than in if you try to do so in its original form. If you want to preserve the formatting (and, let’s face it, copying from the internet into Word or Excel often creates a mess), a simple way is to print to PDF from your browser. This will allow you to keep a hard copy of the forum or data as it appeared originally, and it will also allow you to combine the documents and run rudimentary analyses and searches from Adobe Acrobat. Researchers have also created powerful tools to scrape data for research (here’s a list from Ryerson University of over 60 of such tools). You can also be a participant observer in the community. This means that you would participate in the interactions of the community by asking a question or offering a response. This allows you to gather specific response data. However, above all else, you should avoid deception and not “act like a researcher.” For example, if you were interested in community participants preference for recipe media at the r/recipes subreddit, you might ask that question as a normal participant might: “I was wondering, do you all prefer a printed recipe or watching a video of the recipe?” What you DON’T want to do is the following: “Could you answer these ten questions in a post: What is your age? What is your gender? On a scale of 1-5, how well do you like video recipe sharing? On a scale of 1-5, how well do you like text recipe sharing?...” In fact, in the good post (“what do you prefer”), you can generate a longer discussion (and get more data) if you “prime” the audience with your response. While this does impact the data, recognize that ethnographers and anthropologists have recognized that we are all participant observers when we conduct research in non-virtual life. It’s just in contrast here because in online communities, we truly can be lurkers without influencing natural behavior. What data should I collect? Everything. You should save URLs, first and foremost, but also, scraping the data gives you a permanent record. In looking at a post, you should record the date/time, engagement markers (likes/emojis, comments, shares, retweets), name, any other markers (how long they have been a member, how many posts they have made, their level). You should also capture the breadcrumbs or forum hierarchies (most forums are not just a single place but are organized into various levels or rooms. For example, the World of Warcraft forums have a General, Support, Off-Topic, and at least a dozen other categories). You never know what is going to be important later, so record everything. How do I report on the data? The most important thing in reporting on online community forum participation is that the community posters do not represent the whole of the community. In fact, posts only represent a fraction of the users of the community. The best estimate is called the 1% rule—that is, only 1% of a community actively posts online. In any claim you make in your study, you should avoid over generalized claims based on 100 or 1,000 posts. Make sure you use qualifiers (e.g., most, some, based on this limited data).

You should also define some general terms for your readers. Posts are the messages that a participant makes. These might be different than “comments,” which are messages that follow a post or article. The whole discussion would be a thread (or you could call it a discussion). Are there any other resources that might be helpful? There are thousands of resources on internet research ethics and techniques. Here are four books and an article that I found useful: Consalvo, M., & Ess, C. (Eds.). (2011). The handbook of internet studies (Vol. 14). John Wiley &

Sons. Kozinets, R. V. (2010). Netnography: Doing ethnographic research online. Sage publications. Markham, A. N., & Baym, N. K. (Eds.). (2008). Internet inquiry: Conversations about method.

Sage Publications. McKee, H. A., & Porter, J. E. (2009). The ethics of internet research: A rhetorical, case-based

process (Vol. 59). Peter Lang. Secretary’s Advisory Committee on Human Research Protections (SACHRP). (2013).

Considerations and recommendations concerning internet research and human subjects research regulations. Retrieved from https://www.hhs.gov/ohrp/sites/default/files/ohrp/sachrp/mtgings/2013%20March%2 0Mtg/internet_research.pdf