Big Data

http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com.ezproxy.umuc.edu/login.aspx?direct=true&db=bth&AN=103541034&site=eds-live&scope=site

http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com.ezproxy.umuc.edu/login.aspx?direct=true&db=bth&AN=103541034&site=eds-live&scope=site

http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com.ezproxy.umuc.edu/login.aspx?direct=true&db=heh&AN=122600698&site=eds-live&scope=site

http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com.ezproxy.umuc.edu/login.aspx?direct=true&db=heh&AN=122600698&site=eds-live&scope=site

https://owl.english.purdue.edu/owl/section/1/2/

https://doi.org/10.2308/acch-50921

https://www.tableau.com/learn/training

ISSUES IN ACCOUNTING EDUCATION American Accounting Association Vol. 33, No. 4 DOI: 10.2308/iace-52146 November 2018 pp. 33–46

Using Visualization Software in the Audit of Revenue Transactions to Identify Anomalies

Lauren M. Cunningham The University of Tennessee

Sarah E. Stein Virginia Polytechnic Institute and State University

ABSTRACT: Recent changes in the accounting profession require students to enter the workforce with technical and critical-thinking skills using large datasets. In an audit setting, an important skill is the ability to identify anomalies and

risk factors in the client’s data. This instructional case provides students with experience using visualization to

identify anomalous transactions for further substantive testing based on relationships between financial data

(revenues) and nonfinancial data (weather patterns). Students must also create a memo for the workpaper files that

documents their findings, including recommendations for the audit team to select specific sales transactions for

substantive testing. Aside from gaining experience with Tableau visualization software, this case will improve

students’ problem-solving and analytical skills by encouraging them to work independently and to break a complex

problem into manageable pieces. This case is most applicable for implementation in undergraduate or graduate

auditing or internal auditing courses.

Keywords: audit risks; Big Data; data analytics; data visualization; identifying anomalies; revenues.

THE CASE

Company Background

You were recently promoted to audit senior at your firm, Aoife & Josephine LLP, and one of your primary clients is

Souper Bowl Inc. Souper Bowl (‘‘the company’’) is a privately held business headquartered in Maine, and has a fiscal year-end

of December 31. The company has been in business for nine years and prides itself on offering creative soups at a reasonable

price and that are made with locally sourced ingredients. The most popular soups include sweet potato corn chowder, curried

root vegetable and lentil, and maple-roasted butternut squash. Souper Bowl typically experiences increased sales during winter

months since soup hits the spot on a cold and snowy day. To further encourage sales on days when customers often avoid

venturing outside, the company provides a delivery service and guarantees that soup can be delivered to anyone no matter what

the weather. The company found this strategy to be particularly successful in 2015 when New England (including Maine)

experienced record snowfall during February and March.

Souper Bowl sells their soup at several restaurant locations throughout Maine. The company employs three managers that

direct the day-to-day operations for a group of stores that are organized by approximate geographic region: northern Maine

(Store Type 1), mid-Maine (Store Type 2), and coastal Maine (Store Type 3). Appendix A provides a map of these store

locations. Each manager knows their local market well and has the flexibility to advertise and offer promotions with the overall

goal of increasing sales year over year. If total sales at the end of the year exceed total sales from the prior year for that

manager’s set of locations (i.e., ‘‘Store Type’’), then the manager earns a monetary bonus from the company.

We appreciate the helpful and constructive feedback from Valaria P. Vendrzyk (editor), Ronald F. Premuroso (associate editor), and two anonymous reviewers. We are also grateful to the 2016 Deloitte Foundation Trueblood Seminar for providing us with the inspiration for this case study. Lauren M. Cunningham received financial support from the PwC INQuires Grant for the purpose of integrating technology into accounting programs.

Supplemental material can be accessed by clicking the link in Appendix C.

Editor’s note: Accepted by Valaria P. Vendrzyk.

Submitted: December 2017 Accepted: May 2018

Published Online: May 2018

33

An audit of the company is required to comply with debt covenants related to a large bank loan that the company entered

into when it began operations. Specifically, Souper Bowl must provide audited annual financial statements to the bank within

90 days of the fiscal year-end. The company must also provide unaudited quarterly financial statements to the bank within 45

days of the end of each quarter. The debt contract includes a financial covenant that requires pre-tax income in each quarter to

be greater than zero. If not met, the bank has multiple remedies at its disposal, including calling the loan such that the entire

balance is due immediately, seizing the company’s assets that are posted as collateral, or providing a waiver for the violation.

Souper Bowl’s net income for the year ended December 31, 2016 is $468,810, while net income for the prior year ended

December 31, 2015 was $825,229.

Auditing Revenues

As part of your new role as audit senior, you will be performing a large portion of the planning and testing of sales for the

2016 audit of Souper Bowl. AU-C Section 240.26 states that ‘‘when identifying and assessing the risks of material misstatement due to fraud, the auditor should, based on a presumption that risks of fraud exist in revenue recognition, evaluate which types of

revenue, revenue transactions, or assertions give rise to such risks.’’ During planning for the audit, the partner and manager determined that the following three management assertions represent significant risks for revenues:

(1) recorded sales occurred;

(2) sales are accurately recorded; and

(3) sales are recorded in the proper period.

In prior years, the audit approach relied on random sampling to test revenues. However, the partner wanted to develop more

focused procedures in the current year to hone in on potentially riskier sales transactions. As a result, the plan is to perform

disaggregated sales analytics to identify unusual trends in the daily sales data with the goal of identifying sales on specific days

at specific store locations that should be subjected to substantive testing due to heightened risks. The remainder of the

population would then be sampled using a random sampling approach. 1

Based on your experience from prior audits, you know that Souper Bowl’s daily sales fluctuate with temperature and snow

accumulation. To perform your revenue analytics, you request a file from the client that includes daily sales by store location

for both 2016 (current year) and 2015 ( prior year). You also retrieve daily weather data from the National Oceanic and

Atmospheric Administration’s (NOAA) website for the weather centers closest to Souper Bowl’s store locations. Total revenue

for the current year ended December 31, 2016 is $18.8 million, while total revenue for the prior year ended December 31, 2015

was $19.1 million. The audit team’s workpapers include the following lead sheet for revenue testing, and the total balances for

each year agree to the trial balance and the company’s draft financial statements for 2016.

Souper Bowl Inc. Revenue Lead Sheet December 31, 2016

2016 2015 Change

PBC

% Change

Revenue, Store Type 1 $4,062,390.97 $4,032,383.16 $30,007.81 0.74%

Revenue, Store Type 2 9,331,175.81 9,558,584.07 (227,408.26) �2.38% Revenue, Store Type 3 5,425,421.53 5,546,767.89 (121,346.36) �2.19% Total Revenue $18,818,988.31 $19,137,735.12 $(318,746.81) �1.67%

Your manager stated that Tableau is a popular data visualization tool that your firm recently adopted and she instructed that

you learn how to use it to perform these sales analytics. Since she is busy overseeing the planning and testing of other audit

areas, she wants you to take the first pass and then document your results in a memo for her review. The manager wants you to

provide thoughtful analyses and a thorough exploration of the possible relationships in the data. You are eager to impress her

with your work, especially following your recent promotion to senior.

1 As noted in Christensen, Elder, and Glover (2015), most of the large accounting firms emphasize the use of specific identification testing before random sampling to increase the efficiency and effectiveness of substantive tests. Specific identification testing, also referred to as directed sampling, is usually used to select and test individually significant items and those items identified as having a higher risk of misstatement.

34 Cunningham and Stein

Issues in Accounting Education Volume 33, Number 4, 2018

Requirements

1. Read the articles assigned by your instructor to gain an understanding about how Big Data, data analytics, and new

technologies are transforming external audits. After reading these articles, provide a response to the following two

questions:

a. Discuss at least three specific ways in which Big Data, data analytics, and new technologies can enhance external

audits. How does each item discussed improve the effectiveness and/or efficiency of the audit?

b. On the other hand, what challenges do auditors face when using Big Data, data analytics, and new technologies

during an audit?

2. As noted in the case, auditing standards specifically require auditors to identify revenue recognition as a fraud risk in

most audits. Based on your understanding of the company, what factors may increase the risk of fraudulent financial

reporting in Souper Bowl’s 2016 revenues?

3. Use the daily sales by location as provided by the client (2016 and 2015) and the weather data from NOAA to perform

disaggregated sales analytics in Tableau. Your goal is to develop visualizations that identify potential outliers in the

2016 daily sales data related to the significant risks identified by the partner and manager. Using the memo template in

Appendix B, document your analyses and conclusions as to the specific daily sales from certain locations that you

recommend selecting for focused substantive testing.

Note: Your conclusion needs to be precise enough to pull specific transactions—for example, you would not list the ‘‘month of March’’ in store 1010 because this would result in too many observations to feasibly test. Also, you should not recommend testing observations from 2015. Your engagement team completed that audit in the prior year—instead, you are using 2015 data as a component of your baseline prediction for 2016.

REFERENCE

Christensen, B., R. Elder, and S. Glover. 2015. Behind the numbers: Insights into large audit firm sampling policies. Accounting Horizons 29 (1): 61–81. https://doi.org/10.2308/acch-50921

Using Visualization Software in the Audit of Revenue Transactions to Identify Anomalies 35

Issues in Accounting Education Volume 33, Number 4, 2018

APPENDIX A

Map of Store Locations for Souper Bowl Inc.

36 Cunningham and Stein

Issues in Accounting Education Volume 33, Number 4, 2018

APPENDIX B

Example Memo Template*

Souper Bowl Inc.—December 31, 2016 Disaggregated Revenue Analytics

Purpose: The purpose of this memo is to document plausible trends and expectations for disaggregated revenue data and to identify specific days and locations that warrant further substantive investigation.

Data: We obtained a listing of daily sales by location from the client’s IT system. We tested the details for mathematical accuracy, as summarized in the table below:

Total Sales, 2015 Total Sales, 2016

Store Type 1 $ $

Store Type 2 $ $

Store Type 3 $ $

Total $ $

Procedures: Based on our risk assessment process, we identified the following assertions as significant risks related to revenues/sales:

� Recorded sales occurred. � Sales are accurately recorded. � Sales are recorded in the proper period.

Because Souper Bowl’s operations are solely in the state of Maine, we obtained disaggregated data that reports daily sales

by store location and Store Type. Based on discussions with management and our review of the board of director minutes, we

are unaware of any new store locations or other major changes to operations during the year. Therefore, we expect the prior

year’s revenues to be a reasonable baseline expectation for this year’s revenues (e.g., similar seasonal trends). Because the

business can also be impacted by weather conditions, which vary by year, we also perform analyses that consider changes in

weather patterns to predict expected changes from the prior year’s sales. We performed several analytics to identify unusual

trends as compared to the prior year’s sales, taking weather conditions into consideration. The purpose of these analytics is to

identify specific observations (or specific sets of observations) to select for further substantive testing. The analytics that we

performed are as follows:

� Visualization Analysis #1: Title [Provide a description of the relationship you expected to observe in the data, along with screenshots of the visualization results. Clearly identify (using circles, arrows, etc.) the part of the visualization that leads you to believe that a specific location/day is an anomaly. Ensure that all tables and graphics are properly labeled (x axis, y axis, etc.).]

* Results: [In each of the ‘‘Results’’ sections, include a brief summary of your findings so that your manager can see (in words) the way that you interpret the visualization screenshots.]

� Visualization Analysis #2: Title * Results:

� [The number of analyses that you perform is up to you. Remember that you want to impress your manager, but you also know that the manager’s time is valuable. Therefore, each analysis that you report should offer new information and conclusions (e.g., avoid repeating the same type of analysis with different coloring, shapes, etc., if the conclusions drawn are the same.)

Conclusion: Based on the procedures described above, the audit team will pull supporting sales information to substantively test transactions from the following locations and days:

� This section of the memo can be achieved by using lists or tables, but regardless of the style of presentation, it should clearly describe which item(s) you are recommending that the audit team look into further (based on all the analyses above). For each item, you should reference which analysis # the item comes from. The item should be a specific location on a specific day, or a sample of certain days from a set of observations that exhibit the same unusual trend based on your analyses above (e.g., if you identify an unusual relationship for Q4 for location #1001, but you cannot identify one specific day or set of days that is driving the unusual relationship, you may choose to sample from Q4

Using Visualization Software in the Audit of Revenue Transactions to Identify Anomalies 37

Issues in Accounting Education Volume 33, Number 4, 2018

instead). Remember that it takes time and resources to test each selection, so be strategic in your selections and include justification for your decisions in this section of the memo.

* NOTE: Appendix B is available as a downloadable Word document, see Appendix C for the link.

APPENDIX C

iace-52146_Example Memo Template: http://dx.doi.org/10.2308/iace-52146.s01

38 Cunningham and Stein

Issues in Accounting Education Volume 33, Number 4, 2018

CASE LEARNING OBJECTIVES AND IMPLEMENTATION GUIDANCE

Purpose of the Case

The purpose of this case is to expose students to data analytics by using critical-thinking skills to apply visualization

techniques to a scenario that involves the audit of revenues. Data analytics is the specific process of examining raw data for the

purpose of drawing conclusions about that information (Janvrin and Watson 2017). 2

In auditing, data analytics encompasses

various changes or enhancements to existing procedures such as extracting and using new data sources for analytical

procedures, using new technologies to test 100 percent of the transactions in a population for a specific assertion, employing

artificial intelligence to identify key features in contracts, and using visualization to identify unusual trends and anomalies in a

population (Cao, Chychyla, and Stewart 2015; Cohn 2017; ICAEW 2016; Raphael 2017). These new tools and techniques

allow auditors to analyze much larger datasets and detect trends in the data that might indicate a greater risk for error or fraud

(Rapoport 2016 ). Accounting firms are investing heavily in new technology-based tools for audits, which will significantly

impact the amount of information available to auditors, staffing of audit engagements, and judgments made by auditors in

critical areas of the audit (PCAOB 2017).

These significant improvements to data processing and analytics are placing greater pressure on new graduates to enter the

profession equipped to handle the rapidly changing technology (Janvrin and Watson 2017; Pathways Commission 2012). Our

case responds to calls to integrate data analytics and technology into accounting curriculum (AACSB 2017; PwC 2015). 3

Specifically, this case asks students to use analytical skills to identify potentially anomalous transactions that would require

additional substantive testing. These types of data analytics allow audit teams to reduce sampling risk (i.e., the risk that the

sample is not representative of the population) by honing in on particularly high-risk items in the data that might not have been

selected by using a random sample. As noted in a recent survey, most of the large accounting firms emphasize this type of

specific identification testing before random sampling to increase the efficiency and effectiveness of substantive tests

(Christensen, Elder, and Glover 2015). Moreover, PwC recommends that students who are entering the audit profession gain

skills to ‘‘research and identify anomalies and risk factors in underlying data’’ (PwC 2015, 11). Instead of using approaches

such as Benford’s Law to analyze one set of financial data (Enget, Saucedo, and Wright 2017; Fay and Negangard 2017), 4

our

case requires students to use visualization dashboards to identify outliers based on linear and nonlinear relationships between

two sources of data: financial data (revenues) and nonfinancial data (weather patterns). 5

Our case involves the use of data visualization software to identify outliers in over 21,000 observations of realistically

simulated revenue data. 6

Visualization has gained increasing popularity among public accounting firms and corporations

(Arrowsmith 2017). ‘‘Data visualization is about how to present your data, to the right people, at the right time, in order to

enable them to gain insights most effectively’’ (Marr 2017). Because auditors are time constrained, effective visualization can

be a useful tool in identifying areas of financial information that require additional substantive testing. Visualization software is

often intuitive and quick for both instructors and students to use. As reported in Kokina et al. (2017, 59), ‘‘In our experience,

students get used to Tableau’s interface quickly and are able to create their own visualizations within 5–10 minutes of seeing

the software for the first time.’’ We designed our case to be simple enough for students who have never used Tableau, yet

complex enough for students who have had experience with Tableau.

Beyond experience with data visualization as discussed above, an important aspect of this case is the use of judgment and

critical thinking when developing and interpreting the visualizations. The 2016 ICAEW report on data analytics for external

auditors emphasizes the importance of judgment as it relates to data analytics in the following statement:

2 Janvrin and Watson (2017) provide introductory commentary for a ‘‘Big Data’’ special issue of the Journal of Accounting Education. Their commentary provides many excellent resources for instructors interested in using Big Data in the classroom. Our case can be executed as the only visualization-based project in a course or could be offered as a complement to other Big Data cases used in a course.

3 For example, in the most recent eligibility procedures exposure draft for AACSB accounting accreditation, Standard #5 requires that ‘‘accounting degree programs include learning experiences that develop skills and knowledge related to the integration of information technology in accounting and business. These learning experiences include the development of skills and knowledge related to data creation, manipulation/management, sharing, analytics (including predictive analytics), mining, modelling, reporting, and storage within and across organizations’’ (AACSB 2017, 26).

4 Benford’s Law states that if you randomly select a number from a natural dataset of numbers, the value of the first digit in that number will occur at a predictable frequency. Specifically, numbers in the real world are more likely to have a low number as the first digit than a high number (Enget et al. 2017; Fay and Negangard 2017).

5 Our case is motivated by a real-world example of data analytics used by one of the Big 4 accounting firms whose client operated in the utilities sector. The Big 4 firm used weather patterns as a key source of nonfinancial data that should have a predictable relationship with the client’s utility revenue data.

6 Kokina, Pachamanova, and Corbett (2017) require students to use Excel and Tableau for dashboard visualization in a managerial setting. To provide scale for our dataset, Kokina et al.’s (2017) dataset includes 1,100 observations.

Using Visualization Software in the Audit of Revenue Transactions to Identify Anomalies 39

Issues in Accounting Education Volume 33, Number 4, 2018

Audit quality does not lie in the tools themselves—although it clearly cannot be achieved without tools that are fit for

purpose—rather it lies in the quality of analyses and judgements thereby facilitated. The value is not in the

transformation of the data (however impressive), but in the audit evidence extracted from the conversations and

enquiries that the analytics generates. (ICAEW 2016, 6)

PwC’s report further elaborates that ‘‘it’s not simply a matter of technical skills, but also the ability to think critically using

data’’ (PwC 2015, 17). This case is designed to simulate a real-world experience such that there is no one correct answer, but

instead students must use their judgment to develop a strategy to identify anomalies in the revenue population. The students

must document their logic and findings in a written memo to illustrate their approach. As a result, instructors should expect that

the resulting output will be very different from one student to the next. Aside from introducing students to the use of data

visualization in an audit, this case can be used in conjunction with several topics such as risk assessment, sampling, and the

sales/revenue cycle in an auditing course.

Learning Objectives

The first learning objective of this case is to expand students’ knowledge of various types of data analytics and audit

innovation in practice. Students are required to read and discuss recent articles that illustrate new audit innovations and then

apply their knowledge to a specific setting, using data visualization to identify riskier sales transactions. Our goal is to increase

students’ understanding of and comfort level with data analytics in an audit so that they are better prepared for their new

responsibilities upon entering public accounting.

The second learning objective of this case is to use critical-thinking skills. The ‘‘Implementation Guidance’’ section

describes how instructors can encourage students to work independently (or in small groups) without detailed assistance from

the instructor. This method encourages students to use critical-thinking skills when completing the case. Specifically, we

encourage students to gain an understanding of the data through aggregated high-level analyses, and then to begin identifying

outliers through further disaggregated analyses of the different types of weather patterns, store locations, and time periods. This

learning objective is consistent with AACSB (2017) curriculum guidance to provide learning experiences that require students

to use critical thinking as well as conversations with public accounting professionals, which suggests that students need

exposure to projects that require critical thinking.

The third learning objective of this case is to develop skills specific to data visualization in an audit setting. Data

visualization can be a powerful way to transform data into audit insights for planning and risk assessment or identification of

transactions with greater risk in substantive testing. The instructors of this case required students to learn Tableau—a tool

adopted by many of the accounting firms—to develop visualization skills in a real-world setting. However, this case can be

adapted to use any data visualization tool (described further below). Learning how to work with new software is an important

skill in today’s workplace environment (Janvrin and Watson 2017).

Implementation Guidance

Visualization Software

Prior to the introduction of the case in class, the instructor should encourage students to learn how to use the software

chosen for the case. We chose Tableau because it offers free academic licenses, plenty of free instructional resources, 7

is

covered in textbooks already used in the classroom, 8

and is used by several public accounting firms and companies. Instructor

and student licenses can be requested for the semester at the following website: https://www.tableau.com/academic/teaching.

Instead of offering in-class Tableau tutorials, we instructed our students to complete the Lynda.com basic tutorials for Tableau. 9

The specific instructions provided to our students were as follows:

Find the ‘‘Tableau 10 Essential Training’’ on Lynda.com, as accessed through your university’s website. Some of the key

topics in this training are as follows:

7 Kokina et al. (2017) specifically recommend the Getting Started, Connecting to Data, Visual Analytics, and Dashboards and Stories videos from Tableau’s website (http://www.tableau.com/learn/training). ‘‘In our experience, students learn Tableau quickly through the instructional videos available on the software provider’s website’’ (Kokina et al. 2017, 53). The HUB of Analytics Education also provides several online training videos to get started with Tableau (http://www.hubae.org/tableausoftware).

8 For example, see Chapter 9, ‘‘Data Analytics in Accounting’’ in Richardson, Chang, and Smith (2018) for screenshots and step-by-step guidance to import data into Tableau and create basic visualizations.

9 You can determine whether your school offers a Lynda.com subscription by asking your school’s librarian or subscription manager.

40 Cunningham and Stein

Issues in Accounting Education Volume 33, Number 4, 2018

� Working in Tableau basics: * Lynda, Chapter 2, ‘‘Connecting to a Data Source’’ * Lynda, Chapter 2, ‘‘Joining Related Data Sources’’—to connect the weather data with the daily sales data * Lynda, Chapter 2, ‘‘Creating a Visualization Using Show Me’’ * Lynda, Chapter 2, ‘‘Changing the Order of the Fields in a Visualization’’ * Lynda, Chapter 2, ‘‘Changing the Summary Operation’’—for example, to get daily/weekly/monthly/etc. and averages

vs. totals � Lynda, Chapter 3 ‘‘Displaying the Data Underlying a Workbook’’: This is how you can see which ‘‘suspect’’

observations are in the unusual day/week/quarter, etc. that you have selected. This video shows you how to copy that

data to your clipboard so you can then work with that data in Excel, Word, etc. � Lynda, Chapters 9–11: These videos will give you examples of the different types of visualizations you can make. Do

not forget to stay focused! What are the plausible relationships in the data, and what do you expect to see?

The instructor could also consider offering a hands-on Tableau learning session (in-class or out-of-class) or other Tableau cases

as prerequisite assignments if desired. 10

It is not necessary to use Tableau to implement this case. Several other software companies offer data visualization

packages of varying degrees, including IDEA, Microsoft Power BI, Birst, and Sisense. Alternatively, instructors could require

students to use pivot tables and graphs in Microsoft Excel to make visualizations. The Teaching Notes for this case provide

example screenshots and additional implementation guidance for instructors who will use Tableau.

Topics to Discuss in Class When Introducing the Case to Students

We recommend discussing the following topics with students during class to emphasize key aspects of the case before they

begin working on their own:

� Tableau: Students should watch the instructional videos on the Tableau website and/or execute the Lynda.com tutorials on their own time prior to class on the project introduction day. If students have done this preparation, the only thing to

reinforce in class is that students need to know how to import the data into Tableau. Some key aspects of that process are

discussed below, and can be demonstrated in class with Tableau with screenshots in PowerPoint or with a pre-recorded

video. 11

* Joining data: Data must be joined using both date and weather center location. Joining on only one of these fields will produce incorrect inferences because data will be duplicated. Tableau will most likely auto-join using only the

date field, so students must manually fix the joining feature to also include the weather center location. * Measures versus Dimensions: Upon first importing the data, all numeric fields will be listed as a Measure. However,

the following variables should be moved to Dimensions because they are characteristics across which the data will

most likely vary: Store Location, Store Type, Latitude, and Longitude. The only variables left in Measures should be:

Sales, SNWD, TMAX, and TMIN. * Instructors may also choose to spend time discussing Dual Axes, Filters, and various ways to use Dimensions, since

these aspects may be less intuitive than other Tableau features. The example PowerPoint slides in the Teaching Notes

cover these topics. � Linear Relationships versus Nonlinear Relationships: Students may not fully understand these concepts from prior

classes, or if they do, it may not be in the context of data analysis for auditing. Therefore, instructors may need to spend

time introducing each of these concepts. Consider asking students to describe these concepts in words and then illustrate

them on the board. We recommend also discussing anonymized examples (so as not to reveal potential solutions to the

case) from the sample PowerPoint slides provided in the Teaching Notes.

We did not include these instructions in the Case file for students because instructors may wish to provide more (less)

instruction for students who have very little (advanced) prior experience working with Tableau. We were able to cover all the

topics above in a one-hour lecture, with time for taking questions from students.

10 Since the authors of this case did not offer a hands-on Tableau learning session, we cannot provide specific materials or recommendations for the length of time needed to do so. The ‘‘Getting Started’’ video on Tableau’s website is approximately 35 minutes long. In contrast, the recommended segments in the ‘‘Tableau 10 Essential Training’’ on Lynda.com total approximately 80 minutes. Additional time will be required to follow along with the sample data, and time will vary by person based on prior experience with Tableau or similar software. For additional Tableau cases, consider using the EY Academic Resource Center’s (ARC) resources for The Analytics Mindset. The ‘‘Master Case Guide,’’ which is regularly updated on the EY ARC website, reports which cases include visualization tools. As of early 2018, six of their analytics cases offered an element with visualization.

11 In the Teaching Notes for this case, we include a sample set of PowerPoint slides that instructors may wish to tailor and use to introduce the case.

Using Visualization Software in the Audit of Revenue Transactions to Identify Anomalies 41

Issues in Accounting Education Volume 33, Number 4, 2018

Deciding on Specific Articles for Requirement 1

Requirement 1 in the case asks students to read assigned articles and reflect on the ways in which data analytics enhance

external audits and present challenges for auditors. Since this area is changing rapidly, we purposely did not list specific articles

for this requirement. Instead, we encourage instructors to select the most recent and relevant articles at the time of assigning this

project. To provide an example, we assigned the following articles to students during the Fall 2017 semester:

Required Articles

Cao, M., R. Chychyla, and T. Stewart. 2015. Big Data analytics in financial statement audits. Accounting Horizons 29 (2): 423–429.

Raphael, J. 2017. Rethinking the audit. Journal of Accountancy 223 (4): 28–32.

Optional Article

ICAEW. 2016. Data analytics for external auditors. International Accounting, Auditing & Ethics. Available at: https:// www.icaew.com/en/international-accounting-and-auditing/international-auditing-perspectives/data-analytics-for-

external-auditors

As described in the ‘‘Effectiveness of Case’’ section below, we implemented the case so that one instructor required Requirement 1 and one instructor did not. Students who were not required to complete Requirement 1 reported a lower

‘‘knowledge of Big Data and analytics in an external audit’’ score in the post-case survey than students who did complete Requirement 1. Thus, instructors may obtain different learning outcomes by including (excluding) Requirement 1.

The Instructor’s Role

As the instructor, you should regularly communicate that there is no ‘‘right’’ answer to this case. Your goal should be to facilitate the case by introducing the general concepts discussed in the main case document prior to assigning the case to

students, and then to offer some guidance during the assignment phase (i.e., the time between the introduction of the project and the due date). Remind your students that in the real world, they will be provided with incomplete information about a project

and expected to learn new tools on their own with limited supervision. Therefore, one of the learning objectives of this

assignment is to work independently (or in a small group) to solve problems without the assistance of the instructor. During the assignment phase, it will be tempting to answer questions for your students by showing off your own Tableau skills

or by guiding your students to produce the same visualizations or analyses. Please fight that temptation. As the instructor, you can

encourage your students to think independently and gain confidence with their own analytic skills by grading for logic versus grading for ‘‘right’’ answers. When students have questions during the assignment phase, you can refer students to the PowerPoint slides that were provided when the case was introduced in class (see above). After students attempt their own analyses, you can

offer students to bring their Tableau workbooks to office hours to ask questions. 12

When meeting with students, let your student be

the guide on the planned analyses and ask yourself the following questions when providing guidance or grading:

� Did the student develop a reasonable expectation for the relationship between weather and sales? � Is the visualization effective in showing that relationship?

* Has the student demonstrated that the visualization ‘‘works’’ in the prior year (2015)? * Has the student properly captured a linear relationship versus a nonlinear relationship?

� Did the student interpret anomalies correctly in the visualization for current year data (2016 )? � Did the student clearly document logic, visualization, and the conclusions in a way that would be appropriate for

professional audit workpapers?

We recommend avoiding the temptation to suggest visualizations or analyses that were not already included in the student’s

memo unless the student asks a question about wanting to do a specific type of analyses but needs a small amount of guidance

to begin. 13

Instructors offering unprompted guidance risk conforming all student responses to the same answer. Instructors may

12 The instructor implementing this case as a group project (with 31 groups, 3 students per group) offered groups to stop by during regular office hours with no limits on the number of visits. Generally, most groups stopped by one time and no groups visited more than twice. The instructor implementing this case as an individual project (with 68 students) set up two separate project-only office hours that allowed for 5–10 minutes per student. Students signed up ahead of time and were encouraged to be organized and prepared to make the best use of their time with the instructor. Students were allowed to sign up for both sets of office hours if needed. More than 80 percent of students signed up for the first set of office hours and less than 40 percent signed up for the second set of office hours.

13 For both instructors of this case, the guidance often centered on how to take advantage of the ‘‘Marks’’ and ‘‘Filter’’ features in Tableau and the use of ‘‘Dimensions.’’

42 Cunningham and Stein

Issues in Accounting Education Volume 33, Number 4, 2018

also consider collecting common student questions and distributing general answers to the entire class to ensure that all students

receive similar information about the case.

Effectiveness of Case

Students enrolled in undergraduate and graduate auditing courses at two large state universities completed the case during

the Fall 2017 semester. This case was assigned as a group project in the undergraduate auditing course at one university (93

students) and as an individual project in the graduate auditing course at a different university (68 students). The instructor using

this case as a group project for undergraduates allowed students to form groups of three students and provided six weeks

between the initial case introduction and the due date. This group project represented 15 percent of students’ overall grade in

the undergraduate course. On average, undergraduate students reported that the project took approximately 10 hours for each

person in the group to complete the assignment. The other instructor required students in a graduate course to turn in individual

assignments and allowed three weeks between the initial case introduction and the due date. This individual assignment

represented 15 percent of students’ overall grade in the graduate course. As an individual assignment, this instructor did not

require students to read the assigned articles and respond to the initial questions in Requirements 1 and 2 of the case materials,

but instead asked them to focus on the Tableau analyses and writing the memo. On average, graduate students reported that the

individual assignment took approximately 17.5 hours to complete. Instructors should choose the teaching approach that best fits

the needs of the specific class.

A summary of total time required for the case is provided in Table 1. Student time was based on the post-case survey data

(described below). Instructor time is based on two instructors who were already familiar with Tableau and the case materials;

therefore, this time includes re-familiarizing themselves with Tableau functionality, reviewing the PowerPoint slides (see

above), and creating potential solutions in Tableau that might be provided by students. Instructors not already familiar with

Tableau and the case may require an additional one to two hours of Tableau instructional time using the resources described

above and an additional two hours to familiarize themselves with the case and dataset. In-class time includes approximately one

hour to introduce the case and 30 minutes to debrief the case after grading student projects.

To assess the effectiveness of the case study, students anonymously responded to a pre-case survey before the case was

introduced in class. After students completed the case, instructors debriefed the case in class (refer to the Teaching Notes for

further discussion). Following the case debrief, students anonymously responded to a post-case survey. The post-case survey

included the same five questions as the pre-case survey as well as a series of additional questions to gather student perceptions

of the case. One hundred fifty-six of the 161 students (96.9 percent) completed the pre-case survey, while 152 of the 161

students (94.4 percent) completed the post-case survey. Panel A of Table 2 reports student feedback for the questions asked in

both the pre-case and post-case surveys and also provides a test of differences in mean values. Panel B of Table 2 provides

student feedback for questions included only in the post-case survey.

The combined survey results in Table 2 indicate the case was effective at achieving the learning objectives. In Panel A,

responses were provided on a five-point Likert scale where 1 is ‘‘no knowledge/not comfortable/not interested,’’ 3 is ‘‘some

knowledge/somewhat comfortable/somewhat interested,’’ and 5 is ‘‘high knowledge/very comfortable/very interested.’’ The

results indicate that both undergraduate and graduate students’ knowledge of and comfort with Big Data and analytics increased

significantly between the pre-case and post-case surveys (Q1 knowledge: pre-case total mean of 2.167, post-case total mean of

3.250; Q3 comfort: pre-case total mean of 1.962, post-case total mean of 3.053). Similar increases in undergraduate and

TABLE 1

Time Required for Instructors and Students

Instructor Time Student Time

Ahead of Classa In-Class

Office Hours Total

Case Requirements

1 and 2 Learning Tableau

Creating Visualizations and the Memo Total

Undergraduate (Group) 3 1.5 8 12.5 2 3 5 10

Graduate (Individual) 3 1.5 12 16.5 0 b

5 12.5 17.5

a This time is based on two instructors who are already familiar with Tableau and the case materials. Instructors not already familiar with Tableau and the case may require an additional one to two hours of Tableau instructional time using the resources described in the text and an additional two hours to familiarize themselves with the case and dataset.

b Students in the graduate course were not required to complete Requirements 1 and 2 in the Case file.

Using Visualization Software in the Audit of Revenue Transactions to Identify Anomalies 43

Issues in Accounting Education Volume 33, Number 4, 2018

TABLE 2

Student Feedback

Panel A: Pre- and Post-Case Questions

Pre-Case Mean

(Std. Dev.)

Post-Case Mean

(Std. Dev.) Mean

Difference

Undergrad (n ¼ 89)

Graduate (n ¼ 67)

Total (n ¼ 156)

Undergrad (n ¼ 86)

Graduate (n ¼ 66)

Total (n ¼ 152) Undergrad Graduate Total

Q1: Rate your knowledge of the

use of Big Data and

analytics in an external

audit. a

2.022 2.358 2.167 3.360 3.106 3.250 1.338*** 0.748*** 1.083***

(0.853) (0.865) (0.871) (0.684) (0.726) (0.712)

Q2: Rate your knowledge of

performing data

visualization in Tableau. a

1.225 2.000 1.558 3.093 3.258 3.164 1.868*** 1.258*** 1.607***

(0.559) (1.000) (0.867) (0.835) (0.829) (0.833)

Q3: How comfortable are you

with the use of Big Data

and analytics in an external

audit? b

1.854 2.104 1.962 3.070 3.030 3.053 1.216*** 0.926*** 1.091***

(0.972) (0.923) (0.956) (0.665) (0.784) (0.717)

Q4: How comfortable are you

with performing data

visualization in Tableau? b

1.270 1.776 1.487 3.023 3.288 3.138 1.754*** 1.512*** 1.651***

(0.579) (0.951) (0.799) (0.945) (0.760) (0.877)

Q5: How interested are you in

the topic of Big Data and

analytics in an audit? c

3.191 3.418 3.288 3.302 3.682 3.467 0.111 0.264 0.179

(1.157) (1.183) (1.170) (1.149) (1.084) (1.133)

Panel B: Post-Case Questions Only

Mean (Std. Dev.)

Undergrad (n ¼ 86)

Graduate (n ¼ 66)

Total (n ¼ 152)

Q1: The instructions and background information in the case were clear. d

3.523*** 3.369*** 3.457***

(0.955) (1.206) (1.069)

Q2: The case was interesting. d

3.535*** 3.576*** 3.553***

(1.048) (0.929) (0.995)

Q3: The case was the appropriate level of difficulty for this course. d

3.709*** 3.879*** 3.783***

(0.944) (0.937) (0.941)

Q4: The case and related questions required me to use critical-thinking skills. d

4.349*** 4.409*** 4.375***

(0.748) (0.701) (0.726)

Q5: The case improved my knowledge about the use of Big Data and analytics in an external

audit. d

4.023*** 4.061*** 4.039***

(0.840) (0.959) (0.891)

Q6: The case improved my understanding of how to perform data visualization in Tableau. d

3.919*** 4.197*** 4.039***

(1.008) (0.932) (0.983)

Q7: The case improved my confidence for doing similar analyses on actual clients. d

3.314*** 3.652*** 3.461***

(0.924) (1.030) (0.983)

Q8: I would recommend this case as a learning tool for future auditing classes. d

3.698*** 3.924*** 3.796***

(1.117) (0.997) (1.069)

*** Panel A: Significant mean difference at p , 0.001. *** Panel B: Significantly greater than the mid-point value of 3 at p , 0.001. a

Scale is 1–5, where 1 is ‘‘No knowledge,’’ 3 is ‘‘Some knowledge,’’ and 5 is ‘‘High knowledge.’’ b

Scale is 1–5, where 1 is ‘‘Not comfortable,’’ 3 is ‘‘Somewhat comfortable,’’ and 5 is ‘‘Very comfortable.’’ c

Scale is 1–5, where 1 is ‘‘Not interested,’’ 3 is ‘‘Somewhat interested,’’ and 5 is ‘‘Very interested.’’ d

Scale is 1–5, where 1 is ‘‘Strongly disagree,’’ 3 is ‘‘Neutral,’’ and 5 is ‘‘Strongly agree.’’

44 Cunningham and Stein

Issues in Accounting Education Volume 33, Number 4, 2018

graduate students’ knowledge of and comfort with data visualization are also noted between the pre-case and post-case surveys

(Q2 knowledge: pre-case total mean of 1.558, post-case total mean of 3.164; Q4 comfort: pre-case total mean of 1.487, post-

case total mean of 3.138). We find no statistical difference in pre- versus post-case responses for students’ overall interest in the

topic of Big Data and analytics in an audit; however, we note that student interest was already above the midpoint value of 3

prior to completing the case. 14

In Panel B of Table 2, responses were provided on a five-point Likert scale where 1 is ‘‘strongly disagree,’’ 3 is ‘‘neutral,’’ and 5 is ‘‘strongly agree.’’ The results indicate that both undergraduate and graduate students thought the case included clear instructions and background information (Q1 total mean of 3.457), was interesting (Q2 total mean of 3.553), and was the

appropriate level of difficulty (Q3 total mean of 3.783). Importantly, students agreed that the case required critical thinking (Q4

total mean of 4.375), 15

improved their knowledge about Big Data and analytics in an audit (Q5 total mean of 4.039), and

improved their technical skills for performing visualizations in Tableau (Q6 total mean of 4.039). Finally, students indicated

greater confidence in performing similar analyses on actual clients (Q7 total mean of 3.461) and generally recommend this case

as a learning tool in future auditing courses (Q8 total mean of 3.796 ). The mean values for each of these questions are

significantly different ( p-value , 0.001) from the scale’s mid-point value of 3 using one-sample t-tests as noted in Table 2.

Qualitative responses from students also suggest that this case achieved the learning objectives. Several students liked that

the case simulated a real-world situation and provided an opportunity to tackle a problem that did not have a specific ‘‘correct’’ answer. Other students appreciated the critical thinking and challenge involved with tackling this type of data analysis. Many

students also enjoyed working in Tableau, and learning how to create various types of visualization to identify trends and

outliers in the data. Overall, students provided several positive comments, a few of which are included below:

‘‘I really enjoyed learning Tableau because I have heard about it in practice so it was nice to work on something similar to what I may do in the real world. I felt the overall project helped me think differently than most of my college

courses by using a more analytical mind than just learning facts from a textbook.’’

‘‘I liked how realistic the project felt. It seemed like a real-world application and not just a dummy dataset.’’

‘‘What I liked most about the project was being able to watch our growth from beginning to end. At the start of the project no one in our group was very confident in analyzing Big Data. To see how we progressed over multiple

meetings and practice was very encouraging. At the end of the project I felt like we had come a long way and I had a

newfound confidence about being able to tackle data analytics.’’

‘‘The absence of any formal guidelines for what our visualizations should look like really forced me to be creative and more analytical over the course of the project. Having to figure out what my data was even remotely supposed to look

like was challenging but much more rewarding when I found something that worked.’’

Students also offered recommended improvements to the case if implemented in future auditing courses. Their feedback

primarily centered on providing more instruction about how to use Tableau. For example:

‘‘I wish we had more time to discuss and learn how to use Tableau in class because I think it is such an interesting way to look at data. I think the extra class time and more examples of how to use the tool would be beneficial to students. I

think having an in-class exercise on Tableau would be really interesting and helpful to future students.’’

We purposefully required students to learn Tableau on their own to simulate a likely real-world scenario in which the audit

manager does not know how to use the new technology. However, instructors should consider what works best for their own

class. As discussed previously, certain instructors may want to provide an in-class or out-of-class session using an alternative

dataset to discuss some of the key features in the visualization software.

14 When looking at undergraduate and graduate students separately, we notice that undergraduate students rated their post-case knowledge of Big Data and analytics in an external audit higher than graduate students (Q1 in Panel A), whereas graduate students indicated higher post-case knowledge/ comfort performing visualizations in Tableau (Q3 and Q4 in Panel A). These differences likely exist because undergraduate students were required to complete the background readings in Requirement 1 of the case materials but performed the visualizations in a group, while graduate students were not required to complete the readings and instead performed the visualizations individually. We encourage instructors to weigh these options when deciding the best approach for their own course.

15 Since we did not ask a pre-case question about critical thinking, we cannot draw conclusions about whether the case improved students’ critical- thinking skills. However, we note that of all post-case questions asked, students responded with the strongest agreement that the case required them to use critical-thinking skills.

Using Visualization Software in the Audit of Revenue Transactions to Identify Anomalies 45

Issues in Accounting Education Volume 33, Number 4, 2018

TEACHING NOTES AND STUDENT VERSION OF THE CASE

Teaching Notes and the Student Version of the Case are available only to non-student-member subscribers to Issues in Accounting Education through the American Accounting Association’s electronic publications system at http://www.aaapubs. org/. Non-student-member subscribers should use their usernames and passwords for entry into the system where the Teaching

Notes can be reviewed and printed. The ‘‘Student Version of the Case’’ is available as a supplemental file that is posted with the Teaching Notes. Please do not make the Teaching Notes available to students or post them on websites.

If you are a non-student-member of AAA with a subscription to Issues in Accounting Education and have any trouble accessing this material, please contact the AAA headquarters office at [email protected] or (941) 921–7747.

REFERENCES

Arrowsmith, R. 2017. BlackLine updates software, partners with KPMG. AccountingToday.com (November 28). Available at: https:// www.accountingtoday.com/news/blackline-updates-software-partners-with-kpmg

Association to Advance Collegiate Schools of Business (AACSB). 2017. Exposure Draft #1: Eligibility Procedures and Accreditation Standards for Accounting Accreditation (November 6). Available at: http://www.aacsb.edu/-/media/aacsb/docs/accreditation/ standards/accounting-accreditation-standards-exposure-draft-1.ashx

Cao, M., R. Chychyla, and T. Stewart. 2015. Big Data analytics in financial statement audits. Accounting Horizons 29 (2): 423–429. https://doi.org/10.2308/acch-51068

Christensen, B., R. Elder, and S. Glover. 2015. Behind the numbers: Insights into large audit firm sampling policies. Accounting Horizons 29 (1): 61–81. https://doi.org/10.2308/acch-50921

Cohn, M. 2017. Audit technology evolving quickly at Big Four. AccountingToday.com (November 28). Available at: https://www. accountingtoday.com/news/audit-technology-evolving-quickly-at-big-four-firms

Enget, K., G. Saucedo, and N. Wright. 2017. Mystery, Inc.: A Big Data case. Journal of Accounting Education 38: 9–22. https://doi.org/ 10.1016/j.jaccedu.2016.12.003

Fay, R., and E. Negangard. 2017. Manual journal entry testing: Data analytics and the risk of fraud. Journal of Accounting Education 38: 37–49. https://doi.org/10.1016/j.jaccedu.2016.12.004

Institute of Chartered Accountants in England and Wales (ICAEW). 2016. Data analytics for external auditors. International Accounting, Auditing & Ethics: Available at: https://www.icaew.com/en/international-accounting-and-auditing/international-auditing- perspectives/data-analytics-for-external-auditors

Janvrin, D., and M. W. Watson. 2017. ‘‘Big Data’’: A new twist to accounting. Journal of Accounting Education 38: 3–8. https://doi.org/ 10.1016/j.jaccedu.2016.12.009

Kokina, J., D. Pachamanova, and A. Corbett. 2017. The role of data visualization and analytics in performance management: Guiding

entrepreneurial growth decisions. Journal of Accounting Education 38: 50–62. https://doi.org/10.1016/j.jaccedu.2016.12.005 Marr, B. 2017. The 7 best data visualization tools in 2017. Forbes.com (July 20). Available at https://www.forbes.com/sites/bernardmarr/

2017/07/20/the-7-best-data-visualization-tools-in-2017

Pathways Commission. 2012. Charting a National Strategy for the Next Generation of Accountants. Available at: http://commons.aaahq. org/posts/a3470e7ffa

PricewaterhouseCoopers (PwC). 2015. Data Driven: What Students Need to Succeed in a Rapidly Changing Business World. New York, NY: PwC.

Public Company Accounting Oversight Board (PCAOB). 2017. Standard-Setting Update. (September 30). Washington DC: PCAOB. Available at: https://pcaobus.org/Standards/Documents/Q42017-standard-setting-update.pdf

Raphael, J. 2017. Rethinking the audit. Journal of Accountancy 223 (4): 28–32. Rapoport, M. 2016. Auditing firms count on technology for backup. WSJ.com (March 7). Available at https://www.wsj.com/articles/

auditing-firms-count-on-technology-for-backup-1457398380

Richardson, V. J., C. Chang, and R. Smith. 2018. Accounting Information Systems. 2nd edition. New York, NY: McGraw-Hill Education.

46 Cunningham and Stein

Issues in Accounting Education Volume 33, Number 4, 2018

Course Resources/Project 1 Data Visualization/628 Data Visualization Data File.xlsx