ADV modeling

Eurographics Conference on Visualization (EuroVis) 2015 H. Carr, K.-L. Ma, and G. Santucci (Guest Editors)

Volume 34 (2015), Number 3

An Evaluation of the Impact of Visual Embellishments in Bar Charts

Drew Skau1,4, Lane Harrison2, and Robert Kosara3,4

1Visually Inc. 2Tufts University 3Tableau Research 4UNC Charlotte

A B C 0

100

(a) Bars extend below zero line.

A B C 0

100

(b) Triangle bar chart.

Figure 1: Two examples of embellished charts and abstracted versions of the embellishments.

Abstract As data visualization becomes further intertwined with the field of graphic design and information graphics, small graphical alterations are made to many common chart formats. Despite the growing prevalence of these embellishments, their effects on communication of the charts’ data is unknown. From an overview of the design space, we have outlined some of the common embellishments that are made to bar charts. We have studied the effects of these chart embellishments on the communication of the charts’ data through a series of user studies on Amazon’s Mechanical Turk platform. The results of these studies lead to a better understanding of how each chart type is perceived, and help provide guiding principles for the graphic design of charts.

1. Introduction

In recent years, advertisers, news organizations, non-profits, and even government agencies have begun to use informa- tion graphics to advertise and communicate their messages. These groups use information graphics (commonly called infographics), to convey data or other information to au- diences on the internet or in print publications. Infograph- ics are primarily distributed using blogs and social media, but they are also printed in newspapers or magazines, or embedded in other types of websites. Initially, infographics were very effective at increasing view counts. While they are still exremely effective at driving pageviews, the novelty has worn off and quality of design, as well as having interesting content, has become increasingly important. Data visualiza- tions are often used heavily in these graphics, but they also usually incorporate design elements to help catch viewers’

eyes. In the struggle to rise to the top of the crowd, the data visualizations in infographics often are embellished with ad- ditions and modifications to the raw chart (Figure 2). When these creative changes are made to charts, the general con- sensus is that they can make charts less effective at commu- nicating information. But most of these creative tweaks have never been tested to see how they affect the communication of the data in the charts.

Common wisdom says that many of these embellishments harm the communication of the data by damaging, impeding, or distracting from the pure visual representations of the data (often referred to as chartjunk [TGM83]). For example, bar charts with rounded tops or pointed triangles dont have a horizontal line to help hold the edge of the figure. This may make it harder to see the value represented by the bar, but perhaps there are other ways our visual system is interpret-

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

(a) Rounded ends bar chart.

(b) Capped bar chart.

(c) Overlapping triangles bar chart.

(d) Quadratically increasing area bar chart.

Figure 2: A sampling of charts used in infographics, taken from examples found on Visual.ly [Vis14b].

ing the features of a bar chart. Maybe some changes could even improve our ability to perceive the data in the charts. Our work measures the impact of a set of common embel- lishments on the communication of the data in the chart. The results suggest that some embellishments are better than oth- ers, and allow us to offer advice on which embellishments are appropriate to use in which situations.

This paper discusses related work and gives an overview of the embellishment design space before delving into the study materials, procedure, and results. We conclude with a discussion of the implications of our findings, some advice for designers, and a summary of the research.

2. Related Work

Tools like Datavisual [Woo14], RAW [Des14], In- foactive [Chi14], Visage [Med14], Plotly [Plo14], or Lyra [SH14] make it possible for non-technical designers to build visualizations. These tools often allow the user to ex- port files that are then easy to edit further in design programs like Adobe Illustrator.

Despite their increasing prevalence in infographics, chart embellishments have typically been viewed as decreasing a chart’s ability to communicate the data involved. Edward Tufte has been one of the major drivers behind this point of view, coining the term “chart junk” to describe embel-

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

lishments, and calling them non-data-ink or redundant data- ink [TGM83]. Despite a general lack of empirical evidence supporting this negative view on chart embellishments, it has driven a culture of design simplification and sterilization in the data visualization community. Tufte’s writings encour- age this approach of reducing the data/ink ratio. This may be a good rule of thumb, but it fails to address the real salient features of a chart. By this measure, bars in a bar chart would be better if they were reduced to a single-pixel wide line. There is a balance to be struck between simplicity and prac- ticality, and work by others has begun to investigate this. The work on memorability by Borkin et al. [BVB∗13] and Borgo et al. [BARM∗12] is beginning to decompose capabilities of charts to achieve different tasks with a focus on memorabil- ity and concept comprehension. However, being memorable and comprehensible are not the only goals for a chart; it is also important to communicate data accurately.

In the midst of the controversy, websites like Visual.ly [Vis14b], Visualizing.org [Vis14a], Da- daviz.com [dad14], or ILoveCharts [ILo14] have brought data visualizations merged with design into the public spotlight. They are proof of the increasing occurrences of design merged with data visualization, and their showcases are rife with embellished charts. This points to a stark divide between what is done in practice, and what is encouraged by the theoretical side of the community.

It is clear that an evaluation of embellished charts would provide tangible proof of the impact of designers’ creativity. There are well established ways to evaluate charts. Cleve- land and McGill performed seminal work in 1984 with their studies exploring differences between bar and pie charts for different tasks [CM84]. Specifically, the bar chart portion of their study selects a confounding factor (distance between bars) and examines how that factor has an impact on the chart’s performance for a given task (comparison between bars). They used the results of this to determine best prac- tices when selecting which chart type to use for a given task. Their study has been replicated by Heer et al. using Ama- zon’s Mechanical Turk [HB10], an internet based platform for sourcing work for any scale of human intelligence tasks. Other groups have proven the platform is good for studies on perception [KZ10] and visual decisions [KLcKY11]. The scalability, price, and relatively automated process that Me- chanical Turk provides make it an attractive platform for run- ning browser-based user studies.

The work that comes the closest to evaluating the com- munication abilities of embellished charts mostly addresses memorability. Research from Bateman et al. have sug- gested that people can still interpret embellished charts accu- rately [BMG∗10], and that embellishments may actually im- prove memorability [BVB∗13]. Other research suggests that charts embellished with semantically meaningful objects can have an impact on working memory, long term memory, vi- sual search, and concept grasping [BARM∗12]. This study,

however, also focuses on memorability, not on accuracy of communication.

There is a body of work that specifically investigates the perception of bar charts. Talbot et al. have taken Cleve- land and McGill’s study and delved deeper into the ques- tions of how different bar chart configurations impact accu- racy [TSA14].

There has been another study that looked at the impact of similar design changes on perception accuracy. Zacks et al. compared three versions of charts, a single thin line for each bar, a thicker bar, or a projection of a 3D bar [ZLTS98]. They found that while 3D perspective depth cues lowered ac- curacy, distortions from neighboring elements were a more damaging source of inaccuracies. Correll et al. explored the effectiveness of alternate error bar designs on communica- tion of confidence intervals on bar charts [CG14]. Newman et al. have shown that predictions of averages across a bar chart always end up weighted lower than they should be as a result of the alignment of the bars to the bottom end of the scale [NS12]. Elzer et al. constructed a model of percep- tual task effort aimed at improving communication of the message a bar chart is intended to convey [EGCH06]. Their work shows that having an understanding of the perceptual effects going on in a bar chart is critical to making it com- municate clearly. All of this work points to an incomplete understanding of all of the perceptual issues involved in bar charts.

3. Design Space Overview

The first step to evaluating chart embellishments is defining what they are. We define embellishments as deviations from a baseline chart. In the case of a bar chart, deviations include changes to the shape of the bars, added components, or bars with an altered set of data encodings.

The second step is classifying the different types of em- bellishments. There is a wide range of chart embellishment styles and types, and different embellishments likely have different impacts on the chart. For example, a curved or pointed end to a bar means there is no strong line at the end for the viewer to extend to the value axis. At the same time, a triangular bar has the angle of the edge of the triangle that also communicates the data. Bars with quadratically increas- ing area, or bars that overlap mean the bar’s area does not accurately encode the data. Overlapping bars share charac- teristics with Euler diagrams and could lead to a general mis- interpretation. Bars that extend below the zero point to allow labeling may make comparisons between bars more difficult as the overall height is not proportional to the data. Caps on bars can introduce ambiguity in what signifies the top of the bar (the middle of the cap? the bottom edge? the top edge?). Depending on the qualities of ambiguity that these embellishments may introduce, it could even be possible to use them to represent uncertainty, similar to work by Das-

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

gupta et al. on displaying visual uncertainty in parallel coor- dinates [DCK12]. These possibilities provide motivation for studying the effect of embellishments on graphical percep- tion.

Through an overview of infographics found on Vi- sual.ly [Vis14b], we have identified a set of embellishments that occur frequently. This set serves as a starting point for establishing the impact of embellishments on graphical per- ception.

1. Rounded corners charts (Figure 2a) do not have a strong line at the end for the viewer to mentally extend to the value axis. Sometimes these are rounded due to being a portion of an illustration, or as a part of a pseudo-3D ef- fect.

2. Triangle charts (Figure 1b) have the same issues as the rounded corners charts, however they also lack any ver- tical edges to help judge height. These also add a data encoding, as the angle of the end point changes based on the height (albeit, not with linear proportionality).

3. Capped bars (Figure 2b) come in many forms, but in all cases, they have a wider end, or an end with a stronger color contrast. The change in the visual weight of the bars is the primary change from the baseline chart.

4. Overlapping triangle charts (Figure 2c) generally have some level of transparency so the overlapping regions are visible. The overlapping technique used with these is used almost exclusively with triangular bar charts for some reason. This is likely because of the added visual complexity that comes from overlapping angles rather than overlapping rectangular regions.

5. Quadratically increasing charts (Figure 2d) have shapes ranging from simple rectangles or triangles to an illus- trated figure like the Shrinking Family Doctor [TGM83]. Still, the most common version of these is triangles.

6. Bars that begin before the origin point (Figure 1a) are of- ten illustrations of real world objects, or the bar labels are an extension of the bar itself. This embellishment makes the bar read as being larger than it actually is, possibly causing the value the bar represents to be misinterpreted.

This set of embellishments was identified as worth explor- ing for two reasons: they occur frequently in infographics, and they can be created using standard tools. We conducted an informal survey on the infographics website Visual.ly in November 2014 to pick common embellishments to use in this study.

In addition, all chosen embellishments have some impact on the salient features of the chart in a way that may change our ability to interpret the data. Some of them are isolated versions of a larger set of chart modifications that are done for a certain effect. Some of them are abstracted versions of embellishments that occur frequently in different forms. For example, the capped bar embellishments include a variety of different caps, however the version we chose is abstracted to test the general principle of having more visual weight

at the top of the bar than at the bottom. Triangular charts are extremely prevalent in infographics, at least partly due to easy creation with tools like Microsoft Excel or Piktochart. Rounded bar ends are also common in infographics, some- times as a part of other effects like pseudo 3D cylinders, or often just as rounded ends.

4. Study

The primary goal that motivates measuring the performance of visual communication of charts is communication accu- racy. This is the foundational first step that validates whether any secondary goals like communication speed or informa- tion retention can even be achieved. To realize this goal, we adapt the experiment design from Cleveland and McGill [CM84] and Heer and Bostock’s replication of their experi- ment [HB10], making necessary changes to study the impact of embellishments on communication accuracy.

This study relies on comparisons to a baseline chart style. This chart style uses a grayscale color theme, a clear to read and familiar font (Helvetica), and standardized labels and axes. An example of the baseline style for a bar chart can be found in Figure 3g. Based on our Design Space Overview, Table 1 shows the details of our hypotheses for each bar chart embellishment’s performance as compared to the base- line. Overall, we expected most embellishments to perform roughly equivalent to or worse than the baseline, with a few exceptions for the bars with end caps. We reasoned that the end of the bar farthest from the zero line is the most impor- tant visual portion. Most of the embellishments either dam- age an encoding, or reduce the prominence of the end of the bar farthest from the zero line. The end caps make that por- tion of a bar more pronounced, so it is possible they will improve the communication of the chart’s data.

4.1. Materials

Rather than testing the “in the wild” versions of these charts that often use bright colors, fancy fonts, and all manner of labeling schemes, we recreated versions of the charts that deviate from the baseline in a controlled manner, and by only one criterion at a time (Figure 3). All charts use the same font and labeling technique as the baseline chart (Figure 3g).

The charts display three values each, labeled A, B, and C for reference in the questions. We constrain the bars to three to contain the scope of the study to issues caused directly by the embellishments, avoiding compound issues that may appear when two compared bars have many bars between them (as found by Cleveland and McGill [CM84] and Tal- bot et al. [TSA14]). The minimum value on the y-axis al- ways starts at 0, while the maximum y-axis value is capped at 100. To prevent overly simple judgements where the bars line up with the bottom or top of the y axis, the bar values are generated randomly in the range from 3 to 97.

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

A B C 0

100

(a) Rounded tops bar chart.

A B C 0

100

(b) Triangle bar chart.

A B C 0

100

(c) Capped bar chart.

A B C 0

100

(d) Overlapping triangles bar chart.

A B C 0

100

(e) Quadratically increasing area bar chart.

A B C 0

100

(f) Bars extend below zero line.

A B C 0

100

(g) Baseline bar chart.

Figure 3: Embellished bar charts simplified for use in the study.

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

Absolute Judgement Relative Comparison Modification Figure Hypothesis Result Hypothesis Result Rounded tops 3a equal worse equal worse* Triangles 3b worse worse worse worse* End caps 3c better equal better worse* Overlapping triangles 3d worse worse equal worse* Disproportionate area 3e worse worse* equal worse* Extend below zero 3f worse equal worse equal

Table 1: Hypotheses for each chart embellishment’s performance as compared to the baseline chart (Figure 3g). Results are based on differences in average log-error, with statistically significant differences indicated by * with α = 0.0083.

4.2. Procedure

The study began with an introduction page and a short de- mographic form. This was followed by a page discussing the procedure for answering questions and a block of one type of question (Figure 4). At the end of the first block of ques- tions, there was a short break page reminding the user of the procedure for answering questions and a block of a second type of question. The order of the two blocks of question types was randomized to ensure that there were no learning effects from one question type that would influence the re- sults of the other. The first embellishment type in each block was rotated per participant, also to ensure that there were no learning effects from one embellishment type that would influence the results of the others. The order of the embel- lishment types for the subsequent embellishment types was randomized per participant to ensure that transition order be- tween embellishments had no influence on the results.

For some charts, it may not be immediately obvious that they should be read as bar charts. In these cases, seeing the baseline bar chart could tip viewers off to the pattern of them all being bar charts. In the wild, these charts are often en- countered without any instructions or context to suggest they are bar charts (Figure 2). There are several things we did to ensure viewers come to their own conclusions on how to read the charts.

First, there was no specific mention of bar charts, column charts, bars, or columns throughout the study or participant materials leading up to the study. The charts were merely referred to as charts, and the bars were referred to by their label.

Second, the order that the variations were presented to the participants was controlled. The first embellishment type a participant saw was cycled per participant ensuring equal coverage per type, using the Latin Square mentioned previ- ously. The remaining rows of the square were shuffled ran- domly to distribute the transitions between chart types.

4.2.1. Question Types

The two different question types were designed to test the main tasks associated with bar charts, comparison between bars, and reading the value of a single bar.

Next

Study progress:

In the chart below, what is the value of C?

20

A B C0

100

Figure 4: An example of what a survey participant saw when answering questions. This screen uses the absolute value question and the baseline bar chart. The participant has en- tered “20” into the text entry.

a) In the chart below, what is the value of A? b) In the chart below, what percentage is B of A?

These questions do not address a user’s comprehension of what the chart communicates. We have intentionally left off units and only assigned values to the charts, so that we can focus only on the accuracy of communication and the perception of the charts. It is possible that these embellish- ments could impact higher level comprehension based tasks, however that research is beyond the scope of this study.

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

A B C

Figure 5: Chart used in the relative-question condition did not include a y-axis, to ensure that participants compared the two bars based on their perceived differences rather than numerical estimation.

The questions asking about the absolute value of a single bar (question type a) are simpler to answer, so participants were shown five of these questions for each of the seven em- bellishment types for a total of thirty-five questions in this section. The questions asking the participant to make a com- parison between two bars on the chart (question type b) are more difficult to answer and have more permutations of bar position along the x-axis, so participants were shown eight of these questions for each of the seven embellishment types for a total of 56. In addition, the y-axis is not necessary for this task, and could actually allow the participants to “cheat” by first judging the absolute height of each bar and mathemat- ically computing the percentage. To prevent this, the y-axis was removed from the charts for these questions (Figure 5).

4.3. Results

We recruited 100 participants through Amazon’s Mechani- cal Turk for this experiment. Of the original 100 HITs, three were rejected and re-run by different participants because over one quarter of their answers were incorrect by more than 30%, indicating they were not paying attention or did not fully understand the questions. This resulted in a total of 103 participants with 100 being paid US $2.00 for their participation. It took approximately two days to gather all responses. Average completion time for all participants was 19 minutes and 11 seconds. Our study used a total of 6 em- bellishments (and 1 control, the baseline), and two ques- tion types (absolute/relative), yielding 7 main conditions and 2 sections. Each participant answered questions using all seven chart types, resulting in 35 absolute and 56 relative judgements per participant.

Because our experiment is adapted from previous stud- ies in graphical perception, we follow previous methods for computing errors and confidence intervals [CM84, HB10]. Error was computed using the midmean of log-absolute er- ror (MLAE), and 95% confidence intervals via bootstrap- ping [CM84]. To mititgate the effect of outliers, 6 partic- ipants were removed because their average error exceeded

1.4

1.6

1.8

baseline capped overlapping quadratic rounded triangle extended chart.embellishment

lo g.

er ro

r

Figure 6: Log error for the absolute value task by embellish- ment type.

Table 2: Absolute judgements: summary statistics and Mann-Whitney-Wilcoxon Tests Results comparing embel- lishment types to the baseline. Significant values denoted by * with α = 0.0083.

Embellishment Mean SD p-value baseline 1.41 1.85 capped 1.41 1.68 0.50 overlapping 1.64 1.76 0.048 quadratic 1.70 1.78 0.007∗

rounded 1.67 1.77 0.009 triangle 1.70 1.68 0.012 extended 1.45 1.68 0.562

172% (maximum error for the remaining 94 participants was 45%). This result is in line with Heer and Bostock’s crowd- sourced graphical perception experiments [HB10], and Ma- son and Suri’s study examining the data quality on Mechan- ical Turk [MS12].

Consistent with the previous studies mentioned above, the resulting errors in each question type and embellish- ment group were non-normally distributed. To test the ef- fect of embellishments against the baseline, we compared the error of each to the baseline condition through six Mann- Whitney-Wilcoxon tests. We use a Bonferonni correction to address the problem of multiple comparisons, resulting in an α = 0.0083 required for rejecting the null hypothesis. All tests results and parameters are reported in Tables 2 and 3. To aid visual comparisons, we provide plots of the means and their 95% confidence intervals in Figures 6 and 7.

4.4. Absolute Judgements

For absolute judgments, which involved estimating the value of a given bar, only quadratic bars (m = 1.70, sd = 1.78) per- formed significantly worse than the baseline (m = 1.41, sd = 1.85). One possible explanation is because the area change

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

1.50

1.75

2.00

2.25

2.50

baseline capped overlapping quadratic rounded triangle extended chart.embellishment

lo g.

er ro

r

Figure 7: Log error for relative judgments (comparing two bars) by embellishment type.

Table 3: Relative judgements: summary statistics and Mann- Whitney-Wilcoxon Tests Results comparing embellishment types to the baseline. Significant values are denoted by * with α = 0.0083.

Embellishment Mean SD p-value baseline 1.43 1.85 capped 1.70 1.68 0.0013∗

overlapping 1.82 1.76 < 0.001∗

quadratic 2.33 1.78 < 0.001∗

rounded 1.86 1.77 < 0.001∗

triangle 1.85 1.68 < 0.001∗

extended 1.59 1.66 0.097

in the quadratic bars exaggerates the overall size change and may make it harder to estimate the height of the bar against the value axis. Other embellishments performed sim- ilarly to quadratic in terms of error, including overlapping (m = 1.64, sd = 1.76), rounded (m = 1.67, sd = 1.77), and triangle (m = 1.70, sd = 1.68). Notably, both extended (m = 1.45, sd = 1.68) and capped (m = 1.41, sd = 1.68) are simi- lar to the baseline.

4.5. Relative Judgements

The trends in the relative judgement data were strikingly dif- ferent than the absolute judgement data. Relative judgments involved estimating the percentage of a given bar to another, and only the extended embellishments (m = 1.59, sd = 1.66) performed similarly to the baseline (m = 1.43, sd = 1.85). In all other cases the baseline condition performed signif- icantly better. Specifically, capped (m = 1.70, sd = 1.68), overlapping (m = 1.82, sd = 1.76), rounded (m = 1.86, sd = 1.77), and triangle (m = 1.85, sd = 1.68) performed simi- larly, while quadratic produced much higher error overall (m = 2.33, sd = 1.78).

A B C 0

100

Figure 8: The possible mental operation viewers use to tell the value of an individual bar depends heavily on the shape of the top of the bar and the axis.

5. Discussion

The results of this experiment confirm that common em- bellishments can significantly impact the perceptual perfor- mance of bar charts, and that these impacts differ substan- tially based on the task (i.e., absolute versus relative judge- ments).

Specifically, none of the embellishments tested in this ex- periment performed better at communication of the data than the baseline standardized chart. One notable exception is the capped bar chart, a bar chart with an additional “cap” on top which is wider than the bottom portion of the bar. These performed equally well as, and with slightly lower variance than, the baseline chart for absolute judgement questions. This result suggests that users indeed rely on strong lines at the ends of bars to mentally extend the bar end to the value axis, especially when considering the comparatively poor performance of the embellishments that distort the top of the bar (rounded caps, triangles, etc.) – see Figure 8.

All adaptations except the extended embellishment per- formed significantly worse than the baseline on relative judgements. Even small changes, for example the rounded bar, produced a significantly higher error rate. Confirming the hypothesis of Tufte and others, quadratically-scaled bars led to large errors in relative judgements, even when com- pared to similar embellishment styles (i.e. triangles, over- lapping triangles).

For designers creating charts, this study produces action- able advice. It is advisable to stay away from creating trian- gular bar charts. Triangular charts that overlap and that have quadratically changing areas are especially worth avoiding. While there are no guarantees about shapes other than tri- angles with disporportionately changing areas, these results suggest that it is inadvisable to scale chart elements on two axes simultaneously. This is in line with common wisdom. End caps with a strong horizontal top are not advisable for tasks that involve comparing bars, but are fine (and perhaps better) for absolute judgements. Bars that have a portion ex- tending below the zero point on the value axis seem to be

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

fine to use, assuming the portion that extends is a visibly different color from the value portion of the bar.

The results of this study qualify findings by Borkin [BVB∗13] and Borgo [BARM∗12] suggesting that memorability can be aided by embellishment. Changes to charts that affect the primary chart elements can reduce the communication accuracy of the chart. Increasing the memorability of a chart is certainly a worthwhile pursuit, however it must be balanced with the need to communicate information accurately in the first place.

Several promising areas for future work follow the re- sults of this experiment. Quantifying the impact of embel- lishments on perceptual accuracy establishes a baseline to test the impact of other factors that are at play when design- ing charts. By quantifying the impact of design factors on perception, it is possible to explore and optimize compro- mises that may be struck between accurate communication and high-level design goals.

To increase the reproducability of our study, all study code is included as supplemental material. All materi- als and resulting data have also been made available at https://github.com/dwskau/chart-embellishment.

6. Conclusions

In this paper we present a crowdsourced experiment to in- vestigate the impact of common chart embellishments on the accuracy of absolute and relative judgements in bar charts. The results of this experiment establish that bar chart em- bellishments do indeed have an impact on how well the data within the chart can be communicated. For nearly all tested chart embellishments, even small changes like rounding the top of a bar, led to higher error rate. However, there was one notable exception, the T-shaped capped bar chart.

These results advance our understanding of the intricacies of how we use the visual cues in bar charts. They identify which visual cues in bar charts have the most impact, and establish a basis for exploring the impact of low-level design elements in graphical perception.

References [BARM∗12] BORGO R., ABDUL-RAHMAN A., MOHAMED F.,

GRANT P. W., REPPA I., FLORIDI L., CHEN M.: An empiri- cal study on using visual embellishments in visualization. IEEE Transactions on Visualization and Computer Graphics 18, 12 (2012), 2759–2768. 3, 9

[BMG∗10] BATEMAN S., MANDRYK R. L., GUTWIN C., GEN- EST A., MCDINE D., BROOKS C.: Useful junk?: the effects of visual embellishment on comprehension and memorability of charts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2010), ACM, pp. 2573–2582. 3

[BVB∗13] BORKIN M. M. A., VO A. A. A., BYLINSKII Z., ISOLA P., SUNKAVALLI S., OLIVA A., PFISTER H.: What makes a visualization memorable. IEEE Transactions on Visu- alization and Computer Graphics 19, 12 (2013), 2306–2315. 3, 9

[CG14] CORRELL M., GLEICHER M.: Error Bars Considered Harmful: Exploring Alternate Encodings for Mean and Error. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2142–2151. 3

[Chi14] CHIASSON T.: Infoactive, 2014. [Online; accessed 19- August-2014]. URL: https://infoactive.co/. 2

[CM84] CLEVELAND W. S., MCGILL R.: Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Associa- tion 79, 387 (1984), 531–554. 3, 4, 7

[dad14] DADAVIZ: Data nerds + art lovers = dadaviz, 2014. [On- line; accessed 19-August-2014]. URL: http://dadaviz. com/. 3

[DCK12] DASGUPTA A., CHEN M., KOSARA R.: Conceptu- alizing visual uncertainty in parallel coordinates. In Computer Graphics Forum (2012), vol. 31, Wiley Online Library, pp. 1015– 1024. 4

[Des14] DESIGN D.: Raw, 2014. [Online; accessed 19-August- 2014]. URL: http://raw.densitydesign.org/. 2

[EGCH06] ELZER S., GREEN N., CARBERRY S., HOFFMAN J.: A model of perceptual task effort for bar charts and its role in recognizing intention. User Modelling and User-Adapted Inter- action 16, 1 (2006), 1–30. 3

[HB10] HEER J., BOSTOCK M.: Crowdsourcing graphical per- ception: using Mechanical Turk to assess visualisation design. ACM Human Factors in Computing Systems (2010), 203–212. 3, 4, 7

[ILo14] ILOVECHARTS: I love charts, 2014. [Online; accessed 19-August-2014]. URL: http://ilovecharts.tumblr. com/. 3

[KLcKY11] KIM S., LI S., CHUL KWON B., YI J.: Investigat- ing the Efficacy of Crowdsourcing on Evaluating Visual Decision Supporting System. Proceedings of the Human Factors and Er- gonomics Society 55, 1 (2011), 1090–1094. 3

[KZ10] KOSARA R., ZIEMKIEWICZ C.: Do Mechanical Turks dream of square pie charts? Proceedings of the 3rd BELIV’10 Workshop: BEyond time and errors: novel evaLuation methods for Information Visualization (2010), 63–70. 3

[Med14] MEDIA C. F.: Visage - beautiful, branded content, 2014. [Online; accessed 19-August-2014]. URL: http://visage. co/. 2

[MS12] MASON W., SURI S.: Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods 44, 1 (2012), 1–23. 7

[NS12] NEWMAN G. E., SCHOLL B. B. J.: Bar graphs depict- ing averages are perceptually misinterpreted: The within-the-bar bias. Psychonomic bulletin & review 19, 4 (2012), 601–607. 3

[Plo14] PLOTLY: Plotly, 2014. [Online; accessed 19-August- 2014]. URL: https://plot.ly/. 2

[SH14] SATYANARAYAN A., HEER J.: Lyra: An Interactive Vi- sualization Design Environment. Computer Graphics Forum 33, 3 (2014), 351–360. 2

[TGM83] TUFTE E., GRAVES-MORRIS P.: The visual display of quantitative information. CT Graphics, Cheshire, CT, 1983. 1, 3, 4

[TSA14] TALBOT J., SETLUR V., ANAND A.: Four Experiments on the Perception of Bar Charts. IEEE Transactions on Visual- ization and Computer Graphics 20, 12 (2014), 2152–2160. 3, 4

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.

Skau, Harrison, Kosara / An Evaluation of the Impact of Visual Embellishments in Bar Charts

[Vis14a] VISUALIZING.ORG: visualizing.org | data visualiza- tions, challenges, community, 2014. [Online; accessed 19- August-2014]. URL: http://visualizing.org/. 3

[Vis14b] VISUALLY: Original content marketing for brands | vi- sual.ly, 2014. [Online; accessed 19-August-2014]. URL: http: //visual.ly/. 2, 3, 4

[Woo14] WOO D.: Datavisual, 2014. [Online; accessed 19- August-2014]. URL: http://datavisu.al/. 2

[ZLTS98] ZACKS J., LEVY E., TVERSKY B., SCHIANO D. J.: Reading bar graphs: Effects of extraneous depth cues and graph- ical context. Journal of Experimental Psychology: Applied 4, 2 (1998), 119. 3

c© 2015 The Author(s) Computer Graphics Forum c© 2015 The Eurographics Association and John Wiley & Sons Ltd.