research

R E S E A R CH A R T I C L E

The effect of source credibility on bullshit receptivity

Sandra Ili�c | Kaja Damnjanovi�c

Laboratory for Experimental Psychology,

Faculty of Philosophy, University of Belgrade,

Belgrade, Serbia

Correspondence

Sandra Ili�c, Laboratory for Experimental

Psychology, Faculty of Philosophy, University

of Belgrade, Cika Ljubina 18-20, Belgrade

11000, Serbia.

Email: [email protected]

Summary

Pseudo-profound bullshit pertains to grammatically and syntactically correct but

meaningless sentences, that, due to syntactical correctness appear as made to com-

municate something and research shows that people deem them profound. However,

the effect of differing source credibility on bullshit profoundness evaluations has, to

our knowledge, not yet been tested. We presented participants with pseudo-

profound bullshit alone and with authors of different credibility. In order to partly

replicate and extend on the findings regarding mechanisms of receptivity and sensi-

tivity to bullshit we collected profoundness evaluations for mundane statements and

proverbs, and different measures of analytic thinking. Ascribing credible authors leads

to an increase while ascribing uncredible authors leads to a decrease in profoundness

evaluations. Cognitive reflection protects against the tendency to evaluate any type

of statement as profound and drives better differentiation between pseudo- and con-

ventionally truly profound, while positive views about actively open-minded thinking

enable stronger effects of uncredible authorship.

K E YWORD S

context effect, dual process, pseudo-profound bullshit, source credibility, thinking style

1 | INTRODUCTION

“When you vibrate no need, the Universe senses your vibration of

having what you need. And that is when it will come to you. This prin-

ciple applies to all other areas of your life, as well. [...] When you real-

ize that everything is inside you, that EVERYTHING IS INSIDE YOU

(sic), that's when the release will happen” (Bučevi�c, 2015, p. 77). The excerpt is taken from a book “In the Vortex of Fulfilled

Wishes” written by Ana Bučevi�c (2015), a kinesiologist, motivational

speaker and an author of numerous self-help books, recently accused

of quackery by the Croatian Ministry of Health (“Šok za Anu

Bučevi�c,” 2019). The book, categorized as popular psychology genre,

was aimed at providing readers with better understanding of “the law

of attraction.”1 Nevertheless, the content of the book wasn't based

on the facts, scientific theories or empirical research. On the account

of the author's local popularity her YouTube channel Spirit Safari at

this point in time has 252,000 subscribers and contains videos which

were viewed nearly 130 million times (Safari duha, Bučevi�c, n.d.),

while the aforementioned book that was first published in 2015 had

its seventh edition printed as soon as the following year. Bučevi�c's

books however, although numerous and locally very popular, make up

for only a small part of an ever-growing new age self-help and alterna-

tive medicine book sections of bookstores filled with works from

authors such as Deepak Chopra, Rhonda Byrne, and so on, that are

usually classified as popular psychology.

With increase in popularity of the new age literature rose philoso-

phers' interest in such discourse. In 1986 Harry Frankfurt wrote an

essay “On bullshit” in which he labeled statements created with the

aim to impress, but without concern for their truth value as bullshit

(Frankfurt, 1986). Whereas lies are by nature untrue and as such

require knowledge of the truth from which one deliberately deviates,

truth is completely irrelevant for bullshit (Frankfurt, 1986). Consider

the following example: “(…) the meaning of life is to give the life a

meaning” (Bučevi�c, 2015, p. 37). When one proposes such inference,

according to Frankfurt, “what we are not to understand is that his (sic)

intention is neither to report the truth nor to conceal it” (Frankfurt, 2005; p. 56). This feature of lacking the truth value has

two important implications: such statements cannot be empirically

Received: 4 March 2021 Revised: 28 May 2021 Accepted: 29 May 2021

DOI: 10.1002/acp.3852

Appl Cognit Psychol. 2021;35:1193–1205. wileyonlinelibrary.com/journal/acp © 2021 John Wiley & Sons Ltd. 1193

tested and therefore cannot be refuted (Hopkin & Rosamond, 2018;

Sarajli�c, 2018) and people tolerate it better than lies, at the same time

remaining confident in their ability to recognize it (Frankfurt, 2005).

However, not only do people tolerate bullshit, but they also do not

recognize it, and even consider it profound (Pennycook, Fugelsang, &

Koehler, 2015; Pennycook & Rand, 2020; Pfattheicher & Schindler,

2016; Sterling et al., 2016).

1.1 | Pseudo-profound bullshit

In psychology, researchers were interested in a special type of

bullshit known as the pseudo-profound bullshit and the characteristics

of people who succumb to it (Pennycook, Fugelsang, &

Koehler, 2015). Pseudo-profound bullshit (in further text—bullshit)

pertains to a grammatically and syntactically correct sentence con-

sisting of randomly picked words from a database (e.g., “Perceptive reality transcends subtle truth”). Such sentences, due to their syntacti-

cal and grammatical correctness, inherently appear as they were made

with the purpose of communicating something. However, such sen-

tences only imply but do not contain meaning, truth, nor the concern

for verisimilitude (Pennycook, Fugelsang, & Koehler, 2015). One of

the main characteristics of such sentences is their vagueness that indi-

cates they “may have been constructed to impress upon the reader

some sense of profundity at the expense of a clear exposition of

meaning or truth” (Pennycook, Fugelsang, & Koehler, 2015; p. 550). In

short, bullshit is a statement without discernible meaning that consists

of modern, abstract words, created to impress and not to inform

(Pennycook, Fugelsang, & Koehler, 2015).

1.2 | The mechanisms underlying bullshit receptivity and sensitivity, and the source of the pseudo-profound bullshit

The first empirical study of bullshit was conducted by Pennycook,

Fugelsang, and Koehler (2015) with the aims of: (1) establishing a

measure of bullshit receptivity which pertains to propensity to

evaluate bullshit as profound, that is, as being “of deep meaning; of

great and broadly inclusive significance,” (2) creating a measure of

bullshit sensitivity or the ability to discriminate actually profound

statements, that is, motivational quotations, from bullshit, and

(3) investigating the correlates of these dispositions in order to

tackle the underlying mechanisms which make people susceptible

or spontaneously immune to bullshit. Two such mechanisms have

been proposed, both nested in the paradigm of dual process of rea-

soning (Pennycook, Fugelsang, & Koehler, 2015). The first one,

which might explain receptivity to bullshit is related to response

bias and pertains to the possibility that some people are simply

more prone to approaching any content as true and meaningful,

while other people have higher criteria in terms of expectations

regarding these attributes. Such bias in an empirical sense refers to

the tendency to rate mundane statements such as “winter days can

be very cold” as somewhat profound, and to positive correlations

between evaluations of profoundness of such statements and bull-

shit, and between profundity rating of bullshit and epistemically

suspect beliefs. This mechanism pertains to one's mindset when

approaching content (Pennycook, Fugelsang, & Koehler, 2015) and

cannot explain bullshit sensitivity.

The second mechanism refers to the possibility that people who

rate bullshit as profound do so because they are not capable of

detecting it as bullshit and discerning it from non-bullshit content

which leads to failure to distinguish the vagueness of a statement

from profoundness. In short, some people are simply unaware of the

fact that a stimulus requires additional consideration which makes

them insensitive to bullshit. In an empirical sense this pertains to posi-

tive correlation between bullshit sensitivity and measures of analytic

thinking (Pennycook, Fugelsang, & Koehler, 2015). This mechanism is

related to conflict detection failure which is one of the factors that

causes willing engagement in analytical reasoning processes (Pennycook,

Fugelsang, & Koehler, 2015) and can be applied to both receptivity and

sensitivity.

In addition to these mechanisms, and although the authors con-

sider their study to be ecologically valid as the experience of bullshit

in an experiment is similar to real life because in both situations there

aren't any signs indicating that the presented content is actually bull-

shit, it is also possible, as they state, that the skepticism with which

people approach bullshit in real life is in fact absent “in laboratory” because participants expect that sentences they are presented with in

research have been, by nature, designed to convey some meaning

(Pennycook, Fugelsang, & Koehler, 2015). In other words, high credi-

bility of the source of bullshit, that is, researcher's high level of exper-

tise and trustworthiness (Pornpitakpan, 2004) regarding everything

experiment related, including the presented content, might be what

drove evaluations of bullshit profoundness. Distant empirical support

for this hypothesis comes from various fields of psychological

research. For instance, a preference for wine is influenced by its price

level (Plassmann et al., 2008), preference for a violin is influenced by

the information about who made it (Fritz et al., 2012), even an evalua-

tion of the quality of poetry depends on the reputation of the attrib-

uted author (Bar-Hillel et al., 2012), and so on. Recently, very similar

context effects were registered with bullshit. When attributed to an

author who is famous, or implemented in a vignette or an authored

excerpt from a book, bullshit is deemed more profound than when

presented isolated (Gligori�c & Vilotijevi�c, 2019). The authors call this

effect of varying bullshit context “the labeling effect” (Gligori�c &

Vilotijevi�c, 2019). Although this finding gives very valuable insight into

the fact that the context in which people encounter bullshit influences

their judgment about bullshit, it does not provide information about

what is it about the ascribed authors that drives the changes in pro-

foundness evaluations. As explained above in reference to context of

psychological research, one potential characteristic of sources, that

might have propelled the registered effect, could be their high credi-

bility. Source credibility is defined by two dimensions: expertise which

pertains to a degree to which a source is deemed capable of making

correct assertions, and trustworthiness—the extent to which audience

1194 ILI�C AND DAMNJANOVI�C

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

perceives that those assertions are considered valid by the source

(Hovland et al., 1953; Pornpitakpan, 2004). Research shows that

recipients' beliefs, attitudes, judgments, and behaviors are influenced

by source credibility (for detailed review see Pornpitakpan, 2004). For

instance, commercials associated with a source of high credibility are

rated as more truthful and believable, compared to commercials asso-

ciated with low credibility source (Moore, Hausknecht, & Thamodaran,

1986). So, extending on the findings about labeling, that is, context

effects, and extant research about effects of source credibility, we

tested potential effects of sources' credibility and its' varying levels on

profoundness evaluations of bullshit, as this has not yet been empiri-

cally examined (Ili�c & Damnjanovi�c, 2021a, 2021b).

1.3 | Aim

The aim of the present study was twofold. The first and the main one

was to investigate if profundity evaluations increase, decrease and

remain the same when a credible, uncredible and fictional authors,

respectively, are ascribed to bullshit, in order to empirically answer

the question if credibility of a source indeed influences perceived pro-

foundness of bullshit and to what degree. The second aim comprised

partial replication and further expansion of findings from the first

study on bullshit carried out by Pennycook, Cheyne, et al. (2015). For

these purposes we conducted two experimental studies, preceded by

a pilot study.

2 | PILOT

The pilot study was conducted in order to translate, test and adapt

existing Bullshit Receptivity Scale and Mundane Statements Scale

(Pennycook, Fugelsang, & Koehler, 2015) and to test a new scale con-

sisting of proverbs which would serve as an alternative to

Pennycook's Motivational Quotations Scale, since the proverbs repre-

sent a traditional and more prototypical example of wisdom in local

cultural context. Also, we aimed to replicate the finding that BRS was

negatively correlated to reflective thinking.

2.1 | Method

2.1.1 | Participants

For calculating required sample size power analysis was used. For

detecting a statistically significant difference between mean evalua-

tions of profoundness of BRS, Mundane statements and Proverbs

scale (at p < .05 level), based on the Cohen's f of 0.263 calculated

based on the results obtained by Pennycook, Fugelsang, and

Koehler (2015), with the power of .95, a sample of 39 subjects is

required. Participants (N = 39) were first year psychology students

whose participation was part of the coursework. Data on gender and

age were not collected.

2.1.2 | Materials

Bullshit receptivity. For measuring bullshit receptivity, we adapted Pen-

nycook et al.'s Bullshit Receptivity Scale (BRS; Pennycook, Fugelsang, &

Koehler, 2015). It consisted of 12 bullshit statements (e.g., “Imagination

is inside exponential space time events”) that were selected from the

original 20. Twelve items were used for two reasons: the number of

items needed to be dividable by three, as the main studies involved a

within subjects factor of the ascribed author with three levels. The sec-

ond reason was that the main studies involved a repeated measures

design as well as evaluation of another two scales of the same length

and any larger number of items dividable by three would result in a

considerably lengthier list of items to be evaluated which would render

participation much longer. We used the 10 items from Pennycook and

colleagues' Study 1 (Pennycook, Fugelsang, & Koehler, 2015), and ran-

domly selected another 2 from the remaining 10 items which were

added in subsequent experiments in the same paper. The items were

translated by authors of the present study only from English to Serbian,

and not vice versa as bullshit by definition has no meaning on which a

consensus should be reached. Respondents rated profoundness of all

statements on a five-point Likert scale (1-not at all profound,

2-somewhat profound, 3-fairly profound, 4-definitely profound, 5-very

profound). In instructions we used the same definition of profoundness

as Pennycook, Cheyne, et al. (2015): “Profound is something ‘of deep meaning; of great and broadly inclusive significance’.” The complete

scale is provided in Table S1.

Mundane statements. Respondents also rated profoundness of

12 mundane statements (e.g., “Winter days can be very cold”). Ten out of 12 items comprising the Mundane Statements Scale were

taken from Pennycook, Cheyne, et al. (2015) and two additional state-

ments were created in order to make a scale with an equal number of

statements as BRS. Profoundness was rated on a five-point Likert

scale (Table S2).

Proverbs. Proverbs Scale served as an equivalent to the Motiva-

tional Quotations Scale used by Pennycook, Cheyne, et al. (2015), and

again consisted of 12 items instead of 10. All items were Serbian folk

proverbs (e.g., “There is no tree without a bough, nor a man without a

flaw”) and were selected from Serbian websites listing different types

of folk wisdom common in Serbia. We chose famous proverbs that

represented metaphors conveying a clear meaning conventionally

considered profound (Table S3).

Bullshit sensitivity. Bullshit sensitivity (BS) was calculated by sub-

tracting the mean evaluations of bullshit from mean evaluations of prov-

erbs. The higher difference indicates greater bullshit sensitivity.

Cognitive reflection. Cognitive reflection test (CRT; Frederick, 2005)

measures the ability or disposition to resist giving the first intuitive

response that comes to mind and to reflect on a question, that is—

engage in analytical thinking. Problems comprising this test cue three

types of responses: correct, atypical wrong and typical wrong or heuris-

tic answers. In the present study we used the extended version of the

test which comprises eight tasks, three of which belong to the original

Frederick's test (Damnjanovi�c et al., 2019; Frederick, 2005; Primi

et al., 2016; Toplak et al., 2014).

ILI�C AND DAMNJANOVI�C 1195

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

2.1.3 | Procedure

The experiment was created in OpenSesame software (Mathôt

et al., 2012) and administered in a classroom. After signing the

informed consent, respondents received both verbal and written

instructions, following which the experiment was initiated. Respon-

dents first evaluated the profundity of statements comprising the

BRS, Mundane Statements Scale and Proverbs Scale, and then

proceeded to solving the CRT. Items and tasks were presented in a

randomized order, and on a separate page to avoid later corrections,

reconsideration of answers and the effect of item context. One month

later participants received both verbal and written explanation of the

procedure and the purpose of the study.

2.2 | Results

Internal consistencies of all three scales were satisfactory: Cronbach's

alphas of BRS, Proverbs Scale and Mundane Statements Scale were

.832, .818 and .791, respectively.

On average, respondents rated bullshit as somewhat to fairly pro-

found (M = 2.30, SD = 0.689): 30.8% of respondents gave mean eval-

uations of bullshit as somewhat profound, 50.3% as fairly profound,

while the remaining 17.9% of respondents evaluated the profundity

of bullshit as equal to or higher than 3 (4—definitely profound). No

respondent rated all 12 bullshit as not at all profound.

In comparison to bullshit, mundane statements were evaluated as

less profound (M = 1.44, SD = 0.540), while proverbs were evaluated

as more profound (M = 3.19, SD = 0.622). Repeated measures ANOVA

revealed that the difference between these evaluations was overall sta-

tistically significant (F[2, 76] = 89.009, p < .001, η2 = 0.701). Post hoc

tests with Bonferroni correction showed that differences in rated pro-

foundness between all three levels of this factor were statistically sig-

nificant. Mean difference between evaluations of profoundness of

bullshit and mundane statements was 0.858 (p < .001), and between

bullshit and proverbs �0.892 (p < .001), respectively.

Number of correct answers on CRT ranged from 0 to 7, and mean

number of correctly solved tasks was 3.08 (SD = 1.925). No respon-

dent solved all eight problems correctly. Distribution of correctly

solved tasks is shown in Table S4.

Pearson's correlations between the bullshit receptivity, sensitivity,

number of correct answers on CRT, mean evaluations of profound-

ness of mundane statements and proverbs were not significant, and

are presented in Table 1.

2.3 | Discussion

Results show that BRS, as well as Proverbs Scale andMundane Statements

Scale have good internal consistencies. Respondents have, as expected and

in accordancewith Pennycook, Fugelsang, and Koehler (2015) results, rated

proverbs as the most profound, bullshit as less profound and mundane

statements as least profound. Bullshit statements were evaluated as some-

what to fairly profound, also in accordance with previous findings

(Pennycook, Fugelsang, &Koehler, 2015).

The finding that cognitively more reflective individuals are less recep-

tive to bullshit has not been replicated. Our results suggest that both

respondents with lower and higher scores on CRT are prone to rate bull-

shit as profound, and the same goes for the sensitivity to bullshit. This can,

however, be a consequence of the small sample size, as well as a conse-

quence of using classical version of CRT which might not be a clean mea-

sure of cognitive reflection, but rather a measure of both cognitive

reflection and numerical ability (Böckenholt, 2012; Campitelli &

Gerrans, 2014; Sinayev & Peters, 2015; Thomson & Oppenheimer, 2016;

Welsh et al., 2013). It is therefore possible that the negative correlation

between BRS and cognitive reflection actually exists, but was not

registered due to the sample size, and/or was obscured by individual

differences in numerical ability. Moreover, it is possible that some

respondents have already had experience with some of the CRT prob-

lems, either on domestic news portals or other websites (e.g., “Najkra�ci

test inteligencije,” 2017), or during participation in some other empirical

studies as CRT is frequently used in psychological research (Haigh,

2016; Stieger & Reips, 2016; Thomson & Oppenheimer, 2016). For

these reasons, in the following experiment we used other measures of

analytic thinking including a non-numerical version of CRT.

3 | STUDY 1

The study was conducted in order to test the effect of the credibility

of the assigned author on bullshit receptivity. In addition, we aimed to

further test the relationship between analytic thinking, and receptivity

and sensitivity to bullshit.

3.1 | Method

3.1.1 | Participants

For detecting a significant difference between evaluations of bullshit

presented without and with authors by an F-test (at p = .01 level),

based on ηp 2 of 0.09, as reported by Gligori�c and Vilotijevi�c (2019),

with the power of 0.99, a sample of 65 participants is required.

TABLE 1 Pearson's correlation coefficients (pilot)

BS MSS PS CRTc

BRS �0.692** 0.189 0.164 �0.154

BS �0.141 0.599** 0.250

MSS 0.017 �0.021

PS 0.171

Abbreviations: BRS, bullshit receptivity scale; BS, bullshit sensitivity; MSS,

mundane statements scale; PS, proverbs scale; CRTc, number correct

answers on CRT.

**p < .001.

1196 ILI�C AND DAMNJANOVI�C

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

Please note that the effect size obtained by Gligori�c and Vilotijevi�c (2019)

is based on between subjects design, and the design employed in this

study is within subjects. For this reason, we aimed for a somewhat larger

sample size than indicated by power analysis. The sample (N = 84) con-

sisted of second-year psychology students. As partaking in experiments

is not a mandatory activity for the second-year students of Psychology,

respondents received extra points for participation. Data on gender and

age was not collected.

3.1.2 | Materials

The same Mundane Statements Scale, Proverbs Scale and BRS were

used as in the Pilot study. Participants also solved a nonnumerical

CRT, 16 categorical syllogisms and were presented with the actively

open-minded thinking beliefs scale. Items comprising only BRS were

presented without and with attributed authors of different levels of

credibility. The authors were not ascribed to either mundane state-

ments or proverbs.

Source credibility: Attributed authors were either credible (famous

philosophers and physicists: Nietzsche, Plato, Hawking, Einstein),

uncredible (famous Hollywood actors: Johnny Depp, Bruce Willis, Mel

Gibson, Brad Pitt), or fictional. The reasons for which we chose to

operationalize credible authorship by selecting legendary physicists and

philosophers, aside from minimizing the possibility of respondents not

having heard of the authors pertains to the characteristics of bullshit,

need for the manipulation to be convincing, and the assumption that

the effect of the authors' credibility comes from their expertise and

trustworthiness on the matter. Namely, bullshit is by definition aimed

at impressing upon the listener a sense of profundity while at the same

time it has no meaning, which is a feature that might be interpreted by

participants as overly abstract and elusive meaning. As philosophy is

conventionally associated with the profound and physics deals with the

abstract, authors from these two fields of science are conventionally

considered experts on such matters and are deemed trustworthy. For

uncredible authors we selected famous Hollywood actors because they

are not considered experts on the philosophical and abstract matters,

but also because it is not unusual for public figures in the entertainment

industry to publicly display their philosophical beliefs. For creating

fictional authors names the random name generator was used.

In order to make the manipulation as convincing as possible, the

authors were attributed in a form of a reference, that is—in brackets,

with names of authors followed by a year, place (in case of public

speeches) and source (letter, book, interview) from which the pur-

ported quote was allegedly taken—for example, “Perceptive reality

transcends subtle truth. (Bruce Willis, The fifth element [press confer-

ence], Canes, 2006)” or “(Albert Einstein, Letter to Phyllis Wright,

1936).” All sources were existent—Albert Einstein did send a letter to

Phyllis Wright in 1936. Fictional authors' names were created by a

random name generator (Random name generator: Behind the

Name, 2019, www.behindthename.com/random/). To ensure that

the names of fictional authors were equal in all surface aspects to

the names of credible and uncredible authors, the criteria used for

generating fictional author's names were as follows: create United

States, United Kingdom, German or Greek male first and last name

(no middle name). Unisex names (e.g., Casey), as well as first and last

names which represented actual words (e.g., Cash) were excluded. In

order to minimize the probability of respondents knowing a person

with fictional authors' name, all first and last names were entered in

the Google search engine. All names and surnames for which the

engine returned results were excluded. Names of the authors from

this category were also shown in brackets, but without a year, place

and source—for example, “Perceptive reality transcends subtle truth.

(Montgomery Brigham).” The other three names of fictional authors

were Daniel Blair, Mikkjal Nikephoros and Millard Royden.

In order for authors of all three levels of credibility to be assigned

to each statement three sets of four BRS items were counterbalanced

against the three groups of authors. An example of BRS with ascribed

authors is provided in the Table S5.

Cognitive reflection. The nonnumerical CRT (nnCRT; Thomson &

Oppenheimer, 2016) is used to measure the same ability or disposi-

tion as CRT, but does not require numerical skills and has been used

in 40 times less studies2 which makes it less possible for participants

to have encountered it. It consists of four open ended problems that

facilitate heuristic and correct answers.

Belief bias. Sixteen categorical syllogisms that differed in terms of

believability and validity of conclusions were used as a measure of belief

bias. This bias represents a tendency to judge the validity of conclusions

based on existent beliefs and knowledge on the matter, rather than their

validity (Evans et al., 1983). For this purpose, all syllogisms were divided

into two categories: congruent—believable valid, not believable invalid;

and incongruent—believable invalid, not believable valid.

Participants' task was to judge if the conclusion logically followed

from the premises. According to the authors of nnCRT, problems used

to register belief bias are a good approximation of tasks measuring

cognitive reflection and thus can be used for measuring this disposi-

tion (Thomson & Oppenheimer, 2016). Examples of categorical syllo-

gisms used in the present study are provided in Table S6.

Analytical cognitive style. Actively open-minded thinking beliefs

scale (AOT; Baron et al., 2015) is designed to measure beliefs about

cognitive style that involve reconsidering existing beliefs in light of

new evidence, spending significant time trying to solve a problem

before giving up and taking into account opinions of others when cre-

ating one's own. Individuals who are more prone to analytical style of

thinking are also less prone to belief bias (Macpherson & Stanovich,

2007; Stanovich & West, 1998). The scale consisted of eight items,

evaluated on a five-point Likert scale ranging from 1-completely dis-

agree to 5-completely agree.

3.1.3 | Procedure

The experiment was created in OpenSesame software (Mathôt

et al., 2012) and administered during two experimental sessions in a

classroom. In both experimental sessions prior to initiating the experi-

ments, each respondent signed an informed consent and was given

ILI�C AND DAMNJANOVI�C 1197

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

both oral and written instructions. In the first session participants

were presented with the 16 categorical syllogisms in a random order

and also a lexical decision task (the results of which will not be pres-

ented here). During the second session participants were presented

with the 36 statements comprising the Proverbs Scale, Mundane

Statements Scale and the BRS, also in a random order, and instructed

to judge the profoundness of each statement. This was followed by

the nnCRT, evaluation of bullshit statements with ascribed authors,

and finally the AOT scale, respectively. Prior to evaluation of bullshit

with authors participants were told that they will be shown some of

the statements they evaluated at the beginning of the experiment, but

this time they will be provided with the names of the authors of those

statements. They were then explicitly instructed not to try and

remember their prior evaluations of the same statements but to evalu-

ate the profoundness of each sentence based on how they feel about

them at the moment of presentation. Each item or task during the

experiment was presented on a separate page and without an option

to go back to avoid later corrections of answers. Two weeks later

respondents were given both verbal and written explanation of the

procedure and the purpose of the study.

3.2 | Results

Receptivity to bullshit. Mean evaluations of profoundness of bullshit

statements were lower (M = 2.40; SD = 0.711) compared to prov-

erbs (M = 3.40; SD = 0.772) and higher compared to mundane

statements (M = 1.31; SD = 0.426). Repeated measures ANOVA

showed that the difference between mean profundity evaluations

of different types of statements was statistically significant (F[1.629,

135.214] = 232.874, p < .001, η2 = 0.737).3 Post hoc tests with

Bonferroni correction showed that differences in reported pro-

foundness between the three levels of this factor (type of state-

ment) were statistically significant. Mean difference between rated

profoundness of bullshit and mundane statements was 1.090

(p < .001), while mean difference between rated profoundness of

proverbs and bullshit was 1.000 (p < .001).

Receptivity and sensitivity to bullshit and thinking style. Participants

correctly solved more congruent than incongruent syllogisms (t

[82] = �4.975, p < .001). Because we registered belief bias, in further

analyses we used the number of incorrect answers to incongruent syl-

logisms. As shown in Table 2, all correlations of either BS or BRS with

different measures of analytical thinking were close to zero, and insig-

nificant. Correlation between rated profoundness of bullshit (BRS) and

mundane statements, as an indicator of biased responses, was 0.136

and not statistically significant (p = .218).

Authors' credibility effect. The repeated measures ANOVA showed

that there was a statistically significant interaction between attribu-

tion status (2: with author, without author) and authors' credibility (3:

credible, uncredible, fictional author; F[2, 166] = 28.283, p < .001,

η2 = 0.254). The main effect of the attribution status was statistically

significant (F[1, 83] = 11.035, p = .001, η2 = 0.117), and the same

goes for the main effect of authors' credibility (F[2, 166] = 22.428,

p < .001, η2 = 0.213). Rated profoundness of bullshit changed when

statements were attributed to authors, while the degree and the

direction of the said change depended on the authors' credibility, as

shown in Figure 1.

Post hoc tests with Bonferroni correction showed that when bull-

shit were presented with uncredible authors mean profundity evalua-

tions decreased by 0.114 although insignificantly (p = .099, 95% CI

[�0.022, 0.250]) in comparison to bullshit receptivity to statements

without an ascribed author. Opposite to this, when a credible author

was ascribed mean profundity evaluations significantly increased by

the mean difference of 0.539 (p < .001, 95% CI [0.391, 0.686). Finally,

when bullshit was attributed to a fictional source the insignificant

mean increase in profundity ratings of 0.063 (p = .375, 95% CI

[�0.078, 0.205]) was observed.

3.3 | Discussion

The main goal of this study was to test whether ascribing authors of

different levels of credibility to bullshit affects evaluations of pro-

foundness and how, among student population. The results showed

TABLE 2 Pearson correlation coefficients (study 1)

MSS PS BS nnCRTc nnCRTh wCSic AOT

BRS 0.136 �0.058 �0.700** �0.052 0.075 0.033 �0.071

MSS 0.275* 0.107 0.103 �0.132 �0.131 0.056

PS 0.753** �0.111 0.096 0.132 �0.083

BS �0.045 0.019 0.072 �0.013

nnCRTc �0.905** �0.213 0.181

nnCRTh 0.185 �0.129

wCSic �0.521**

Note: Cronbach's alphas are reported on the diagonal in brackets.

Abbreviations: AOT, actively open minded thinking beliefs scale; BRS, bullshit receptivity scale; BS,

bullshit sensitivity; nnCRTc, number correct answers on nnCRT; nnCRTh, number of heuristic answers on

nnCRT; PS, proverbs scale; S, mundane statements scale; wCSic, number of incorrect answers on

incongruent categorical syllogisms.

*p < .05. **p < .001.

1198 ILI�C AND DAMNJANOVI�C

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

that attributing bullshit to authors leads to changes in profundity rat-

ings, and that the direction and intensity of those changes depends on

the level of the ascribed authors' credibility. Attribution of bullshit to

Hollywood actors led to slight and insignificant decrease in evalua-

tions. Similarly, ascribing fictional authors led to insignificant increase

in profundity ratings. Finally, attributing the bullshit to a credible

author was followed by the significant increase in profundity evalua-

tions. While bullshit statements without authors were evaluated as

somewhat to fairly profound, when attributed to a famous philoso-

pher of physicist, they were judged to be on average slightly more

than fairly profound. In other words, credible source makes us

more susceptible to bullshit, and, opposite to what might be expected,

an uncredible source does not protect us from falling prey to it.

The second aim of this study was to test the relationship between

different test and self-report measures of analytic thinking, and receptiv-

ity and sensitivity to bullshit. The results showed that those who are less

biased by beliefs report themselves to be more actively open-minded,

which is in accordance with previous research regarding this relationship

(Macpherson & Stanovich, 2007; Stanovich & West, 1998) and have

better scores on cognitive reflection test, although this correlation was

borderline significant (p = .053). Correlations of these measures with

receptivity and sensitivity to bullshit were close to zero. In short, these

findings imply that both intuitive and reflective or analytical reasoners

are receptive (or sensitive) to bullshit. However, these findings might be

due to the fact that the distributions of correct answers on nnCRT and

incongruent syllogisms were negatively skewed. More precisely, almost

half of participants (47%) solved at least seven out of eight incongruent

syllogisms correctly, and 25% gave correct answers to all incongruent

syllogisms. Similarly, 80% of students gave correct answers on at least

three out of four tasks in nnCRT, and almost 40% solved all tasks cor-

rectly. Taking into account that not only our data, but also previous

research on comparability of data obtained from the student and non-

student populations suggests that responses of students are more homo-

geneous extending inferences based on the results of this study to adult

population in general (Peterson, 2001) would not be justified.

For this reason, we conducted the second study in which we col-

lected the data on a more representative sample from the general

population.

4 | STUDY 2

The second study was conducted in order to test the generalizability

of the effect of ascribed author credibility obtained in the Study 1 to

general adult population. We again tested the relationship between

test and self-report measures of analytic thinking, and BRS and BS,

but created additional five tasks of the same type as CRT, in order to

increase the test reliability.

4.1 | Method

4.1.1 | Participants

Participants (N = 199, Female = 53.3%, aged 18–76, mean

age = 42.22, SD = 14.463) were recruited via Facebook advertising.

The description of the sample structure by sex and the level of educa-

tion is presented in the Table S7. Participants received no reimburse-

ment for participation. Only participants who answered all questions

were included in the final sample.

4.1.2 | Materials

Participants were again presented with bullshit without and with the

same ascribed authors as in Study 1. They also again evaluated pro-

foundness of proverbs and mundane statements (only without

ascribed authors), and were presented with AOT, CRT, nnCRT, along

with five newly created cognitive reflection tasks.

Merged cognitive reflection test

In order to increase the reliability of the measure of cognitive reflection we

constructed a merged CRT (mCRT) comprising 17 items: three from the

original three-item CRT (Frederick, 2005), five from extensions to the origi-

nal test (Primi et al., 2016; Toplak et al., 2014), four from nonnumerical CRT

(Thomson & Oppenheimer, 2016), and five new tasks. All new tasks, along

with the correct and heuristic answers are provided in the Table S8.

4.1.3 | Procedure

The survey was created using Sosci Survey (Leiner, 2019) and admin-

istered online. After indicating that they agree with the terms of par-

ticipation, participants read written instructions and proceeded to

F IGURE 1 The effect of attribution status (2: without, with) and source credibility (3: uncredible, fictional, credible) on profoundness evaluations of bullshit. The range of possible mean evaluations of profoundness (y-axis) is shortened so that differences between evaluations without and with ascribed authors and error bars based on standard errors of the mean would be visible

ILI�C AND DAMNJANOVI�C 1199

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

complete the survey. After evaluating profoundness of bullshit, mun-

dane statements and proverbs, participants completed 17 tasks com-

prising mCRT, then again evaluated profoundness of bullshit

statements with ascribed authors of different levels of credibility, and

finally, completed the AOT scale. Within each of these four blocks

items/tasks were presented in a randomized order.

4.2 | Results

4.2.1 | Bullshit receptivity

Participants' evaluations of mundane statements, proverbs and bull-

shit differed significantly (F[1.809, 358.256] = 272.862, η2 = 0.579).4

The mundane statements were rated as the least profound (M = 1.85,

SD = 0.857), followed by the bullshit (M = 2.45, SD = 0.932), and

finally proverbs which were rated as the most profound (M = 3.50,

SD = 0.911). Means, standard deviations, skewness and kurtosis for

each item of each scale are presented in Tables S1–S3. All simple main

effects were significant (p < .001) and are presented in Table 3, along

with standard errors, significance and confidence intervals.

4.2.2 | Bullshit receptivity, sensitivity to bullshit and thinking style

Bullshit receptivity correlated significantly and positively with the

mean evaluations of profoundness of mundane statements and

proverbs, number of heuristic answers on mCRT (mCRTh), and nega-

tively with BS, AOT and the number of correct answers on mCRT

(mCRTc). BS was calculated as the difference between mean evalua-

tions of proverbs and bullshit statements; therefore, the higher score

indicates higher bullshit sensitivity. This measure correlated signifi-

cantly and positively with mCRTc. Pearson correlation coefficients for

all measures are provided in the Table 4.

4.2.3 | The effect of authors' credibility

Means and standard deviations of evaluations of profundity by attri-

bution status and authors' credibility are presented in Table 5.

The repeated measures ANOVA revealed significant interaction of

the attribution status and the authors' credibility factors (F[1.845,

365.337] = 24.790, p < .001, η2 = 0.111).5 Main effects of both the attri-

bution status (F[1, 198] = 5.744, p = .017, η2 = 0.028), and the authors'

credibility(F[1.921, 380.290] = 6.462, p = .002, η2 = 0.032) were

significant.6

As can be seen on Figure 2, evaluations of profoundness of bullshit

significantly decreased when uncredible authors were ascribed and

increased when credible authors were ascribed. When a statement was

attributed to a fictional author, the evaluations remained statistically the

same. Post hoc comparisons using Bonferroni correction showed that

the mean differences between without and with author conditions were

0.131 (p = .002, 95% CI [0.048, 0.214]), �0.062 (p = .129, 95% CI

[�0.141, 0.018]), and �0.275 (p < .001, 95% CI [�0.371, �0.179]) when

an uncredible, fictional, and credible authors were ascribed, respectively.

TABLE 3 Simple main effects of the statement type (study 2)

MD SE p CI lower CI upper

Proverbs—Mundane 1.644 0.064 <.001 1.489 1.800

Bullshit—Mundane 0.595 0.066 <.001 0.436 0.755

Proverbs—Bullshit 1.049 0.082 <.001 0.851 1.247

Abbreviations: CI, 95% confidence interval; MD, mean difference; p, significance level; SE, standard error.

TABLE 4 Pearson correlation coefficients and internal consistencies (Study 2)

BRS MS PS BS mCRTc mCRTh AOT

BRS (0.914) 0.460** 0.213** �0.638** �0.449** 0.400** �0.217**

MS (0.924) 0.474** 0.003 �0.214** 0.203** �0.301**

PS (0.943) 0.616** �0.097 0.139* �0.260**

BS 0.285** �0.213** �0.029

mCRTc (0.801) �0.910** 0.332**

mCRTh �0.293**

AOT (0.596)

Note: Cronbach's alphas are reported on the diagonal in brackets.

Abbreviations: AOT, actively open minded thinking beliefs scale; BRS, bullshit receptivity scale; BS, bullshit sensitivity; mCRTc, number correct answers on

mCRT; mCRTh, number of heuristic answers on mCRT; PS, proverbs scale.

*p = .05.

**p < .001.

1200 ILI�C AND DAMNJANOVI�C

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

Finally, we calculated the measures of the effect of ascribing

credible and uncredible authors to bullshit. Because ascribing the

credible author led to increased evaluation of profoundness the

effect of credible author was calculated by subtracting the mean

profoundness evaluations of bullshit without author from mean

evaluations of bullshit with credible author. The effect of

uncredible author was calculated in the opposite direction (without

author minus with uncredible author), as attribution of bullshit to

such author led to decrease in evaluations. This way the higher pos-

itive difference indicates the greater effect in both cases. The

repeated measures ANOVA revealed that the effect of credible

author was significantly larger, by a mean difference of 0.144 (F[1,

198] = 5.064, p = .026, η2 = 0.025). We correlated these effects

with the achievement on mCRT, score on AOT and bullshit sensitiv-

ity. As can be seen in Table 6, the only significant and positive cor-

relation was registered between the effect of the uncredible author

and the score on AOT (r = .184, p = .009).

5 | GENERAL DISCUSSION

The present study represents a partial replication and an extension to

Pennycook, Cheyne, et al.'s (2015) findings regarding the mechanisms

of receptivity and sensitivity to bullshit, and the first study in which

credibility of an author attributed to bullshit was varied systematically.

5.1 | The source

We presented students and adults from general population with bull-

shit with and without ascribed authors of different levels of credibility.

Explicit information about the source of the statement alters evalua-

tions in such a way that when bullshit is attributed to a credible

author the profoundness ratings increase, while ascribing uncredible

source has an opposite effect. It seems that when it comes to wisdom,

it is more important who said something than what was said. The fact

that the change occurs during the same experimental session, that is—

within 15 min, further supports this assumption. Theoretical and social

implications of these findings are multiple and interrelated.

As postulated by researchers who investigated mechanisms

underlying bullshit receptivity and sensitivity, evaluating profoundness

of bullshit statements requires scrutiny, so one would be able to

detect that bullshit is meaningless (Pennycook, Cheyne, et al., 2015).

In the realm of models of dual processing, successful detection that a

certain statement is bullshit is based on detection of conflict and sub-

sequent deliberative judgment. More precisely, the detection that the

meaning is not where it is conventionally expected (in communicative

statements) pertains to the detection of the conflict between the cued

intuitive and the normative answer. Such conflict detection followed

by decoupling is considered to be the central sequence in successful

reasoning, in all proposed models, whether they fall into the cluster of

default-interventionist (serial; Evans, 2010; Kahneman, 2003;

Stanovich, 2011), parallel (De Neys & Glumicic, 2008; Sloman, 1996),

or semi-parallel models (Bago & DeNeys, 2017; De Neys, 2014; Pen-

nycook, Fugelsang, & Koehler, 2015). When conflict is detected and

monitored, Type 2 processes can play their part and evaluate bullshit

statements. However, the mere capacity to detect conflict is not

enough, as engagement in deliberate reasoning takes will, that is

decoupling, and only if finished successfully.

TABLE 5 Mean evaluations of bullshit with and without ascribed authors of different levels of credibility (study 2)

Attribution status

Without With

Authors' credibility M SD M SD

Uncredible 2.46 1.037 2.33 1.077

Fictional 2.46 1.007 2.52 1.059

Credible 2.42 1.041 2.69 1.143

Abbreviations: M, mean; SD, standard deviation.

F IGURE 2 The effect of attribution status (2: without, with) and source credibility (3: uncredible, fictional, credible) on profoundness evaluations of bullshit. The range of possible mean evaluations of profoundness (y-axis) is shortened so that differences between evaluations without and with ascribed authors and error bars based on standard errors of the mean would be visible

TABLE 6 Pearson correlations of the effects of ascribing credible and uncredible authors with the mCRT achievement, scores on AOT and bullshit sensitivity (study 2)

mCRTc AOT BS

EunCA 0.005 0.184** 0.066

ECA �0.010 0.059 0.027

Abbreviations: AOT, actively open-minded thinking beliefs scale; BS,

bullshit sensitivity; ECA, effect of ascribing credible author; EunCA, effect

of ascribing uncredible author to bullshit; mCRTc, number correct answers

on mCRT. **p < .01.

ILI�C AND DAMNJANOVI�C 1201

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

Unreliable or uncredible source of the statement could serve as

the contextual trigger of willingness to engage in Type 2 processing.

However, although uncredible author does cue decrease in evaluation

of profoundness, mean evaluations remain in the range between

somewhat and fairly profound. This might be due to the methodologi-

cal decision to operationalize uncredible authorship by using public

appearances of Hollywood actors as references. Namely, although

actors are not experts regarding abstract content, they might not be

perceived as truly uncredible sources. This is indicated by the findings

from the Study 1 in which effect of uncredible authors wasn't signifi-

cant, but also by findings of the Study 2 in which effect of credible

author was larger. Moreover, it is also possible for the knowledge

about the source of a statement to have an opposite effect on reason-

ing rendering the processing even more biased. This was consistently

registered as ascribing credible author to meaningless sentences led

to increase in evaluations regardless of the sample. Related social

implications of the finding that an evaluation of whether or how much

a content without discernible meaning is wise depends on the credi-

bility of the ascribed author are widespread and obvious. As previ-

ously explained, inherent nature of bullshit makes it irrefutable

(Hopkin & Rosamond, 2018; Sarajli�c, 2018). Consequence of this fea-

ture is that the person who produces bullshit is enabled to waive and

avoid responsibility for the events that occur as “repercussions” of

their claims (Sarajli�c, 2018), which also might be part of the reason

why bullshit is so prevalent today (Frankfurt, 2005; Pennycook,

Cheyne, et al., 2015). Consider the following kicker statement from an

article by a well-known Serbian medical doctor and psychotherapist,

which served him as an argument against the introduction of a law

which would ban corporal punishment of children: “Just as it is to

some extent true that ‘violence breeds violence’, it is also true that

‘non-violence breeds violence’” (Milivojevic, 2011). This sentence has

all the characteristics of bullshit proposed so far from both the psy-

chological and the philosophical stance: adding a negation sign to it

does not change its plausibility (Cohen, 2002), it has nothing to do

with the truth as truth is here irrelevant (Frankfurt, 2005; Pennycook,

Cheyne, et al., 2015), but it is not a lie, as it is in fact a logical tautol-

ogy (Frankfurt, 2005; Pennycook, Cheyne, et al., 2015), which makes

it irrefutable (Sarajli�c, 2018). The content prevents the listener from

constructing relevant epistemic and practical premises as it does not

give reasons either for or against a certain course of action

(Sarajli�c, 2018), while speaker's intention to convince a listener of

something is quite clear (Cohen, 2002; Frankfurt, 2005;

Sarajli�c, 2018), and it provides him—the speaker, as well as his sup-

porters, with enough space to claim that he never said something

untrue or at least that he did not have such an intention

(Sarajli�c, 2018). This last characteristic is precisely what makes bullshit

more dangerous and socially corrosive than lies (Hopkin &

Rosamond, 2018; Sarajli�c, 2018), and it is even more so when the

author is by vocation a person of authority, as is the case with this

example where the author, simply by being a psychotherapist and a

doctor, holds an expert authority in the realm of childhood psycholog-

ical development or upbringing, as well as everything that is more or

less directly health-related. In short, for upbringing advice

psychotherapists are just as valid credible sources as are physicists

and philosophers when it comes to transcendental and abstract con-

tent such as bullshit, and a credible source leads to overrating the

degree to which meaningless content is considered wise.

5.2 | The recipient

We presented participants with bullshit along with mundane state-

ments, proverbs and different self-report and test measures of think-

ing style in order to replicate and extend the findings by Pennycook,

Cheyne, et al. (2015) regarding mechanisms of receptivity and sensi-

tivity to bullshit. The findings based on a student sample do not sup-

port the assumptions that some people are simply more prone to

evaluate anything as profound including bullshit and that individual

differences in thinking style drive differences in receptivity or sensitiv-

ity to bullshit. However, results obtained on a less restrictive sample

of adults from general population do. Namely, we registered higher

evaluations of profoundness of mundane statements compared to the

student sample, as well as significant correlation between participants'

evaluations of bullshit and mundane statements. These findings taken

together indicate that at least some participants' receptivity to bullshit

is due to the response bias. The second proposed mechanism of

receptivity to bullshit pertains to a lack of ability to detect that pres-

ented sentence is in fact bullshit (Pennycook, Cheyne, et al., 2015).

Our findings also support this assumption as participants who had

higher scores on mCRT not only rated bullshit as less profound but

were also more sensitive to bullshit. AOT as a self-report measure of

thinking style only correlated negatively with bullshit receptivity,

while significant negative correlation with bullshit sensitivity was not

registered. Moreover, while participants with more correct answers

on merged CRT rated bullshit and mundane statements as less pro-

found, those who reported themselves as more prone to analytic

thinking style were less inclined to rate any type of statement as pro-

found. Opposite to this, intuitive reasoning leads to response bias as

participants who had more heuristic answers on mCRT also gave

higher evaluations of profoundness regardless of the type of state-

ment. In short, it seems that people who are more inclined to reason

analytically better distinguish truly profound statements from bullshit

and, opposed to intuitive reasoners, show less response bias, while

participants who perceive themselves more as analytical thinkers are

less inclined to rate any content as profound.

Finally, we calculated the effects of ascribing credible and

uncredible authors to bullshit, and correlated these effects with other

collected measures. Interestingly, the correlations between the effects

of authorship and sensitivity to bullshit were not significant. In other

words, participants who judge profundity based largely on authorship,

and during this process apparently do not deliberate over the meaning

of the statement, are both those who are good and those who are bad

at differentiating truly meaningless or pseudo-profound from truly

profound, sensical, and at times ambiguous content such as proverbs.

Taking into account that in order to be profound (i.e., of deep mean-

ing, etc.) a statement must have meaning in the first place, and that

1202 ILI�C AND DAMNJANOVI�C

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

both participants who are more and less able to detect bullshit, that -

is—discern truly profound and meaningful from pseudo-profound and

meaningless are largely affected by source credibility, it seems that

information about the source overrides individual differences in bull-

shit sensitivity, and directly biases evaluations. Moreover, participants

with higher scores on AOT were under a greater influence of

uncredible authorship i.e., their evaluations of the profoundness

of bullshit decreased more after attribution to uncredible source.

AOT, although somewhat similar to other measures of analytical

thinking, differs due to the focus on consideration of new evidence

and revision of one's current beliefs (Baron, 2000). For example, those

with high scores on CRT might consider something more carefully and

analytically, but those with high AOT will actively search for reasons

to overturn their existing beliefs (Stenhouse et al., 2018), which is

what we registered after we have presented participants with new

evidence, that is, information about the uncredible source of the

statement.

Overall, cognitive reflection indeed serves as a protective factor

against response bias and drives better differentiation between

pseudo- and conventionally truly profound. However, better detec-

tion of bullshit does not protect from falling prey to information about

authorship, while AOT enables even stronger effects of uncredible

authorship.

5.3 | Limitations and future directions

In the present study we aimed to shed light on contextual factors that

might be in effect when evaluating profoundness of meaningless

statements. Extending on Pennycook, Cheyne, et al.'s (2015) inference

that people might rate bullshit as profound simply because they

expect that in a scientific study in which the key task is to evaluate

profoundness of a statement they will be presented with meaningful

statements, we explicitly varied the credibility of authors to whom we

attributed bullshit. Taking into account that the effect of uncredible

author was not registered among student population indicating faulty

manipulation of authors' credibility, in the future studies a data driven

approach might come in handy for resolving this issue. Another possi-

ble, and more structured way of manipulating authors' credibility in

future studies is to cross and systematically vary the two dimensions

of credibility: expertise and trustworthiness. This would provide

insight into whether potentially expertise or trustworthiness, that is,

degree to which audience perceives assertions made by communica-

tor to be the ones that he considers valid, drive the effects of source

credibility. Also, the question that remains is whether ascribing

authors of different levels of credibility is only effective when

ascribed to sentences with no discernible meaning, or the source of

the statement is more important than the meaning (or lack of it) of the

statement regardless of its' type. Systematical variation of both author

credibility and type of statements to which authors are ascribed might

elucidate the characteristics of statements that drive observed effects

of source reliability, over and above the individual differences in mea-

sured dispositions.

ACKNOWLEDGEMENT

Life is not a tiptoe through the tulips.

CONFLICT OF INTEREST

The authors have no conflict of interest to declare.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are openly available in

Mendeley Data at http://doi.org/10.17632/xmp29vsb5d.1, reference

number 10.17632/xmp29vsb5d.1.

ORCID

Sandra Ili�c https://orcid.org/0000-0002-5145-0197

Kaja Damnjanovi�c https://orcid.org/0000-0002-9254-1263

ENDNOTES 1 The law of attraction is a pseudoscientific concept that pertains to the

belief that positive or negative thoughts attract positive or negative

experiences into a person's life, respectively. Although the term first

appeared a decade earlier, it was first articulated as a law in an essay by

Prentice Mulford cca 1887. In 2006, with the release of the movie The

Secret, and the book of the same title a year later, it received renewed

and widespread attention. 2 According to Google Scholar at the time of the study the article propos-

ing alternate CRT version (Thomson & Oppenheimer, 2016) has been

referenced 75 times, while only Frederick's (2005) original short CRT has

been referenced 3068 times. 3 As Mauchly's test of sphericity was significant X2(2) = 21.185, p < .001)

a Greenhouse–Geisser correction of the degrees of freedom was

used (ε = 0.815). 4 As Mauchly's test of sphericity was significant (X2(2) = 21.932, p < .001)

a Greenhouse–Geisser correction of the degrees of freedom was

used (ε = 0.905). 5 As Mauchly's test of sphericity was significant (X2(2) = 17.270, p < .001)

a Greenhouse–Geisser correction of the degrees of freedom was

used (ε = 0.923). 6 As Mauchly's test of Sphericity was significant (X2(2) = 8.311, p = .016)

a Greenhouse–Geisser correction of the degrees of freedom was

used (ε = 0.960).

REFERENCES

Bučevi�c, A. (n.d.). Safari duha [YouTube Channel]. YouTube. Retrieved from

https://www.youtube.com/c/SAFARIDUHA/about

Bago, B., & DeNeys, W. (2017). Rise and fall of conflicting intuitions during

reasoning. In G. Gunzelmann, A. Howes, T. Tenbrink, & E. J. Davelaar

(Eds.), Proceedings of the 39th annual conference of the cognitive science

society (pp. 87–92). Cognitive Science Society. Bar-Hillel, M., Maharshak, A., Moshinsky, A., & Nofech, R. (2012). A rose by

any other name: A social-cognitive perspective on poets and poetry. Judg-

ment & Decision Making, 7(2), 149–164. https://doi.org/10.1037/

e633962013-382

Baron, J. (2000). Thinking and deciding. Cambridge University Press.

Baron, J., Scott, S., Fincher, K., & Metz, S. E. (2015). Why does the cogni-

tive reflection test (sometimes) predict utilitarian moral judgment

(and other things)? Journal of Applied Research in Memory and Cogni-

tion, 4(3), 265–284. https://doi.org/10.1016/j.jarmac.2014.09.003

Behind the Name. Random name generator [Computer software]. (2019).

Retrieved from https://www.behindthename.com/random/

ILI�C AND DAMNJANOVI�C 1203

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

Böckenholt, U. (2012). The cognitive-miser response model: Testing for

intuitive and deliberative reasoning. Psychometrika, 77(2), 388–399. https://doi.org/10.1007/s11336-012-9251-y

Bučevi�c, A. (2015). U vorteksu ostvarenih želja: kreirajte vlastitu stvarnost

uz pomo�c Zakona privlačenja i svjedočite čudima u svom životu. Kontakt plus.

Campitelli, G., & Gerrans, P. (2014). Does the cognitive reflection test

measure cognitive reflection? A mathematical modeling approach.

Memory and Cognition, 42(3), 434–447. https://doi.org/10.3758/ s13421-013-0367-9

Cohen, G. A. (2002). Deeper into bullshit. In S. Buss & L. Overton (Eds.),

The contours of agency: Essays on themes from Harry Frankfurt

(pp. 321–340). MIT Press.

Damnjanovi�c, K., Novkovi�c, V., Pavlovi�c, I., Ili�c, S., & Panteli�c, S. (2019). A

cue for rational reasoning: Introducing a reference point in cognitive

reflection tasks. Europe's Journal of Psychology, 15(1), 25. https://doi.

org/10.5964/ejop.v15i1.1701

De Neys, W. (2014). Conflict detection, dual processes, and logical intui-

tions: Some clarifications. Thinking & Reasoning, 20(2), 169–187. https://doi.org/10.1080/13546783.2013.854725

De Neys, W., & Glumicic, T. (2008). Conflict monitoring in dual process

theories of thinking. Cognition, 106(3), 1248–1299. https://doi.org/10. 1016/j.cognition.2007.06.002

Evans, J. S. B. (2010). Intuition and reasoning: A dual-process perspective.

Psychological Inquiry, 21(4), 313–326. https://doi.org/10.1080/

1047840X.2010.521057

Evans, J. S. B., Barston, J. L., & Pollard, P. (1983). On the conflict between

logic and belief in syllogistic reasoning. Memory & cognition, 11(3),

295–306. https://doi.org/10.3758/bf03196976 Frankfurt, H. (1986). On bullshit. Raritan Quarterly Review., 6(2), 81–100. Frankfurt, H. (2005). On bullshit. Princeton University Press.

Frederick, S. (2005). Cognitive reflection and decision making. Journal of

Economic Perspectives, 19(4), 25–42. https://doi.org/10.1257/

089533005775196732

Fritz, C., Curtin, J., Poitevineau, J., Morrel-Samuels, P., & Tao, F. C. (2012).

Player preferences among new and old violins. Proceedings of the

National Academy of Sciences of the United States of America, 109(3),

760–763. https://doi.org/10.1073/pnas.1114999109 Gligori�c, V., & Vilotijevi�c, A. (2019). “Who said it?” How contextual infor-

mation influences perceived profundity of meaningful quotes and

pseudo-profound bullshit. Applied Cognitive Psychology, 34, 535–542. https://doi.org/10.1002/acp.3626

Haigh, M. (2016). Has the standard cognitive reflection test become a vic-

tim of its own success? Advances in cognitive psychology, 12(3), 145.

https://doi.org/10.5709/acp-0193-5

Hopkin, J., & Rosamond, B. (2018). Post-truth politics, bullshit and bad

ideas: ‘Deficit Fetishism’ in the UK. New Political Economy, 23(6), 641– 655. https://doi.org/10.1080/13563467.2017.1373757

Hovland, C. I., Janis, I. L., & Kelley, H. H. (1953). Communication and

persuasion.

Ili�c, S., & Damnjanovi�c, K. (2021a). Philosophical and psychological

approaches to bullshit. Theoria, Beograd, 64(1), 161–172. https://doi. org/10.2298/theo2101161i

Ili�c, S., & Damnjanovi�c, K. (2021b). Bullshit and credibility; Mendeley data

[dataset].

Kahneman, D. (2003). A perspective on judgment and choice: Mapping

bounded rationality. American Psychologist, 58(9), 697–720. https:// doi.org/10.1037/0003-066X.58.9.697

Leiner, D. J. (2019). SoSci Survey (Version 3.1.06) [Computer software].

Available at https://www.soscisurvey.de

Macpherson, R., & Stanovich, K. E. (2007). Cognitive ability, thinking dispo-

sitions, and instructional set as predictors of critical thinking. Learning

and Individual Differences, 17(2), 115–127. https://doi.org/10.1016/j. lindif.2007.05.003

Mathôt, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-

source, graphical experiment builder for the social sciences. Behavior

Research Methods, 44(2), 314–324. https://doi.org/10.3758/s13428- 011-0168-7

Milivojevic, Z. (2011). Telesno kaznjavanje dece—da ali razumno [Child

corporal punishment—yes but reasonably]. Nova srpska politicka misao.

Retreived from http://www.nspm.rs/kulturna-politika/telesno-

kaznjavanje-dece-da-ali-razumno.html?alphabet=l

Moore, D. L., Hausknecht, D., & Thamodaran, K. (1986). Time compres-

sion, response opportunity, and persuasion. Journal of Consumer

Research, 13(1), 85–99. https://doi.org/10.1086/209049 Najkra�ci test inteligencije. (2017). 3 pitanja otkrivaju da li ste genije

[Shortest IQ test: 3 questions reveal if you are a genious]. Retreived

from https://www.b92.net/zivot/vesti.php?yyyy=2017&mm=09&

dd=27&nav_id=1307725

Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J., & Fugelsang, J. A.

(2015). On the reception and detection of pseudo-profound bullshit.

Judgment and Decision making, 10(6), 549–563. Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015). What makes us

think? A three-stage dual-process model of analytic engagement.

Cognitive Psychology, 80, 34–72. https://doi.org/10.1016/j.cogpsych. 2015.05.001

Pennycook, G., & Rand, D. G. (2020). Who falls for fake news? The roles

of bullshit receptivity, overclaiming, familiarity, and analytic thinking.

Journal of Personality 88(2), 185–200. https://doi.org/10.2139/ssrn. 3023545

Peterson, R. A. (2001). On the use of college students in social science

research: Insights from a second-order meta-analysis. Journal of

Consumer Research, 28(3), 450–461. https://doi.org/10.1086/

323732

Pfattheicher, S., & Schindler, S. (2016). Misperceiving bullshit as profound

is associated with favorable views of Cruz, Rubio, trump and conserva-

tism. PLOS ONE, 11(4), e0153419. https://doi.org/10.1371/journal.

pone.0153419

Plassmann, H., O'Doherty, J., Shiv, B., & Rangel, A. (2008). Marketing

actions can modulate neural representations of experienced pleasant-

ness. Proceedings of the National Academy of Sciences of the United

States of America, 105(3), 1050–1054. https://doi.org/10.1073/pnas. 0706929105

Pornpitakpan, C. (2004). The persuasiveness of source credibility: A critical

review of five decades' evidence. Journal of Applied Social Psychology, 34

(2), 243–281. https://doi.org/10.1111/j.1559-1816.2004.tb02547.x Primi, C., Morsanyi, K., Chiesi, F., Donati, M., & Hamilton, J. (2016). The

development and testing of a new version of the cognitive reflection

test applying item response theory (IRT). Journal of Behavioral Decision

Making, 29(5), 453–469. https://doi.org/10.1002/bdm.1883

Sarajli�c, E. (2018). Bullshit, truth, and reason. Philosophia, 47(3), 865–879. https://doi.org/10.1007/s11406-018-9990-9

Sinayev, A., & Peters, E. (2015). Cognitive reflection vs. calculation in deci-

sion making. Frontiers in Psychology, 6, 532. https://doi.org/10.3389/

fpsyg.2015.00532

Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psy-

chological Bulletin, 119(1), 3–22. https://doi.org/10.1037/0033-2909. 119.1.3

Stanovich, K. (2011). Rationality and the reflective mind. Oxford University

Press.

Stanovich, K. E., & West, R. F. (1998). Individual differences in rational

thought. Journal of Experimental Psychology: General, 127(2), 161.

https://doi.org/10.1037/0096-3445.127.2.161

Stenhouse, N., Myers, T. A., Vraga, E. K., Kotcher, J. E., Beall, L., &

Maibach, E. W. (2018). The potential role of actively open-minded

thinking in preventing motivated reasoning about controversial sci-

ence. Journal of Environmental Psychology, 57, 17–24. https://doi.org/ 10.1016/j.jenvp.2018.06.001

1204 ILI�C AND DAMNJANOVI�C

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense

Sterling, J., Jost, J. T., & Pennycook, G. (2016). Are neoliberals more sus-

ceptible to bullshit? Judgment and Decision Making, 11(4), 352–361. Stieger, S., & Reips, U. D. (2016). A limitation of the cognitive reflection

test: Familiarity. PeerJ, 4, e2395. https://doi.org/10.7717/peerj.

2395

Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate

form of the cognitive reflection test. Judgment and Decision Making, 11

(1), 99–113. Toplak, M., West, R., & Stanovich, K. (2014). Assessing miserly information

processing: An expansion of the cognitive reflection test. Thinking &

Reasoning, 20, 147–168. https://doi.org/10.1080/13546783.2013.

844729

Welsh, M., Burns, N., & Delfabbro, P. (2013). The cognitive reflection test:

How much more than numerical ability? Paper presented at the Pro-

ceedings of the 35th Annual Meeting of the Cognitive Science Society,

Berlin, Germany (pp. 1587–1592). Šok za Anu Bučevi�c (2019). Spli�canka dobila kaznenu prijavu Ministarstva

zdravstva zbog navodnog nadriljiječnjištva [Shock for Ana Bučevi�c: citizen

of Split received a criminal report from the Ministry of Health for

alleged quackery]. Slobodna Dalmacija. Retrieved from https://

slobodnadalmacija.hr/mozaik/zivot/sok-za-anu-bucevic-splicanka-

dobila-kaznenu-prijavu-ministarstva-zdravstva-zbog-navodnog-

nadrilijecnistva-595044

SUPPORTING INFORMATION

Additional supporting information may be found online in the

Supporting Information section at the end of this article.

How to cite this article: Ili�c, S., & Damnjanovi�c, K. (2021). The

effect of source credibility on bullshit receptivity. Applied

Cognitive Psychology, 35(5), 1193–1205. https://doi.org/10.

1002/acp.3852

ILI�C AND DAMNJANOVI�C 1205

10990720, 2021, 5, D ow

nloaded from https://onlinelibrary.w

iley.com /doi/10.1002/acp.3852 by Florida International U

niversity, W iley O

nline L ibrary on [08/05/2023]. See the T

erm s and C

onditions (https://onlinelibrary.w iley.com

/term s-and-conditions) on W

iley O nline L

ibrary for rules of use; O A

articles are governed by the applicable C reative C

om m

ons L icense