Discussion
A Wikipedia-based Method to Support Creative Idea Generation: The Role of Stimulus Relatedness
KAI WANG AND JEFFREY V. NICKERSON
KAI WANG (kaiwan@kean.edu) is an assistant professor of management in the School of Management and Marketing at Kean University. His research domains include using information systems to support creative work and crowdsourcing innovation. His work appears in Creativity and Innovation Management and Computers in Human Behavior.
JEFFREY NICKERSON (jnickerson@stevens.edu) is a Professor and Associate Dean of Research in the School of Business, Stevens Institute of Technology. His research and teaching interests include creativity support systems, autonomous tools, and collective intelligence. He is currently the principal investigator of three NSF-funded projects related to creativity and productivity in online communities. He has published in MIS Quarterly and ACM Transactions on Computer-Human Interaction.
ABSTRACT: Providing stimuli may facilitate idea generation. Creativity theories often suggest that stimuli unrelated to the problem task will improve creativity, but empirical studies have yielded inconsistent results. We propose a Wikipedia-based approach that is able to identify stimuli at different levels of relatedness. Specifically, we use hypertext links in two sections of the Wikipedia article of a focal concept to identify closely related concepts. Repeating this procedure leads to increasingly remote con- cepts. Using this approach to obtain stimulus concepts, we examine the effect of stimulus relatedness on idea generation. Our results show that stimulus relatedness is positively related to idea quantity and idea usefulness. While creativity theories often suggest using unrelated stimuli to promote idea novelty, results of this experimental study indicate that remotely related stimuli, not unrelated stimuli, tend to improve idea novelty. Because Wikipedia covers knowledge in almost all disciplines, our Wikipedia- based approach can be used to discover appropriate stimuli and thereby support creative work in most domains of knowledge.
KEY WORDS AND PHRASES: creativity, idea generation, creativity support systems, creativity stimulation, association in creativity, Wikipedia.
Introduction
Organizations develop new ideas, products, and services in order to survive and succeed [4, 38, 40]. There is a research stream within the IS discipline that focuses on creativity support systems (CSS), information technology-based tools that
Journal of Management Information Systems / 2019, Vol. 36, No. 4, pp. 1284–1312.
Copyright © Taylor & Francis Group, LLC
ISSN 0742–1222 (print) / ISSN 1557–928X (online)
DOI: https://doi.org/10.1080/07421222.2019.1661095
enhance the creative output of individuals or groups [3, 30, 51, 68]. A variety of approaches have been developed in CSS research, including guiding people through steps in a creative process [24, 46, 68], using mind maps [2, 47], facilitating creative techniques [32], and supporting group collaboration [2, 23, 30, 39, 56]. One particularly interesting approach is to provide stimuli to inspire new ideas, such as existing example solutions [3, 62], analogies [3, 39], concepts [42, 47, 64], and pictures [37, 46, 65, 66]. Are external stimuli always beneficial for creativity? Studies on idea generation indicate that showing concepts or design examples can sometimes constrain thinking and reduce creativity by inducing fixation: designers can be so attracted to examples that they find it difficult to fully explore the design space [8, 10, 28, 41]. It is clear that creativity support systems need algorithms that are cautious and selective in identifying stimuli for supporting creative work. One important property of a stimulus is the degree to which it is related to
a creative task, that is, stimulus relatedness [58]. Some creativity theories suggest that a stimulus that is less related to a creative task may lead to novel associations and increase idea creativity [48, 52]. This notion has some empirical support, for example, in engineering design and group brainstorming [13, 14, 19, 39]. However, there are also studies showing that the cognitive distance of stimuli from a domain is not related to the creativity of resulting ideas or solutions [46, 63]. Two recent studies even show that exposure to remote examples [3] or citing conceptually distant solutions [12] is associated with lower creativity. In order to better under- stand the effects of stimuli, and to build effective creativity support systems, there is a need to find stimuli along a spectrum from highly related to unrelated, as well as to understand the potential effects of such calibrated stimuli. This study aims to develop an automatic way of providing stimuli for creative
ideation, and to improve the understanding of how stimulus relatedness influences idea generation. The next section provides a review of past theoretical and empiri- cal research that addresses the use of stimuli to promote creativity. Our hypotheses on the effect of stimulus relatedness are then presented. Afterwards, we introduce and validate a Wikipedia-based approach for finding stimuli that are related to an initial concept along a spectrum of relatedness. Two experiments used this new approach to find stimuli of a range of relatedness, and then tested their effects on creative idea generation. The implications for future research in information sys- tems and practice are discussed subsequently.
Background
Creativity is typically defined as the generation of products or ideas that are novel (or original) and appropriate (or useful) [2, 3, 28, 40]. Here a review is provided on theoretical and empirical research on creativity and stimulus relatedness. We briefly discuss cognitive theories on creativity, then we explain how stimuli may influence idea generation, with a focus on the role of stimulus relatedness.
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1285
Cognitive Theories of Creativity
In this section, we focus on those cognitive theories that have direct implications on using stimuli in creative idea generation. Ideas are commonly considered as pro- ducts of existing information in minds [52, 58]. Consequently, theories of idea generation are often based on theories of memory processes. Two well-cited theories on memory processes are the Search of Associative Memory theory (SAM) [57] and the Adaptive Control of Thought (ACT) theory [5, 6]. Both theories claim that long term memory is an associative network of memory units. Short term memory, or working memory, has limited capacity and contains ele- ments that can be thought of as search cues. That is, these elements are sources of activation that probe long term memory. The probability of activating certain memory units (chunks in ACT; images in SAM, no visual representation implied) is based on the association strength between the search cue and the memory units. SAM emphasizes the retrieval plan, which specifies a series of search and recovery operations. A retrieval plan can be changed as search proceeds. The retrieval plan is used to determine how to choose or combine cues and what cues are used at each stage of search. Different search cues may be used at different stages of search. ACT builds on the spreading-activation theory of semantic processing and assumes that a stimulus will activate some concept node in the semantic network in mind, and the activation automatically spreads to other concepts across the network based on associations among memory units [16]. Consider the example of a particular creative task: coming up with new ideas to
promote a hotel. According to SAM, when a person is faced with this task, working memory will necessarily contain the information related to this task. If a stimulus word, for example cooking, is presented, the word may be added to the existing information in working memory. A retrieval plan is generated in working memory that uses both task information and cooking as search cues to probe long term memory. SAM posits that search is focused on information that is strongly con- nected to all the search cues, in this case, both the task and cooking. Therefore, different features and associations about cooking may be activated, such as the concepts of menu, recipe and diet. In addition, more idiosyncratic memories may be triggered: for example, of cooking shows that are framed as athletic contests. The activated knowledge is evaluated and if appropriate, will be used for ideation. For example, an idea might be generated about a weekly menu with healthy choices; or, perhaps, introducing a competitive walk on hotel grounds before or after meals as part of an exercise regime. If this round of search does not turn out successful, the newly activated information, combined with old information in working memory, will be used to generate the next search cues, leading to a new round of search, unless a decision to terminate search has been made. According to ACT theory, knowledge in mind has a baseline level of activation,
which will be enhanced by external stimuli. Using the aforementioned example, the word cooking will activate some chunk in declarative knowledge based on associa- tion strength. For example, the features and associations of cooking are likely
1286 WANG AND NICKERSON
activated. The activated knowledge will then be used for further processing or ideation. Such activation can automatically spread into other chunks based on associations. Consequently, additional ideation based on further associations of cooking is possible. Notwithstanding the differences, the SAM model and the ACT theory depict a similar overall picture of the memory retrieval process where knowledge is activated based on the association strength between search cues and the knowledge units. Using concepts to stimulate ideation involves both conscious and subconscious aspects. Deliberate attention to and use of concepts takes a conscious effort. The activation or retrieval of knowledge in mind and its spreading contain automatic and subconscious processes. Based on SAM, a theory of idea generation was developed, called Search for Ideas in
Associative Memory (SIAM) [52]. According to the SIAM model, idea generation is a repeated search process with two stages. First, a search cue, such as the problem definition or a previous idea, is used as to activate certain knowledge in long term memory. Such activation is probabilistic: the activation is dependent on the strength of the association between the search cue and the knowledge. Then, in the second stage, the activated knowledge is combined or processed by working memory to generate ideas. A similar theory is called the cognitive network model of creativity (CNM) [58]. This theory also assumes the existence of long term memory as an associative network of knowledge and the existence of working memory containing activated knowledge. The CNM theory argues that, in problem solving, the diversity of external stimuli increases the disparity among activated knowledge, which tends to increase the creativity of solutions. However, the number of stimuli per unit time and the disparity among activated knowledge also increase cognitive load, which can in turn inhibit creative thinking. While the SIAM theory is quite detailed in elucidating all the steps in the idea generation process, the CNM theory stresses the role of knowledge distance and cognitive load in creativity.
Influence of Stimuli on Idea Generation
There are two major ways of using stimuli in idea generation: priming and deliberate conscious use of stimuli. Priming is defined as presenting stimuli to activate certain mental representations of concepts, attitudes, or beliefs that affect the behavior on a later task [22, 55]. Priming typically affects people subcon- sciously: people are unaware that the stimuli are activating mental representations and do not know the intent of priming [22]. It has been shown that being exposed to example uses of objects led people to associate the objects with certain functions, making it difficult to come up with other functions for the objects [1, 34]. In one study, the participants were primed with a computer game [22]. In this game, people selected and arranged words into headlines that emphasize achievement, such as “scholar aspires for honor.” The participants who experienced this achievement priming generated more creative ideas in group electronic brainstorming, compared to neutral priming. In another study, the researchers used a similar scrambled-
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1287
sentence task to prime either prosocial or efficiency norm with related words [55]. Then, the participants generated ideas for an open-ended problem and the ideas matched the prosocial or efficiency norm primed earlier. Priming using such games may activate the semantic content related to the prime (such as achievement or prosocial norm), which is used in later tasks [22]. However, it is also possible that the priming introduces a subconscious goal (such as achievement) and affects the motivation and effort [22]. Unlike priming, in the second approach to using stimuli people are consciously
aware of the intent of using stimuli. People deliberately consider stimuli as inputs in the ideation process. The following studies, as well as our study, use this second approach. Design examples can lead people to focus on familiar categories and schemas and generate designs of limited originality [8, 28, 29, 61]. This narrow focus can be attributed to the tendency towards taking the path of least cognitive resistance [28, 61]. The fixation effect is especially salient when stimuli are common rather than novel [54, 60, 71]. Some empirical studies on engineering design show the positive effect of remote
stimuli (examples or words) on creativity [13, 14]. Similarly, it is found that analogies can transfer information or relational structures from distant stimuli and lead to more creative outcomes [19, 35, 39]. In addition, it is found that when people are exposed to novel or paradigm-modifying ideas (serving as remote stimuli), they tend to generate such ideas [32, 46, 59, 60, 71]. However, the notion that distant stimuli promote creativity is challenged by many studies. In an engineering design experiment, the patents that were moderately dissimilar were more useful as analogies that stimulate ideas [31], which means that example solutions that were too dissimilar were not very useful. In a study on generating marketing campaign proposals for a beer company, people with access to campaign proposals for dissimilar products generated less creative ideas, compared to people with access to campaign proposals for similar products [3]. In another study, the authors analyzed hundreds of design concepts on an online innovation platform that tracks connections to sources of inspiration [12]. They found that conceptually closer sources are more beneficial for design creativity, compared to conceptually far sources. One disadvantage of using remote stimuli is that such stimuli are often not recognized as relevant [31, 63]. Therefore, there might be an optimal range of stimulus relatedness, within which stimuli are neither too close nor too far to be beneficial [2, 3, 19]. However, to our best knowledge, this notion has not been empirically tested, perhaps due to the difficulty in obtaining stimuli of a spectrum of relatedness. In addition, there are studies showing that the cognitive distance of stimuli has
little impact on the creativity of resulting ideas or solutions [25, 46]. In an experi- ment on the generation of new ice cream flavors, words and pictures that were random or closely related to the task led to the same level of idea creativity [46]. In studying cross-industry innovation, some researchers analyzed the impact of the cognitive distance between the acquired knowledge and the problem to be solved [25]. They found no direct correlation between the cognitive distance and the radicalness of innovation.
1288 WANG AND NICKERSON
In summary, creativity-related theories often indicate the benefit of using stimuli that are distant or unrelated to a creative task. But this notion is both supported and refuted in empirical studies, so no definitive conclusion can be drawn. In addition, previous studies typically have three limitations. First, since different studies often have different ways of defining stimulus relatedness, remote stimuli in one study could be considered moderately related or even closely related in another study [31]. This inconsistency may contribute to the conflicting results. Second, the literature rarely distinguishes unrelated stimuli (as may happen through random selection of concepts from a wide range of knowledge domains) from remotely related stimuli (that can be connected to the focal task through a couple of associative steps). It is presupposed that people will consider and use unrelated stimuli in a sensible way, even though this might not occur [12, 69]. Third, previous experimental studies typically manually collect or generate stimuli (e.g., [10, 45, 46, 49, 58]). Such manual effort is often difficult to perform, and so can lead to heuristics or short cuts that can potentially bias the selection of stimuli. There are a few computational approaches for searching stimuli to support creativity [2, 3, 65, 66], but they typically focus on a specific domain and rely on a database that has been built in advance. There is a lack of generalized and automatic approaches for finding stimuli along a spectrum of relatedness. In this article, we propose a simple and automatic way to find stimuli of various levels of relatedness and, based on it, test our theoretical predictions of the effect of stimulus relatedness, which are explained in the following sections.
Hypotheses Development
A stimulus can be closely, moderately, remotely related, or unrelated to a focal ideation topic. The stimulus relatedness can affect the number of ideas generated, as well as idea novelty and usefulness. First, we contend that the number of ideas generated is affected by stimulus relatedness. The responsiveness to a stimulus is positively related to the similarity between a stimulus and the current cognitive state [17, 48]. The less related a stimulus is, the less likely it is similar to the currently considered concepts or categories. Therefore, it is less likely for people to respond to an unrelated stimulus, internalize it, and use it to stimulate ideas. Adopting the terms of SIAM, using unrelated stimuli as search cues may result in cognitive failure which might terminate the idea generation process. In the perspec- tive of the cognitive network model of creativity, considering a less related stimulus would activate distant knowledge, adding to the disparity among active knowledge and hence increases cognitive load [58]. The amount of cognitive resources for ideation is reduced, which tends to lower the number of ideas. In other words, it is difficult to jump among and connect remote areas in the cognitive network and use the associations to generate ideas on the focal topic. Consistent with this set of arguments, some studies show that original or irrelevant stimuli tend to reduce the number of ideas [37, 69]. Typically ideation consists of the generation of multiple
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1289
preliminary ideas and the development of a final idea [10]. In this case, we contend that the number of preliminary ideas is positively related to stimulus relatedness.
Hypothesis 1: There is a negative relationship between stimulus relatedness and the number of ideas generated.
Stimulus relatedness affects the usefulness of ideas generated. Relevant stimuli can easily activate knowledge that is more applicable to the focal ideation topic [2, 3]. In other words, related stimuli are connected to the target problem in meaningful ways, thereby contributing to the production of useful ideas. Related stimuli tend to promote the search within the current idea category [3] and thus possibly anchor the ideation on existing useful ideas and improve them by adding relevant concepts, features and mental frameworks [10]. Similarly, it is argued that familiar stimuli can improve idea usefulness by infusing meaning, clarity, and legitimacy into ideas [10, 38]. In contrast, we argue that unrelated stimuli reduce idea quantity (as argued in hypothesis 1) and idea usefulness. There are no obvious connections between unrelated stimuli and a focal topic. Consequently people find difficulty in applying the knowledge activated by the stimuli and developing appropriate ideas [2, 3]. The resulted ideas might be unique yet less useful. Indeed, in technology innovation, trying new components and new combinations leads to less useful inventions on average [27]. An unrelated stimulus might even point to an unproductive path that leads to meaningless ideas. In summary, high stimulus relatedness is associated with both high number of ideas and high level of idea usefulness. Combining the two effects, when generating preliminary ideas, highly related stimuli lead to larger number of useful ideas. With a high number of useful raw ideas, the usefulness of the final idea tends to be high as well.
Hypothesis 2: There is a positive relationship between stimulus relatedness and idea usefulness.
As argued in the path of least resistance model [28, 70] and the cognitive network model of creativity [58], when people solve problems or generate ideas, they tend to use familiar examples or previous solutions as a starting point and include many properties from these examples or solutions in ideas developed. Thus, without external stimuli, people often come up with unoriginal ideas. If a stimulus is closely related to the creative task, the stimulus tends to activate knowledge that is also highly related to the task. This highly related knowledge would probably be activated even when people consider the creative task without the stimulus. Consequently, compared to using no stimuli, a closely related stimulus is unlikely to introduce many new elements and improve idea novelty. Consistent with SIAM, when a stimulus is remotely related to a task, the knowledge activated by it will be less related to the task as well. The less relevant knowledge may lead to some original ideas because it adds new cognitive elements and potentially brings in new perspectives. Similarly, it is argued that exposure to new information can help reduce design fixation and improve creativity [63]. Furthermore, the cognitive network model of creativity argues that original solutions result from new
1290 WANG AND NICKERSON
connections among previously unconnected knowledge [58]. Because remotely related stimuli can activate less related knowledge, they increase the possibility of forming such new connections in knowledge. Therefore, remotely related stimuli tend to result in higher novelty in final ideas.
Hypothesis 3: Using stimuli that are remotely related to a creative task leads to higher idea novelty, as compared to using no stimuli.
We posit that totally unrelated stimuli are less effective in promoting idea novelty. The SIAM model suggests that for a stimulus to be effective, it first needs to activate knowledge in mind, and then the active knowledge needs to be processed to produce ideas [52]. Failure in either step would make the stimulus ineffective. When generating ideas on a creative task, people’s minds are necessarily oriented towards elements and associations of the task. Therefore, people are less responsive to a semantically unrelated stimulus [17, 48]. If the stimulus is indeed considered, according to the CNM model, activating very distant knowledge adds to the disparity among active knowledge and hence increases cognitive load [58]. Therefore, the amount of cognitive resources for ideation is reduced, which tends to inhibit idea generation. For example, although people can try to connect unre- lated stimuli to a creative task through analogies, the literature suggests that it is difficult to make analogies from stimuli that are too far away [3, 31]. According to the SIAM model, when people use a stimulus but fail to generate ideas, they may stop using the stimulus or even terminate idea generation all together [52]. Furthermore, even if an unrelated stimulus leads to some preliminary ideas, such ideas might be shallow or unusable [12, 69]. Consequently, these ideas might not be selected for further development and hence have less impact on the ideation process. In summary, when a stimulus is very distant to the point of being unrelated, the activated knowledge is less likely to be used in meaningful ways and the resulting preliminary idea, if any at all, is less likely to be used for further development into a final idea. Therefore, it can be assumed that remotely related stimuli are more effective in promoting idea novelty than both closely related stimuli and unrelated stimuli. Consequently, there is an inverted U-shape relation- ship between stimulus relatedness and the novelty of the final idea.
Hypothesis 4: There is an inverted U-shape relationship between stimulus relatedness and idea novelty.
New Method for Finding Stimuli of Different Levels of Relatedness
We propose an approach that generates different levels of stimuli by following associative links among interconnected webpages in Wikipedia. Wikipedia is the most popular collaborative resource of conceptual knowledge and is useful for measuring different levels of stimulus relatedness [33, 71]. These relatedness measures are all based on the assumption that hypertext links between Wikipedia
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1291
pages indicate semantic relatedness. While other knowledge bases might also be used as starting points (for example, WordNet [26] or CYC [43]), Wikipedia covers essentially all disciplines in many different languages, and so can potentially be used to generate stimuli for almost all problem domains in popular languages. For example, Wikipedia is able to provide concepts related to highly specialized terms, such as micro black hole and guanosine triphosphate. Furthermore, the entries in Wikipedia are interconnected through hypertext links. This allows automatic asso- ciation using computer programs. We use the word link to denote a hypertext link that is in the Introduction section
(before the Contents list) or See also section in a Wikipedia page. We are making two assumptions. First, links usually connect to concepts that are closely related to the initial concept. For example, in the Wikipedia page called innovation, links point to the Wikipedia pages called idea, product, and process, all highly related to innovation. Second, closely related concepts probably appear as links in a Wikipedia page. Using the example above, concepts that are highly related to innovation (such as creativity) probably appear as links in the Wikipedia page of innovation. Based on these two assumptions, we can use Wikipedia links to automatically identify closely related concepts. By iterating this procedure, con- cepts can be found at different levels of relatedness. For instance, starting with the concept innovation, the Wikipedia page for innovation has a link to the Wikipedia page for product. Now starting with the Wikipedia page product, there is a link to the Wikipedia page for raw material, and so on. From innovation to product to raw material, concepts are less and less related to the initial concept innovation. By automating these steps using a computer program, stimuli of various levels of relatedness can be found automatically. We assume that the more associative steps taken by following the links, the less related the concepts are, on average. The next section tests this assumption.
Testing the New Method
A computer program in Python was written to automatically identify links in a Wikipedia page. The program is iterative so that it finds concepts that are spreading out from an initial concept through hypertext linkages, for example, from innovation to product, to raw material, then to lumber. These concepts are labeled as 1st degree concepts, 2nd degree concepts, and 3rd degree concepts, respectively. In our program, we remove 1st degree concepts from 2nd degree concepts, and so on, such that there is no overlap between different degrees of concepts. For any given initial concept, there are often hundreds of 2nd degree concepts and thousands of 3rd degree concepts. To shorten the list of concepts, the computer program selects the concepts whose Wikipedia pages have the largest number of hypertext links. For example, if there are 300 2nd degree concepts, then those 300 concepts are ranked in terms of the number of hypertext links in their respective Wikipedia pages. Only the 30 top ranked concepts were selected. Our
1292 WANG AND NICKERSON
observation is that a Wikipedia page with many links tends to be about a well- known concept (such as meat). In contrast, a Wikipedia page with few links is probably about a concept that few people know about: for example there is an article on tixel. Our program selects concepts with more hypertext links so that they tend to be well known and thus useful in stimulating ideas. This selection method is not the only one or necessarily the best one, but it has the virtue that it is deterministic and will select the same stimuli each time we run the program. Random selection of concepts would result in too much variability and make the method difficult to replicate. Additionally, a Python program was written to search random Wikipedia books.1
A Wikipedia book is a container for a collection of articles. For example, Nobel laureates is a Wikipedia book corresponding to the webpage http://en.wikipedia. org/wiki/Book:Nobel_laureates. The Python program uses a function provided by Wikipedia that retrieves at random one Wikipedia book (http://en.wikipedia.org/ wiki/Special:Random/Book). The reason why random Wikipedia books are used as random stimuli instead of just random Wikipedia articles is because Wikipedia books are more likely to be well known than just random Wikipedia articles. For example, there are many Wikipedia articles are about relatively obscure mountains and rivers, and these are not likely to be useful for stimulating creativity. In testing the effectiveness of the new method, we tested both concepts known to
the general public and concepts used only by certain professions. Six initial con- cepts in total were used, two common objects (brick and kitchen), two concepts in materials science (materials science and polymer engineering) and two information systems concepts (management information systems and business intelligence). From each initial concept, the computer program identified all the 1st degree concepts, 30 2nd degree concepts and 30 3rd degree concepts (the method used to select the 30 concepts was explained before). Also, another Python program identified 30 random Wikipedia books as random concepts. All the relatedness measurements were made on the scale of 1 to 7 (1 being totally unrelated, 7 being highly related). As an example, every concept found for the concept “brick” was evaluated with regard to its relatedness to “brick.” For the common object concepts, because they are known to the general public, concept relatedness was judged by ten workers employed through Amazon Mechanical Turk, an online work market- place. For the concepts in materials science, the relatedness evaluation was done by two Ph.D. students in that field. For the concepts in management information systems, the relatedness evaluation was performed by two Master’s students in the field. The ratings were averaged across different raters after checking the level of inter-rater agreement. The results are shown in Table 1. First degree concepts are closely related to brick
(Mean = 6.10, SD = 0.94) while 2nd, 3rd degree concepts and random concepts are less and less related (in this order, Mean = 4.14, SD = 1.83; Mean = 3.04, SD = 1.48; Mean = 1.36, SD = 0.41). Based on a one-way analysis of variance (ANOVA), the relatedness is significantly different across different groups (F(3,102) = 49.8, p < 0.001). For all the
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1293
six concepts tested, all the t-tests between adjacent groups of concepts show significant difference in relatedness (p < 0.05). Because we ran three t-tests for each topic, we further used the Benjamini-Hochberg procedure as the correction for multiple compar- isons [9]. The three p-values for the three t-tests were ranked from the lowest to the highest. Each of these three p-values then was compared to the corresponding Benjamini-Hochberg critical value (0.05/3, 0.10/3, and 0.05, respectively). For all of our six topics, our p-values are smaller than the corresponding Benjamini-Hochberg critical values. Consequently, all the comparisons in Table 1 are significant using this correction procedure. Therefore, the same pattern of decreasing relatedness is found for all six initial concepts. These findings suggest that 1st degree concepts are closely related to the initial concept, 2nd degree concepts are moderately related, 3rd degree concepts are remotely related, while random concepts are unrelated.
Study 1
Methods
Study Design and Participants
In this experiment, two hundred USA-based workers from Amazon Mechanical Turk were employed to generate ideas for designing a mobile app for improving the physical fitness of college students. The maximum time on task allowed was 30 minutes. Each worker was offered and paid one US dollar for completing the task, and was told in the task description that the best idea would be rewarded with a $50 bonus. On average, these participants were 33.6 years old (SD = 10.8 years), spent 1003 seconds on the task (SD = 404 seconds). A total of 61.7 percent of the participants were female. The Wikipedia concept physical fitness was used to identify 1st, 2nd, and 3rd degree
concepts. The computer program collected seven 1st degree concepts, fifty 2nd degree concepts, and fifty 3rd degree concepts (only the top 50 concepts with the highest number of hypertext links in their Wikipedia pages were selected). Also collected were 50 random Wikipedia book concepts. This experiment adopted a between-subject design where the workers were randomly assigned into 5 conditions, each condition with 40 participants. In all conditions, each worker was asked to generate some preliminary ideas about designing a fitness app for college students, then provide one final idea. In the control condition, the workers saw no Wikipedia concepts. In the unrelated condition, each worker contributed some preliminary ideas, then saw 3 random Wikipedia book concepts (randomly chosen from the 50 random concepts collected) as stimuli. An example of using an unrelated stimulus was provided to the participants (Online Supplemental Appendix 1 has the full instructions). After each stimulus, the workers were asked to generate additional preliminary ideas based on that specific stimulus. At the end, the workers were asked to provide a final idea for us to consider. We chose this design to mimic design environments that often practice a process of generating initial ideas and then developing a final idea [10]. The close, moderate, and remote condition were the same as the unrelated condition, except that
1294 WANG AND NICKERSON
T ab le
1 . C o n ce p t R el at ed n es s
In it ia l C o n ce p t (R el ia b il it y o f
re la te d n es s d at a,
A N O V A
re su lt fo r re la te d n es s)
C o n d it io n
M ea n (S D )
T -t es t w it h th e p re v io u s g ro u p
E x am
p le
C o n ce p ts
B ri ck
(I C C (1 ,1 0 ) = 0 .9 5 .
A N O V A :
F (3 ,1 0 2 ) = 4 9 .8 , p < 0 .0 0 1 .)
1 s t d e g re e
6 .1 0 (0 .9 4 )
A d o b e , C la y
2 n d d e g re e
4 .1 4 (1 .8 3 )
t( 4 4 ) = 4 .8 , p < 0 .0 0 1
C a st le , C a rp e n tr y
3 rd d e g re e
3 .0 4 (1 .4 8 )
t( 5 6 ) = 2 .6 , p = 0 .0 1 3
U n iv e rs ity , M in in g
R a n d o m
1 .3 6 (0 .4 1 )
t( 3 3 ) = 6 .0 , p < 0 .0 0 1
M ic ro fa ct o ry , T h o m a s Is la n d
K itc h e n
(I C C (1 ,1 0 ) = 0 .9 7 .
A N O V A :
F (3 ,1 0 7 ) = 5 7 .9 3 , p < 0 .0 0 1 .)
1 s t d e g re e
5 .9 4 (1 .1 9 )
S in k,
D is h w a sh
e r
2 n d d e g re e
3 .7 6 (2 .0 6 )
t( 4 9 ) = 4 .5 4 , p < 0 .0 0 1
S p o ila
g e , C a n n in g
3 rd d e g re e
2 .3 2 (1 .7 8 )
t( 4 2 ) = 3 .5 0 , p = 0 .0 0 1
T e ch
n o lo g y,
A rt m o ve
m e n t
R a n d o m
1 .2 2 (0 .4 4 )
t( 3 3 ) = 5 .9 1 , p < 0 .0 1
M ic ro fa ct o ry , T h o m a s Is la n d
M a te ri a ls
S ci e n ce
(I C C (2 ,2 ) = 0 .8 6 .
A N O V A :
F (3 ,1 1 6 ) = 1 0 5 .8 8 , p < 0 .0 0 1 .)
1 s t d e g re e
6 .1 8 (0 .8 7 )
M a tt e r, M in e ra lo g y
2 n d d e g re e
5 .2 5 (1 .4 1 )
t( 4 8 ) = 3 .0 9 , p = 0 .0 0 3
M a n u fa ct u ri n g , M e d ic in e
3 rd d e g re e
3 .7 5 (1 .8 5 )
t( 5 4 ) = 3 .5 3 , p < 0 .0 0 1
F o o d , B o ta n y
R a n d o m
1 .8 8 (1 .1 9 )
t( 4 9 ) = 4 .6 6 , p < 0 .0 0 1
T h e W h ite
V ik in g , 2 9 th
A rm
y P o ly m e r E n g in e e ri n g
(I C C (2 ,2 ) = 0 .7 8 .
A N O V A :
F (3 ,9 4 ) = 4 1 .6 6 , p < 0 .0 0 1 .)
1 s t d e g re e
6 .3 1 (1 .0 0 )
P o ly m e r S ci e n ce
, E n g in e e ri n g
2 n d d e g re e
4 .9 5 (1 .7 6 )
t( 2 0 ) = 2 .8 6 , p = 0 .0 1
A ro m a tic s,
C o rn
3 rd d e g re e
3 .4 3 (1 .5 9 )
t( 5 7 ) = 3 .5 0 , p < 0 .0 0 1
E n e rg y,
P h a rm
a ce
u tic a ls
R a n d o m
1 .6 0 (0 .4 6 )
t( 3 4 ) = 6 .0 6 , p < 0 .0 0 1
T ru ck e rs , W ill ia m
G re in e r
M a n a g e m e n t In fo rm
a tio
n S ys te m s
(I C C (2 ,2 ) = 0 .7 4 .
A N O V A :
F (3 ,1 1 1 ) = 3 8 .7 , p < 0 .0 0 1 .)
1 s t d e g re e
5 .5 2 (1 .1 6 )
A cc o u n tin
g , B u si n e ss
R u le
2 n d d e g re e
4 .7 8 (1 .2 8 )
t( 5 3 ) = 2 .2 6 , p = 0 .0 2 8
A u d iti n g , C lo u d C o m p u tin
g 3 rd d e g re e
3 .2 1 (1 .2 4 )
t( 5 8 ) = 4 .8 1 , p < 0 .0 0 1
B io fu e l, In d u st ri a l R e vo
lu tio
n R a n d o m
2 .5 7 (0 .9 2 )
t( 5 4 ) = 2 .2 8 , p = 0 .0 2 7
P e n a n g , S e a n F ra se
r
B u si n e ss
In te lli g e n ce
(I C C (2 ,2 ) = 0 .8 0 .
A N O V A :
F (3 ,1 1 6 ) = 8 4 .1 , p < 0 .0 0 1 .)
1 s t d e g re e
6 .4 0 (0 .6 8 )
A n a ly tic s,
B u si n e ss
R e p o rt in g
2 n d d e g re e
5 .2 3 (1 .4 3 )
t( 4 2 ) = 4 .0 4 , p < 0 .0 0 1
H a d o o p , In fo rm
a tio
n 3 rd d e g re e
3 .1 6 (1 .4 0 )
t( 5 8 ) = 5 .6 9 , p < 0 .0 0 1
L o g ic , L in u x G a m in g
R a n d o m
2 .3 2 (0 .7 3 )
t( 4 4 ) = 2 .8 9 , p = 0 .0 0 6
T h e B la ck
P a n th e r,
M a ry
R o b in so
n
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1295
the stimuli were randomly chosen 1st, 2nd, and 3rd degree concepts, respectively. Altogether, there are three types of ideas: initial preliminary ideas (before seeing any stimuli), stimulated preliminary ideas (upon seeing stimuli), and final ideas. The control condition does not have stimulated preliminary ideas because no stimuli were given.
Dependent Variables
The experiment generated 200 final ideas. On average, each idea has 99 words (SD = 55). The final ideas were evaluated by two professional app developers with regard to idea novelty and usefulness. Both app developers have college degrees and at least four years’ experience in mobile app development. Therefore, they have experience on both the customer side (as college students) and on the designer side (as app developers). We notice that Dean et al. [20] advocate for evaluating ideas in four dimensions: novelty, feasibility, relevance and specificity. In practice, when people do adopt this method, they often omit specificity and instead focus on the former three dimensions [22]. In our measurement, idea usefulness is similar to feasibility plus relevance in Dean et al. [20]. Specifically, in our idea evaluation, novelty is defined as the degree to which an idea is original and paradigm modifying [20]. Usefulness is the degree to which the idea is feasible and effective in improving college students’ physical fitness. Novelty and usefulness were rated on the scale of 1 to 7 (1 being not novel/useful at all, 7 being highly novel/useful). The two raters first browsed the online Apple app store to become familiar with existing fitness apps. Afterward, they independently rated all the final ideas, which were randomly ordered. The intraclass correlation coefficients show sufficient levels of agreement in their ratings, therefore their ratings were averaged (ICC(2,2) = 0.79 for both novelty and usefulness). The average scores are called final novelty and final usefulness, denoting the novelty and usefulness of the final ideas. In addition, all the 1956 preliminary ideas were evaluated on novelty and useful-
ness by one of the raters. To ensure the quality of his ratings, the second rater also evaluated 100 preliminary ideas. For those 100 preliminary ideas, two raters agree on 87 ideas in novelty assessment and 86 ideas on usefulness assessment. Agreement means the scores are no more than 1 point different. The two raters discussed and reconciled their differences in the evaluation of those 100 ideas before the first rater evaluated all the remaining preliminary ideas. The number of preliminary ideas generated by each worker upon the exposure to three concepts is called stimulated fluency. Stimulated novelty is the number of novel preliminary ideas generated upon seeing three concepts (novelty score larger than 4, the scale midpoint). Stimulated usefulness is defined in the same manner.
Independent Variables
The relatedness of all the concepts to physical fitness was evaluated by 20 workers from Mechanical Turk (ICC(1,20) = 0.97). The scores from different raters were
1296 WANG AND NICKERSON
averaged. The average of the relatedness of the three concepts that each participant saw was calculated and then centered for the regression purpose. This centered variable is called concept relatedness.
Control Variables
The number of preliminary ideas generated at the beginning without seeing any concepts is called initial fluency. Initial novelty is defined as the number of novel preliminary ideas (novelty score being more than 4, the scale midpoint) generated before seeing any concepts. Initial usefulness is defined similarly. At the end of the survey, all the participants were given a set of questions measuring knowledge of physical fitness, knowledge of mobile apps, and intrinsic motivation in this task. The questions for knowledge of physical fitness and mobile apps were adapted from the domain-specific consumer knowledge scale [44, 50]. We used the consumer knowl- edge scale because the participants were from the general public and therefore more similar to potential consumers, instead of developers, of the designed app. Knowledge of mobile apps and physical fitness have a Cronbach’s alpha of 0.848 and 0.880, respectively. Intrinsic motivation was measured with two items: “did you find the task interesting” and “was it enjoyable to work on” [2]. For both the measures of knowledge and intrinsic motivation, a Likert scale was used with 1 defined as strongly disagree and 5 defined as strongly agree. Having a high level of intrinsic motivation is considered important for creative work [40]. In this experiment, the intrinsic motivation of the participants was indeed high (M = 4.33, SD = 0.64).
Results
The relatedness of different degrees of concepts is shown in Table 2. A one-way ANOVA shows that concept relatedness is different across different groups (F(3,153) = 108.3, p < 0.001). Using the Benjamini–Hochberg procedure as the correction for multiple comparisons, the three t-tests are still significant. Again, concept relatedness decreases with the number of associative steps taken, as predicted. The descriptive statistics and correlations between all the main variables are shown in
Table A in Online Supplemental Appendix 3. Concept relatedness is correlated with stimulated fluency (r(158) = 0.239, p < 0.01). This is aligned with hypothesis 1 which
Table 2. Concept Relatedness for Study 1
Concepts Mean (SD) T-test with the previous group Example concepts
1st degree 6.76 (0.23) Nutrition, Bodybuilding 2nd degree 5.20 (1.10) t(48) = 8.77, p < 0.001 Fruit, Immune system 3rd degree 3.57 (1.33) t(95) = 6.67, p < 0.001 Surgery, Fungus Random 1.71 (0.79) t(80) = 8.50, p < 0.001 George Washington, Gospel
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1297
indicates a positive relationship between concept relatedness and idea quantity. Concept relatedness is also correlated with stimulated usefulness (r(158) = 0.453, p < 0.01) and final usefulness (r(158) = 0.158, p < 0.05). This is aligned with hypothesis 2 that proposes a positive relationship between concept relatedness and final usefulness. These correla- tions alone are not sufficient support for the hypotheses but they do provide information that is consistent with the results of the regressions, reported below. Hypothesis 4 proposed an inverted U-shape between stimulus relatedness and final novelty. Our results show that the correlation between concept relatedness and final novelty is not significant (r(158) = −0.096, p = 0.227). Idea novelty and idea usefulness in different conditions are presented in Table 3.
A one way ANOVA tests show that final novelty and final usefulness are both different across conditions (F(4,195) = 3.452, p = 0.009 for novelty; F(4,195) = 2.668, p = 0.034 for usefulness). ATukey post hoc test shows that final usefulness in the close condition (M = 4.988, SD = 0.755) is higher than the moderate condition (M = 4.275, SD = 1.311, p = 0.015). A t-test shows that final novelty in the remote condition (M = 4.75, SD = 1.11) is significantly higher than that in the control condition (M = 3.85, SD = 1.06; t (78) = 3.70, p < 0.001). This is consistent with hypothesis 3. The remote condition has 30 final ideas that score higher than 4 (the scale midpoint) in novelty. The control, close, moderate and unrelated conditions have 15, 20, 25, and 27 such ideas. Regression analysis is used to further test all the hypotheses while considering
control variables, including time on task, knowledge of mobile apps, knowledge of physical fitness, and intrinsic motivation (Table 4). Four dummy variables (close, moderate, remote, unrelated) represent the conditions in the experiment. For the control condition, all these dummy variables are set to zero. In the analyses, we first regressed a dependent variable on the control variables, such as time on task and knowledge of mobile apps. These variables form the first block. The second block contains either dummy variables for the experimental conditions or concept relat- edness and its squared term (for testing inverted U-shape relationship). The change in explained variance (ΔR2 in Table 4) indicates whether the experimental condi- tions explain any variance in a dependent variable, over and above the variance explained by the control variables.
Table 3. Idea Novelty, Usefulness, and Quantity in Different Conditions in Study 1
Condition
Final Novelty
Mean (SD)
Final Usefulness Mean (SD)
Stimulated Fluency
Mean (SD)
Stimulated Novelty
Mean (SD)
Stimulated Usefulness Mean (SD)
Control 3.85 (1.06) 4.53 (0.98) NA NA NA Close 4.35 (1.29) 4.99 (0.76) 8.70 (1.24) 3.45 (2.15) 4.53 (2.14) Moderate 4.55 (1.34) 4.28 (1.31) 8.23 (1.85) 4.38 (2.20) 3.38 (2.19) Remote 4.75 (1.11) 4.61 (0.91) 7.53 (2.29) 3.55 (2.74) 1.65 (1.64) Unrelated 4.69 (1.34) 4.51 (0.97) 7.35 (3.02) 4.43 (3.00) 2.03 (2.01)
1298 WANG AND NICKERSON
T ab le
4 . R eg re ss io n A n al y si s T es ti n g th e H y p o th es es
in S tu d y 1
D ep en d en t V ar ia b le
S ti m u la te d
F lu en cy
(H 1 )
F in al
U se fu ln es s
(H 2 )
S ti m u la te d
U se fu ln es s
F in al
N o v el ty
(H 3 )
F in al
N o v el ty
(H 4 )
S ti m u la te d
N o v el ty
In te rc e p t
5 .2 9 3 ** *
3 .7 4 5 ** *
− .0 2 4
2 .7 3 6 ** *
3 .5 3 7 ** *
3 .3 1 9 *
B lo c k 1 : C o n tr o l V a ri a b le s
T im
e o n ta sk
.0 3 7
− .0 0 2
.0 3 5
.0 2 9 *
.0 3 7 *
.0 4 3
K n o w le d g e o f m o b ile
a p p s
.5 2 1 †
.2 5 8 *
.3 6 1
− .0 7 2
− .1 1 4
.3 5 1
K n o w le d g e o f p h ys ic a l fit n e ss
− .2 1 4
− .1 8 7
− .1 3 7
.0 4 8
.0 5 0
− .5 9 7 †
In tr in si c m o tiv a tio
n .2 0 9
.0 1 7
.1 2 5
.1 4 1
.1 1 2
.0 6 4
In iti a l flu
e n cy
− .0 4 4
In iti a l u se
fu ln e ss
.2 9 6 ** *
.2 8 4 *
In iti a l n o ve
lty .1 8 2 *
.2 6 6 **
.3 8 6 *
B lo c k 2
C o n ce
p t re la te d n e ss
.3 2 4 **
.1 0 2 *
.6 0 2 ** *
− .0 7 0
− .0 9 4
C o n ce
p t re la te d n e ss
sq u a re d
.0 1 2
.0 3 2
.1 3 9 **
− .0 2 5
.0 1 2
C lo se
co n d iti o n
.3 9 6
M o d e ra te
co n d iti o n
.6 7 5 *
R e m o te
co n d iti o n
.8 2 6 **
U n re la te d co
n d iti o n
.7 2 5 *
R 2
.1 0 6
.1 8 0
.2 9 4
.1 3 8
.1 2 1
.0 8 6
F F (7 ,1 5 2 ) = 2 .5 7 9 *
F (7 ,1 5 2 ) = 4 .7 6 8 ** *
F (7 ,1 5 2 ) = 9 .0 2 4 ** *
F (9 ,1 9 0 ) = 3 .3 8 6 **
F (7 ,1 5 2 ) = 2 .9 9 0 **
F (7 ,1 5 2 ) = 2 .0 3 4 †
Δ R 2 (Δ F )
.0 7 5 (6 .3 6 2 )* *
.0 3 8 (3 .5 6 6 )*
.2 5 3 (2 7 .2 0 7 )* **
.0 4 6 (2 .5 3 2 )*
.0 1 2 (1 .0 5 7 )
.0 0 5 (. 4 5 1 )
N o te s: N
= 2 0 0 . V al u es
re p re se n t u n st an d ar d iz ed
re g re ss io n co ef fi ci en ts , w it h st an d ar d er ro rs
in p ar en th es es . † p < .1 0 . * p < .0 5 . * * p < .0 1 . * * * p < .0 0 1 ; tw o -t ai le d p -v al u es .
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1299
As Table 4 indicates, concept relatedness does explain significant variance over and above the variance explained by the control variables, for stimulated fluency (B = .324, t = 3.539, p = .001), stimulated usefulness (B = .602, t = 7.246, p < 0.001) and final usefulness (B = .102, t = 2.534, p = .012). Therefore, concept relatedness is indeed positively related to the number of ideas and idea usefulness. Hypotheses 1 and 2 are supported. Supporting Hypothesis 3, the regression shows that the remote condition explains significant variance in final novelty beyond the variance explained by the control variables (B = .826, t = 2.855, p = 0.005). When final novelty and stimulated novelty are used as dependent variables, concept
relatedness and its quadratic term have nonsignificant coefficients and do not explain significant variance beyond the control variables. The results provide no evidence for Hypothesis 4 that predicts an inverted U-shape relationship. The remote condition has the highest final novelty (M = 4.75, SD = 1.11), but it is not significantly higher than the moderate condition (M = 4.55, SD = 1.34; t(78) = 0.728, p = 0.469) or the unrelated condition (M = 4.69, SD = 1.34; t(78) = 0.227, p = 0.821).
Study 2
We conducted a second study to test the generalizability of our results. Specifically, we recruited college students, instead of Mechanical Turk workers, as participants and used a different ideation task. In addition, the second study made some changes to addresses three potential concerns about study 1. First, in study 1, the number of candidate concepts is different across conditions. For example, in the close condi- tion, the stimulus concepts were randomly selected from 7 first degree concepts (there are only 7 first degree concepts). In the moderate condition, the stimulus concepts were randomly selected from 50 second degree concepts. It is possible that the number of candidate stimuli has an effect on idea diversity and potentially idea novelty. In study 2, we randomly selected stimuli from the same number of candidate stimuli across conditions. Second, in study 1, the participants in the experimental conditions generated additional preliminary ideas based on stimulus concepts, while the control condition did not have this step. The absence of this step might have meant that those in the control condition exerted less effort. It is also possible that they might have exerted equal or more effort until they became stuck due to lack of external stimulation. To create more symmetry in the conditions, in study two, the participants in the control condition were asked to generate three extra sets of preliminary ideas without external stimuli. This step was parallel to the step in the experimental conditions where the participants generated three sets of ideas based on three stimuli. Third, in study 1 the stimuli are all regular Wikipedia concepts except that the unrelated concepts are Wikipedia book concepts. It is unclear whether Wikipedia book concepts are somehow different from regular Wikipedia concepts, which could result in undesirable variability. In study 2, we used the fifth degree concepts as unrelated stimuli to eliminate this concern.
1300 WANG AND NICKERSON
Methods
The design of study 2 is highly similar to study 1, with the following differences. In this experiment, two hundred undergraduate students from a university on the east coast of the United States were recruited as participants. They were instructed to generate ideas on designing a mobile app for online shopping by college students. The maximum time on task allowed was 60 minutes. The concept online shopping was used to identify 1st, 2nd, 3rd, and 5th degree concepts. There are twenty 1st degree concepts. The computer program also collected twenty 2nd, 3rd, and 5th degree concepts by selecting the top twenty concepts with the highest number of hypertext links in their Wikipedia pages. One of the two idea evaluators is different from study 1. The intraclass correlation
coefficients for the ratings of the two raters are ICC(2,2) = 0.72 and 0.66 for novelty and usefulness, respectively. These numbers indicate acceptable levels of interrater agreement [15]. The relatively modest level of agreement is not uncom- mon in creativity judgment (as shown in [7, 10]), perhaps because raters have different experience bases and hence different associative networks. Out of the two hundred final ideas generated, eight ideas were considered by both raters as irrelevant. Therefore, only 192 participants’ data were used. On average, each idea has 103 words (SD = 49). Online Supplemental Appendix 2 describes the instructions of the survey. On average, the student participants were 21.82 years old (SD = 1.41 years), spent 1,752 seconds on the task (SD = 718 seconds). Fifty-two percent of the participants were female. The average score of intrinsic motivation was 3.91 (SD = 0.98). We measured the participant’s knowledge of shopping apps using the same scale mentioned earlier with a Cronbach’s alpha of 0.847.
Results
The relatedness of different degrees of concepts is shown in Table 5. A one-way ANOVA shows that concept relatedness is different across different groups (F(3,76) = 42.6, p < 0.001). Using the Benjamini–Hochberg procedure as the correction for
Table 5. Concept Relatedness for Study 2
Concepts Mean (SD)
T-test with the previous group Example concepts
1st degree 5.86 (0.96) Credit card, Website 2nd degree 4.34 (1.37) t(34) = 3.97, p < 0.001 Internet Explorer, Shopping
streets 3rd degree 2.87 (1.34) t(38) = 3.36, p = 0.002 Concert hall, Emoji 5th degree 2.06 (0.66) t(28) = 2.36, p = 0.026 Shakespeare, Soviet scientists
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1301
multiple comparisons, the three t-tests are still significant. The four groups corre- spond closely to, moderately, remotely related concepts, and unrelated concepts. The descriptive statistics and correlations between all the main variables are
shown in Table B in Online Supplemental Appendix 3. Concept relatedness is correlated with stimulated fluency (r(150) = 0.195, p < 0.01). This is aligned with hypothesis 1 which indicates a positive relationship between concept relatedness and idea quantity. Concept relatedness is also correlated with stimulated usefulness (r(150) = 0.272, p < 0.01), final usefulness (r(150) = 0.178, p < 0.05). This is aligned with Hypothesis 2 that proposes a positive relationship between concept relatedness and final usefulness. These correlations alone are not sufficient support for the hypotheses but they do provide information that is consistent with the results of the regressions, reported in the following section. Hypothesis 4 proposes an inverted U-shape between stimulus relatedness and final novelty. Our results show that the correlation between concept relatedness and final novelty is not significant (r(150) = −0.041, p = 0.612), again consistent with the regression results. Idea novelty and idea usefulness in different conditions are presented in
Table 6. Final novelty does not appear different considering all the conditions (: F(4,187) = 1.491, p = 0.207). The control, close, and unrelated conditions all have ANOVA 18 final ideas scoring higher than 4 in novelty. The moderate and remote conditions both have 21 such final ideas. A one way ANOVA shows that final usefulness is significantly different across conditions (F (4,187) = 2.638, p = 0.035). A Tukey post hoc test shows that final usefulness is higher in the close condition (M = 3.946, SD = 0.949) than the unrelated condition (M = 3.338, SD = 1.140, p = 0.048). In the control condition, there are three steps generating additional preliminary ideas,
parallel to idea stimulation by three Wikipedia concepts in other conditions. Stimulated novelty for the control condition refers to the number of novel preliminary ideas generated in these steps. An ANOVA indicates that stimulated novelty is different across conditions (F(4,187) = 3.805, p = 0.005). A t-test shows that stimulated novelty is significantly higher in the remote condition (M = 2.89, SD = 2.54), compared to the control condition (M = 1.83, SD = 1.66; t(59) = 2.136, p = 0.037).
Table 6. Idea Novelty, Usefulness, and Quantity in Different Conditions in Study 2
Condition
Final Novelty
Mean (SD)
Final Usefulness Mean (SD)
Stimulated Fluency
Mean (SD)
Stimulated Novelty
Mean (SD)
Stimulated Usefulness Mean (SD)
Control (n = 40) 4.04 (1.17) 3.83 (1.02) 5.43 (1.62) 1.83 (1.66) 2.63 (1.60) Close (n = 37)
4.05 (1.21) 3.95 (0.95) 6.89 (1.85) 2.11 (1.90) 3.54 (1.61)
Moderate (n = 39) 4.40 (1.42) 3.58 (0.70) 5.97 (1.72) 2.51 (2.06) 2.62 (2.02) Remote (n = 36) 4.60 (1.01) 3.46 (0.94) 6.11 (1.79) 2.89 (2.54) 2.75 (1.76) Unrelated (n = 40) 4.14 (1.25) 3.34 (1.14) 5.60 (1.50) 3.45(2.10) 1.95(1.54)
1302 WANG AND NICKERSON
Using the same method as study 1, regression analysis is used to further test all the hypotheses considering the control variables (shown in Table 7).2 As Table 7 indicates, concept relatedness does explain significant variance over and above the variance explained by the control variables, for stimulated fluency (B = .180, t = 2.026, p = .045), stimulated usefulness (B = .280, t = 3.152, p = 0.002) and final usefulness (B = .099, t = 1.943, p = .054). Therefore, concept relatedness is indeed positively related to the number of ideas and idea usefulness. Hypotheses 1 and 2 are supported. The regression shows that the remote condition explains marginally significant
variance in final novelty (B = .511, t = 1.944, p = 0.053) and significant variance in stimulated novelty (B = 0.970, t = 2.084, p = 0.039) beyond the variance explained by the control variables. So Hypothesis 3 is supported in part. When final novelty and stimulated novelty are used as dependent variables, the quadratic term of concept relatedness has nonsignificant coefficients. The results provide no evidence for Hypothesis 4 that predicts an inverted U-shape relationship. The remote condi- tion has the highest final novelty (M = 4.60, SD = 1.01), but it is not significantly higher than the moderate condition (M = 4.40, SD = 1.42; t(69) = 0.709, p = 0.481) or the unrelated condition (M = 4.14, SD = 1.25; t(74) = 1.753, p = 0.084).
Discussion
Influence of Stimuli on Idea Quantity and Usefulness
Our first hypothesis states that the number of preliminary ideas is positively related to stimulus relatedness. For study 1 and 2, the correlation and regression analysis con- sistently support this hypothesis. As argued earlier, it is difficult to respond to a less related stimulus and generate many ideas based on it. This result is also consistent with previous studies where irrelevant or original stimuli led to lower number of ideas [37, 69]. Hypothesis 2 states that final usefulness is positively related to stimulus related- ness. In both study 1 and 2, correlation and regression analysis alike show this positive relationship for both final usefulness and stimulated usefulness. The exception is that the regression coefficient for concept relatedness is only marginally significant for final usefulness in study 2. Overall, there is support for Hypothesis 2. Highly related stimuli bring relevant, appropriate and meaningful associations into the thinking process, leading to useful ideas. Our results are consistent with Berg [10] where familiar concepts led to more useful ideas.
Influence of Stimuli on Idea Novelty
Hypothesis 3 contends that remotely related stimuli are effective in increasing idea novelty, compared to using no stimuli. Regression results support the hypothesis in both experiments. In study 2, although the regression coefficient for the remote condition is only marginally significant for final novelty, the coefficient is
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1303
T ab le
7 . R eg re ss io n A n al y si s T es ti n g th e H y p o th es es
in S tu d y 2
D ep en d en t V ar ia b le
S ti m u la te d
F lu en cy
(H 1 )
F in al
U se fu ln es s
(H 2 )
S ti m u la te d
U se fu ln es s
F in al
N o v el ty
(H 3 )
S ti m u la te d
N o v el ty
F in al
N o v el ty
(H 4 )
S ti m u la te d
N o v el ty
In te rc e p t
5 .7 5 2 ** *
3 .9 0 5 ** *
.6 5 6
3 .1 9 9 ** *
.9 3 7
3 .4 3 2 ** *
2 .3 9 7 *
B lo c k 1 : C o n tr o l V a ri a b le s
T im
e o n ta sk
0 0
.0 0 1 **
0 0
0 0
K n o w le d g e o f sh
o p p in g a p p s
− .2 9 5 †
− .0 5 0
.0 3 2
− .0 8 2
− .0 3 6
− .0 6 6
− .0 4 2
In tr in si c m o tiv a tio
n − .0 6 7
− .0 8 4
.0 3 3
.2 5 8 **
.1 9 2
.1 9 7 *
.0 3 8
In iti a l flu
e n cy
.3 8 6 ** *
In iti a l u se
fu ln e ss
.2 0 7 **
.4 3 1 ** *
In iti a l n o ve
lty .4 8 1 ** *
.6 9 1 ** *
.4 9 2 ** *
.7 6 6 ** *
B lo c k 2
C o n ce
p t re la te d n e ss
.1 8 0 *
.0 9 9 †
.2 8 0 **
− .0 6 8
− .3 5 7 **
C o n ce
p t re la te d n e ss
sq u a re d
.1 0 6 †
.0 1 0
.0 4 7
− .0 4 2
− .0 0 1
C lo se
co n d iti o n
− .2 5 8
− .1 2 9
M o d e ra te
co n d iti o n
.1 7 0
.4 2 1
R e m o te
co n d iti o n
.5 1 1 †
.9 7 0 *
U n re la te d co
n d iti o n
.0 1 7
1 .4 8 2 **
R 2
.1 8 5
.1 0 4
.2 2 9
.2 2 1
.1 8 2
.2 1 1
.1 6 7
F F (6 ,1 4 5 ) =
5 .4 7 8 ** *
F (6 ,1 4 5 ) =
2 .7 9 3 *
F (6 ,1 4 5 ) =
7 .1 9 4 ** *
F (8 ,1 8 3 ) =
6 .4 9 8 ** *
F (8 ,1 8 3 ) =
5 .1 0 3 ** *
F (6 ,1 4 5 ) =
6 .4 7 4 ** *
F (6 ,1 4 5 ) =
4 .8 4 9 ** *
Δ R 2 (Δ F )
.0 5 8 (5 .1 7 1 )* *
.0 2 9 (2 .3 6 7 ) †
.0 7 2 (6 .7 5 3 )* *
.0 3 8 (2 .2 1 8 ) †
.0 8 1 (4 .5 4 1 )* *
.0 1 7 (1 .5 7 5 )
.0 6 4 (5 .6 0 4 )* *
N o te s:
N = 1 9 2 . V al u es
re p re se n t u n st an d ar d iz ed
re g re ss io n co ef fi ci en ts , w it h st an d ar d er ro rs
in p ar en th es es .
† p < .1 0 . * p < .0 5 . * * p < .0 1 . * * * p < .0 0 1 ; tw o -t ai le d p -v al u es .
1304 WANG AND NICKERSON
significant for stimulated novelty. Overall, the results are in line with the common notion that remotely related stimuli introduce new cognitive elements, reduce fixation, and lead to high novelty. Hypothesis 4 suggests that final novelty has an inverted-U shape relationship
with stimulus relatedness. The regression analyses in study 1 and 2 show no evidence for this relationship, either for stimulated novelty or final novelty. Stimulated novelty is the number of stimulated preliminary ideas that are novel (scoring higher than 4). Therefore, stimulated novelty can be influenced by two variables: the number of stimulated ideas and the average novelty of stimulated ideas. We conducted a post-hoc analysis by calculating the average novelty of stimulated ideas. In both experiments, the average novelty of stimulated ideas is negatively correlated with stimulus relatedness (r(158) = −0.227, p < 0.01 for study 1; r(150) = −0.206, p < 0.05 for study 2). Therefore, the less related the stimuli, the higher the average novelty of the preliminary ideas. However, as hypothesis 1 states, low stimulus relatedness also leads to a lower number of preliminary ideas. Combining these two effects, the relationship between stimulus relatedness and stimulated novelty becomes complicated. While this relationship is negative in study 2 (r(150) = −0.200, p < 0.05), the correlation is non- significant in study 1 (r(158) = −0.095, p = 0.233). In both studies, stimulated novelty is highest in the unrelated condition while final
novelty is highest in the remote condition. The discrepancy suggests that people may not integrate every preliminary idea in their final ideas. We noticed that unrelated stimuli may lead to preliminary ideas that are superficial or less meaningful and consequently not used to generate final ideas. For example, in study 1 about fitness apps, after a participant saw the unrelated stimulus assault rifles, a preliminary idea was include gun safety tips in the app. This idea was not included in the final idea, presumably due to low relevance. Because unrelated stimuli might result in preliminary ideas that are abandoned later, high stimulated novelty in the unrelated condition does not necessarily translate to high final novelty. Our data also show that the difference in final novelty across conditions is not
large enough to lead to a significant inverted-U shape relationship, even though the trend seems to be consistent with this shape. There might be two reasons behind the small differences in final novelty. First, people can still generate novel ideas using 1st and 2nd degree concepts. The semantic network in long term memory is very richly connected. While a Wikipedia concept may have ten 1st degree concepts, a concept in mind may be directly connected to hundreds of other concepts. Therefore, one step of association in mind can still be a semantic leap. As a result, it is still possible for people to start with a 1st degree concept and generate novel ideas. Second, it is usually difficult to generate extremely novel ideas. Consequently, most ideas have moderate levels of novelty. The literature also provides little direct evidence of an inverted-U shape relationship, even though the theoretical argument has been made before [2, 3, 19, 31].
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1305
Comparing Our Research with Priming Studies
Priming is typically through subconscious mechanisms, therefore different from our approach of deliberate conscious use of stimuli. Even though our study shows that the effect of unrelated stimuli is limited in our context, priming with unrelated stimuli can promote creativity. For example, people generate more creative ideas after they play an unrelated scrambled sentence game that emphasizes achievement [22]. This outcome might result from activated knowledge related to achievement in the semantic network, or elevated expectancy of achievement through motivational mechanisms. While the exact mechanism seems unclear [22], it is safe to assume that priming may work through different mechanisms than deliberate conscious use of stimuli. In our study, because we provided a clear example of using stimuli as cognitive stimulation, the effects of stimuli are likely (at least in part) due to cognitive mechanisms. In general, however, it is also possible that the effects of stimuli are through a mix of various mechanisms. In future experimental studies, it might be possible to measure the cognitive, emotional and motivational influence on the use of stimuli of different degrees, which would further elucidate the mechanisms and provide in-depth under- standing of various possibilities of using stimuli in ideation.
Implications for Practice
Our results suggest that remotely related stimuli are more likely to promote idea novelty than unrelated stimuli. Therefore, creative professionals are well advised to search remotely related stimuli for inspiration. Our approach for concept search is able to find remote stimuli, as well as other levels of stimuli from Wikipedia. This approach for finding stimuli can be included in creative work processes or can be explicitly built into creativity support systems. Another implication for practice in creative work is related to the development of preliminary ideas. Our study shows that people tend to abandon highly novel preliminary ideas. Such preference against novelty exists both in our participants and in highly educated scientific communities. A recent study shows that highly novel scientific research proposals systematically got lower evaluation scores [11], which would increase the chance of rejecting such proposals. While it is certainly sensible to consider the relevance and appropriateness of a preliminary idea, such a prevalent bias against novelty can hurt the effectiveness of using external stimuli and harm creative work. Therefore, when using external stimuli in creative work, people might be encouraged to give highly novel nascent ideas more consideration. It may be worthwhile to nurture such ideas through a sequential process, encouraging several rounds of improvement before making a final judgment.
Implications for Future Research
Existing theories on idea generation (such as SIAM and CNM) typically focus on the cognitive aspect. As our discussion on priming studies shows, idea generation
1306 WANG AND NICKERSON
can also be influenced by a change in emotion, attitude, or motivational level. To our knowledge, these additional aspects are not well-integrated into existing the- ories of idea generation, which might hinder our understanding of stimuli use in creative tasks. One source of theory may lie in cognitive science: ACT-R 5.0 is an updated version of the ACT theory and aims at an integrated theory of mind [6]. In this theory, there are different modules in mind, such as a declarative module containing semantic memory, a production unit containing production rules, and an intentional module containing goal information. It might be possible to build on this model and further explain how an external stimulus might activate not only semantic memory but also production rules and goal information, which further influences ideation. It might be possible to add some explanation of how stimuli influence emotions and in turn ideation. In short, an integrated theory of idea generation considering different aspects might greatly improve our understanding of creative work – and our ability to aid it. In our hypothesis development, we argued that unrelated stimuli may increase
cognitive load, which in turn limits idea quantity and idea novelty if the increased load reduces success in finding relevant ideas. Since we did not explicitly measure cognitive load, we have not provided direct empirical evidence for this argument. Psychologists have measured cognitive load through perceived mental effort [53] and pupillary response [36]. In that we have proposed and tested a way of finding stimuli of varying degrees of relatedness, it should be possible to apply this technique to directly study the relationship between stimulus relatedness and cognitive load in ideation tasks. Such studies may provide valuable insights into the phenomenon of using stimuli to promote idea generation. Some additional research directions are suggested by our results. In our experi-
ments, each participant saw exactly three stimulus concepts. In future research, a different number of stimuli might be tested. It is an open question whether using fewer stimuli would lead to the same results: it could be that even just one external stimulus is enough to break someone out of a fixation. With regard to using more than three stimuli, it may be that there are diminishing returns to extra stimuli. Conversely, more stimuli might increase the chances of novel idea being generated, and therefore might lead to larger differences across conditions. Such larger differences might in turn cast more light on the proposed inverted U-shape relation- ship between stimulus relatedness and idea novelty. It is possible that stimulus relatedness affects idea novelty through a different non-linear function, or that relatedness has different effects at different stages of ideation. It may be useful to examine different strategies for using the stimuli generated
through our method. For example, it might be effective if the stimulus is used by following a step-by-step process of analogical thinking [42] or other creativity techniques [18, 62, 67]. In generating stimuli, we only used the associative links in Wikipedia. It may also be fruitful to use the category structure in Wikipedia to search stimuli to support creative work, because category structure may have an embedded ontology which is useful in generating ideas [33].
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1307
Given the proposed method allows control of stimuli distance, it becomes possi- ble to use a sequence of stimuli at different distances. For example, much like simulated annealing, stimuli might first be provided at far distances to shake fixation and generate novelty. Subsequent stimuli might be provided at closer distances in order to trigger an increase in idea usefulness. That is, instead of trying to stimulate full-blown creative ideas in a single step, a system might seek to first generate one dimension of creativity, novelty, followed by the other dimension of creativity, usefulness. Researchers might compare this Wikipedia-based method with other methods of
supporting creative work with stimuli, such as case-based reasoning [3] and online methods based on metaphor, such as Yossarian Lives (https://yossarian.co/) and analo- gical idea generation by crowds [72]. Case-based reasoning method depends on the development of an information system that stores existing solutions. The rich details in such existing solutions may affect creativity differently than the vaguer prompts derived from Wikipedia. Yossarian Lives provides words and images that are meta- phorically related to an initial concept; these might lead to associations and emotions both. Yu et al. [72] suggested the use of online crowds to find analogies to aid ideation. Comparing our method with these methods may lead to a better understanding of the role of stimuli in creative work, which in turn may improve the design of creativity support systems.
Conclusions
Providing stimuli is a common approach in creativity support systems. This article proposes a new approach to generating stimuli that can be used to support creative work. Specifically, starting with an initial Wikipedia concept, hypertext links in Wikipedia pages are followed iteratively to find concepts of decreasing levels of relatedness. Consistent with our predictions, stimulus relatedness is positively related to idea quantity and idea usefulness. When people were exposed to remotely related concepts, they generated ideas of higher novelty, compared with seeing no stimuli. By contrast, unrelated concepts do not consistently improve idea novelty. Our research suggests a systematic way of catalyzing creativity by generating stimuli that are not random, but instead are related to the creative task to different degrees.
Supplemental Material
Supplemental data for this article can be accessed on the publisher’s website.
Funding
The authors want to acknowledge the support from the National Science Foundation under grants 0968561, 1422066, 1442840, and 1717473.
1308 WANG AND NICKERSON
NOTES 1. The Python programs that we used in the study are in the following repository:https://
github.com/bertramman/JMIS 2. Compared to Table 4 in study 1, Table 7 has an additional column representing the
regression of stimulated novelty on experimental conditions. This is to test hypothesis 3. Unlike study 1, study 2’s control condition has three additional ideation steps parallel to stimulus presentation in other conditions. Thus, study 2’s control condition has a counterpart “stimulated novelty”, based on these additional ideation steps. There is no such “stimulated novelty” in study 1’s control condition. Consequently, it is not feasible to do the same regression in Table 4 in study 1.
REFERENCES 1. Adamson, R.E. Functional fixedness as related to problem solving: A repetition of
three experiments. Journal of Experimental Psychology, 44, 4 (1952), 288–291. 2. Althuizen, N.; and Reichel, A. The effects of IT-enabled cognitive stimulation tools on
creative problem solving: A dual pathway to creativity. Journal of Management Information Systems, 33, 1 (2016), 11–44.
3. Althuizen, N.; and Wierenga, B. Supporting creative problem solving with a case-based reasoning system. Journal of Management Information Systems, 31, 1 (2014), 309–340.
4. Amabile, T.M. Creativity in Context. Boulder, CO: Westview, 1996. 5. Anderson, J.R. The Architecture of Cognition. Cambridge, MA: Harvard University
Press, 1983. 6. Anderson, J.R.; Bothell, D.; Byrne, M.D.; Douglass, S.; Lebiere, C. and Qin; Y. An
integrated theory of the mind. Psychological Review, 111, 4 (2004), 1036–1060. 7. Baer, M.; Leenders, R.T.A.; Oldham, G.R.; and Vadera, A.K. Win or lose the battle for
creativity: The power and perils of intergroup competition. Academy of Management Journal, 53, 4 (2010), 827–845.
8. Baughman, W.A.; and Mumford, M.D. Process-analytic models of creative capacities: Operations influencing the combination-and-reorganization process. Creativity Research Journal, 8, 1 (1995), 37–62.
9. Benjamini, Y.; and Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57, 1 (1995), 289–300. 10. Berg, J.M. The primal mark: How the beginning shapes the end in the development of
creative ideas. Organizational Behavior and Human Decision Processes, 125, 1 (2014), 1–17. 11. Boudreau, K.J.; Guinan, E.C.; Lakhani, K.R.; and Riedl, C. Looking across and
looking beyond the knowledge frontier: Intellectual distance, novelty, and resource allocation in science. Management Science, 62, 10 (2016), 2765–2783. 12. Chan, J.; Dow, S.P.; and Schunn, C.D. Do the best design ideas (really) come from
conceptually distant sources of inspiration? Design Studies, 36, (2015), 31–58. 13. Chan, J.; Fu, K.; Schunn, C.D.; Cagan, J.; Wood, K.L.; and Kotovsky, K. On the
benefits and pitfalls of analogies for innovative design: Ideation performance based on analogical distance, commonness, and modality of examples. Journal of Mechanical Design, 133, 8 (2011), 081004. 14. Chiu, I.; and Shu, H. Investigating effects of oppositely related semantic stimuli on
design concept creativity. Journal of Engineering Design, 23, 4 (2012), 271–296. 15. Cicchetti, D.V. Guidelines, criteria, and rules of thumb for evaluating normed and standar-
dized assessment instruments in psychology. Psychological Assessment, 6, 4 (1994), 284–290. 16. Collins, A.M.; and Loftus E.F. A spreading-activation theory of semantic processing.
Psychological Review, 82, 6 (1975), 407–428. 17. Coskun, H.; Paulus, P.B.; Brown, V.; and Sherwood, J.J. Cognitive stimulation and
problem presentation in idea-generating groups. Group Dynamics: Theory, Research, and Practice, 4, 4 (2000), 307–329.
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1309
18. Couger, J.D.; Higgins, L.F.; and McIntyre, S.C. (Un)structured creativity in informa- tion systems organizations. MIS Quarterly, 17, 4 (1993), 375–394. 19. Dahl, D.W.; and Moreau, P. The influence and value of analogical thinking during new
product ideation. Journal of Marketing Research, 39, 1 (2002), 47–60. 20. Dean, D.L.; Hender, J.M.; Rodgers, T.L.; and Santanen, E.L. Identifying quality, novel,
and creative ideas: Constructs and scales for idea evaluation. Journal of the Association for Information Systems, 7, 10 (2006), 646–699. 21. Dennis, A.R.; and Valacich, J.S. Electronic brainstorming: Illusions and patterns of
productivity. Information Systems Research, 10, 4 (1999), 375–377. 22. Dennis, A.R.; Minas, R.K.; and Bhagwatwar, A.P. Sparking creativity: Improving
electronic brainstorming with individual cognitive priming. Journal of Management Information Systems, 29, 4(2013),195–215. 23. Dennis, A.R.; Valacich, J.S.; Connolly, T.; and Wynne, B.E. Process structuring in
electronic brainstorming. Information Systems Research, 7, 2 (1996), 268–277. 24. Elam, J.J.; and Mead, M. Can software influence creativity? Information Systems
Research, 1, 1 (1990), 1–22. 25. Enkel, E.; and Gassmann, O. Creative imitation: exploring the case of cross industry
innovation. R and D Management, 40, 3 (2010), 256–270. 26. Fellbaum, C. WordNet: An Electronic Database. Cambridge, MA: MIT Press, 1998. 27. Fleming, L. Recombinant uncertainty in technological search. Management Science,
47, 1 (2001), 117–132. 28. Finke, R.A.; Ward, T.B.; and Smith, S.M. Creative Cognition: Theory, Research and
Application. Cambridge, MA: MIT Press, 1992. 29. Finke, R.A. Creative insight and preinventive forms. In R.J. Sternberg and J.E. Davidson
(eds.), The Nature of Insight. Cambridge, MA: MIT Press, 1995, pp. 255–280. 30. Fjermestad, J.; and Hiltz, S.R. An assessment of group support systems experimental
research: Methodology and results. Journal of Management Information Systems, 15, 3 (Winter 1998–99), 7–149. 31. Fu, K.; Chan, J.; Cagan, J.; Kotovsky, K.; Schunn, C.; and Wood, K. The meaning of
“near” and “far”: The impact of structuring design databases and the effect of distance of analogy on design output. Journal of Mechanical Design, 135, 2 (2013), 021007. 32. Garfield, M.J.; Taylor, N.J.; Dennis, A.R.; and Satzinger, J.W. Research report:
Modifying paradigms - Individual differences, creativity techniques, and exposure to ideas in group idea generation. Information Systems Research, 12, 3 (2001), 322–333. 33. Genc, Y.; Sakamoto, Y.; and Nickerson, J.V. Discovering context: Classifying tweets
through a semantic transform based on Wikipedia, In D. Schmorrow and C. Fidopiastis (eds). Foundations of Augmented Cognition: Directing the Future of Adaptive Systems, Lecture Notes in Computer Science, Berlin: Springer, 2011, pp.484–492. 34. German, T.P.; and Barrett, H.C. Functional fixedness in a technologically sparse
culture. Psychological Science, 16, 1 (2005), 1–5. 35. Gick, M.L.; and Holyoak, K.J. Analogical problem solving. Cognitive Psychology, 12,
3 (1980), 306–355. 36. Granholm, E.; Asarnow, R.F.; Sarkin, A.J.; and Dykes, K.L. Pupillary responses index
cognitive resource limitations. Psychophysiology, 33, 4 (1996), 457–461. 37. Guo, J.; and McLeod P.L. The impact of semantic relevance and heterogeneity of
pictorial stimuli on individual brainstorming: An extension of the SIAM model. Creativity Research Journal, 26, 3 (2014), 361–367. 38. Hargadon, A.B.; and Douglas, Y. When innovations meet institutions: Edison and the
design of the electric light. Administrative Science Quarterly, 46, 3 (2001), 476–501. 39. Hender, J.M.; Dean, D.L.; Rodgers, T.L.; and Nunamaker, J.F. An examination of the
impact of stimuli type and GSS structure on creativity: Brainstorming versus non-brainstorming techniques in a GSS environment. Journal of Management Information Systems, 18, 4 (2002), 59–85. 40. Hennessey, B.A.; and Amabile, T.M. Creativity. Annual Review of Psychology, 61,
(2010), 569–598. 41. Jansson, D.; and Smith, S. Design fixation. Design Studies; 12, 1 (1991), 3–11.
1310 WANG AND NICKERSON
42. Knoll, S.W.; and Horton, G. Changing the perspective: Using a cognitive model to improve thinklets for ideation. Journal of Management Information Systems, 28, 1 (2011), 85–114. 43. Lenat, D.B. CYC: A large-scale investment in knowledge infrastructure.
Communications of the ACM, 38, 11 (1995), 33–38. 44. Luo, L.; and Toubia, O. Improving online idea generation platforms and customizing
the task structure on the basis of consumers’ domain-specific knowledge. Journal of Marketing, 79, 5 (2015), 100–114. 45. MacCrimmon, K.R.; and Wagner, C. Stimulating ideas through creativity software.
Management Science, 40, 11 (1994), 1514–1532. 46. Malaga, R.A. The effect of stimulus modes and associative distance in individual
creativity support systems. Decision Support Systems, 29, 2 (2000), 125–141. 47. Massetti, B. An empirical examination of the value of creativity support systems on
idea generation. MIS Quarterly, 20, 1 (1996), 83–97. 48. Mednick, S.A. The associative basis of the creative process. Psychological Review, 69,
3 (1962), 220–232. 49. Miron-Spektor, E.; Gino, F.; and Argote, L. Paradoxical frames and creative sparks:
Enhancing individual creativity through conflict and integration. Organizational Behavior and Human Decision Processes, 116, 2 (2011), 229–240. 50. Mitchell, A.; and Dacin, P.A. The assessment of alternative measures of consumer
expertise. Journal of Consumer Research, 23, 3 (1996), 219–239. 51. Mueller-Wienbergen, F.; Mueller, O.; Seidel, S.; and Becker, J. Leaving the beaten
tracks in creative work – A design theory for systems that support convergent and divergent thinking. Journal of the Association for Information Systems, 12, 11 (2011), 714–740. 52. Nijstad, B.A.; and Stroebe, W. How the group affects the mind: A cognitive model of
ideation in groups. Personality and Social Psychology Review, 10, 3 (2006), 186–213. 53. Paas, F.; Tuovinen, J.E.; Tabbers, H.K.; and Van Gerven, P.W.M. Cognitive load
measurement as a means to advance cognitive load theory. Educational Psychologist, 38, 1 (2003), 63–71. 54. Perttula, M.K.; and Sipila, P.P. The idea exposure paradigm in design idea generation.
Journal of Engineering Design, 18, 1(2007),93–102. 55. Postmes, T.; Spears, R.; Sakhel, K.; and De Groot, D. Social influence in
computer-mediated communication: The effects of anonymity on group behavior. Personality and Social Psychology Bulletin, 27, 10 (2001), 1243–1254. 56. Potter, R.E.; and Balthazard, P. The role of individual memory and attention processes
during electronic brainstorming. MIS Quarterly, 28, 4 (2004), 621–643. 57. Raaijmakers, J.G.W.; and Shiffrin R.M. Search of associative memory. Psychological
Review, 88, 2(1981),93–134. 58. Santanen, E.L.; Briggs, R.O.; and de Vreede, G. Causal relationships in creative
problem solving: Comparing facilitation interventions for ideation. Journal of Management Information Systems, 20, 4 (2004), 167–197. 59. Satzinger, J.W.; Garfield, M.J.; and Nagasundaram, M. The creative process: The
effects of group memory on individual idea generation. Journal of Management Information Systems, 15, 4(1999),143–160. 60. Sio, U.N.; Kotovsky, K.; and Cagan, J. Fixation or inspiration? A meta-analytic review
of the role of examples on design processes. Design Studies, 39, (2015), 70–99. 61. Smith, S.M.; Ward, T.B.; and Schumacher, J.S. Constraining effects of examples in
a creative generation task. Memory and Cognition, 21, 6(1993),837–845. 62. Stein, E.W.; and Zwass, V. Actualizing organizational memory with information
systems. Information Systems Research, 6, 2 (1995), 85–117. 63. Tseng, I.; Moss, J.; Cagan, J.; and Kotovsky, K. The role of timing and analogical
similarity in the stimulation of idea generation in design. Design Studies, 29, 3 (2008),203–221. 64. Viriyayudhakorn, K.; Kunifuji, S.; and Ogawa, M. A comparison of four association
engines in divergent thinking support systems on Wikipedia. In T. Theeramunkong, S. Kunifuji, V. Sornlertlamvanich, and C. Nattee (eds.), Knowledge, Information, and Creativity Support Systems. Heidelberg: Springer Berlin, 2011, pp. 226–237.
A WIKIPEDIA-BASED METHOD TO SUPPORT IDEATION 1311
65. Wang, H.-C.; Cosley, D.; and Fussell, S.R. Idea expander: Supporting group brain- storming with conversationally triggered visual thinking stimuli. In K. Inkpen, and C. Gutwin (eds.), Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, New York: ACM, 2010, pp. 103–106. 66. Wang, H.-C.; Fussell, S.R.; and Cosley, D. From diversity to creativity: Stimulating
group brainstorming with cultural differences and conversationally-retrieved pictures. In P. Hinds, J. Tang, and J. Wang (eds.) Proceedings of the 2011 ACM Conference on Computer Supported Cooperative Work, New York: ACM, 2011, pp. 265–274. 67. Wang, K. Factors influencing the adoption and effective use of creativity techniques in
business settings: An exploratory study. Engineering Management Journal, 26, 4 (2014), 29–37. 68. Wang, K.; and Nickerson, J.V. A literature review on individual creativity support
systems. Computers in Human Behavior, 74, (2017), 139–151. 69. Wang, K.; Nickerson, J.V.; and Sakamoto, Y. Crowdsourced idea generation: The
effect of exposure to an original idea. Creativity and Innovation Management, 27, 2 (2018), 196–208. 70. Ward, T.B. Structured imagination: The role of category structure in exemplar
generation. Cognitive Psychology, 27, 1 (1994), 1–40. 71. Witten, I.; and Milne, D. An effective, low-cost measure of semantic relatedness
obtained from Wikipedia links. In R. Bunescu, E. Gabrilovich, and R. Mihalcea (eds.), Proceeding of AAAI Workshop on Wikipedia and Artificial Intelligence: An Evolving Synergy, Chicago, IL: AAAI Press, 2008, pp. 25–30. 72. Yu, L.; Kraut, R.E.; and Kittur, A. Distributed analogical idea generation with multiple
constraints. In D. Gergle, and M.R. Morris (eds.), Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing, New York: ACM, 2016, pp. 1236–1245.
1312 WANG AND NICKERSON
Copyright of Journal of Management Information Systems is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
- Abstract
- Introduction
- Background
- Cognitive Theories of Creativity
- Influence of Stimuli on Idea Generation
- Hypotheses Development
- New Method for Finding Stimuli of Different Levels of Relatedness
- Testing the New Method
- Study 1
- Methods
- Study Design and Participants
- Dependent Variables
- Independent Variables
- Control Variables
- Results
- Study 2
- Methods
- Results
- Discussion
- Influence of Stimuli on Idea Quantity and Usefulness
- Influence of Stimuli on Idea Novelty
- Comparing Our Research with Priming Studies
- Implications for Practice
- Implications for Future Research
- Conclusions
- Supplemental Material
- Funding
- Notes
- References