Creative writing 6

Creativity_Cognition_and_Knowledge_An_Interactio..._----_4_Creativity_Relational_Knowledge_and_Capacity_Why_Are_Humans_So_Cre....pdf

Home >English homework help >Article writing homework help >Creative writing 6

Creativity depends on using our mental processes to create novelty. How- ever, for novelty to be creative it must be effective, either in some practical sense, or in that it establishes a coherent set of relations where no relations, or a less coherent set of relations, existed before. Creativity may be defined therefore as the production of effective novelty through the operation of our mental processes. It is a truism that human beings are the most creative systems that we know. They are more creative than any machines invented so far. They are also more creative than other animals, and they become more creative as they develop. This is not to say that children are not cre- ative, as they may well be less afraid of novelty than most adults, but the power of creative processes generally increases with development. Our problem then is to ask what it is about adult humans that confers on them such unique creative power.

An answer is suggested by Dartnall (1994), who draws attention to the fact that creativity depends on explicit representation. He points out that creativity depends on more than being able to do things. It is also neces- sary to represent what has been done. In a similar way Clark and Karmiloff-Smith (1993) distinguish between implicit and explicit knowl- edge. Implicit knowledge is sufficient to perform a task but the perform- ance cannot be modified solely by the operation of mental processes. A person who has learned a particular skill may perform well without being able to describe how the performance is executed, just as an animal that has learned how to find food is evidently not aware of what it does. Both

Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

Graeme S. Halford and William H. Wilson

Creativity, Cognition, and Knowledge : An Interaction, edited by Terry Dartnall, Greenwood Publishing Group, Incorporated, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/michstate-ebooks/detail.action?docID=3000849. Created from michstate-ebooks on 2018-08-10 05:38:28.

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

cases depend on procedural knowledge, but there is no declarative knowl- edge (Anderson, 1983), meaning that the performer is unable to describe, or declare, what is done. Such performances can only be changed by further experience. Components of the skill can be modified or rearranged, or the animal can learn to run in a different direction. However, such modification requires further experience, with external input, including feedback, and mental processes alone cannot achieve it. The golfer cannot perfect his swing solely by thinking about it, and there is solid experimental evidence that lower animals cannot reverse their habits on the basis of a single sig- nal. They only change their performances incrementally, a little at a time, under the continued influence of concrete experience. Such automatic, inac- cessible, unmodifiable performance is generally termed “implicit” (Clark & Karmiloff-Smith, 1993).

Explicit processes on the other hand are mentally accessible and modifi- able (Clark & Karmiloff-Smith, 1993). The transition from implicit to explicit cognitive processes has been explained in terms of representational redescription (Karmiloff-Smith, 1990). This means that implicit knowledge is represented in a progressively more abstract manner, thereby becoming more accessible to other cognitive processes. For example, a young child might say and comprehend both “a dog” and “the dog,” but not recognize that “a” and “the” are related, or have the grammatical concept of the arti- cle (Karmiloff-Smith, 1990). Similarly, young children might be able to draw houses, but if asked to draw a “funny” house they make only super- ficial changes, leaving the essential structure of the drawing unchanged. Karmiloff-Smith interprets this as indicating that the child has drawing skills that are not accessible and cannot be modified by the operation of internal cognitive processes alone.

Boden (1990) proposes that creativity entails “breaking out” of old con- ceptual frameworks and creating new ones. Dartnall (1994) interprets this as breaking out of one rule set, and creating a new one. For this to happen we need explicit knowledge of the rules under which we operate. At the implicit level we are subject to the rules, but at the explicit level we can articulate the rules and modify them. Animals that are phylogenetically more primitive lack this ability. For example, a beaver can build a dam, but has no explicit knowledge of dam building, so a beaver is not an engineer. Even if a beaver and an engineer were to build precisely the same dam on some occasion, their mental processes would be very different. The differ- ences would become apparent if modifications had to be made to the dam- building procedure, and the more fundamental the modifications, the more the difference would become apparent. We will present empirical evidence of comparative performances of various animals and humans later.

We can conclude, then, that creativity requires explicit representation, in the sense that it is accessible to, and modifiable by, other cognitive processes, without the necessity of external input. However, the argument

154 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

so far has been essentially descriptive, but now we want to consider how the differences we have been discussing can be defined at a more theoreti- cal level. For this purpose we will apply the theory of representational ranks (Halford, Wilson, & Phillips, 1998b; Halford, Wilson & Phillips, 1998a; Halford, Phillips & Wilson, 2001) which is an extension of relational com- plexity theory (Halford et al., 1998b). Representational rank theory pro- vides a complexity metric in which different levels of cognitive functioning are distinguished. It also provides a way of identifying the essential proper- ties of higher cognitive processes. We will consider how effective each of these levels of cognitive processing will be in mediating creativity. Then we will attempt to provide at least a rudimentary explanation for differences in creativity between animals and humans, between children and adults, and between humans and machines.

TYPES OF REPRESENTATIONS

To develop our argument we first need to summarize the levels of repre- sentation, defined by Halford et al. (1998b) and the extension by Halford et al. (2001). These are shown schematically in Figure 4.1, together with possible implementations in neural nets based on the STAR model (Halford, Wilson, Guo, Gayler, Wiles, Stewart, 1994; Wilson, Halford, Gray, & Phillips, in press). The ranks correspond to levels of conceptual complexity, and new psychological processes emerge at each level. Ranks 0 and 1 are restricted to the associative mode, and embody mechanisms traditionally identified with associationist psychology. Ranks 2–6 permit processing of explicit relations, and the higher cognitive processes depend on these levels.

Rank 0: Performance Without Representations

This level includes reflexes, stimulus-response associations, and perceptuo- motor skills, insofar as they are not guided by a representation of the task. These processes can be captured in neural net models by pattern associa- tors, or two-layered nets, as shown in Figure 4.1. That is, there is an asso- ciation between input and output, without internal representation, and the association is either acquired incrementally through the operation of a learning algorithm, or is hard-wired. The acquisition is not influenced by other cognitive processes, and is not under strategic control in the way that higher cognitive processes are.

Psychologically, Rank 0 corresponds to elemental association, which is a link between two entities, without intervening representation. For example, a two-object discrimination might consist of the following two associations:

Element triangle → R+ (respond) Element square → R– (don’t respond, avoid)

155Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

FIGURE 4.1 Representations of six different ranks, with possible Parallel Distributed Processing (PDP) representation.

Rank Representation Neural Net Specification

0 No representation

1 Wholistic representation (no explicit relations)

2 Unary relation

3 Binary relation

4 Ternary relation

5 Quaternary relation

6 Quinary relation

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

This is the most primitive level of functioning, and presumably occurs in all animals. However, the term “elemental” should not be confused with the everyday meaning of “elementary,” and this level can produce some quite complex performances, such as transitivity of choice (McGonigle & Chalmers, 1984; Wynne, 1995). However, it does not have the properties of higher cognitive processes, and it would not support creativity, in the sense that we have defined the term.

Rank 1: Computed, Nonstructural Representations

At this level there is internal representation, but it is undifferentiated, and is unrelated to other representations. Many features, and quite a large amount of information, can be represented, but no relations with semantic interpretations are defined between components of the representation.

Psychologically, this level corresponds to configural association, in which the link between cue and response is not direct, but is modified by another entity. One example would be conditional discrimination tasks. Partici- pants, either animals or humans, might be trained to choose a circle and avoid a triangle on a white background, and this might be reversed on a striped background (i.e., triangle becomes positive and circle negative). The contingencies that must be learned can be expressed this way:

circle, white → + triangle, white → – circle, striped → – triangle, striped → +

Each stimulus (circle, triangle, white, striped), taken singly, is equally asso- ciated with positive or negative responses, so the task cannot be acquired through elemental association (Halford, 1993; Rudy, 1991). Consequently the discrimination must be based on configurations, such as “circle&white,” “circle&striped,” and so on. This performance requires internal representa- tion of the configurations, and there is evidence that rats are capable of it, although it depends on maturation of the hippocampus (Rudy, 1991). Con- ditional discrimination is isomorphic to the exclusive-OR (XOR).

Another example of this level of performance would be the representa- tion of objects in space in the manner that has been observed in five-month- old infants by Baillargeon (1995; but see also Rivera, Wakeley, & Langer, 1999). Infants represent features of objects, such as height, and their dis- tance from other objects, but there is no evidence that they can relate such representations to other representations. They can represent perceptible fea- tures, but presumably have no explicit representations of relations. Other

157Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

examples of computed nonstructural representations are prototypes and image schemas. Prototypes (Rosch & Mervis, 1975) are the basis of natu- ral categories, and can be learned by infants of a few months of age (Quinn, Eimas, & Rosenkrantz, 1993). They can be modeled by three-layered nets (McClelland & Rumelhart, 1985; Quinn & Johnson, 1997). Image schemas, such as self-motion, animate-motion, support and containment (Mandler, 1992) are also examples of this level of cognition.

Performances at this level can be captured by three-layered nets, com- prising input, output, and hidden layers, as shown in Figure 4.1. Configural learning tasks such as conditional discrimination can be performed by using units in the hidden layer which represent conjunctions of features such as “circle&white” (Schmajuk & DiCarlo, 1992), just as hidden units can be used to represent conjunctions in the exclusive-OR task. In spatial tasks, hidden units could represent various features of objects such as height and distance from other objects.

These features are represented by activations in one or more of the hid- den units. They do not have a relational structure, such as conjunction (circle,white), so it is not possible to answer a query such as “what is com- bined with circle?” They are therefore more primitive, and less accessible, than the relational representations at Ranks 2–6.

Ranks 2–6: Relational Representations

Phillips, Halford and Wilson (1995) have argued, on the basis of the work of Codd (1990), that essential properties of higher cognitive processes, including data structures such as lists and trees, can be captured by a model based on the processing of relations. Relational representations have properties such as explicit representation and accessibility to other cognitive processes which enable them to be processed recursively. Phillips et al. (1995) have argued that representational redescription can be explained at least in part by the transition from associative to relational rep- resentations.

This argument was developed further by Halford et al. (2001) who defined the properties of relational knowledge that are essential to higher cognitive processes. We will summarize this argument here to provide a basis for our account of creativity.

In general, an n-ary relation R(a 1 ,a

2 ,...,a

n ) is a subset of the Cartesian

product S 1 ×S

2 ×...×S

n . Representation of a relation requires a set of bindings

between a relation symbol, R, and the arguments (a 1 ,a

2 ,...,a

n ). The binary

relation BIGGER-THAN(whale,dolphin) is a binding between the relation symbol BIGGER-THAN and the arguments “whale” and “dolphin.” Rela- tional representations have the following properties:

Symbolization means that the link between the arguments of a relation is explicitly symbolized (e.g., the link between “whale” and “dolphin” is

158 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

explicitly symbolized by the relation symbol LARGER-THAN), whereas associative links are all the same and do not carry a label. This makes a rela- tion accessible to other cognitive processes, so that a relational instance can be an argument to another relation. For example:

BECAUSE(LARGER-THAN(whale,dolphin), AVOIDS(dolphin,whale)).

Higher-order relations have relations as arguments, whereas first order relations have objects as arguments.

Omni-directional access means that, given all but one of the components of a relation, we can access (i.e., retrieve) the remaining component. For example, given the relational schema R(a,b), any of the following can be performed:

R(a,–) → b R(–,b) → a –(a,b) → R

Given a relational instance such as MOTHER-OF(woman,child), and given MOTHER-OF(woman,–) we can access “child,” whereas given MOTHER- OF(–,child) we can access “woman,” and given –(woman,child) we can access “MOTHER-OF.” When we are dealing with more than one rela- tional instance, the answers will not always be unique. For example, MOTHER-OF(woman,–) may yield child, toddler, infant, baby, teenager, and so on. Because omni-directional access seems to be inherent in relations they have more flexible access than elemental and configural associations, which are not inherently bi-directional.

Role representation means that relational knowledge entails representa- tion of argument roles or slots, independent of specific instances. Thus BIGGERTHAN(–,–) entails representation of slots for a larger and a smaller entity.

Decomposability of relations means that relations can be composed of simpler relations. A decomposable relation is one that can be written as a conjunct of instances of relations of lower arities. For example, the ternary relation MONOTONICALLY-LARGER(a,b,c) can be decomposed into >(a,b) & >(b,c) & >(a,c). Not all relations are decomposable (Halford et al., 1998b, Sections 3.4.3 and 4.2.8).

Relational systematicity means that certain relations imply other rela- tions. For example, >(a,b) → <(b,a), and sells (seller,buyer,object) → buys (buyer,seller,object). This is really a case of higher-order relation; for exam- ple implies (>(a,b), <(b,a)).

Dimensionality of relations means that each argument of R can be instan- tiated in more than one way, and therefore represents a source of variation, or dimension. An n-ary relation may be thought of as a set of points in N-dimensional space. The number of arguments, N, corresponds to the number of dimensions in the space defined by the relation. In terms of sets,

159Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

a unary relation R on a set S is a subset of S. It is the set of objects {x ∈ S | R(x) is true}. Similarly, a binary relation on a set S is a subset of the Carte- sian product S×S of elements of S, a ternary relation is a subset of S×S×S of elements of S, and so on. Each set contributes a dimension of the Cartesian product. This is the basis for our complexity metric (Halford et al., 1998b). The relation symbol can contribute an extra dimension if it is a variable (i.e., the relation changes) within the task.

Relational representations can represent the knowledge structures that are important to higher cognition, as follows.

Proposition. A proposition is defined as the smallest unit of knowledge that can have a truth value. Let {R

1 , R

2 ...} be the collection of all valid rela-

tions. Each R i is a subset of some Cartesian product, which we refer to as

C i . A true proposition, then, is a relational instance, that is, a member of

one of the relations R i . A false proposition is an “expressible non-fact”—

that is, a member of one of the complements C I \R

I of the relations R

i , and

not a member of any of the R i . For example, the true proposition bigger-

than(dog, mouse) is an instance of the “bigger-than” relation. We choose to regard “bigger-than” as a subset of the Cartesian product animals × ani- mals. The false proposition bigger-than(mouse, dog) has the property that (mouse, dog) belongs to animals × animals \ bigger-than. Each proposition is a point in the space defined by union of the Cartesian products C

i .

An attribute is a relation symbol with one argument. An attribute value is an instance of a unary relation. An attribute is therefore distinct from a feature as defined earlier, because an attribute has the properties of rela- tional knowledge, whereas a feature does not. Thus the attribute wags- tail(dog) is a unary relational instance, and has the properties of relational knowledge, including omni-directional access. If we ask “what is (an) attribute of dog”(i.e., ?(dog)) we can answer “wags-tail” and if we ask “what wags its tail?”(i.e., wags-tail(?)) an answer is “dog.” Features are dis- tinct from attributes in this respect. Representation of features plays a role in determination of responses, but the representation is not accessible to other cognitive processes. A good model for this is the representation of fea- tures in the hidden layer of three-layered nets. The representation consists of an activation in one or more hidden units, and is a variable that deter- mines the activations in the output layer. However, the structure of the rep- resentations in the hidden layer is not typically accessible to other cognitive processes of the performer. It might be accessible to researchers (e.g., by performing cluster analyses of hidden-layer activations [Elman, 1990]) but it is not accessible to the participant.

Operations on relations. Operations on relations are adapted from those defined in the theory of relational databases (Codd, 1990). They include select, project, join, add, delete, union, intersect, and difference

160 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

(Halford et al., 2001; Phillips et al., 1995). A model of relational knowl- edge that includes these operations can capture important properties of higher cognitive (Halford et al., 2001). These operations permit informa- tion stored in relational knowledge structures to be accessed and manipu- lated in flexible and powerful ways.

Planning and analogy. Relational representations are essential to plan- ning, or the organization of a sequence of actions to achieve a goal, and to analogy. Planning, which has been explicitly modeled by VanLehn and Brown (1980), Greeno, Riley and Gelman (1984) and Halford, Smith, Dickson, Maybery, Kelly, Bain, and Stewart (1995), entails representing relations between components of a task. Analogy entails mapping relations between a base or source and a target (Gentner, 1983; Gick & Holyoak, 1983; Holyoak & Thagard, 1989). Both of these will be considered in more detail later.

We can now consider the specific properties of Ranks 2–6.

Rank 2: Representations with Two Components

At this level of representation there are two components that can be related to one another. This is the lowest level at which propositions can be repre- sented. The simplest proposition requires representation of a relation-symbol and one argument, which is a Rank 2 representation. Where a relation- symbol represents a state, such as happy(John), the proposition corre- sponds to a unary relation.

Unary relations, R(x), have one argument. The proposition big(dog) is a binding between the unary relation “big” and one argument. The relation can be interpreted as expressing a state or an attribute. Other cases of propositions with one argument can be interpreted as class membership, for example, dog(fido). A binding between a variable and a constant can also be expressed as a unary relation, for example, HEIGHT(1 meter). This proposition represents a binding between a variable, height, and an argu- ment, which can take a range of values (Halford, Wiles, Humphreys, & Wilson, 1993; Smolensky, 1990).

Halford et al. (1994) and Halford et al. (1998b) have shown that relation symbol-argument bindings can be modeled using tensor product networks. Each tensor product representation comprises a vector representing the rela- tion symbol, and a vector representing each argument, as shown in Figure 4.1. The binding units (whose values correspond to the computation of the tensor product) are interpreted as activations or as weights. Where they are inter- preted as activations, bindings can be changed dynamically and the relation may be changed in all-or-none fashion without external input. For example, we can change happy(John) to sad(John). One component of the representa- tion (John) remains the same, but when the other component is changed a

161Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

new binding is formed. This means representations can be changed in the course of reasoning tasks. They are not dependent on incremental change as a function of experience. Where activation units are interpreted as weights, the only change mechanism is incremental adjustment of weights under the influence of environmental input. In memory models (Humphreys, Bain, & Pike, 1989), binding units are interpreted as weights, implying that the change is incremental rather than dynamic.

Several important new properties emerge at Rank 2 that are not possible with lower rank representations. These include ability to represent propo- sitions, relations, and variables. This is a sizeable step towards explicit, con- ceptual thought. There is also increased content independence, because a relation symbol expresses an idea that is independent of its argument to some extent: large(x) means the same irrespective of what x is. Further properties emerge at each of the higher ranks, as we will see.

Rank 3: Representations with Three Components

Representations of this rank have three components. With associative binding this level is exemplified by the three-way associations between cue, target and context which have been important in memory retrieval theories (Humphreys et al., 1989); for example. What did you have for breakfast (cue) on Sunday (context)? Bacon and eggs (target).

Binary relations, or relations with two arguments, can be represented, with one component representing the relation symbol and two components representing the arguments. For example, bigger(–,–) has two arguments, which can represent any pair of objects such that the first is bigger than the second. Univariate functions, f(a) = b, and unary operators, such as change- sign{(x,–x)}, can also be represented at this level.

At Rank 3, more complex variations between components can be repre- sented. The binary relation R(x,y) represents the way x varies as a function of y, and vice versa, neither of which is possible with Rank 2 representa- tions. Propositions with two arguments can be represented: for example, loves(Joe, Jenny).

Rank 4: Representations with Four Components

At this level, concepts based on ternary relations can be represented. These include the “love-triangle,” in which two persons, x and y, both love a third person, z. Concepts such as transitivity and class inclusion entail core repre- sentations that are ternary relations (Andrews & Halford, 1998; Halford, 1993; Halford et al., 1998b). This level requires one component of the rep- resentation for the relation symbol and three to represent arguments.

The number of possible relations between elements increases again with ternary relations: R(x,y,z) represents three binary relations, R1(x,y), R2(y,z)

162 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

and R3(x,z), as well as the three-way relation, R(x,y,z), which is not defined in a binary relation. This means that with a ternary relation, but not with unary or binary relations, it is possible to compare x with y,z, and y with x,z, and z with x,y. It thus becomes possible to compute the effects on x of variations in y,z, and so on. More complex interactions can be represented than at the lower levels, which increases the flexibility and power of thought.

Bivariate functions and binary operations may also be represented at this level. A binary operation is a special case of a bivariate function. A binary operation on a set S is a function from the set S×S of ordered pairs of ele- ments of S into S, that is S×S → S. For example, the binary operation of arithmetic addition consists of the set of ordered pairs of {..(3,2,5),.. (5,3,8),..}.

Rank 5: Representations with Five Components

At this level, quaternary relations, R(w,x,y,z), may be represented, with one component representing the relation symbol and the other four the argu- ments. An example would be proportion; a/b = c/d expresses a relation between the four variables a,b,c, and d. It is possible to compute how any element will vary as a function of one or more of the others. With a quater- nary relation all the comparisons that are possible with ternary relations can be made, as well as four-way comparisons; the effect on w of variations in x,y,z, the effects on x of variations in w,y,z, and so on. Quaternary relations can also be interpreted as functions or as operations. A trivariate function is a special case of a quaternary relation. It is a set of ordered 4-tuples (a,b,c,d) such that for each (a,b,c) there is precisely one d such that (a,b,c,d ∈ f). Qua- ternary relations may be interpreted as a composition of binary operations. For example, (a + b) × c = d is a quaternary relation. As with Rank 4 there do not appear to be any concepts using associative representations.

Rank 6: Representations with Six Components

At this level there would be a relation symbol and five arguments. The psychological existence of this level is speculative, and if it exists, it is prob- ably available only for a minority of adults, which makes evidence about it difficult to obtain. It would permit processing of quinary relations that would enable people to reason about relations between systems, each of which is composed of compositions of binary operations. If we assume that theories are composed of mathematical expressions each of which inte- grates a set of binary operations, then representation of the relation between theories amounts to representing the relation between structures defined by a composition of binary operations. This means that the ability to work within a theory would require Rank 5 reasoning, whereas ability to deal with relations between theories would require Rank 6.

163Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

There is no mathematical limit to the number of arguments that a relation can have, but there may be psychological limits to the complexity of rela- tions that can be computed. There is evidence that normal human cognition entails representations only up to Rank 5 (Halford, 1993; Halford et al., 1998b). That is, adult humans appear to be limited to processing one qua- ternary relation in parallel, though a minority of adults can process quinary relations under optimal conditions.

Concepts more complex than Rank 5 are handled by segmentation and chunking (Halford et al., 1994; Halford et al., 1998b). Segmentation means that complex tasks can be decomposed into smaller segments that can be processed serially. The development of serial processing strategies permits complex tasks to be performed without exceeding limits on the amount of information that can be processed in parallel. Chunking means that repre- sentations of high dimensionality, represented by tensor products of high rank, are recoded into fewer dimensions, with tensor products of lower rank. An example would be velocity, defined as v=s/t (velocity = distance/time). This is three-dimensional, but it can be recoded into one dimension (as occurs when we think of speed as the position of a pointer on a dial). However, this does not mean that all processing loads can ultimately be reduced to that for a one-dimensional concept, because conceptual chunking results in loss of representation of some relationships. For example, when velocity is chunked as one dimension, the three-way relation between v, s, and t is no longer rep- resented, so that changes in v as a function of s and/or t cannot be computed without returning to the three-dimensional representation, which entails the higher processing load.

The arguments of chunked relational representations can be unpacked, to provide lower level relations. Consider, for example, a moderately complex concept such as that temperature difference is the cause of heat flow. We can represent this as cause(temperature difference,heat flow). However, this is really a higher-order relation, and the arguments temperature difference and heat flow are chunked representations of lower-order relations. Thus temperature difference can be unchunked to greater(temperature1,temper- ature2) and heat-flow can be unchunked to flow(heat,conductor,high tem- perature,low temperature), that is, heat flows along a conductor from high to low temperature. Thus the concept can be represented as a higher-order binary relation, one argument of which is a lower-order binary relation, and the other is a lower-order quaternary relation. This hierarchical structure can be modeled within the tensor product formalism mentioned earlier (Halford et al., 1994; Halford et al., 1998b; Wilson et al., 2001).

A major component of expertise is to have knowledge organized into effi- cient hierarchical structures in this way. This enables humans to operate within the limitation that no more than one quaternary (or, in some cases, quinary) relation can be processed in parallel. Thus the expert may not

164 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

process relations more complex than those processed by the novice. A per- son with a knowledge of physics can understand heat flow without pro- cessing anything more complex than a quaternary relation at one time, as illustrated above. A novice might experience a higher processing load by attempting, unsuccessfully, to consider relations between all variables at once. One function of expertise is to maintain processing load at manage- able levels.

However, there are limits to the decomposability of relations in this way. Chunked relations become inaccessible, as noted above, some relations can- not be decomposed without loss of information (Halford et al., 1998b, Sec- tion 3.4.3) and individuals may not have appropriate chunking and unchunking or segmentation strategies available. A more subtle, but in some ways more serious, problem is that some representation of the structure of the concept is needed for strategies to be developed. This notion is incorpo- rated into the TRIM model (Halford et al., 1995) to be considered later.

THOUGHT AND RANK OF REPRESENTATION

Rank of representation is a measure of the power and flexibility of thought. Higher rank representations permit higher levels of relations to be represented, and permit more of the structure of a concept or situation to be processed in thought. Rank is analogous to the number of facets of a sit- uation that can be viewed simultaneously.

Rank of representation has been associated with phylogenetic and onto- genetic development. We will briefly review evidence of the type of repre- sentation that is within the capacity of each level in the evolutionary scale, and at each age range in human development.

Representational Rank of Animals and Children

Levels of representation equivalent to those defined here have been related to species differences by Holyoak and Thagard (1995) and by Halford et al. (1998b). Rank 0, equivalent to simple associations, appear to be possible for all species of animals, albeit with varying degrees of efficiency. Mammals appear capable of Rank 1, as indicated by the fact that rats are capable of configural learning (Rudy, 1991). Rank 2 appears to be possible for mon- keys, because they can recognize the binding between attributes and objects. For example, they can learn to choose which of two objects is like a sample. If they are shown, say, an apple as sample object, and required to choose between an apple and, say, a hammer, they can learn to choose the apple. This could be done of course by associative learning, which is Rank 0. How- ever, they can transfer to a new task in which, say, the sample is a hammer, and the choices are a banana and a hammer. Transfer of the principle implies

165Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

they can represent a binding between an attribute (hammer-like) and a spe- cific object. This can be interpreted as a Rank 2 representation. They can represent a dynamic binding between an attribute and an object.

Chimpanzees appear capable of representing a binary relation between objects, which implies Rank 3 representations (Halford et al., 1998b; Halford et al., 2001; Premack, 1983). For example, they can learn that if the sam- ple is XX, they should choose AA rather than BC. If the sample is XY, they should choose BC rather than AA. They learn to choose a pair of objects that has the same relation as the sample. This implies they can represent binary relations, a Rank 3 representation.

As Holyoak and Thagard (1995) have shown, such performances are really a form of analogical reasoning. The animals are sensitive to the fact that an attribute, or a relation, in the sample is the same as in the correct choice item. Analogical reasoning entails dynamic binding (Halford et al., 1993), so these performances indicate that monkeys and apes are capable of relational representations with dynamic binding. This interpretation is born out by studies of what has been called “reversal learning” (Bitterman, 1960). Suppose an animal is trained in a discrimination in which, say, cir- cle is positive, and triangle is negative. Then, without any warning, the con- tingency is reversed, so triangle is now positive and circle negative. Then when that discrimination is learned the reversal process is repeated, so cir- cle becomes positive again, and triangle negative, and so on. The question of interest is whether animals learn to make successive reversals more and more quickly, indicating that they “catch on” to the reversal principle. Species differ considerably in the rate at which they learn to do this. Apes will learn to make a reversal immediately after the contingency is reversed, and monkeys also learn, but at a slower rate, while other mammals learn more slowly still. Birds, reptiles and amphibians show less benefit from training with successive reversals, while fish show no benefit at all. This performance clearly differentiates between species at different levels of the phylogenetic scale. As with other tasks that make this differentiation, it requires representation of the relations in the task, together with dynamic binding. That is, it requires recognition that there is always one positive and one negative stimulus, together with the ability to change the mapping of positive/negative into the stimuli dynamically. The higher animals can do this, but the lower animals can only relearn the discrimination incremen- tally. They are not capable of dynamic change. This paradigm is a subset of so-called “learning set” acquisition (Harlow, 1949, 1959) in which the structure underlying a set of isomorphic discrimination tasks is acquired. It can be interpreted as acquisition of relational knowledge, combined with analogical mapping of a problem into the relational representation (Hal- ford, 1993; Halford, Bain, Maybery, & Andrews, 1998).

It appears then that ability to represent relations and to change the rep- resentation dynamically is a fact or which differentiates species at different

166 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

points in the phylogenetic scale. Putting it another way, the capacity for cre- ative thought is a recent evolutionary development, which is present only in the highest non-human animals, and then in rudimentary form.

Children’s representational abilities have been reviewed elsewhere (Hal- ford, 1993; Halford et al., 2001; Halford et al., 1998b). Neonates are capa- ble of Rank 0 associative learning, but there is evidence that Rank 1 repre- sentations are possible at five months. This is indicated by infants’ awareness of properties of recently vanished objects (Baillargeon, 1995) and ability to form prototypes (Quinn et al., 1993; Quinn & Johnson, 1997). Rank 2 rep- resentations are possible at approximately one year, when infants become capable of treating the hiding place of an object as a variable, and recogniz- ing category memberships explicitly, which in turn leads to their under- standing referents of words; for example “Doggy gone” can be represented as GONE(doggy), which is a relation symbol with one argument.

Rank 3 representations appear to be possible at age two, because children of this age can make discriminations based on explicit representation of binary relations. One consequence is that they can perform proportional analogies of the form A:B::C:D, provided the content is familiar to them.

Rank 4 representations become possible at a median age of five years, and they open up a wide range of new performances, including transitive inference, hierarchical classification and inclusion, certain kinds of hypoth- esis testing, concept of mind, and many others. Transitivity entails a ternary relation, because premises R(a,b) and R(b,c) are integrated into the ordered triple, monotonic-R(a,b,c). Twenty percent of four-year-olds and 40 percent of five-year-olds can perform it (Andrews & Halford, 1998). Class inclu- sion and the part-whole hierarchy are essentially ternary relations. A class inclusion hierarchy has three components: a superordinate class, a subclass and a complementary class (e.g., fruit, divided into apples and non-apples). Part-whole hierarchies are similar, and comprise a whole divided into a part and a complementary part. Though the age of attainment of these concepts has been controversial, it appears that children have difficulty processing tasks that entail ternary relations until approximately five years. This issue has been discussed in greater detail elsewhere (Halford, 1992, 1993; Hal- ford & Leitch, 1989; Hodkin, 1987).

Representations of Rank 5 are typically understood at about age 11, as evidenced by understanding of proportion and a number of other concepts, including understanding of the balance scale. This entails representing the interaction of four factors: weight and distance on the left and weight and distance on the right.

Capacity to represent relational concepts increases with phylogenetic level, and also with age. There is evidence that processing load is a function of the complexity of relations being processed (Halford, Maybery, & Bain, 1986; Maybery, Bain, & Halford, 1986; Posner & Boies, 1971). The rea- son becomes apparent from distributed representation models, and it is due

167Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

to the number of binding units being an exponential function of the num- ber of arguments (Halford, 1993; Halford et al., 1998b).

Relational Complexity and Creativity

As we saw earlier, creativity depends partly on more explicit representa- tion of the relational structure of a task. It is a matter of being able to change the structure, rather than being subject to the structure. This can be achieved using the higher rank representations, because of three factors: dynamic binding, higher-order relations, and ability to change strategies. We will consider these in turn.

Dynamic binding means that relational representations can be changed without incremental learning. Consider a representation of the proposition likes(John,Tom). We can change that to hates(John,Tom). Under the tensor product and similar models of relation symbol argument bindings mentioned earlier, this can be done by changing the vector on the relation symbol units. The result is a different pattern of activation on the binding units, while the argument vectors remain unchanged. The representations of John and Tom have remained constant, but the relation between them has been changed. This is a simple example of a principle that applies to all relational represen- tations with dynamic bindings, and it can have quite spectacular effects. We have shown how it can be employed in analogical and other forms of rea- soning (Halford, 1993; Halford et al., 1993; Wilson et al., 2001).

Representations of Rank 3 and above can support higher-order relations. The sentence “It’s good that Peter returned” can be expressed as GOOD(RETURNED(Peter)), and includes the higher-order relation symbol GOOD, which has one argument, the relation symbol RETURNED which itself has one argument. It is a Rank 3 representation, because it requires three vectors, representing GOOD, RETURNED, and Peter. It is not possi- ble to represent higher-order relations below Rank 3. They can of course be represented by higher rank representation. For example, “Tom hit Peter because he kissed Wendy” can be expressed as CAUSE(kiss(Peter,Wendy), hit(Tom,Peter)). In this case CAUSE is a higher-order relation symbol whose arguments are two, two-valued relation symbols. It can be represented by the higher-order binary relation CAUSE(kiss/Peter/Wendy,hit/Tom/Peter). The arguments of this higher-order relation can be unchunked to the first- order binary relations kiss(Peter,Wendy) and hit(Tom,Peter).

Higher-order relations permit more abstract correspondences to be rec- ognized. For example CAUSE(LOVE(Jane,Mark),JEALOUS(Michael, Mark)) is structurally identical to the example in the previous paragraph, and one can be mapped into the other solely on the basis of structural cor- respondence (Holyoak & Thagard, 1989). The mapping can be made inde- pendently of similarity of lower-order relations or arguments. Thus the level of abstraction, or content-independence increases with higher structural

168 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

complexity, and higher rank, because higher-order relations can be repre- sented. This enables analogies to be formed between situations that are structurally similar but are very different in content, and this is an impor- tant component of creativity.

Finally, ability to represent the structure of a task permits strategies to be developed. A computational model of this process has been developed in the domain of transitive inference. The implication is that understanding, which entails ability to represent the structure of a task, can guide the devel- opment of efficient skills, strategies, and procedures.

The Transitive Inference Mapping Model (TRIM) (Halford et al., 1995) simulates the acquisition of transitive inference strategies in children and adults. It is a self-modifying production system model using the PRISM (Ohlsson & Langley, 1986) modeling language. The base language is LISP.

The development of adequate strategies depends on ability to represent an ordered set of three elements. Without this ability, inadequate strategies result. To acquire such a strategy spontaneously, as most children do, is undoubtedly creative, but it depends on ability to represent ternary rela- tions. The development of a strategy depends on a representation of a con- cept of an ordered set, which has the following components:

• Understanding of one or more antisymmetric binary relations (e.g., larger than, better than). The child need not, and would not, know that they are asymmetric and binary. The child just needs to know examples of relations that have this property.

• Each element in the set occurs once and only once in an ordered set.

• End elements have the same relation to all other elements; for example, a>b, a>c, a>d in the string a,b,c,d.

• The position of an internal element is defined by relations to elements on either side of it; for example, b<a, b>c.

• The same relation must exist between all pairs, both adjacent and nonadjacent; for example, a>b, b>c,..., a>c, and so on.

Any ordered set of at least three elements instantiates these properties. It is sufficient therefore for the child to know any concrete instance of an ordered set of three or more elements. The three bears, the three billygoats gruff, or a set of blocks ordered for size, would suffice. An “abstract” rep- resentation is not required.

Transitive inference strategies develop under the dual constraints of the concept of order and problem solving experience. When the child is pre- sented with the problem of ordering elements, he or she first compares the known relations to the ordered set, which provides a “template” or concept of order. The comparison process is performed by analogical mapping. The correct order is determined by a valid and consistent mapping. No other cri- terion is required. Once this is determined, means-end analysis is used to

169Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

determine an operator that will produce the correct ordering of the prob- lem elements. Then a production is devised that incorporates the current problem situation in its condition side, and the correct operator in its action side. The production then fires and an ordering results.

If a production already exists that has the current situation in its condi- tion side, and if the production is above threshold strength, it fires. There is then no need to build a new production. If more than one production has conditions that match the current situation, the strongest production fires. Feedback is provided about the correctness of the ordering. Positive feed- back increases the strength of the production, while negative feedback weakens it.

The model shows how procedural knowledge, in the form of a transitive inference or ordering strategy, develops under the guidance of relational knowledge. There is creativity in the development of the strategy, because the model begins without any strategy. It is equipped initially with basic cognitive processes such as memory storage and retrieval, means-end analy- sis, and analogical mapping. From these basic tools, under the stimulus of problem solving experience with feedback, it develops new strategies. To do this however it requires relational knowledge, in the form of a representa- tion of an ordered set. This ordered set provides a template for the strate- gies that are developed.

To summarize this section, creative reasoning depends on ability to form dynamic representations of relations. This ability appears to have evolved relatively recently, and continues to develop throughout childhood. The complexity of relations that can be represented influences the conceptual power of creative reasoning, but is obtained at the cost of higher process- ing loads for more complex relations. In the next section we will examine more specific models of creativity using representations of this type.

REPRESENTATIONS IN COMPUTERS

Given that we can define the representations of biological intelligence in terms of the complexity of relations represented, we should be able to apply the same metric to computers.

Models of Creative Processing

One aspect of creativity is the retrieval of an idea that relates formerly unre- lated things. This can be illustrated using one of the practice items from the Remote Associates Test for creativity (Mednick, 1962). What word relates “rat,” “blue,” and “cottage”? (cheese). These words are not normally seen as related, but all have a common association with cheese. Another example is supplied by Rubin and Wallace (1989), as analyzed by Humphreys, Wiles, and Bain (1993), and Wiles, Halford, Stewart, Humphreys, Bain, and Wilson

170 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

(1994): name a mythical being that rhymes with post (ghost). Here the target element, ghost, is weakly associated with either the cue “mythical being” or the cue “rhymes with post,” but is quite strongly retrievable to the two cues. These examples may not meet the effectiveness criterion, because the set of relations created by the retrieval of cheese, or ghost, is not particularly coherent. Nevertheless the memory processes entailed in these tasks probably have much in common with those used by a painter producing new juxta- positions of colors and forms, a poet composing images, or a scientist see- ing a connection between apparently unrelated ideas. The memory model of Humphreys et al. (1989) has been demonstrated to handle this type of memory retrieval using Rank 3 tensor product representations.

Transfer of a relation from one domain to another is another aspect of creativity. Analogy is fundamental to creative thought in many domains (Holyoak & Thagard, 1995) and recent work has documented its use in sci- entific creativity (Dunbar, 2001). The scientific use of analogy is well illus- trated by the Rutherford analogy between the structure of the hydrogen atom and the structure of the solar system, as analyzed by Gentner (1983). Here the essential insight was recognizing that the structure of the atom entailed the principles of orbital motion, as exemplified in the solar system. This entails transferring the system of relations between solar and planetary bodies to the nucleus and electron. This can be modeled using the Struc- tured Tensor Analogical Reasoning (STAR) model (Halford et al., 1995; Wilson et al., in press).

In the STAR model relation symbol-argument bindings are represented as tensor products, as shown in Figure 4.2. The proposition MOTHER- OF(woman,baby) is represented as a Rank 3 tensor product. The three vec- tors represent the relation symbol MOTHER-OF and its arguments, “woman” and “baby.” As with the memory model of Humphreys et al. (1989), representations are superimposed. Therefore the bindings MOTHER-OF(mare,foal), MOTHER-OF(cat,kitten), as well as LOVES (mother,baby), . . . , LARGER-THAN(mare,foal) are all superimposed on the same tensor product.

Simple proportional analogies, such as woman:baby::mare:? can be solved by entering the base arguments, “woman” and “baby” into the representa- tion, and the output is a relation symbol bundle representing “MOTHER- OF,” “FEEDS,” “LARGER-THAN,” and so on. That is, it is a vector equiv- alent to the linear sum of vectors representing each of these outputs. In the next step this relation symbol bundle is the input, together with the first argument of the target, “mare,” as shown in Figure 4.2. The output is an argument bundle that includes “foal.” Possible solutions can be recognized by one or more cleanup processes, the simplest of which entails computing the inner product of vectors representing candidate solutions with this out- put. There is also an auto-associative clean-up process that performs the task in parallel (Chappell & Humphreys, 1993).

171Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

This model can solve all the major classes of analogies (Halford et al., 1994; Wilson et al., in press), but we will consider how the model would solve a simplified and idealized version of the Rutherford analogy.

We will assume that the structure of the base, the solar system, is known completely, but that the structure of the atom is incompletely known. We will further assume that the atom was known to comprise two major com- ponents, a nucleus and an electron, that there was a mutual attraction rela- tion between them, and a size difference (the nucleus was known to be of larger mass). Most important of all, we assume that the correspondence between the atom and the solar system had not been previously recognized. This recognition was at the core of the creative contribution. The simula- tion of the analogy in the STAR model begins with this information coded in a tensor product.

The discrepancies between this hypothetical situation and the actual state of Rutherford’s knowledge at the time are less relevant than the idea of starting with an incomplete representation of a target domain, finding a base domain to which it is analogous, transferring relations from the base to the target, then making candidate inferences about the target. It is this creative use of analogy that we want to simulate, the Rutherford analogy being a convenient and well-known example.

172 Creativity, Cognition, and Knowledge

FIGURE 4.2 Simple proportional analogies performed by the Structured Tensor Analogical Reasoning model.

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

Recognition of the solar system as a potential base is a memory retrieval process in the model. The relation symbols ATTRACTS and DIFFERENT- SIZE in the representation of the target serve as retrieval cues. If these are used as input to the representation of the task, the output is a tensor prod- uct of vectors representing all sets of arguments of these relation symbols. For example, when ATTRACTS is used as input, the output will include the tensor product of vectors representing “sun” and “planet,” plus other pairs of arguments of ATTRACTS.

When arguments of a potential base are recognized, they can be used as input. If the input is “sun” and “planet” the output will include ATTRACTS, LESS-MASSIVE-THAN, ORBITS-AROUND, and so on. These relation symbols become candidate inferences for the target, leading to the hypothesis that the electron orbits around the nucleus.

This simplified account of how analogical reasoning is simulated in the STAR model was designed to illustrate the main argument of this section, that a computational process can produce an output which is inherently creative, using representations of relations of the kind defined earlier. In this case a computational model has been constructed which can retrieve a base, the solar system, which is a potential analog of the target. Relations in the base are transferred to the target, and become candidate inferences for the target. The higher rank representations permit more powerful and more flexible reasoning, and also permit representations to be changed dynami- cally in the course of cognitive processing.

The Rutherford analogy illustrates the point that creativity can depend on representation of complex relations in an explicit form, that is accessi- ble to other cognitive processes. Expertise brings with it the ability to make the best use of processing capacity, so that a physicist can represent com- plex relations such as orbital motion efficiently as chunks, using minimum processing capacity. However, the processes of chunking and segmentation do not remove the need to represent complex relations. The recognition of the analogy between the atom and the solar system would have necessitated representation of some complex relations in parallel. In our simulation, it depends on representing quaternary relations in parallel.

Much less exalted cases of creativity require representation of relations in a similar way. When a four-year-old child is first working out how to cope with monotonic series, he has to use his representations creatively to develop efficient, reliable strategies and skills. If, for example, he notices that not only is his next oldest sibling taller than he is, but the oldest sib- ling is taller still, he has to work out some strategy or procedure for mak- ing reliable inferences about who is taller than whom. There is definite cre- ativity in this performance, the fact that all normal human beings go through it notwithstanding. It is what Boden (1990) calls “P-creative.” Our research (Halford et al., 1992; Halford et al., 1995) shows that such cre- ativity depends on ability to represent relations with an appropriate degree

173Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

of complexity. We suggest that development of strategies for transitive inference and serial order tasks requires ability to represent ternary rela- tions in parallel. If the child can only represent binary relations, inadequate strategies result.

ACQUISITION OF RELATIONAL KNOWLEDGE

Perhaps the most intriguing question of all is how relational knowledge develops. Unfortunately comparatively little is known about acquisition of relational knowledge, as distinct from associative knowledge. However, there have been attempts to model and investigate this process (Halford et al., 1998a; Holland, Holyoak, Nisbett, & Thagard, 1986; Holyoak, Koh, & Nisbett, 1989). There are two ways of formulating the question, which we will briefly consider.

The first is to consider the transition from Rank 1 neural nets to Rank 2 neural nets. This is basically to change from a three-layered net to a tensor net. So far we know of no published algorithm that could make that tran- sition. It is possible that some extension of cascade correlation, in which new hidden units are recruited as required to handle extra dimensions of the task, might be worth exploring in relation to this question. Another possi- bility would be the use of genetic algorithms. There are hybrid models, such as the tensor recurrent net of Phillips (1994), that certainly show promise. Then there have been interesting attempts to extend three-layered nets to handle relational information (Hinton, 1990; Phillips, 1994; but see also Halford et al., in preparation). But research so far has been more concerned with modeling phenomena that belong to particular levels (ranks), than with making the transition between levels. In our work we have been attempting to define the levels with more precision, using the concepts of relational complexity and representational rank. We believe this is a pre- requisite to understanding the transition process.

The second approach would be to take advantage of the transition as it occurs in nature. Actually this phenomenon is well established, but the lit- erature has been curiously reluctant to recognize its significance. We refer here to so-called “learning set” acquisition, discussed earlier. In this para- digm a series of isomorphic discrimination tasks are learned. Learning becomes progressively faster as more problems are experienced. Thus the participants, humans or (other) animals, learn something more general, more “abstract” than the properties of individual stimuli. This is often called “learning to learn” or “learning set.” Yet, despite the fact that the phenomenon goes back to Harlow (1949) and a few attempts at modeling notwithstanding (e.g., Levine, 1975; Restle, 1958) there is almost no under- standing of how improvement across problems occurs. It is standardly attributed to “rule learning” (e.g., Kendler, 1995), which explains nothing. As mentioned previously, we have proffered the explanation that learning

174 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

set entails acquisition of relational knowledge, together with analogical mapping of the relational representation from one problem to another (Halford, 1993; Halford et al., 1998a). If this interpretation is correct, it means that so-called learning set acquisition actually amounts to relational knowledge acquisition occurring in nature, right before our eyes! So why not study it? The only reason that occurs to us is the inhibitory effect of neo-behaviorist attitudes, together with the rather uninformative label “learning set,” that stand in the way of cognitive analyses of the phenome- non. Whatever the cause, the inhibition of work on this problem is very regrettable, because it means we lose the opportunity to study the forma- tion of abstract, relational processes as they occur.

CONCLUSION

We suggest that human creativity depends on our ability to represent complex relations in parallel, in a way that is explicit and accessible to other cognitive processes. The kinds of relations required are defined in terms of the number of arguments, or number of separately varying, inter- acting components, that are related. The representations required are defined in terms of rank. The model can be neatly implemented in a PDP architecture, using tensor products, which enables many of the properties of each rank to be simulated.

In less technical terms, creativity depends on having an adequate cognitive “window” on sets of interacting processes. If a person or, more generally, a system cannot represent the relations between the relevant interacting processes, creative, non-routine, or non-algorithmic thought is not possible.

DISCUSSION

TD: You’ve already incorporated new material into the chapter in response to some of my earlier questions, but I do have one more question for you.

You say that creativity requires explicit representations that are accessible to other cognitive processes. More specifically, you say that it requires the ability to represent and recursively modify explicit, complex relations in parallel. You discuss this in the context of your theory of representational ranks and provide a hierarchy of cognitive processes that is defined in terms of the complexity of the relations that can be handled at each level.

Your main model of these ranks is a connectionist one—and here I have prob- lems. The inner states of connectionist systems are notoriously hard to access and explicitate, so I wonder how explicit and accessible your representations really are. You say that they are explicit in the sense that they are explicitly symbolized. Your example is LARGER-THAN(whale,dolphin). Here the relation is explicitly symbol- ized by “LARGER-THAN,” which you say is accessible to other cognitive processes, as in

BECAUSE(LARGER-THAN(whale,dolphin), AVOIDS(dolphin, whale))

175Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

But these are symbolisms that you have given to the system. To what extent are they, or the relations that they express, explicitly represented in the states of the system? And so, to what extent are they accessible to the system and to other systems run- ning other cognitive processes?

GH/WW: We agree that accessibility is an issue in neural net models. Thus the representations in the hidden layer of a lot of three-layered nets are not semantically interpretable, as Smolensky (1988) has pointed out. Entities that are represented do not retain their identity in the representation, and the components are not recover- able, so the representation is not compositional. Consider, for example, the config- ural discrimination task discussed earlier. A neural net model might include hidden units that represent the configurations circle/white, triangle/white, and so on. The activations on these units are important for computing the output, but they are only accessible to the net by means of the inputs. That is, activation of a set of inputs is transmitted, via variable connections, to the hidden units, the activations on which are transmitted via another set of variable weights, to the output units. But, as we mentioned previously, the net cannot query the hidden units to ask questions such as “what is combined with circle when the output is positive?” Researchers may be able to study hidden unit activations by techniques such as clustering analysis, but this information is not accessible to the organism whose cognitions are being mod- eled by the net, as we all seem to agree.

However our contention is that representations in tensor product nets are more accessible than those in the hidden unit of a three-layered net of the kind that might be used to model conditional discrimination. Thus the binary relational instance MOTHER-OF(woman,baby) is represented by the tensor product of three vectors, representing MOTHER-OF, woman, and baby. The three components are each rep- resented by a vector, that corresponds to activations on a set of neural units. Their composition into the relational instance is represented by the three-dimensional matrix or tensor product, corresponding to activation of units representing both the components and their binding. Thus the components, MOTHER-OF, woman, and baby retain their identity in the composite representation. Furthermore they are acces- sible to the system, because the model includes operations for recovering the compo- nents from the tensor product representation. As illustrated in Figure 4.2, if the input is “woman” and “baby” the output will be MOTHER-OF plus other predicates bound to these arguments. Or again, if the input were MOTHER-OF and “woman” the output would be “baby.” The STAR model routinely accesses tensor product rep- resentations in this way. The representations are also modifiable, by changing the vec- tor on the predicate units, as indicated earlier. Thus tensor product representations of relations, as used in the STAR model, are accessible in a way that, say, representations in a three-layered net model of conditional discrimination are not.

What makes our representations accessible is that they are symbols, whereas rep- resentations in a typical three-layered net model are inaccessible because they are not symbols. To avoid confusion here, it is important to note that accessibility of cognitive processes does not depend on whether they are modeled by neural nets, but on whether they are symbolic. Cognitive processes modeled by neural nets are not inherently inaccessible. Neural nets can be used to model symbols, and processes for accessing them, which we have done with our tensor product formalism. It is important not to confuse the cognitive processes modeled, in this case symbols and

176 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

processes for operating on them, with the architecture of the model, in this case neu- ral nets. To illustrate, people can be aware of symbols like “whale.” When this occurs, the neurophysical reality underlying their awareness of the symbol “whale” is a pattern of neural activity in (parts of) their brain. The fact that we are unable to interpret the neural patterns directly does not invalidate their awareness of and access to the symbol “whale.” In our, much simpler, model of some aspects of cog- nition, similarly, the fact that symbols are encoded as binding unit patterns in ten- sor product networks does not invalidate our view that these structures encode accessible symbols. It is not the neural patterns, or the neural net patterns, that peo- ple are aware of, but the symbols they represent. To argue otherwise is to confuse different levels of description.

While we would certainly not contend that tensor product models have solved all the problems of modeling symbolic processes in neural nets, we do believe that they have taken a significant step in that direction. It is true of course that in achieving this we have sacrificed the learning functions of three-layered nets. Consequently, we have to agree that in our model the representations are supplied to the net by the modelers. Therefore representations are not emergent properties of tensor product nets, as they are with three-layered nets. So at the present time the two types of neu- ral net architectures are most appropriately seen as complementary.

REFERENCES

Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.

Andrews, G., & Halford, G. S. (1998). Children’s ability to make transitive inferences: The importance of premise integration and structural complexity. Cognitive Development, 13, 479–513.

Baillargeon, R. (1995). A model of physical reasoning in infancy. In C. Rovee- Collier & L. P. Lipsitt (Eds.), Advances in infancy research (Vol. 9, pp. 305–371). Norwood, NJ: Ablex.

Bitterman, M. E. (1960). Toward a comparative psychology of learning. American Psychologist, 15, 704–712.

Boden, M. A. (1990). The creative mind: Myths and mechanisms. London: Weidenfeld and Nicholson.

Chappell, M., & Humphreys, M. S. (1993). Analysis of a neural network with application to human memory modelling. Fourth Australian Conference on Neural Networks.

Clark, A., & Karmiloff-Smith, A. (1993). The cognizer’s innards: A psychological and philosophical perspective on the development of thought. Mind and Language, 8, 487–519.

Codd, E. F. (1990). The relational model for database management: Version 2: Reading, MA.: Addison-Wesley.

Dartnall, T. H. (1994). Creativity, thought and representational redescription. In T. H. Dartnall (Ed.), Artificial Intelligence and creativity: An interdisciplinary approach. Dordrecht, Netherlands: Kluwer Academic Publishers.

Dunbar, K. (2001). The analogical paradox: Why analogy is so easy in naturalistic settings, yet so difficult in the psychological laboratory. In D. Gentner,

177Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

K. J. Holyoak, & B. Kokinov (Eds.), The analogical mind: Perspectives from cognitive science (pp. 313–334). Cambridge, MA: MIT Press.

Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179–211. Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy.

Cognitive Science, 7, 155–170. Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer.

Cognitive Psychology, 15, 1–38. Greeno, J. G., Riley, M. S., & Gelman, R. (1984). Conceptual competence and

children’s counting. Cognitive Psychology, 16, 94–143. Halford, G. S. (1992). Analogical reasoning and conceptual complexity in cognitive

development. Human Development, 35, 193–217. Halford, G. S. (1993). Children’s understanding: The development of mental

models. Hillsdale, NJ: Erlbaum. Halford, G. S., Bain, J. D., Maybery, M., & Andrews, G. (1998). Induction of

relational schemas: Common processes in reasoning and complex learning. Cognitive Psychology, 35, 201–245.

Halford, G. S., & Leitch, E. (1989). Processing load constraints: A structure- mapping approach. In M. A. Luszcz & T. Nettelbeck (Eds.), Psychological development: Perspectives across the life-span (pp. 151–159). Amsterdam: North-Holland.

Halford, G. S., Maybery, M. T., & Bain, J. D. (1986). Capacity limitations in children’s reasoning: A dual task approach. Child Development, 57, 616–627.

Halford, G. S., Maybery, M. T., Smith, S. B., Bain, J. D., Dickson, J. C., Kelly, M. E., & Stewart, J. E. M. (1992). Acquisition of reasoning: A computational model of strategy development in transitive inference. University of Queensland, Australia: Department of Psychology, Centre for Human Information Processing and Problem Solving.

Halford, G. S., Phillips, S., & Wilson, W. H. (2001). Structural complexity in cognitive processes: The concept of representational rank. Manuscript in preparation.

Halford, G. S., Smith, S. B., Dickson, J. C., Maybery, M. T., Kelly, M. E., Bain, J. D., & Stewart, J. E. M. (1995). Modeling the development of reasoning strategies: The roles of analogy, knowledge, and capacity. In T. Simon & G. S. Halford (Eds.), Developing cognitive competence: New approaches to cognitive modelling (pp. 77–156). Hillsdale, NJ: Erlbaum.

Halford, G. S., Wiles, J., Humphreys, M. S., & Wilson, W. H. (Eds.). (1993). Parallel distributed processing approaches to creative reasoning: Tensor models of memory and analogy. In T. H. Dartnall, S. Kim, R. Levinson, F. Sudweeks, D. Subramanian (Eds.), Artificial Intelligence and creativity, papers from the 1993 Spring Symposium Technical Report. Menlo Park, CA: AAAI Press.

Halford, G. S., Wilson, W. H., Guo, J., Gayler, R. W., Wiles, J., & Stewart, J. E. M. (1994). Connectionist implications for processing capacity limitations in analogies. In K. J. Holyoak & J. Barnden (Eds.), Advances in connectionist and neural computation theory, Vol. 2: Analogical connections (pp. 363–415). Norwood, NJ: Ablex.

Halford, G. S., Wilson, W. H., & Phillips, S. (1998a). Processing capacity defined by relational complexity: Implications for comparative, developmental, and cognitive psychology. Behaviorial and Brain Sciences, 21, 803–831.

178 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

Halford, G. S., Wilson, W. H., & Phillips, S. (1998b). Relational processing in higher cognition: Implications for analogy, capacity and cognitive development. In K. Holyoak, D. Gentner, & B. Kokinov (Eds.), Advances in analogy research: Integration of theory and data from the cognitive, computational, and neural sciences (pp. 57–73). Sofia, Culgaria: New Bulgarian University.

Harlow, H. F. (1949). The formation of learning sets. Psychological Review, 42, 51–65.

Harlow, H. F. (1959). Learning set and error factor theory. In S. Koch (Ed.), Psychology: A study of a science: Vol. 2. General systematic formulations, learning, and special processes (pp. 492–537). New York: McGraw-Hill.

Hinton, G. E. (1990). Mapping part-whole hierarchies into connectionist networks. Artificial Intelligence, 46, 47–75.

Hodkin, B. (1987). Performance model analysis in class inclusion: An illustration with two language conditions. Developmental Psychology, 23, 683–689.

Holland, J. H., Holyoak, K. J., Nisbett, R. E., & Thagard, P. R. (1986). Induction: Processes of inference, learning and discovery. Cambridge, MA: Bradford Books/MIT Press.

Holyoak, K. J., Koh, K., & Nisbett, R. E. (1989). A theory of conditioning: Inductive learning within rule-based hierarchies. Psychological Review, 96, 315–340.

Holyoak, K. J., & Thagard, P. (1989). Analogical mapping by constraint satisfaction. Cognitive Science, 13, 295–355.

Holyoak, K. J., & Thagard, P. (1995). Mental leaps. Cambridge, MA: MIT Press. Humphreys, M. S., Bain, J. D., & Pike, R. (1989). Different ways to cue a coherent

memory system: A theory for episodic, semantic and procedural tasks. Psychological Review, 96, 208–233.

Humphreys, M. S., Wiles, J., & Bain, J. D. (1993). Memory retrieval with two cues: Think of intersecting sets. In D. E. Meyer & S. Kornblum (Eds.), Attention and performance XIV: A silver jubilee. Hillsdale, NJ: Erlbaum.

Karmiloff-Smith, A. (1990). Constraints on representational change: Evidence from children’s drawing. Cognition, 34(1), 57–83.

Kendler, T. S. (1995). Levels of cognitive development. Mahwah, NJ: Erlbaum. Levine, M. (1975). A cognitive theory of learning. Hillsdale, NJ: Erlbaum. Mandler, J. M. (1992). How to build a baby: II. Conceptual primitives.

Psychological Review, 99, 587–604. Maybery, M. T., Bain, J. D., & Halford, G. S. (1986). Information processing

demands of transitive inference. Journal of Experimental Psychology: Learning, Memory and Cognition, 12, 600–613.

McClelland, J. L., & Rumelhart, D. E. (1985). Distributed memory and the representation of general and specific information. Journal of Experimental Psychology: General, 114, 159–188.

McGonigle, B., & Chalmers, M. (1984). The selective impact of question form and input mode on the symbolic distance effect in children. Journal of Experimental Child Psychology, 37, 525–554.

Mednick, S. A. (1962). The associative basis of the creative process. Psychological Review, 69, 220–232.

Ohlsson, S., & Langley, P. (1986). PRISM tutorial and manual. Irvine, CA: University of California.

Phillips, S. (1994). Strong systematicity within connectionism: The tensor-recurrent network. Sixteenth Annual Conference of the Cognitive Science Society.

179Creativity, Relational Knowledge, and Capacity: Why Are Humans So Creative?

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .

Phillips, S., Halford, G. S., & Wilson, W. H. (1995, July). The processing of associations versus the processing of relations and symbols: A systematic comparison. Seventeenth Annual Conference of the Cognitive Science Society, Pittsburgh, PA.

Posner, M. I., & Boies, S. J. (1971). Components of attention. Psychological Review, 78, 391–408.

Premack, D. (1983). The codes of man and beasts. Behavioral and Brain Sciences, 6, 125–167.

Quinn, P. C., Eimas, P. D., & Rosenkrantz, S. L. (1993). Evidence for representations of perceptually similar natural categories by 3-month-old and 4-month-old infants. Perception, 22, 463–475.

Quinn, P. C., & Johnson, M. H. (1997). The emergence of perceptual category representations in young infants: A connectionist analysis. Journal of Experimental Child Psychology, 66, 236–263.

Restle, F. (1958). Towards a quantative description of learning set data. Psychological Review, 65, 77–91.

Rivera, S. M., Wakeley, A., & Langer, J. (1999). The drawbridge phenomenon: Representational reasoning or peceptual preference? Developmental Psychology, 35, 427–435.

Rosch, E., & Mervis, C. B. (1975). Family resemblences: Studies in the internal structure of categories. Cognitive Psychology, 7, 573–605.

Rubin, D. C., & Wallace, W. T. (1989). Rhyme and reason: Analysis of dual retrieval cues. Journal of Experimental Psychology: Learning, Memory and Cognition, 15, 698–709.

Rudy, J. W. (1991). Elemental and configural associations, the hippocampus and development. Developmental Psychobiology, 24, 221–236.

Schmajuk, N. A., & DiCarlo, J. J. (1992). Stimulus configuration, classical conditioning, and hippocampal function. Psychological Review, 99, 268–305.

Smolensky, P. (1988). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1–74.

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159–216.

VanLehn, K., & Brown, J. S. (1980). Planning nets: A representation for formalizing analogies and semantic models of procedural skills. In R. E. Snow, P. A. Federico, & W. E. Montague (Eds.), Aptitude learning and instruction. Vol. 2. Cognitive process analyses of learning and problem solving (pp. 95–137). Hillsdale, NJ: Erlbaum.

Wiles, J., Halford, G. S., Stewart, J. E. M., Humphreys, M. S., Bain, J. D., & Wilson, W. H. (1994). Tensor models: A creative basis for memory retrieval and analogical mapping. In T. H. Dartnall (Ed.), Artificial Intelligence and creativity: An interdisciplinary approach (pp. 147–161). Dordrecht: Kluwer.

Wilson, W. H., Halford, G. S., Gray, B., & Phillips, S. (2001). The STAR-2 model for mapping hierarchically structured analogs. In D. Gentner, K. Holyoak, & B. Kokinov (Eds.), The analogical mind: Perspectives from cognitive science (pp. 125–159). Cambridge, MA: MIT Press.

Wynne, C. D. L. (1995). Reinforcement accounts for transitive inference performance. Animal Learning and Behavior, 23, 207–217.

180 Creativity, Cognition, and Knowledge

C op

yr ig

ht ©

2 00

2. G

re en

w oo

d P

ub lis

hi ng

G ro

up , I

nc or

po ra

te d.

A ll

rig ht

s re

se rv

ed .