FOR AKADEMIK ONLY

February 2, 2007 15:37 WSPC/169-ACS 00084

Advances in Complex Systems, Vol. 9, No. 4 (2006) 315–335 c© World Scientific Publishing Company

COOPERATION, COLLECTIVES FORMATION AND SPECIALIZATION

CHRISTOPH HAUERT

Program for Evolutionary Dynamics, Harvard University, One Brattle Square, Cambridge MA 02138, USA

christoph [email protected]

Received 11 September 2005 Revised 30 April 2006

Cooperation in spatial evolutionary game theory has revealed various interesting insights into the problem of the evolution and maintenance of cooperative behavior. In social dilemmas, cooperators create and maintain a common resource at some cost to them- selves while defectors attempt to exploit the resource without contributing. This leads to classical conflicts of interest between the individual and the community with the prisoner’s dilemma as the most prominent mathematical metaphor to describe such sit- uations. The evolutionary fate of cooperators and defectors sensitively depends on the interaction structure of the population. In spatially extended populations, the ability to form clusters or collectives often supports cooperation by limiting exploitation to the cluster boundaries but often collectives formation may also inhibit or even eliminate cooperation by hindering the dispersal of cooperators. Another attempt at resolving the conflict of interest allows individuals to drop out of unpromising public enterprises and hence changes compulsory interactions into voluntary participation. This leads to a cyclic dominance of cooperators, defectors and loners that do not participate and gives rise to oscillatory dynamics which again subtly depends on the population structure. Here we review recent advances in the dynamics of cooperation in structured popula- tions as well as in situations where cooperative investments vary continuously. In such continuous games, the evolutionary dynamics driven by mutation and selection can lead to spontaneous diversification and specialization into high and low investing individuals which provides a natural explanation for the origin of cooperators and defectors.

Keywords: Evolutionary game theory; spatial structure; social dilemmas; synchroniza- tion; oscillations; evolutionary branching; tragedy of the commons; tragedy of the com- mune; public goods games; prisoner’s dilemma; voluntary participation; continuous games.

1. Introduction

The emergence of cooperative behavior has challenged evolutionary biologists for decades. According to Darwinian selection, any behavior that benefits others but

315

February 2, 2007 15:37 WSPC/169-ACS 00084

316 C. Hauert

not the individual itself should be doomed and vanish. This is in obvious contrast with the abundance of cooperation in nature ranging from bacterial colonies to animal and human societies [5,11]. Recently, the problem of cooperation even made it onto the prominent list of the 25 most important unresolved scientific questions published in Science [50]. Following the seminal works of Trivers [59] as well as Axelrod and Hamilton [1] much theoretical effort has been expended on the under- standing of the evolution of cooperative behavior based on the game theoretical model of the prisoner’s dilemma. In the prisoner’s dilemma, two interacting indi- viduals can either cooperate, which provides a benefit β to the co-player at a cost γ to the cooperator (with β > γ), or defect, which neither provides benefits nor incurs costs. Thus, irrespective of what the co-player does, it is always advantageous to choose defection. Consequentially the two players end up with nothing instead of the favorable reward for mutual cooperation — and hence the dilemma.

The prisoner’s dilemma can be easily extended to interactions among groups of N individuals. Such interactions are usually referred to as public goods games [31] — a term that originated in economics — but often they are also termed N -player prisoner’s dilemmas [3, 27], the Tragedy of the Commons [17], or free- rider problems [42]. In a typical public goods experiment a group of six players is endowed with one dollar each. Every player is then offered the opportunity to invest the dollar into a common pool knowing that the total amount in the common pool will be tripled by the experimenter and equally shared among all members of the group irrespective of their contributions. Realizing that each invested dollar returns only 50 cents to the investor, rational players will withhold their contributions and attempt to free ride on the other players’ contributions. Consequentially, the group foregoes the public good and fails to increase the initial capital even though they could have tripled it if everybody had cooperated. This is in stark contrast to experimental findings. For example, humans display a surprisingly high readiness to cooperate in prisoner’s dilemma [40, 64] or public goods interactions [12]. This calls for a better theoretical understanding of the problem in order to identify mechanisms that can help to overcome the dilemma.

In formal terms, the payoffs for defectors and cooperators in a group with k cooperators are given by PD(k) = rkc/N and PC (k) = PD(k) − c, respectively, where c denote the costs of cooperation and r the multiplication factor of the common pool, i.e. of the total contributions. A true public goods interaction is characterized by the facts that (i) groups of cooperators must outperform groups of defectors but (ii) in mixed groups defectors must outperform cooperators. This requires 1 < r < N . Otherwise, for r < 1 groups of cooperators would be worse off than groups of defectors, whereas for r > N the dilemma would be resolved because every invested dollar returns more than one dollar to the investor and thus cooperation would become the dominant solution. However, also note that defectors still outperform cooperators in mixed groups but now they can further improve their payoff by also switching to cooperation. For pairwise interactions (N = 2) the prisoner’s dilemma and public goods approach become identical with

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 317

the following transformation: β = rc/2 and γ = (r/2 − 1)c, i.e. γ specifies the net or effectively arising costs from the act of cooperation [28].

In order translate these game theoretical considerations into an evolutionary setting, Maynard Smith and Price [38] simply and ingeniously related payoffs from interactions with other members of the population to fitness, i.e. to the reproductive success of individuals. Consider a large population with a fraction x cooperators and 1−x defectors. In well-mixed populations, i.e. where individuals interact in ran- domly formed groups of size N , the average payoff of cooperators fC and defectors fD is given by

fC = k=N−1∑

k=0

( N − 1

k

) xk(1 − x)N−1−kPC (k + 1), (1a)

fD = k=N−1∑

k=0

( N − 1

k

) xk(1 − x)N−1−kPD(k). (1b)

The evolutionary fate of cooperators, i.e. the time evolution of the frequency of cooperators x is then determined by the replicator dynamics [30, 58]:

ẋ = x(fC − f̄ ), (2) where f̄ = xfC + (1 − x)fD denotes the average population payoff. In prisoner’s dilemma and public goods interactions fC < f̄ holds for all x. For pairwise interac- tions this is readily verified for fC = c(xr−1) and f̄ = cx(r/2(x+1)−1) with r < 2. Thus, in the absence of supporting mechanisms selection works against coopera- tors and drives them to extinction. This outcome changes upon considering more sophisticated strategies which are able to condition their behavior on past inter- actions with the same individuals (direct reciprocity [59]) or on the reputation of other individuals (indirect reciprocity [47]; reward and punishment [52]) but these are other lines of research that will not be considered here.

The failure of cooperation is also based on the assumption that populations are unstructured such that individuals interact in randomly formed groups. This conclusion no longer holds if interactions are restricted to a limited local neigh- borhood such as in spatially structured populations [44]. Spatial extension enables cooperators to thrive by forming compact clusters or collectives and thereby reduc- ing exploitation by defectors [18, 62]. This can lead to the emergence of intriguing spatio-temporal patterns and critical phase transitions [56] as reviewed in Sec. 2. Further reasons why cooperation fails are the compulsory interactions and lack of alternatives. In nature, individuals often have the capacity to refuse to participate in common enterprises. Such voluntary prisoner’s dilemma or public goods games can be modeled by introducing a third strategic type, the loners [23]. Loners are risk averse and refuse to participate in the public endeavor and instead prefer to rely on some autarkic resource. This results in a rock-paper-scissors type cyclic dominance of the three strategic types of cooperators, defectors and loners. In spatial settings, the cyclic dominance drives fascinating spatio-temporal patterns, which includes

February 2, 2007 15:37 WSPC/169-ACS 00084

318 C. Hauert

traveling waves, but, moreover, raises interesting issues related to synchronization of oscillations across spatial dimensions [55]. This is reviewed in Sec. 3.

The prisoner’s dilemma and public goods games represent the most stringent form of social dilemmas [7] because cooperation is dominated by defection. Social dilemmas are characterized by a conflict of interest between the individuals and the community but the actual severity of the dilemma can vary. In general, cooperation is not necessarily dominated by defection such that cooperators and defectors can co-exist in snowdrift type interactions [54] even in unstructured populations. Intro- ducing spatial extension again enables cooperators to form clusters but the cluster shape and dynamics under these relaxed conditions are surprisingly different from spatial prisoner’s dilemma or public goods games. In fact, spatial structure often turns out to be detrimental to cooperation by reducing the equilibrium fraction of cooperators as compared to well-mixed populations [24]. This is reviewed in Sec. 4 together with a generalized framework to study cooperation in social dilemmas, which is based on discounted and synergistically enhanced values of accumulated cooperative benefits [26].

In biological systems interactions tend not to be as clear-cut as black and white or cooperate and defect and thus in many situations it might be more appro- priate to allow for a continuous range of degrees of cooperation. Evolutionary changes in the degree of cooperation are driven by mutation and selection. In pris- oner’s dilemma and public goods games this change has little effect on the evo- lutionary outcome in unstructured populations: cooperative investments decrease over time and eventually disappear. However, as before, this changes when intro- ducing spatial extension such that higher investors can survive by forming com- pact clusters and thereby minimizing exploitation by lower investors. Eventually, this process converges to an equilibrium level of cooperative investments [35, 36]. In contrast, considering snowdrift type interactions in a setting with continuous degrees of cooperative investments leads to much richer dynamics. Based on the fact that cooperators and defectors co-exist in the traditional setting, one might naively expect that in unstructured populations intermediate investment levels would evolve. Although this is one possible outcome, a far more intriguing sce- nario can occur where two distinct phenotypic clusters of high and low investors evolve and co-exist. This suggests a natural explanation for the evolutionary ori- gin of cooperators and defectors [10]. Thus, in some situations, natural selec- tion may actually favor clear-cut behavioral patterns. Interestingly, this outcome with highly distinct degrees of cooperative investments runs against the accepted notion of fairness in human interactions and may give rise to social tension — a scenario that was termed the Tragedy of the Commune [10]. This is reviewed in Sec. 5.

Section 6 concludes this review by putting the theoretical results into perspective with experimental findings and by suggesting potential applications and verifica- tions of the theory in biologically relevant settings with an outlook on promising directions for future theoretical and empirical investigations.

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 319

2. Collectives Formation and Phase Transitions

In unstructured populations cooperators readily disappear in prisoner’s dilemma (N = 2) or public goods games. Recall that for true public goods games, the multiplication factor r must be smaller than the interaction group size N (for r > N cooperation becomes dominant). In contrast, in structured populations cooperators can thrive already for r < N in both prisoner’s dilemmas [29, 44, 45,] and public goods games [28]. The limited local interactions enables cooperators to form collectives or clusters and thereby reducing exploitation by defectors (see Fig. 1).

The evolutionary dynamics in structured populations can be modeled as follows: first, a focal individual is randomly drawn and its payoff or fitness is determined by a single interaction within its neighborhood. If the interaction group size N is smaller than the neighborhood size (including the focal individual), then N − 1 random neighbors are selected for the interaction (plus the focal individual). Sec- ond, randomly pick one of the focal individual’s neighbors and similarly determine its fitness from a single interaction within its respective neighborhood. Finally, a probabilistic comparison of the two payoffs determines which individual’s offspring replaces the focal individual. In the following we use a particularly simple compar- ison where the neighbor’s offspring replaces the focal individual with a probability proportional to the payoff difference provided that the neighbor performed better than the focal individual and with probability zero otherwise (for other updating mechanisms see, for example, Refs. 19, 21 and 56). This represents an individual

multiplication factor r 3 3.2 3.4 3.6

0

0.2

0.4

0.6

0.8

1

fr eq

ue nc

ie s

of s

tr at

eg ie

s

(a) (b) (c)

Fig. 1. (Color online) Spatial structure with limited local interactions enables cooperators to thrive in public goods interactions by forming compact clusters and thereby reducing exploitation by defectors. (a) Frequency of cooperators (�) and defectors (�) as a function of the multiplication factor r of the public good. Individuals interact on a square lattice (100 × 100) within the Moore neighborhood (the eight neighbors reachable by a chess-kings-move) in groups of N = 5. For r < rC ≈ 3.01 cooperators vanish, for rC < r < rD ≈ 3.68 cooperators and defectors co-exist in dynamical equilibrium and for r > rD cooperation dominates and reaches fixation. (b, c) Typical snapshots of lattice configurations where cooperators (blue/dark grey) and defectors (red/light grey) co-exist. (b) For small r, i.e. close to the extinction threshold of cooperators (r = 3.1) and (c) for large r close to the extinction threshold of defectors (r = 3.65).

February 2, 2007 15:37 WSPC/169-ACS 00084

320 C. Hauert

based analogy of the replicator dynamics [30] and actually recovers it in the limit of infinite population size and increasing neighborhood sizes. All simulations pre- sented in this article are based on this update procedure. Unstructured populations correspond to networks where every individual is linked to every other member of the population (fully connected graph).

In spatial public goods games, defection reigns for low r just as in unstructured populations. For r above a critical threshold, rC , cooperators quickly increase in fre- quency and co-exist with defectors until above another critical threshold, rD < N , defectors are driven to extinction (Fig. 1(a)). In unstructured populations, the tran- sition from dominant defection to dominant cooperation occurs only for r > N . Near the extinction threshold of cooperators and defectors, the rare strategy forms iso- lated clusters (Fig. 1(b, c)). The clusters move randomly across the lattice and can coalesce or divide. This suggests that the system becomes equivalent to a branching and annihilating random walk [4] which exhibits a critical phase transition belong- ing to the universality class of directed percolation [37]. This has been confirmed for both spatial prisoner’s dilemmas [55, 57] and spatial public goods games [56].

While critical phase transitions are exiting for physicists, their presence alone may not be exceedingly important in biologically relevant scenarios. However, they do have substantial implications with far reaching consequences. For example, this demonstrates that small changes in one parameter can have tremendous effects on the equilibrium state of a system. But more importantly still, this indicates that in vulnerable systems it might be difficult, if not intrinsically impossible, to define characteristic scales in time and space that allow to fully understand the sys- tems dynamics in empirical settings because both spatial and temporal correlation lengths diverge when approaching the critical threshold. This might be particularly relevant in conservation biology dealing with species interactions at the edge of extinction.

The results for spatial prisoner’s dilemmas and public goods games promoted and supported the general conclusion that spatial structure is capable of promoting and maintaining cooperation. The next section, however, demonstrates that spatial structure may not be as universally beneficial because in many biologically relevant situations spatial structure can actually be detrimental to cooperation.

3. Synergy and Discounting of Cooperation

The abundance of the puzzle of cooperative behavior in human and animal soci- eties is reflected in numerous game theoretical models to address this problem. Apart from the aforementioned prisoner’s dilemma, public goods games, free-rider problems [42], or the Tragedy of the Commons [17], closely related scenarios are described by Snowdrift games [54] or by-product mutualism [6]. Despite the vari- ety, all models actually share a common pattern: all represent conflicts of interest between the individuals and the group — only the severity of the conflict varies. Such situations are generally referred to as social dilemmas [7]: cooperators produce

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 321

a valuable common good at some cost to themselves while defectors attempt to exploit the resource without contributing. Thus, groups of cooperators are better off than groups of defectors. However, in any mixed group, defectors outperform cooperators and hence the dilemma.

An encompassing framework to model cooperation in social dilemmas has recently been proposed based on synergistic enhancement or discounting of the value of accumulated benefits [26]. In groups of N interacting individuals, coopera- tors produce a beneficial common resource b at a cost c to themselves. The common resource is evenly shared among all group members regardless of whether they con- tributed or not. If there are several cooperators in a group, the actual value of additional benefits may be synergistically enhanced or discounted by a factor w. The first cooperator produces a benefit b/N for every member of the group, the second increases everyones benefit by b/N w, the third by b/N w2, etc. Thus, the total payoffs for defectors and cooperators in a group with k cooperators are

PD(k) = b

N (1 + w + w2 + · · · + wk−1) = b

N

1 − wk 1 − w , (3a)

PC (k) = PD(k) − c. (3b)

Note that neither discounting nor synergy involve temporal components refer- ring to potential future benefits (temporal discounting is often considered in eco- nomics [13]). Discounting of benefits occurs, for example, in foraging yeast cells. They produce and secret an enzyme to lyse their environment and thereby produce a valuable common food resource, which is prone to exploitation by other cells. A single cell may be better off producing the enzyme (prevent starvation) if no one else does. However, if an increasing number of cells secrete the enzyme the value of the additional benefits decreases until further increases become useless because of the cells limitations of food intake [15]. Similarly, accumulated benefits can be synergistically enhanced in situations where cooperators produce substances for chemical reactions [16].

In unstructured populations, this system can be fully analyzed by inserting Eq. (3) into Eqs. (1) and (2). Apart from the two trivial fixed points x∗0 = 0 and x∗1 = 1 there may exist a third interior fixed point x

∗ 2 = [1 − (cN/b)1/(N−1)]/[1 − w],

depending on the parameter values. This results in a natural classification of social dilemmas which recovers the four basic scenarios of evolutionary dynamics [48]. (i) Defection is dominant if cN/b > 1 and cN/b > wN−1 holds. This corresponds to interactions of the type of prisoner’s dilemmas or public goods games. (ii) Coop- erators and defectors co-exist at an equilibrium level x∗2 if 1 > cN/b > w

N−1. The fact that cooperators persist, reflects the relaxed conditions of the social dilemma as represented by snowdrift type interactions where the rare type is always favored. (iii) Cooperation dominates if cN/b < 1 and cN/b < wN−1 holds. Even though the social dilemma is completely relaxed, it still holds that defectors outperform coop- erators in mixed groups. However, defectors would be even better off by switching

February 2, 2007 15:37 WSPC/169-ACS 00084

322 C. Hauert

(a) (b) (c)

Fig. 2. (Color online) Effects of spatial structure in social dilemmas. (a) Phase plane diagram depicting parameter regions where spatial structure (square lattice, Moore neighborhood) pro- motes (blue, +) or inhibits (red, −) cooperation or has no effect (white) for interactions in groups of size N = 5 as compared to unstructured populations. The saturation of the colors indicates the strength of the effect. The phase plane is divided into four regions corresponding to social dilemmas of different severity. The dash-dotted line separates regions where benefits are discounted (left) and synergistically enhanced (right). Region (i) corresponds to public goods type interactions where defection dominates. For a small region, spatial clustering enables cooperators to thrive (◦ marks the settings in Fig. 1(b)). In the region of snowdrift type interactions (ii) cooperators and defectors co-exist in unstructured populations. The dashed line indicates equilibrium states with equal proportions of cooperators and defectors in well-mixed populations (◦ marks the settings in panels (b, c)). (b) typical lattice configuration of cooperators (blue/dark grey) and defectors (red/light grey) where spatial structure can again be beneficial to cooperation and is boosted to 90% instead of 50% in unstructured populations (b = 7.62, c = 1, w = 0.8). (c) spatial structure often turns out to be detrimental to cooperation with 40% cooperators as compared to 50% in unstructured populations (b = 80, c = 1, w = 0). In region (iii) cooperation is dominant in the form of by-product mutualism. Spatial structure has no effect on the evolutionary success of coopera- tion. In the region of bi-stability (iii) the basin of attraction of the cooperative state is considerably enhanced. Variations of the group size N do not alter the qualitative results and essentially only

change the boundary between (ii) and (iii) or (i) and (iv), respectively. For N = 2 the boundary is simply a vertical line such that all regimes have equal size, whereas for N → ∞ the boundary converges to the horizontal line with cN/b = 1 such that the co-existence and bi-stability regions disappear.

to cooperation (just as everybody else). This corresponds to by-product mutualism. (iv) If wN−1 > cN/b > 1, the system is bi-stable and the social dilemma presents itself as a coordination problem such that cooperation is favored only if it is already common (x > x∗2). These four dynamical domains are shown in Fig. 2(a).

This encompassing framework for modeling social dilemmas provides an ideal vantage point to investigate effects of population structure on the evolutionary fate of cooperators from a broader and more general perspective (Fig. 2) [21]. If defection dominates, i.e. in public goods type interactions, spatial structure enables cooperators to survive by forming clusters in order to reduce exploita- tion by defectors, which is in perfect agreement with the previous section (c.f. Fig. 1). However, it also becomes clear that the clustering advantage of coop- erators is rather limited such that cooperators survive only within a restricted parameter range where the cooperative benefits significantly exceed the incurring

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 323

costs (e.g. b � cN/2 for w = 1). Conversely, if cooperation dominates, spatial structure has no effect on the evolutionary outcome and cooperation invariably reaches fixation. In the region of bi-stability, spatial structure significantly extends the basin of attraction of the cooperative end state. The intuitive reason for this increase is the fact that in spatial settings cooperators need to exceed the thresh- old frequency only locally. If this condition is satisfied anywhere, a cooperative cluster forms that grows and eventually takes over the entire population. Most interesting effects of spatial structure, however, are observed whenever cooper- ators and defectors co-exist in unstructured populations. Not only does it lead to complex dynamics [33, 34] but more generally, affects the equilibrium levels of cooperators and defectors. Under these relaxed conditions of the social dilemma, spatial structure can be advantageous for cooperators, too, but quite intrigu- ingly it often turns out to be detrimental to cooperation by lowering the fraction of cooperators as compared to unstructured populations or by even eliminating cooperation altogether [24]. Ironically, the ultimate reason for the inhibition of cooperation actually lies in the maintenance of cooperation in unstructured pop- ulations, i.e. because in snowdrift type interactions it is best to adopt a different strategy than the co-players. This mechanism prevents the formation of compact clusters of cooperators and, instead, cooperators congregate in small filament like patterns.

Near the extinction threshold of cooperators, the relevant mechanisms driving the pattern formation process can be intuitively summarized as follows: in public goods type interactions cooperators lower exploitation by forming compact clusters that “minimize” interactions with defectors. Conversely, in snowdrift type inter- actions, cooperators form filament-like clusters that “maximize” interactions with defectors.

4. Cyclic Dominance and Synchronization

In nature, individuals often have the capability to abstain from or refuse to partic- ipate in joint endeavors by relying on a modest autarkic resource. Such risk averse individuals can be modeled by introducing a third strategic type, the loners [23]. Loners prefer a small but fixed income

PL = σc (4)

with 0 < σ < r − 1, such that the loners payoff is better than the payoff for mutual defection but worse than the payoff for mutual cooperation, PC (N ) = (r − 1)c. Thus, loners provide an escape hatch out of states of mutual defection and economic stalemate, which operates under full anonymity and without requiring preferential assortment or sophisticated strategic responses involving anticipation, condition- ing or reward and punishment mechanisms [52]. In the traditional formulation of public goods games (see Sec. 1 and 2) the loner option leads to a variable number

February 2, 2007 15:37 WSPC/169-ACS 00084

324 C. Hauert

S ≤ N of actual participants in the public goods interaction [22]. A single indi- vidual willing to participate in the public goods game has no choice but to act as a loner.

In evolving populations, the replicator dynamics describes the evolutionary change of the frequencies of the strategies cooperate, defect and loner with rel- ative abundances x, y and z (x + y + z = 1):

ẋ = x(fC − f̄ ), (5a) ẏ = y(fD − f̄ ), (5b) ż = z(fL − f̄ ), (5c)

where the average payoffs for cooperators, defectors and loners are given by

fD = σcz N−1 + r

x

1 − z (

1 − 1 − z N

N (1 − z) )

, (6a)

fC = fD − c (

1 + (r − 1)zN−1 − r N

1 − zN 1 − z

) , (6b)

fL = σc, (6c)

and an average population payoff of

f̄ = σc − c ((1 − z)σ − (r − 1)x) (1 − zN−1). (7)

For a detailed derivation, see Ref. 22. A close inspection of fC and fD reveals that for r > 2 cooperation can be favorable (fC > fD) for sufficiently large z. This results in a rock-scissors-paper type dominance of the three strategies: if cooperators abound, defection is favored; whereas if defectors prevail, it pays to abstain and choose the loner option; but if loners dominate, small interaction groups can form and re- establish cooperation. Experiments with students have confirmed these results [51] and similar findings were reported for closely related scenarios focussing on social welfare [49] and conflict resolution [41].

Interestingly, for prisoner’s dilemma interactions (N = 2) the loner option is unable to maintain cooperation in infinite and unstructured populations (Fig. 3). Random shocks may lead to brief intermittent burst of cooperation but the dynam- ics drives the system back to a state with all loners. In contrast, in public goods games (N > 2) a fixed point Q appears in the interior of the simplex S3 spanned by the three strategies. Q is a center (neutrally stable) and surrounded by closed orbits which lead to periodic oscillations of cooperators, defectors and loners. The period and amplitude of the oscillations is determined by the initial configuration. Along any trajectory, the average payoffs for cooperators, defectors and loners are all equal and reduce to the loner’s payoff σ [22]. Thus, in the long run everybody fares equally well than if the public goods game had not existed. However, also note that if the loner’s option had not existed, then everybody would have been worse off and ended up with nothing at all.

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 325

e z e y

e x

e z e y

e x

Q

(a) (b)

Fig. 3. The rock-scissors-paper type dominance of the three strategies cooperation, defection and loner is reflected in the heteroclinic cycle along the boundary of the simplex S3. (a) For voluntary prisoner’s dilemma interactions (N = 2) in unstructured populations, the interior of S3 consists of homoclinic cycles starting and ending in the homogenous state with all loners (r = 1.8, c = 1, σ = 1). (b) In public goods games (N > 2) a neutrally stable interior fixed point Q appears, which is surrounded by closed periodic orbits (N = 5, r = 3, c = 1, σ = 1). Thus, in unstructured, infinite populations cooperators, defectors and loners co-exist and their frequencies oscillate periodically.

All oscillatory trajectories are structurally unstable and any kind of stochastic disturbances (e.g. arising from finite population sizes) are sufficient to eventually drive the system to the boundary of S3 and end in one of the three homogenous states along the heteroclinic cycle. Thus, the long-term maintenance of cooperation requires additional stabilization. This can be achieved in different ways [23], e.g. through modifications of the dynamics, changes in the updating procedure, or by considering structured populations (Fig. 4). The spatial structure of lattice pop- ulations suppresses global oscillations and replaces them with uncorrelated local oscillations. This prevents the build up of fluctuations that can lead to extinctions. In addition, the loner option largely extends the range where cooperators persist (c.f. Figs. 1(a) and 4(a)). For small multiplication factors r < σ + 1 loners are clearly the best option. For σ + 1 < r < rL all three strategies co-exist in dynamical equilibrium. Note that in compulsory public goods games, i.e. in absence of loners, such values of r would almost invariably end up with everybody defecting. The cyclic dominance of cooperators, defectors and loners gives rise to traveling waves propagating across the lattice. For clusters of cooperators the loners provide pro- tection against exploitation restricting the detrimental impact of defectors to one side while cooperators can expand into the loners’ territory. Above the threshold rL loners vanish and cooperators are able to thrive through cluster formation alone. Thus, the dynamics reverts the voluntary public goods game back to a compulsory interaction. The extinction of loners exhibits another critical phase transition in the universality class of directed percolation [56]. The robustness of directed percolation

February 2, 2007 15:37 WSPC/169-ACS 00084

326 C. Hauert

multiplication factor r 2 2.2 2.4 2.6 2.8 3. 3.2 3.4 3.6

0

0.2

0.4

0.6

0.8

1

fr eq

ue nc

ie s

of s

tr at

eg ie

s

(a) (b) (c)

Fig. 4. (Color online) Spatial structure is capable of stabilizing the co-existence of cooperators, defectors and loners in prisoner’s dilemma as well as public goods interactions. (a) Frequency of cooperators (�), defectors (�) and loners (�) as a function of the multiplication factor r of the public good. Individuals interact on a square lattice (100 × 100) within the Moore neighborhood in groups of N = 5 with the loners payoff σ = 1. For r < σ + 1 = 2 loners dominate because they outcompete even groups of cooperators. All three strategies co-exist in dynamical equilibrium for σ + 1 < r < rL ≈ 3.18. Note that in absence of loners, cooperation would be doomed and vanish (cf. Fig. 1). For r > rL the persistence of cooperation no longer hinges on the presence and protection of loners and cooperators thrive through cluster formation. Finally, for r > rD ≈ 3.68 cooperators dominate and drive defectors to extinction. (b, c) Typical snapshots of lattice config- urations where cooperators (blue/dark grey), defectors (red/light grey) and loners (yellow/white) co-exist. (b) Near the extinction threshold of defectors (r = 2.1) and (c) close to the threshold where cooperators can survive on their own (cf. Fig. 1) but still crucially depend on the presence of loners (r = 2.9).

transitions is nicely demonstrated by noting that the extinction of cooperators and defectors in the compulsory public goods game (see Sec. 1) left a homogenous state behind, whereas the extinction of loners occurs on an inhomogeneous and fluctuat- ing background of cooperators and defectors.

Interestingly, spatial structure is even able to stabilize cooperation in the volun- tary prisoner’s dilemma [29, 55]. It is important to note, however, that the stabilizing effects are intrinsically linked to the actual structure of the population. In particular, the above lattice configuration prevents synchronized oscillations because the spa- tial separation allows for uncorrelated fluctuations in different areas of the lattice. This can be compromised in spatial structures that include long range interactions such as small-world networks [63] where far reaching connections can induce global synchronization [29, 55]. To exemplify this, consider the extreme case of random regular graphs [2]. Such graphs are generated by randomly assigning neighbors to each site under the constraint that every site ends up with the same number of neighbors while excluding self and double connections. For small multiplication fac- tors r the results for random regular graphs (Fig. 5(a)) are in close agreement with lattice populations (cf. Fig. 3(a)). However, in contrast to the lattice results, the co-existence of the three strategies persists only for a small range of r (2 < r < 2.2 for N = 5) and above defectors reign. Only for significantly larger r, cooperators reappear and co-exist with defectors. The frequency of cooperators quickly increases

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 327

2 2.5 3. 3.5 4. 4.5 5. multiplication factor r multiplication factor r

fr eq

ue nc

ie s

of s

tr at

eg ie

s

2 2.5 3. 3.5 4. 4.5 5.

0

0.2

0.4

0.6

0.8

1

Fig. 5. Long-range interactions in structured populations can promote global synchronization of periodic fluctuations induced by the cyclic dominance of the three strategies cooperation, defection and loner in voluntary public goods games. (a) Frequency of cooperators (�), defectors (�) and loners (�) as a function of the multiplication factor r of the public good. Individuals are arranged on a random regular graph (104 sites) where each site has eight neighbors and interacts in groups of N = 5 with the loners payoff σ = 1. As on square lattices (cf. Fig. 4), loners reign for r < σ + 1 = 2 and for r > σ + 1 the three strategies co-exist. However, already for r > rL ≈ 2.2 the co-existence breaks down and leads to a homogenous state with all defectors. Only for r > rC ≈ 3.6 cooperators can persist and co-exist with defectors but, interestingly, loners are absent. Finally, for r > rD ≈ 4.6 cooperation again becomes dominant and reaches fixation. (b) Average, minimal and maximal frequency of cooperators for the same settings as in panel (a). For σ + 1 < r < rL the fluctuations in the frequency of cooperators rapidly increases with r, indicating global synchronization of the oscillations in the frequencies of the three strategies. For r > rL the fluctuations eventually lead to the extinction of one strategy with another strategy following swift and resulting in a homogenous state. In most cases defectors reach fixation but occasionally the system could also end up in states with all defectors or all loners, depending on which strategy vanished first. All simulations are randomly initialized with strategy frequencies close to the interior equilibrium point Q of the unstructured model (cf. Fig. 3), if it exists, or equal proportions otherwise.

with r until they eventually displace the defectors. These surprising results are bet- ter understood by considering the fluctuations of the strategies (Fig. 5(b)). In the region where all three strategies co-exist, the minima and maxima of the strategy frequencies rapidly increase with r. This clearly indicates global synchronization of the local fluctuations that are driven by the cyclic dominance of cooperators, defec- tors and loners. The building up of these global fluctuations eventually leads to the extinction of one strategy and the system approaches a homogeneous absorb- ing state. Which strategy disappears first is essentially random but the basin of attraction of the three absorbing states depends on the parameters, the initial con- figuration and the population size. For example, for the configuration in Fig. 5 in all cases loners disappeared first, which seals the fate of cooperators and leads to an

February 2, 2007 15:37 WSPC/169-ACS 00084

328 C. Hauert

end state with all defectors. Other outcomes where cooperators or loners prevailed were rarely observed in other simulation runs. Also note that this setting is quite different from a system of coupled oscillators because individuals will alter their strategy only in response to changes in their neighborhood.

Apart from altering the stabilization versus synchronization capabilities, the details of the population structure also affect the characteristics of strategy extinc- tions, i.e. the type of phase transitions. For random regular graphs and small world networks, the lack of spatial correlations results in a linear decrease of cooperators in the prisoner’s dilemma indicating a mean-field transition [29].

5. Specialization and the Origin of Cooperators and Defectors

Instead of focussing on pure cooperators and pure defectors, it often seems more natural to assume continuously varying degrees of cooperative investments. This could be, for example, the time and effort expended by cooperators in producing the common good, such as for the enzyme production and secretion in yeast cells (see Sec. 3). The strategy of an individual is then given by a real number x between zero and an upper limit xmax which specifies the individual’s investment in the common enterprise. The theory of social dilemmas is easily translated to settings with contin- uous strategies. For example, in the continuous prisoner’s dilemma, an x-strategist facing a y-strategist obtains the payoff Q(x, y) = B(y) − C(x), where B(y) deter- mines the benefits that accrue to the x-strategist as a function of the co-player’s investment level y and C(x) denotes the costs incurring to the x-strategist based on its own investment level x. B(x) and C(x) are both assumed to be monotonously increasing functions with B(0) = C(0) = 0, i.e. zero investments (pure defection) incur no costs but produce no benefit either, and B(x) > C(x) at least for small x. The latter condition creates the classical prisoner’s dilemma for players with dif- ferent investment levels. Obviously, the investment levels decrease over time and converge to zero, because an individual can only improve its payoff by reducing the incurring costs and hence by decreasing the investment level x. In spatially structured populations, however, this conclusion no longer holds and instead inter- mediate investment levels can evolve [35]. This approach is not further pursued in the following but, in lieu, a slightly different model is considered that proves to be more general and displays much richer dynamics.

Returning once more to the foraging yeast cells, it is evident that the contin- uous prisoner’s dilemma fails to appropriately capture these interactions because every yeast cell also profits from their own investments into enzyme production and secretion. In this situation, the cooperative benefits depend on the investments of all participants. In order to keep the model simple and transparent, the following exposition focusses on pairwise interactions but it is important to note that the theory is easily generalized to interacting groups of N individuals [10]. For pair- wise interactions, the payoff of an x-strategist against a y-strategist is then given by P (x, y) = B(x + y) − C(x). In that case, the näıve expectation would be that

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 329

intermediate investment levels evolve (because of B(x) > C(x) for small x) but the dynamics turns out to be much more interesting.

The evolution of the investment levels can be analyzed by using adaptive dynam- ics [8, 14, 39, 46]. The two strategies x and y represent the cooperative invest- ment levels of residents and mutants, respectively. In the framework of adaptive dynamics, i.e. in the limit of small mutations, the mutant is either unable to invade or it invades and replaces the resident. For rare mutations, mutants always face a homogeneous resident population. The replicator dynamics [30] then states that the growth rate or invasion fitness of a rare mutant y in a resident pop- ulation x is given by fx(y) = P (y, x) − P (x, x). Note that because y is rare, mutant-mutant and resident-mutant interactions, i.e. P (y, y) and P (x, y), can be neglected. The evolutionary change of trait x is then governed by the selection gra- dient D(x) = ∂fx/∂y|y=x = B′(2x) − C′(x) which leads to the canonical equation of adaptive dynamics:

ẋ = mD(x), (8)

where m depends on the population size and accounts for the mutational process driving changes in x. For constant population sizes m reduces to a constant and setting m = 1 merely rescales time. Trait values where the selection gradient van- ishes (D(x∗) = 0) are termed singular strategies x∗. If no singular strategy exists, then the investment level x either always decreases (D(x) < 0) or always increases (D(x) > 0) until the boundaries of the investment interval [0, xmax] are reached. The former reflects prisoner’s dilemma type interactions whereas the latter corresponds to a continuous variant of by-product mutualism.

If a singular strategy x∗ exists, it is convergent stable and hence an attractor of the evolutionary dynamics if dD(x)/dx|x=x∗ = 2B′′(2x∗)−C′′(x∗) < 0. Conversely, if dD(x)/dx|x=x∗ > 0 holds, x∗ is a repellor and the investment levels evolve away from x∗ (Fig. 6(c)). Depending on the initial configuration x0, the investment level increases (x0 > x∗) or decreases (x0 < x∗). Assuming that an attractor x∗ exists, then all initial investment levels in the vicinity of x∗ converge to x∗. Subsequent evolutionary changes, however, depend on whether x∗ is also evolutionarily stable, i.e. whether it represents a maximum or a minimum of the invasion fitness fx(y). If it is a maximum (∂2fx(y)/∂y2|y=x∗ = B′′(2x∗) − C′′(x∗) < 0), then x∗ is evo- lutionarily stable (Fig. 6(b)). This outcome corresponds to the näıve expectation where stable intermediate investment levels evolve. Yet, if x∗ turns out to be a fitness minimum (∂2fx(y)/∂y2|y=x∗ > 0) then x∗ is called an evolutionary branch- ing point because a population of x∗-strategists can be invaded by both higher and lower investing mutants (Fig. 6(a)). Eventually, the population splits into two types of individuals characterized by high investments (cooperators) and low investments (defectors). It is important to note that this diversification does not require any external tuning because the dynamics keeps the population at the fitness minimum until suitable mutants occur to finally escape that unfortunate state. For quadratic cost and benefit functions, this system can be fully analyzed [10]. In the case of

February 2, 2007 15:37 WSPC/169-ACS 00084

330 C. Hauert

evolutionary branching, the two investment levels continue to diverge until both reach the boundaries of the range of investments. This spontaneous social diversifi- cation and specialization provides a natural explanation for the evolutionary origin of cooperators and defectors.

A celebrated avatar of social dilemmas is the Tragedy of the Commons [17] which basically states that any common resource is bound to be overexploited — at least unless rigorous control mechanisms are introduced. In the present context, whenever evolution favors co-existence of cooperators and defectors over homoge- nous populations with uniform intermediate investments, another kind of dilemma emerges, which has been termed the Tragedy of the Commune [10]. In communal enterprises, unequal contributions to the common good are against the accepted notion of fairness. Low investors are perceived as social parasites which creates tensions and has a high potential for conflicts.

According to preliminary simulations, spatial structure has similar effects on the dynamics in continuous games [25] as in games with a discrete strategy set (Sec. 3) [26]. In agreement with simulations of the continuous prisoner’s dilemma [35], spatial structure promotes and supports cooperation for parameter ranges where defection dominates, i.e. investment levels converge to zero, in unstructured

Fig. 6. Evolutionary dynamics of investment levels for continuously varying degrees of cooper- ation in unstructured populations (darker shades indicate higher frequencies of certain invest- ment levels). Individuals interact in pairs and the benefits depend on the joint investment levels but the costs are determined solely by the individuals’ investment. (a) Evolutionary branch- ing results in two distinct phenotypic clusters of high investors (cooperators) and low investors (defectors). First the trait evolves towards the convergent stable singular investment level x∗ (dashed line) but x∗ is not evolutionarily stable and thus adjacent strategies can invade and induce phenotypic diversification of the strategies. (b) Stable intermediate investment levels evolve if x∗ is both convergent and evolutionarily stable. (c) If x∗ is a repellor, the evolu- tionary end state depends on the initial conditions and the population either evolves to full cooperation or full defection (two simulations shown). Parameters: quadratic cost and benefit functions B(z) = b2z

2 + b1z, C(x) = c2x 2 + b1x where z = x + y denotes the joint invest-

ment of both players. (a) b2 = −1, b1 = 7.5, c2 = −1.5, c1 = 7 resulting in x∗ = 0.5; (b) b2 = −1.5, b1 = 7, c2 = −1, c1 = 4.6 leading to x∗ = 0.6; (c) b2 = −0.5, b1 = 3.4, c2 = −1.5, c1 = 4 again yielding x∗ = 0.6.

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 331

0 0.2 0.4 0.6 0.8 1.0 0

2.104

4.104

6.104

8.104

1.105

investment level

ge ne

ra ti

on s

(a) (b) (c)

Fig. 7. Evolutionary dynamics of the distribution of investment levels for continuously vary- ing degrees of cooperation in spatially structured lattice populations with Moore neighborhood (darker shades indicate higher frequencies of certain investment levels). Apart from the spatial structure, the setting is identical to Fig. 6(a). (a) Just as in unstructured populations, the invest- ment level first evolves towards the singular level x∗, which is predicted to be shifted to x∗ = 0.729 due to the spatial structure. Once the investment levels approach x∗, selection pressure decreases and the distribution widens and eventually again splits into two strategic types of high investors (cooperators) and low investors (defectors). (b) Snapshot of a typical lattice configuration after t = 3 × 104 generations (lighter shades indicate higher investors). The average investment level is close to x∗ with 0.7005 (range: 0.42–0.84) and the population is about to undergo evolutionary branching. (c) Lattice configuration after branching at t = 105 generations. In this evolution- ary end state, the high and low investing types essentially engage in traditional snowdrift type interactions.

populations. The stabilizing effects of spatial structure reduces the parameter range where evolutionary branching occurs, but quite intriguingly, the range now includes areas where defection dominates otherwise. This indicates that evolutionary branch- ing also occurs in the spatial continuous prisoner’s dilemma. In the case of evolu- tionary branching, the average investment level first converges to the singular strat- egy x∗. Once the population is near x∗, the variance in investment levels increases and eventually the population again splits into distinct high and low investors (Fig. 7). At this point, the population essentially consists of two strategic types and similar pattern formation processes are observed as discussed for the discrete strategy set of cooperators and defectors (Secs. 2 and 3). The cluster shape and distribution is again determined by the corresponding interactions in unstructured populations.

6. Summary and Conclusions

Evolutionary game theory applied to the problem of cooperation in populations with different kinds of interaction structures represents a fascinating field for studying emergent phenomena including pattern formation and specialization in a behav- ioral context. Population structures with limited numbers of interaction partners have profound effects on the evolutionary fate of cooperators and defectors. For the

February 2, 2007 15:37 WSPC/169-ACS 00084

332 C. Hauert

most stringent form of social dilemmas, spatial structure promotes cooperation by enabling cooperators to form clusters and thereby reducing exploitation (prisoner’s dilemma or public goods type interactions). However, under relaxed conditions of the social dilemma, spatial structure often inhibits cooperation by actually prevent- ing the formation of compact clusters (snowdrift type interactions) and, instead, favors highly dispersed and filament-like patterns. Microorganisms appear to be ideal model systems to test these hypotheses. Promising candidates include pro- duction of replication enzymes in RNA phages [60, 61], foraging in yeast [15] or antibiotic resistance in bacteria [43].

In voluntary public goods interactions, the cyclic dominance of the three behav- ioral strategies (cooperate, defect and loner) induces oscillatory dynamics. The efficiency of the loner option has been demonstrated in human experiments [51]. Population structure again supports cooperation but the details of the structure have intricate consequences on the dynamics. Long-range connections (such as on small world networks and random regular graphs) can induce and promote global synchronization, which sometimes leads to disastrous amplification of local oscilla- tions. This often prevents co-existence of all three strategies and usually eliminates cooperation. Cooperators thrive only under more favorable conditions that allow co-existence with defectors in absence of loners, i.e. returning to compulsory inter- actions. Conversely, lattice structures stabilize the system and support co-existence of all three strategies. The spatial separation of the individuals prevents global synchronization and, instead, gives rise to traveling waves sweeping across the lat- tice. Promising experimental systems, which naturally exhibit this rock-scissors- paper type dominance, include mating strategies in the side-blotched lizard (Uta stansburiana) [53] and competition in E. coli strains [32].

Game dynamics in structured populations suggests interesting links to con- densed matter physics. Specifically, this includes cluster motion and dynamics that relate to random walks, critical phase transitions in the universality class of directed percolation as well as diverging fluctuations that are characteristic to the Ising model.

In nature, cooperation may not always be an all or nothing decision but rather a continuous range of cooperative investment levels. The surprisingly rich evolu- tionary dynamics of such continuous games is nicely captured by the framework of adaptive dynamics. Most importantly, however, evolution may lead to specialization and spontaneous splitting of the population into high and low investing individuals, i.e. emerging cooperators and defectors, instead of homogeneous populations with uniform investment levels. This represents the Tragedy of the Commune because, at least in humans, unequal contributions in a communal enterprise are consid- ered unfair and result in a superimposed dilemma with quite some potential for escalating conflicts. This contrasts with the Tragedy of the Commons, which states that public resources are doomed to be overexploited. Again, microbial systems appear to be promising candidates to study cooperative investment levels including cooperative polymorphisms [9].

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 333

For systems with such complex dynamics, it is often challenging to get an intu- itive understanding of the relevant processes. Visualization of the spatio-temporal patterns or the dynamic changes of continuous strategies helps to understand the collectives formation and evolutionary dynamics as well as to provide insights and inspirations for further explorations. A suite of interactive tutorials allows to repro- duce and verify essentially all results reported in this review and moreover to check the robustness of the conclusions by altering numerous parameters [20].

Acknowledgments

Helpful comments on the manuscript by Martin A. Nowak are gratefully acknowledged.

References

[1] Axelrod, R. and Hamilton, W. D., The evolution of cooperation, Science 211, 1390– 1396 (1981).

[2] Bollobás, B., Random Graph (Academic Press, New York, 1995). [3] Boyd, R. and Richerson, P. J., The evolution of reciprocity in sizeable groups, J.

Theor. Biol. 132, 337–356 (1988). [4] Cardy, J. L. and Täuber, U. C., Field theory of branching and annihilating random

walks, J. Stat. Phys. 90, 1–56 (1998). [5] Colman, A. M., Game Theory and Its Applications in the Social and Biological Sci-

ences (Butterworth-Heinemann, Oxford, 1995). [6] Connor, R. C., Altruism among non-relatives: Alternatives to the ‘Prisoner’s

Dilemma’, TREE 10(2), 84–86 (1995). [7] Dawes, R. M., Social dilemmas, Ann. Rev. Psychol. 31, 169–193 (1980). [8] Dieckmann, U. and Law, R., The dynamical theory of coevolution: A derivation from

stochastic ecological processes, J. Math. Biol. 34, 579–612 (1996). [9] Doebeli, M. and Hauert, C., Models of cooperation based on the prisoner’s dilemma

and the snowdrift game, Ecol. Lett. 8, 748–766 (2005). [10] Doebeli, M., Hauert, C. and Killingback, T., The evolutionary origin of cooperators

and defectors, Science 306, 859–862 (2004). [11] Dugatkin, L. A., Cooperation Among Animals: An Evolutionary Perspective (Oxford

University Press, Oxford, 1997). [12] Fehr, E. and Gächter, S., Altruistic punishment in humans, Nature 415, 137–140

(2002). [13] Fudenberg, D. and Maskin, E., The folk theorem in repeated games with discounting

or with incomplete information, Econometrica 54(3), 533–554 (1986). [14] Geritz, S. A. H., Kisdi, E., Meszéna, G. and Metz, J. A. J., Evolutionarily singular

strategies and the adaptive growth and branching of the evolutionary tree, Evol. Ecol. 12, 35–57 (1998).

[15] Greig, D. and Travisano, M., The Prisoner’s Dilemma and polymorphism in yeast SUC genes, Biol. Lett. 271, S25 – S26 (2004).

[16] Hammes, G. G., Enzyme Catalysis and Regulation, (Academic Press, New York, 1982).

[17] Hardin, G., The tragedy of the commons, Science 162, 1243–1248 (1968). [18] Hauert, C., Fundamental clusters in spatial 2 × 2 games, Proc. Roy. Soc. London B

268, 761–769 (2001).

February 2, 2007 15:37 WSPC/169-ACS 00084

334 C. Hauert

[19] Hauert, C., Effects of space in 2 × 2 games, Int. J. Bifurc. Chaos 12, 1531–1548 (2002).

[20] Hauert, C., Virtuallabs: Interactive tutorials on evolutionary game theory, http://www.univie.ac.at/virtuallabs (2005).

[21] Hauert, C., Spatial effects in social dilemmas, J. Theor Biol. (in print) (2006). [22] Hauert, C., De Monte, S., Hofbauer, J. and Sigmund, K., Replicator dynamics in

optional public goods games, J. Theor. Biol 218, 187–194 (2002). [23] Hauert, C., De Monte, S., Hofbauer, J. and Sigmund, K., Volunteering as red queen

mechanism for cooperation in public goods games, Science 296, 1129–1132 (2002). [24] Hauert, C. and Doebeli, M., Spatial structure often inhibits the evolution of cooper-

ation in the snowdrift game, Nature 428, 643–646 (2004). [25] Hauert, C. and Doebeli, M., Adaptive dynamics and game theory in structured pop-

ulations, (in preparation) (2005). [26] Hauert, C., Michor, F., Nowak, M. and Doebeli, M., Synergy and discounting of

cooperation in social dilemmas, J. Theor. Biol. 239, 195–202 (2006). [27] Hauert, C. and Schuster, H. G., Effects of increasing the number of players and

memory size in the iterated prisoner’s dilemma: A numerical approach, Proc. Roy. Soc. London B 264, 513–519 (1997).

[28] Hauert, C. and Szabó, G., Prisoner’s dilemma and public goods games in different geometries: Compulsory versus voluntary interactions, Complexity 8(4), 31–38 (2003).

[29] Hauert, C. and Szabó, G., Game theory and physics, Am. J. Phys. 73(5), 405–414 (2005).

[30] Hofbauer, J. and Sigmund, K., Evolutionary Games and Population Dynamics (Cambridge University Press, Cambridge, 1998).

[31] Kagel, J. H. and Roth, A. E., (eds.), The Handbook of Experimental Economics (Princeton University Press, Princeton, 1995).

[32] Kerr, B., Riley, M. A., Feldman M. W. and Bohannan, B. J. M., Local dispersal promotes biodiversity in a real-life game of rock-paper-scissors, Nature 418, 171–171 (2002).

[33] Killingback, T. and Doebeli, M., Spatial evolutionary game theory: Hawks and doves revisited, Proc. Roy. Soc. London B, 263, 1135–1144 (1996).

[34] Killingback, T. and Doebeli, M., Self-organized criticality in spatial evolutionary game theory, J. Theor. Biol. 191, 335–340 (1998).

[35] Killingback, T., Doebeli, M. and Knowlton, N., Variable investment, the continuous prisoner’s dilemma, and the origin of cooperation, Proc. Roy. Soc. London B, 266, 1723–1728 (1999).

[36] Le Gaillard, J.-F., Ferrière, R. and Dieckmann, U., The adaptive dynamics of altruism in spatially heterogenous populations, Evolution 57, 1–17 (2003).

[37] Marro, J. and Dickman, R., Nonequilibrium Phase Transitions in Lattice Models (Cambridge University Press, Cambridge, 1999).

[38] Maynard Smith, J. and Price, G., The logic of animal conflict, Nature 246, 15–18 (1973).

[39] Metz, J. A. J., Geritz, S. A. H., Meszena, G., Jacobs, F. J. A. and van Heerwaar- den, J. S., Adaptive dynamics: A geometrical study of the consequences of nearly faithful replication, in Stochastic and Spatial Structures of Dynamical Systems, eds. van Strien, S. J., and Verduyn Lunel, S. M. (North Holland, Amsterdam, 1996), pp. 183–231.

[40] Milinski, M. and Wedekind, C., Working memory constrains human cooperation in the prisoner’s dilemma, Proc. Natl. Acad. Sci. 95, 13755–13758 (1998).

February 2, 2007 15:37 WSPC/169-ACS 00084

Cooperation, Collectives Formation and Specialization 335

[41] Miller, R. R., No play: A means of conflict resolution, J. Pers. Soc. Psych. 6(2), 150–156 (1967).

[42] Molander, P., The prevalence of free riding, J. Conflict Resolution 36, 756–771 (1992). [43] Neu, H. C., The crisis in antibiotic resistance, Science 257, 1064–1073 (1992). [44] Nowak, M. A. and May, R. M., Evolutionary games and spatial chaos, Nature 359,

826–829 (1992). [45] Nowak, M. A. and May, R. M., The spatial dilemmas of evolution, Int. J. Bifurc.

Chaos 3(1), 35–78 (1993). [46] Nowak, M. A. and Sigmund, K., The evolution of stochastic strategies in the prisoner’s

dilemma, Acta Appl. Math. 20, 247–265 (1990). [47] Nowak M. A. and Sigmund, K., Evolution of indirect reciprocity by image scoring,

Nature 393, 573–577 (1998). [48] Nowak, M. A. and Sigmund, K., Evolutionary dynamics of biological games, Science

303, 793–799 (2004). [49] Orbell, J. H. and Dawes, R. M., Social welfare, cooperators’ advantage, and the

option of not playing the game, Ame. Soc. Rev. 58, 787–800 (1993). [50] Pennisi, E., How did cooperative behavior evolve?, Science 309, 93 (2005). [51] Semmann, D., Krambeck, H.-J. and Milinski, M., Volunteering leads to rock-paper-

scissors dynamics in a public goods game, Nature 425, 390–393 (2003). [52] Sigmund, K., Hauert, C. and Nowak, M. A., Reward and punishment, Proc. Natl.

Acad. Sci. 98, 10757–10762 (2001). [53] Sinervo, B. and Lively, C. M., The rock-paper-scissors game and the evolution of

alternative male strategies, Nature 380, 240–243 (1996). [54] Sugden, R., The Economics of Rights, Co-operation and Welfare (Blackwell, Oxford

and New York, 1986). [55] Szabó, R. and Hauert, C., Evolutionary prisoner’s dilemma with optional participa-

tion, Phys. Rev. E 66, 062903 (2002). [56] Szabó, G. and Hauert, C., Phase transitions and volunteering in spatial public goods

games, Phys. Rev. Lett. 89, 118101 (2002). [57] Szabó, G. and Tőke, C., Evolutionary prisoner’s dilemma game on a square lattice,

Phys. Rev. E 58, 69–73 (1998). [58] Taylor, P. and Jonker, L., Game dynamics and evolutionarily stable strategies, Math.

Bio. Sci. 40, 145–156 (1978). [59] Trivers, R. L., The evolution of reciprocal altruism, Q. Rev. Biol. 46, 35–57 (1971). [60] Turner, P. E. and Chao, L., Prisoner’s dilemma in an RNA virus, Nature 398, 441–443

(1999). [61] Turner, P. E. and Chao, L., Escape from prisoner’s dilemma in RNA phage φ6, Am.

Nat. 161, 497–505 (2003). [62] van Baalen, M. and Rand, D. A., The unit of selection in viscous populations and

the evolution of altruism, J. Theor. Biol. 193, 631–648 (1998). [63] Watts, D. J. and Strogatz, S. H., Collective dynamics of ‘small world’ networks,

Nature 393, 440–442 (1998). [64] Wedekind, C. and Milinski, M., Human cooperation in the simultaneous and the

alternating prisoner’s dilemma: Pavlov versus generous tit-for-tat, Proc. Natl. Acad. Sci. 93, 2686–2689 (1996).