eco 512

Optimal control theory is a technique being used increasingly by aca- demic economists to study problems involving optimal decisions in a mul- tiperiod framework. This textbook is designed to make the difficult subject of optimal control theory accessible to economists while at the same time maintaining rigor. Economic intuition is emphasized, and examples and problem sets covering a wide range of applications in economics are pro- vided. Theorems are clearly stated, and their proofs carefully explained. The development of the text is gradual and fully integrated, beginning with simple formulations and progressing to advanced topics such as control parameters, jumps in state variables, and bounded state space. For greater economy, optimal control theory is introduced directly, without recourse to the calculus of variations. The connection with the latter and with dy- namic programming is explained in a separate chapter.

A secondary purpose of the book is to draw a parallel between optimal control theory and static optimization. The first chapter provides an ex- tensive treatment of constrained and unconstrained maximization, with emphasis on economic insight and applications. Starting from basic con- cepts, it derives and explains important results, including the envelope theorem and the method of comparative statics. This chapter may be used for a short course in static optimization.

The book is largely self-contained. No previous knowledge of differ- ential equations is required.

Optimal control theory and static optimization in economics

Optimal control theory and static optimization in economics

DANIEL LEONARD

NGO VAN LONG

1 CAMBRIDGE UNIVERSITY PRESS

CAMBRIDGE UNIVERSITY PRESS Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, Sao Paulo, Delhi

Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK

Published in the United States of America by Cambridge University Press, New York

www.cambridge.org

Information on this title: www.cambridge.org/9780521331586

© Cambridge University Press 1992

This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press.

First published 1992 Reprinted 1993, 1994, 1995, 1996, 1998

A catalogue record for this publication is available from the British Library

ISBN 978-0-521-33158-6 hardback ISBN 978-0-521-33746-5 paperback

Transferred to digital printing 2009

Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party Internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate. Information regarding prices, travel timetables and other factual information given in this work are correct at the time of first printing but Cambridge University Press does not guarantee the accuracy of such information thereafter.

Contents

Preface page

1 Static optimization 1.1 Unconstrained optimization, concave and convex

functions 1.2 Optimization under equality constraints: the method

of Lagrange 1.3 Comparative statics 1.4 Optimization under inequality constraints: nonlinear

programming 1.5 Economic applications of nonlinear programming 1.6 The special case of linear programming Appendix Exercises

2 Ordinary differential equations 2.1 Introduction 2.2 Definitions and fundamental results 2.3 First-order differential equations 2.4 Systems of linear FODE with constant coefficients 2.5 Systems of two nonlinear FODE Appendix Exercises

3 Introduction to dynamic optimization 3.1 Optimal borrowing 3.2 Fiscal policy 3.3 Suboptimal consumption path 3.4 Discounting and depreciation in continuous-time models Exercises

4 The maximum principle 4.1 A simple control problem 4.2 Derivation of the maximum principle in discrete time 4.3 Numerical solution of an optimal control problem in

continuous time

ix

1

1

20 43

52 67 70 74 79

87 87 88 91 95

100 111 113

117 118 119 120 121 124

127 127 129

133

vi Contents

4.4 Phase diagram analysis of optimal control problems 4.5 Economic interpretation of the maximum principle 4.6 Necessity and sufficiency of the maximum principle Exercises

The calculus of variations and dynamic programming 5.1 The calculus of variations 5.2 Dynamic programming: discrete-time, finite-horizon

problems 5.3 Dynamic programming in continuous time Exercises

The general constrained control problem 6.1 The set of admissible controls 6.2 Integral constraints 6.3 The maximum principle with equality constraints only 6.4 The maximum principle with inequality constraints 6.5 Necessity and sufficiency theorems: the case with

inequality and equality constraints 6.6 Concluding notes Exercises

Endpoint constraints and trans versality conditions 7.1 Free-endpoint problems 7.2 Problems with free endpoint and a scrap value function 7.3 Lower bound constraints on endpoint 7.4 Problems with lower bound constraints on endpoint

and a scrap value function 7.5 Free-terminal-time problems without a scrap value

function 7.6 Free-terminal-time problems with a scrap value

function 7.7 Other trans versality conditions 7.8 A general formula for trans versality conditions 7.9 Sufficiency theorems 7.10 A summary table of common transversality conditions 7.11 Control parameters Exercises

Discontinuities in the optimal controls 8.1 A classical bang-bang example 8.2 The beekeeper's problem 8.3 One-sector optimal growth with reserves

137 151 161 165

169 169

173 182 184

187 187 190 192 198

210 218 218

221 222 226 229

235

240

244 247 248 251 253 253 259

263 263 267 274

Contents

8.4 Highest consumption path 8.5 Concluding comments Exercises

Infinite-horizon problems 9.1 Optimality criteria 9.2 Necessary conditions 9.3 Sufficient conditions 9.4 Autonomous problems 9.5 Steady states in autonomous infinite-horizon

problems 9.6 Further properties of autonomous infinite-horizon

problems Exercises

10 Three special topics 10.1 Problems with two-state variables 10.2 Trade in capital goods: jumps in the state variables 10.3 Constraints on the state variables Exercises

Bibliography Index

277 281 282

285 285 287 288 289

294

298 304

307 307 310 332 342

345 351

Preface

As the range of problems tackled by economists expands, the curriculum of economics programs follows. Questions of choice in dynamic economic models are often an integral part of such programs. The most useful tech- nique for dealing with these questions is optimal control theory. It was developed in the late 1950s as an outgrowth of the centuries-old calculus of variations, and it has been traditional to present an exposition of the latter as a preliminary to this more modern technique. Here we break with this tradition on the grounds that there is nothing to be learned from the calculus of variations that cannot be learned from optimal control theory, whereas the converse is not true. Our approach emphasizes the links be- tween the methods of classical programming and those of optimal control theory. For this reason we begin with a thorough and lengthy exposition of static optimization techniques: unconstrained, equality-constrainted, and inequality-constrained problems (Chapter 1). After presenting some simple solution techniques for differential equations and their qualitative analysis through phase diagrams (Chapter 2), we proceed with a very short and informal chapter introducing various concepts related to optimiza- tion in dynamic models (Chapter 3). Chapter 4 describes the optimal con- trol format for dynamic optimization problems and the core of its solu- tion procedures, known as the maximum principle. We have attempted to make the reader's first encounter with a standard control problem as lim- pid as possible by relegating all complications to a later stage and empha- sizing the links with the Lagrangean methods of static optimization. Chap- ter 5 diverges from the main line of argument to give a very brief account of the calculus of variations and the related method of dynamic program- ming. Chapter 6 deals with a much more general control problem, which involves several types of constraint. Chapter 7 extends the results by al- lowing for various boundary conditions at the beginning or the end of the planning horizon. Chapter 8 concentrates on a special class of models that might elicit discontinuities in the controls. Chapter 9 considers infinite- horizon problems, and Chapter 10 treats three separate topics.

The book is intended to be a very detailed exposition of static and dy- namic optimization, beginning at an elementary level. (Some knowledge of calculus and matrix algebra is needed, but these are reviewed in the

IX

x Preface

appendixes to Chapters 1 and 2.) The presentation gradually builds up to a degree of sophistication sufficient for readers to understand these topics as treated in most economic journal articles. Indeed, after a thorough study of the material presented here, including the exercises, readers should be able to use the techniques in their own research. Theorems and definitions are stated rigorously, but most proofs are chosen for their heuristic value.

The book is intended for university economists who feel a need to ex- pand the array of techniques at their disposal without wishing to invest too much time in the study of more rigorous mathematical derivations. It can be used for self-instruction. Alternatively, the book can be used for a graduate course in economic optimization with more emphasis on the beginning or the end of the book, depending on students' backgrounds and the place of the course in the overall graduate program. It is essential that students attempt the exercises if they are to acquire a thorough grasp of the subject. The Bibliography includes a selected list of articles and monographs, including several volumes of collected papers that empha- size the use of optimal control theory in economics.

Many students and colleagues have contributed to this volume directly or indirectly; we are particularly grateful to Jeffrey Bernstein, Richard Cornes, Bruce Forster, Murray Kemp, T. H. Lou, the late Richard Mann- ing, Frank Milne, John Pitchford, Hans-Werner Sinn, Mark Tippet, Sa- bine Toussaint, Stephen Turnovsky, and Neil Vousden. Our final thanks are reserved for Jan Anthony and Silvana Tomasiello, who produced many versions of the typescript, and for Mary Racine, who edited it.

CHAPTER 1

Static optimization

In this chapter we deal with problems involving the choice of values for a finite number of variables in order to maximize some objective. Sometimes the values the variables may take are unrestricted; at other times they are restricted by equality constraints and also by inequality constraints. In the course of the presentation an important class of functions will emerge; they are called concave functions and are closely associated with "nice" maximum problems. They will be encountered throughout this book. For this reason we weave the concept of concavity of functions through the exposition of maximization problems. This is done to suit our purposes, but concave functions have other important properties in their own right.

The notation we use is fairly standard. If in doubt, the reader should refer to the appendix to this chapter, which also contains a reminder of the basic notions of multivariate calculus and some matrix algebra needed to follow the exposition.

1.1 Unconstrained optimization, concave and convex functions

In what follows we assume all functions to have continuous second-order derivatives, unless otherwise stated. Strictly speaking, all domains of defi- nitions should be open subsets of the multidimensional real space so that no boundary problems arise.

1.1.1 Unconstrained maximization

Consider the problem of finding a set of values xx, x2,.. •, xn to maximize the function f(xu...,xn). We often write this as

Maximize / ( x ) , (1.1) X

where x is understood to be an ^-dimensional vector. We refer to the problem of (1.1) as an unconstrained maximum because no restrictions are placed on x.

Necessary conditions. Suppose we find a solution to this problem and denote the optimal vector by x*. Consider an arbitrarily small deviation

1

2 1 Static optimization

from x*, say dx. If we have a maximum at x*, then / must not increase for any dx.

The change in / is approximated by

df=ZfXi(x*)dXi. i

Clearly, df<0 if we have a maximum at x*. Furthermore, suppose we found some dx vector such that df< 0; then by using the deviation (—dx) we would obtain an increase in / . Therefore, it must be that for any dx vector, df is equal to zero. The only way this can be achieved for arbi- trary deviations is to require each derivative fXj(x*) to vanish. Formally,

f(x) reaches a maximum at x* implies fx.(x*) = 0, i = 1,..., n. (1.2)

This is called the first-order condition. Several remarks must now be made. First, the above reasoning, hence (1.2), also applies to minimization prob- lems. Second, we have been lax in defining a maximum. We should have distinguished a global maximum from a local maximum. We say that f(x) reaches a global maximum at x* if f(x*) > f(x) for all x on its do- main of definition (assumed to be an open set). We say f(x) reaches a local maximum at x* if /(x*) > f(x) for all x "close" to x* (i.e., for all x within 6 units of distance from x*, where 6 is some positive number). The local maximum is a much weaker concept than the global one. However, because our argument relies on arbitrarily small deviations from x*, it applies to both cases. The first-order condition (1.2) follows from the existence of a maximum; hence, it is a necessary condition for a maxi- mum, but it is not the only one, as we now show. As we noted previously, condition (1.2) is necessary for a local minimum as well. The following condition, called the second-order necessary condition, takes a different form for a maximum than for a minimum.

To establish it we must take a Taylor's expansion (with remainder) of the function / about the point x*:

/(x*+tfx) = /(x*) + S fx.(x*)(dXi) i = \

or in vector notation (see the Appendix for details),

/(x*+rfx) = /(x*) + (rfx)'./x(x*)+i(rfx)'-/xx.(x*)-(rfx) + - + i ? , (1.3b)

where dx is small enough (i.e., |rfx| < 5) that higher-order terms vanish relative to second-order terms.

1.1 Unconstrained optimization 3

Suppose again that we have a (at least local) maximum, that is, /(x*) > /(x*+tfx), Vcfx, |rfx| <6. Then /x(x*) = 0, and neglecting terms higher than the second order we have

/(x*+tfx)-/(x*) = i(rfx)Vxx,(x*).tfx < 0, because x* is a maximum.

Since (tfx)'-/XX'(x*) • (dx) is negative or zero for all small deviation vectors d\9 the Hessian matrix of/evaluated at x* must be negative-semidefinite. This is the second-order necessary condition:

/(x) reaches a maximum at x* implies /xx(x*) is negative-semidefinite. (1.4)

Again, (1.4) applies to global as well as local maxima.

Sufficient conditions (for a local maximum). It is unfortunately not pos- sible to state conditions that are both necessary and sufficient for a func- tion to reach a maximum. We can, however, easily provide sufficient con- ditions:

If fx.(x*) = 0, / = 1,..., n, and/XX'(x*) is negative-definite, then f(x) reaches a local maximum at x*. (1.5)

To prove this we shall consider again Taylor's expansion in (1.3) and let dx->0, so that the second-degree term dominates those of higher order while the first-degree term vanishes; we obtain / ( x * + d x ) < / ( x * ) , thus establishing x* as a local maximum.

1.1.2 Global results and concave functions

When we seek a maximum in an economic problem, it is most often a global one. Indeed, it is little comfort to know that we are doing the best we can but only if considering policies which differ minutely from the current one (local optimum). It is also clear that we will not be able to characterize a global maximum with conditions on the values of the func- tion and its derivatives at the maximum itself; we will need to place re- strictions on the overall shape of the function, restrictions that apply everywhere on the domain of definition, which we denote by X.

Consider the exact form of Taylor's expansion to the second degree: there exists a point x, on the line segment between x and x such that

/(x) = /(x) + ( x - x r . / x ( x ) + i ( x - x r . H ( x , ) . ( x - x ) , (1.6)

where H(x,) denotes the Hessian matrix of/, evaluated at the point \t. If we were to restrict our attention to functions with a negative-semidefinite

4 1 Static optimization

matrix everywhere on its domain of definition, then the last term of (1.6) would be guaranteed to be nonpositive for any x, and the requirement that x be a global maximum (i.e., /(x) — f(x) < 0 Vx eX) would be equivalent to the first-order condition /x(x) = 0. We now formalize this argument.

Definition 1.1.1. A function with continuous second-order derivatives defined on a convex set X is concave if and only if its Hessian matrix is negative-semidefinite everywhere on its domain of definition X.

Theorem 1.1.1. Let f(x) be a concave function; then it reaches a global maximum at x if and only if /x(x) = 0.

Definition 1.1.1 applies only to functions with continuous second-order derivatives. It is useful to have a more general definition of concavity that does not require this assumption.

Definition 1.1.2. A function /(x) with continuous first-order derivatives defined on a convex set X is concave if and only if

/ ( x 2 ) - / ( x 1 ) < ( x 2 - x 1 ) , - / x ( x 1 ) ,

for all Xj,x2on X.

Note that Definition 1.1.2 is less stringent than Definition 1.1.1 in terms of differentiability restrictions, since it requires continuity only for the first derivatives; this is the only difference between the two definitions. Indeed, if we assume that the function has continuous second-order de- rivatives, we can see that the two definitions are equivalent simply by writing down the exact form of Taylor's expansion. Given two arbitrary points Xi and x2, there exists a point x, between them such that

/(x 2 ) = /(x1) + (x2~x1r-/x(x1) + i ( x 2 - x 1 ) , - H ( x / ) . ( x 2 - x 1 ) ,

/ ( x 2 ) - / ( x 1 ) - ( x 2 ^ x 1 ) ' - / x ( x 1 ) = i ( x 2 - x 1 ) , - H ( x / ) . ( x 2 - x 1 ) < 0 .

The geometric interpretation is simply that a tangent plane to the graph of f(x) must remain everywhere above the graph, the equation for the tangent plane at xx being

^ = /(x1)-H(x-x1)'-/x(x1).

This is illustrated in Figure 1.1a for functions of one variable. Defini- tion 1.1.2 does not cover functions that have "kinks" and as such are not differentiable everywhere. To admit this case, a more general definition is needed.

1.1 Unconstrained optimization

f(x)

tf(X l) + ( l - t ) f ( x 2 )

(a)

(b)

tangent

graph

Figure 1.1

Definition 1.1.3. A function fix) defined on a convex set X is concave if and only if

f(xt)^tf(xl) + (l-t)f(x2),0^t^hallxl,x2mX,

where xt = txi + (l — t)x2.

If a function satisfies Definition 1.1.2, it also satisfies Definition 1.1.3. To see this we state Definition 1.1.2 in two instances:

6 1 Static optimization

/ ( x 2 ) - / ( x , ) < ( x 2 - x , ) ' . / x ( x , ) and

/(xO-AxJ^iXi-Xty-Uxt), where

xt = txl-\-(l-t)x2 for some/, 0 < / < l . Since

x2—xt = t(x2—Xi) and Xj — x, = — (1 — t)(x2—Xj), we have

/(x 2 )-/(x,)</(x 2 -x 1 ) , -/ x (x,),

f(xl)-f(xt)^-(l-t)(x2-x1Y'fx(^)' Multiplying the first inequality by (1 —t), the second by t, and adding yields (with 0 < t < 1)

//(X l) + ( l - / ) / ( x 2 ) - / ( x , ) < 0 ,

which was to be proved. Note that no differentiability properties are required in Definition 1.1.3.

The geometric interpretation of this definition is that a line (or chord) joining two points of the graph always lies below the graph, since the left-hand side of the inequality represents the value of / at a convex com- bination of Xj and x2 and the right-hand side is the same convex combina- tion of the values of the function at Xj and x2 - hence the height of the point on the chord above x,. This is illustrated in Figure 1.1b for functions of one variable.

Concave functions have many notable properties; Theorem 1.1.2 lists some of the most useful ones.

Theorem 1.1.2

(i) Let f(x) be a concave function and k > 0 a constant; then kf(x) is a concave function.

(ii) Let f(x) and g(x) be concave functions; then f(x) + g(x) is itself a concave function.

(iii) Let f(x) be a concave function; then the upper contour set de- fined by B(x) ss {x e Rn | /(x) > /(x)} is a convex set.

(The converse of (iii) is not truel)

The proofs of these results are straightforward; for instance, (iii) re- quires that we show that if f(xx) > /(x) and /(x 2 ) > / ( x ) , it follows that f(xt) > / ( x ) ; this is obvious from Definition 1.1.3.

1.1 Unconstrained optimization 7

Strictly concave functions: unique global maximum. While concave func- tions have the property that a solution of the first-order condition yields a global maximum, this does not ensure the uniqueness of that solution: a concave function may reach its global maximum at several points. For example, the following function is concave, but the first-order condition admits as a solution any point between 1 and 2; thus, the function reaches a global maximum at any x* such that 1 < x* < 2.

fx-0.5x2, x < l , / ( * ) = ] 0.5, 1 < J C < 2 ,

[ ( x - l ) - 0 . 5 ( x - l ) 2 , 2<x.

Other examples will be encountered in Section 1.1.5. It is sometimes desirable to place more restrictions on the function so

that if a maximum exists, it is the unique global maximum. We use this as a means of introducing a subclass of concave functions called strictly concave functions. Definitions 1.1.2 and 1.1.3 are adapted by simply re- quiring strict inequalities.

Definition 1.1.3'. A function /(x) defined on a convex set X is strictly concave if and only if

f(xt)>tf(xl) + (l-t)f(x2), 0 < f < l ,

for all xl,x2inX, where Xj^x2and xt = txi + (l — t)x2.

Definition 1.1.2'. A function/(x) with continuous first-order derivatives defined on a convex set X is strictly concave if and only if

/ ( x 2 ) - / ( x 1 ) < ( x 2 - x 1 ) , - A ( x 1 )

for all Xj and x2 in X, where Xj 5* x2.

It is obvious from Definition 1.1.2' that fx(xx) = 0 is necessary and suf- ficient for Xj to be the unique global maximum of that function / .

We cannot claim that functions with continuous second-order derivatives are strictly concave if and only if their Hessian matrix is negative-definite, because some strictly concave functions have a Hessian matrix which be- comes negative-semidefinite at some points. One instance is f(xux2) = —(JCJ)4— (x2)

2, which is negative-definite everywhere but at xx = 0, when it is negative-semidefinite. We must be content with the following theorem.

Theorem 1.1.3. A function that is defined on a convex set X and has a negative-definite Hessian matrix everywhere on X is strictly concave.

The reader is invited to prove this result using Definition 1.1.2'.

8 1 Static optimization

1.1.3 Unconstrained minimization and convex functions

Results for minimization problems are just mirror images of those for maximization problems and are obtained by replacing /(x) by —/(x). Thus, the first- order necessary condition for a local minimum at x* is

/.(x*) = 0, I = 1 , . . . , / I , (1.7)

and the second-order necessary condition is

/XX'(x*) is positive-semidefinite. (1.8)

The sufficient conditions for a local minimum at x* are

/x(x*) = 0 and /xx(x*) is positive-definite. (1.9)

Similarly, we have to define convex functions in order to obtain global results on minimization. Corresponding to Definitions 1.1.1, 1.1.2, and 1.1.3 we now have the following (results on strictly convex functions are indicated in parentheses).

Definition 1.1.4. A function with continuous second-order derivatives defined on a convex set is (strictly) convex if and only if its Hessian matrix is positive-semidefinite (if its Hessian matrix is positive-definite).

Definition 1.1.5. A function/(x) with continuous first-order derivatives defined on a convex set X is (strictly) convex if and only if

/ ( x 2 ) - / ( x 1 ) > ( x 2 - x 1 ) , - / x ( x 1 ) for all x1,x2inA

r,

( / ( x 2 ) " " / ( x i ) > ( x 2 ~ x i ) , , / x ( x i ) for all x1,x2inA r, wherex 1 ^x 2 ).

Definition 1.1.6. A function /(x) defined on a convex set X is (strictly) convex if and only if

f(xt)^tf(xl) + (l-t)f(x2), 0 < f < l , a l l x ^ i n J T ,

(f(xt)<tf(xl) + (\-t)f(x2)i 0 < / < l , Mxux2mXi x ^ x 2 ) .

Theorem 1.1.4

(i) Let /(x) be a convex function and k > 0 a constant; then kf(x) is a convex function.

(ii) Let f(x) and g(x) be convex functions; then /(x)-hg(x) is itself a convex function.

(iii) Let f(x) be a convex function; then the lower contour set defined by W(x) = [xeRn\f(x)<:f(x)} is a convex set. (The converse of (iii) is not true!)

(iv) Let f(x) be a (strictly) convex function; then — fix) is a (strictly) concave function.

1.1 Unconstrained optimization 9

f(x)

f(x2) tf(Xl) + ( l - t ) f ( x 2 )

f(X!>

f(xt)

(a)

(x2 - x ^ f C x ^

(b)

Figure 1.2

(v) A linear function is both convex and concave but not strictly either.

Definitions 1.1.5 and 1.1.6 are illustrated in Figure 1.2 for convex func- tions of one variable.

1.1.4 Geometric representation

Figures 1.3a and 1.3b represent the graphs of a concave and a convex function, respectively. It is important to realize that a concave function

10 1 Static optimization

f ( X i , X 2 ) f

Figure 1.3

need not have a maximum, nor a convex function a minimum. If they do, then one is somewhat dome-shaped and the other bowl-shaped. It is then obvious that a rod connecting two points of the dome remains under it (Definition 1.1.3), while such a rod connecting two points of the bowl remains above its walls (Definition 1.1.6). It is clearly inconvenient to rely on three-dimensional diagrams; instead, we most often use level curves. We know that if a function is concave, its upper contour sets are convex sets. We use this information in Figure 1.4a to draw some level curves of a concave function, where the arrows indicate directions of increase of the function and one convex upper contour set is hatched. We can also verify that Definition 1.1.3 is satisfied: the function takes on the value c at points A and B\ thus, it takes on a higher value at a point between

1.1 Unconstrained optimization 11

f ( x 1 , x 2 ) > c

f(Xi , X 2 ) < C

f ( x 1 , x 2 ) = c

(a)

f ( x 1 , x 2 ) =

(b)

x 2 * f(Xj , X 2 ) = C

(c)

Figure 1.4

them, D9 which is naturally within the convex upper contour set. A simi- lar picture emerges for a convex function in Figure 1.4b, where a lower contour set is hatched and the arrows indicate directions of increase of the function. Finally note that a contour curve such as the one in Figure 1.4c cannot correspond to a concave (or a convex) function since it delin- eates no convex set on either side of it.

A word of warning is in order. Because concave functions have convex upper contour sets but some other functions do too, we cannot rely on

12 1 Static optimization

this contour curve representation to characterize concave functions ex- actly. For many purposes, however, it will be adequate.

1.1.5 Numerical examples and some useful functional forms

It is useful to develop some "feel" for the concavity properties of func- tions so as to avoid always running back to the definitions. The knowledge of a few simple functions along with the composition rules already out- lined and some more to follow is very helpful. We first list a few functions and the conditions for their concavity and/or convexity. The reader is in- vited to check these as exercises, using mainly Definitions 1.1.1 and 1.1.4.

n

/ ( x ) = I I (*/)a/ is concave for x > 0 i = \

n if and only if a, > 0, V/, and 2 a, < 1. (1.10)

/ = i

/(x) = (tf0+tfjXjH \-anxn) a, defined when

tfo+tf^jH hanxn>0is concave if and only if 0 < a < 1; it is convex if and only if a > 1 or a < 0. (1.11)

/(x) = x'-A-x is concave if and only if A is negative - semidefinite; it is convex if and only if A is positive - semidefinite. (1.12)

n

/(x) = 2 ai ln(*/ + ff/) is concave whenever it is defined i = \

(i.e., */ +#, > 0, all /) if and only if a, > 0, V/; it is similarly convex if and only if a, < 0, Vi. (1-13)

Theorem 1.1.5. An increasing concave function of concave functions is concave.

Proof Let W(x\...9x

n)mV(Ul(xl)9...9U n(xn)),

where xl denotes a vector of arbitrary dimension, V is increasing and concave in all Ul jointly, and Ul is concave in x', V/. We use the standard notation for convex combinations: z, = Zzj-h (1 - t)z2, 0 < t < 1, W(xl...ix?)=V(U

1(xj)9...9U n(x?))

^V(tU\x\)^{\-t)U\x\)9.^9tU n(xnx)^(\-t)U

n(xn2))9 because all Ul are concave and V is increasing,

^tV(U\x\)9...,U n(x^)) + (l-t)V(Ul(x12)i...iU

n(xn2))9

1.1 Unconstrained optimization 13

by the concavity of V,

Theorem 1.1.6. Let /(x) be a function of n variables and let z = —x and h(z) = f(x); then if f(x) is concave (convex), so is h(z).

The proof is obvious using, for instance, Definition 1.1.3. As an example, f(x) = 1 — e~x is concave in x; hence, h(z) = 1 — ez is concave in z, where z = -x.

We now consider a few numerical examples that may or may not pos- sess a global maximum.

Example 1.1.1. Let/(x) = x'-A-x + a'-x, where

- 1 1 5j'

It is concave since A is negative-definite and the linear term does not af- fect concavity. To find a maximum, set the first-order derivatives to zero and solve: fx = -2xx+x2-\ = 0 and f2 = Xi~2x2+5 = 0 yield Xj = l, x2 — 3, the point at which / reaches its global maximum.

As we mentioned earlier, a function may reach its global maximum at many points; that is, the solution may not be unique. This is illustrated in the following example of a concave but not strictly concave function.

Example 1.1.2. Let f(x) = {xl)°' 3(x2)

0'7-0.3x^0.7x2. We know that this function is defined and concave for all x positive (e.g., use (1.10)):

/ 1 = 0.3U1)" a 7U2)°-7-0.3 = 0,

/ 2 = 0.7(x1) a 3(x2)-°-

3-0.7 = 0.

These first-order conditions have many solutions; namely, any x satis- fying xx — x2 is a solution. The global maximum value of / is zero and the upper part of its graph is shaped like the inside of a tunnel.

Example 1.1.3: saddle point. In this example we emphasize the idea that a function may be concave in all its variables but not necessarily concave in those variables jointly. We also introduce the concept of a saddle point. The example involves the function f(xu x2) = — (xx)

2+axxx2—(x2) 2 for

various values of a. Case (a). Let f(xx,x2) = — (xx)

2—(x2) 2; then

A = - 1 0.5 0.5 - 1

and a =

H-T - 2 °1 0 - 2 '

14 1 Static optimization

Figure 1.5

the function is concave, and (0,0) is the global maximum. This is illus- trated in Figure 1.5a. We proceed to "stretch" this function by introduc- ing ever-increasing mixed terms.

Case(b). Let f(xux2) = ~(xl) 2+xlx2-(x2)

2; then

- 2 1 H =

1 - 2

1.1 Unconstrained optimization 15

the function is still concave, and (0,0) is still a global maximum. The stretching is shown in Figure 1.5b.

Case (c). Let f{xux2) = — (Xi) 2+2xlx2—(x2)

2; then

- [ i ->]• the function is still concave, but the first-order conditions only imply xx = x2; thus, there are many points at which the function reaches a global maximum. We again have a tunnel shape: the stretching has been carried out to an extent that we have a tubular shape with a horizontal top line. Note also that |H| = 0. Further stretching will destroy concavity, as we now see.

Case (d). Let f(xux2) = -(x1) 2-\-3xlx2-(x2)

2; then

L 3 - 2 j ' the function is no longer concave in (xux2) because |H| = — 5, although it is still concave in xx and x2 individually. The solution of the first-order conditions still is (0,0), but we can no longer claim that it is a maximum. It is not a minimum either, but what we call a saddle point. The level curves are drawn in Figure 1.5d; the two straight lines corresponding to / = 0 delineate four regions, and when we move from region I to III the origin appears to be a minimum, but when we cross the origin while mov- ing from region II to IV it appears as a maximum. This is the essential property of a saddle point configuration: it appears as a maximum in some directions and as a minimum in others. These directions need not be the axes as Figure 1.5d shows. Thus if we cross the origin following any one axis, it appears as a maximum with respect to that variable, which is as it should be since / is concave in xx and concave in x2, separately. We have drawn a three-dimensional representation of the graph in Fig- ure 1.5e. The additional mixed term has lifted the ends of the tunnel; it does look something like a saddle. A mountain pass is another, less com- mon description.

1.1.6 Some economic applications

We are now able to tackle any economic problem in which the objective is to maximize some objective and where the entities to be chosen are many while their choice is unrestricted. One such problem is profit maxi- mization by a competitive firm, to which we now turn.

Let / ( * ! , . . . , x n ) be the output obtainable from input levels xu...,xn. If output price is /?, the price of input / is wh and some fixed cost is k, the maximization of profit reduces to choosing (xu...,xn) to maximize

16 1 Static optimization

n

pf(xl9...,xn)- £ WiXt-k. i = \

If we assume that / is concave, the global maximum will be the solu- tion of the n equations

The first term is the rate of increase in output per unit of input / at the margin (called the marginal physical product of input /) multiplied by the output price; this is called the marginal value product of input / (MVP/ for short). The above condition equates it to the price of input /; thus, the price of the input is equal to the contribution to revenue made by the marginal unit. This seems sensible, yet fails to relate maximization of profit to the concavity of the profit expression. We now seek to clarify this relationship. In general, economic sense dictates that if MVP, > wh we would gain by increasing the input level; conversely, MVP/ < W/ would lead us to decrease input. Suppose now that / is concave and indeed that fa < 0 for all /; then the derivative of / with respect to xf decreases when X; increases; hence, if xt were to rise above the level x* indicated by the first-order condition, wt would exceed MVP/ and we would bring xt back down. Similar reasoning shows that if xt strays below that level, we should bring it back up. If, in contrast, fu were positive at x*, that point could not be a maximum, for an increase in xt from x* would increase MVP/ above wt and induce further increases in xh Indeed, with a strictly convex production function we could reach an arbitrarily large profit; in other words, the problem would have no solution. This possibility should al- ways be kept in mind for any problem in analytical economics, since we work with unspecified functional forms and a precise solution is never derived. As a way of illustrating this point we consider profit maximiza- tion when the production function is homogeneous.

Homogeneous production functions and returns to scale. Suppose q = f(x) is a production function that is homogeneous of degree h (see the Appendix for definitions). From a starting point of x units of input, sup- pose that we scale the operations up by a factor of / > 1, that is, employ tx units of input; we will obtain an output f(t\) = (t)hf(x), and hence we will have scaled up output by a factor (t)h. Depending on the value of h, this factor (t)h will be larger or smaller than t and output will increase more or less than the input vector. More precisely,

h < 1 -• (t)h<t: f(x) exhibits decreasing returns to scale.

h = 1 -» (t)h=t: f(x) exhibits constant returns to scale.

h > 1 -• (t)h>t: f(x) exhibits increasing returns to scale.

1.1 Unconstrained optimization 17

Theorem 1.1.7. Let f(x) be homogeneous of degree h9 positively valued, and concave; then 0 < h < 1.

Proof. Since fj is homogeneous of degree (h — 1), Euler's theorem yields

Jlxifij = (h-l)fj. i

Multiplying by xj9 summing, and applying Euler's theorem again yields

2 2xixjfij = (h-l)j;xjfj = (h-l)hf. J i J

The left-hand side is the quadratic form x'Hx, where H is the Hessian matrix of / . Concavity of / ensures that it is nonpositive; hence, / > 0 implies A(A-1)<0. •

Note that our argument does not establish that if / is homogeneous of degree h and positively valued, then it is concave if and only if h < 1. This is because on the left-hand side of the preceding equation the values xt and Xj are from the same vector at which f^ is evaluated, a weaker re- quirement than Definition 1.1.1 of concavity. As a counterexample con- sider the function/(x1,Ar2) = (x1

2+X2)1/4, x{>09 J C 2 > 0 . It is positively valued and homogeneous of degree \ but not concave since its upper con- tour sets are clearly not convex sets.

We are now ready to examine the implications of alternative assump- tions regarding the degree of homogeneity of the production function on the profit of the firm. In what follows we assume that all x are positively valued and /(x) > 0, unless otherwise indicated.

If the profit expression ir = pf(x) — w'-x has an unconstrained maxi- mum, it will satisfy the necessary conditions

pfi(\) = wh I = 1,...,/I. (1.14)

If x* solves equation (1.14), multiplying by xf9 summing, and applying Euler's theorem yields

P^xfMx*)=^wixf9 i i

hpf(x*) = w'-x*.

Substituting in the profit expression, we get

7T = ( l - / 0 / 7 / ( X * ) .

Therefore, at x*, profit will be positive, zero, or negative, depending on whether there exist decreasing, constant, or increasing returns to scale, respectively. In the case of increasing returns, first note that the objective function cannot be concave (if it were, h could not exceed 1, by Theorem

18 1 Static optimization

1.1.7) and Theorem 1.1.1 fails us. Furthermore, as in the proof of Theorem 1.1.7, we can show that

x*'-H(x*)-x* = (/z-l)/z/(x*)>0, since h>\.

This demonstrates that H(x*) is not negative-semidefinite and violates the second-order necessary condition for a maximum. Let us remark that increasing returns to scale are often associated with unbounded profit and as such are not consistent with the hypothesis of a price-taking firm. The case of decreasing returns poses no special problems, since we can assume that / is concave, but the case of constant returns to scale is more difficult to handle, although the profit expression is concave under the additional assumption of concavity for / . The problem is with the first- order conditions (1.14):

pfi(\) = wh / = 1,...,/!.

Recall that under constant returns, ft is homogeneous of degree 0; there- fore, if a vector x satisfies these conditions, so will any vector t-x, t > 0. The profit made with any of these vectors remains zero. The scale of oper- ations is thus indeterminate and profit nil. This defect becomes a virtue when in some general equilibrium models such as those of international trade the focus is on the performance of each industry and the number and size of firms in each industry are not a matter of concern. There is, however, a further difficulty with the constant returns to scale assump- tion for an individual competitive firm. The problem is that for an arbi- trary set of prices, equation (1.14) usually does not admit a solution, as we now demonstrate. Let /(x) and w be fixed throughout, and suppose that at some price/7* (1.14) admits a vector x* as a solution; then ir(x*) = 0 and tx* is also a solution, t > 0; this is a global maximum, since we as- sumed 7r to be concave. Now consider another output price, say p; the profit expression can be written as

7r = /?/(x)-w'-x

= (p-p*)f(x) + [p*f(x)-w'.x]. (1.15)

We know that the second term in (1.15) has a global maximum of zero at x* (and at tx*), but if p > p* we can make the first term infinitely large by increasing t; hence, there is no maximum, and the first-order necessary conditions (1.14) do not hold anywhere (if they did, a global maximum would exist by concavity of IT). Note that an arbitrary x value may well make profit negative even in this case. Conversely, suppose that p<p*; then the second term has a global maximum of zero at tx*, but the first term can be only negative or zero. Hence, the maximum is found at the

1.1 Unconstrained optimization 19

lower bound x = 0 (i.e., t = 0), but this is not an unconstrained maximum and again the necessary first-order conditions (1.14) fail to have a solu- tion. Note that when p < p*, any vector x > 0 yields a negative profit.

In order to get a more intuitive grasp of these results, consider a firm with two inputs. The equations (1.14) are pf\(xux2) = wx and pf2(xu x2) = w2. However, if / is homogeneous of degree 1, then fx and f2 are homo- geneous of degree 0. Consequently, these derivatives are simply functions of a single argument x2/xx (the only one that matters, since scale is irrele- vant), and both determine a value for it; unless the prices are in a particu- lar configuration, these values will differ and no solution exists. The exact relationship is that output pricep be equal to the unit cost function c(l, w); see the definition of cost functions in Section 1.2.3. Let us now briefly illustrate these results with a numerical example.

Example 1.1.4

f(xux2) = 2(Xl) l/2(x2)

l/\ w1 = l, w2 = 2.

Equations (1.14) are

p(x1)~ l/2(x2)

l/2=l and p(xx) l/2(x2r

l/2 = 2, or

X\/*2 = P2 and xl/x2 = 4/p 2.

Therefore, (1.14) is satisfied if and only if p = p* = y/2; then the optimal input mix is xx — 2x2, the scale is arbitrary, and profit is zero, a global max- imum. If, however, /?</?*, say p=l, then ir = 2(xi)l/2(x2)

l/2—xx — 2x2 and letting (Xi/x2)

l/2=u, TT = X2[ — U2+2U — 2]. The bracketed expres- sion is always negative and so is profit. Finally, if p>p*, say p = 2, then ir = x2[—u

2+4u — 2]. This bracketed expression reaches a positive max- imum of 2 when u = 2, that is, xx = 4x2, and by letting x2 be large we can generate arbitrarily large profits. Finally, note that an arbitrary choice of u may generate a negative profit, for example, u = 4, even with /?*</? = 2.

To gain some geometric insight into the matter, try to visualize the graph of a linearly homogeneous function of two variables. Because of the property f(txu tx2) = tf(xux2) we see that the graph is "ruled from the origin"; a half-line from the origin to any point of the graph lies on the graph in its entirety. Visualize now the graph of input costs C = wxxx + w2x2; it is a plane going through the origin. Let us now draw the graph of pf(xux2) for low/? values; it lies entirely below the cost plane: profit is everywhere negative. As p rises, the graph comes into contact with the plane, but it does so along an entire half-line from the origin. At

20 1 Static optimization

this value of /?, say /?*, profit is maximized anywhere along that half-line and is equal to zero. As p goes above /?*, the graph rises; it intersects the cost plane, and profit can be negative or positive depending on the choice of inputs. However, as the scale of operations is increased (moving away from the origin), the gap between the graph and the plane can be made arbitrarily large and so can profit: there is no maximum.

This concludes our brief survey of the economic applications of uncon- strained optimization. Many economic problems involve constraints of some sort; this is taken up in the next section.

1.2 Optimization under equality constraints: the method of Lagrange

Economic agents typically face problems of choice subject to constraints. In many cases these take the form of equality constraints. Typical ex- amples are the budget constraint of a utility-maximizing consumer or the resource constraints of a whole economy (land, labor, capital). This yields the classical equality-constrained problem:

Find x*,..., x* that maximize (alternatively minimize) f(xu..., xn) subject to

gl(xl9...,xn) = 0 (1.16)

gm(xu...,xn) = 0, m<n.

We call / the objective function and gJ, j = 1,..., m, the constraints. We require that there be fewer constraints than there are choice variables. Our task is to choose among all feasible vectors x, that is, those satisfying the m-dimensional vector constraint g(x) = 0, the vector(s) that yield(s) the highest value for / ( x ) .

Although in a few simple cases it would be possible to use each con- straint to eliminate one variable from the objective function and thereby obtain an unconstrained problem, we normally do not do this. Instead, we choose the seemingly more cumbersome method of introducing new variables, called multipliers, and solve for all variables at once. One minor reason for preferring this approach is that it preserves the structure and symmetry of the problem. The major reason is that these new variables, the multipliers, will be shown to provide important information on the sensitivity of the solution to parameter changes and on the operation of economic forces. Furthermore, this approach will be seen to be the proto- type of methods used to solve more complicated problems such as non- linear programming and optimal control.

1.2 Optimization under equality constraints 21

1.2.1 The method of Lagrange: Necessary conditions

Consider problem (1.16). We introduce m new variables called Lagrange multipliers (one for each constraint) denoted by Xl5...,Xm and form a new function called a Lagrangean,

m

£(\u...,\nnxu...9xn) = f(xu...,xn) + 2 \j-g J(xu...9xn), (1.17)

or more compactly,

£(A,x) = /(x) + A'-g(x). (1.17')

We can then state the main result.

Theorem 1.2.1. Let x* be a solution to problem (1.16) and let the mxn matrix dg(x*)/dx' = [dgJ(x*)/dXj] have rank m (this is known as the rank condition). Then there must exist a unique set of values Xj,..., Xm such that

d£

d£ d\m

d£ dxi

d£ dxn

= g\x*

= gm(x

= /„<*

=u*

•) = o ,

•) = o,

m

•)+SX/^(x*) =

*)+SXy4(X*) =

= 0,

= 0,

(1.18)

or more compactly (see the Appendix for matrix derivative notation),

£ x = g(x*) = 0 and £ x = /x(x*) + g'x(x*).X = 0. (1.18')

(Note that g'x is the transpose of dg/dx' and thus n x m.)

Proof. We need to show that the column vector /x(x*) of (1.18') can be expressed as a linear combination of the columns of the (nxm) matrix G'(x*) = gx(x*), the weights being identified as the multipliers X of (1.18). We shall prove that the nx(m + l) matrix [G'(x*)i/X(x*)] has at most rank m. Once we have done this our assumption that G(x*) has rank m (the rank condition) implies the desired result; that is, there exists a vec- tor X such that /x(x*) + G'(x*) • X = 0.

22 1 Static optimization

We shall use the implicit function theorem to prove that any feasible vector x° such that [G'(x°);/ X (x

0 )] has rank ra + 1 cannot yield a con- strained maximum for (1.16). Suppose that the rank of the above matrix is (ra + 1); then we can assume without loss of generality that the first (m +1) rows of [G'(x°) ; / x ( # ) ] are linearly independent. By the implicit function theorem it follows that the set of m + 1 equations

gJ(xu...,xn) = 0, y = l , . . . , r a ,

f(xu...,xn)-z = 0,

in n + 1 variables xu x2, • • •, xn and z, which by feasibility of x° is known to have a solution (Xi,x2,...,x£,z°), where z° = / ( x ° ) , also admits as a solution (xux2,...,xn9z), for any arbitrarily chosen {xm+2,...,xn,z) within a rectangular region around (x°, z ° ) :

xf-b < Xj < xf+ 6, j = m + 2 , . . . , n,

z 0 - < 5 < z < z ° + 5 ,

where 5 is some positive number. Note that (xux2i . - . , x m + 1 ) depend on the choice of (xm+2,..., xn, z), and in particular we can choose z = z°+d. Then / ( x ° ) = z ° < z = / ( x ) and g(x°) = 0 = g(x). Hence, x° does not yield a constrained maximum. Therefore, the rank of our (m + l)xn matrix is m. This completes the proof of the theorem. •

Remark (a). The assumption that G(x*) has rank m also guarantees that the vector X that satisfies (1.18') is unique. If the rank condition is relaxed, it is indeed possible (but not certain) that multipliers satisfying (1.18) exist, but they are not unique. To see why, suppose rank G(\*) = r<m; our proof that at a maximum, rank [G'(x*) i /x(x*)] < m +1 still applies, but this matrix can now have rank r or r + 1 (the only two possibilities). If its rank is r, then again /x(x*) can be expressed as a linear combination of the columns of G'(x*), but the weights (the X's) are no longer unique. If the rank is r + 1 , it is impossible to find a vector X such that (1.18) is satis- fied. To illustrate the latter case, we present the following example:

Maximize z = f(xux2) = x2—e* l~l

subject to (x\n-x\n)2 = 0.

In the positive quadrant the constraint can be represented by a 45° line, and for any z, the level curve of the objective function takes the form x2 = z 4- e

Xl~\ Clearly, the constrained maximum occurs at (xl9x2) = (1,1), but at that point fx = —l, / 2 = 1 , £i = g2 = 0

a n d it is not possible to find any X such that (1.18) is satisfied.

1.2 Optimization under equality constraints 23

As an illustration of the case in which the X's are not unique, consider any problem for which the constraints are tangent to one another at the constrained maximum:

Maximize ln(x1jc2x3)

subject to x 1 + x 2 + x 3 = 3 a n d 0.5(X\+X2 + xl) = \.5.

The first constraint, a plane, and the second constraint, a sphere cen- tered at the origin, are tangent at the point (1,1,1), which by symmetry is clearly the maximum. In addition to the constraints, the first-order con- ditions are

xirl - X] — \2Xj = 0, / = 1,2,3.

At the maximum xt = 1, but 1 = Xj + X2 is the only restriction placed on the multipliers; there are therefore many such X values, while a t x * = ( l , l , l ) ,

G(x*) = - 1 - 1 - 1 - 1 - 1 - 1

has rank 1 < 2.

Remark (b). We can provide a more intuitive derivation of conditions (1.18). There is a constrained maximum at x*, by assumption. Therefore, any small feasible change, that is, a small movement along the constraints, cannot improve the value of the objective function. This is the crux of the argument; the rest follows from taking first-order approximations and using linear algebra.

We represent a small movement by the differential notation dx. If it is feasible, it does not change the value of the constraint; that is, it induces a zero small change in the vector g: d% = 0. We stated earlier that such a change in x could not improve / ; therefore, it must also induce df= 0. In compact form,

(dg = )^^-'dx = 0 => tf/=/x,(x*).tfx = 0. (1.19)

The geometric meaning of the rank condition is that it rules out singu- larities (e.g., cusps, multiple points, isolated points) on the constraint surface, so that around x* the curves differ but little from their tangents. G(x*)-t/x = 0 then puts an effective restriction on the change in x which takes into account all constraints and not one of them can be dispensed with. In other words, it makes precise our "small movement along the constraints."

Equation (1.19) more explicitly requires that all (dxu . . . , dxn) that satisfy

24 1 Static optimization

dxi dxn : (1.20a)

must also satisfy

W / = > ^ * . + - + ^ * . = 0. (1.20b)

Thus, if (dxi,..., dxn) solves the system of equations (1.20a), it must also satisfy equation (1.20b). Therefore, the last equation adds nothing to what is already contained in (1.20a), and we must be able to duplicate it with an appropriate weighted sum of the equations of (1.20a). These weights are the multipliers. In the language of linear algebra, the vector of coeffi- cient of (1.20b) and the vectors of coefficients of equations (1.20a) are linearly dependent. We have shown that the coefficients of the last equa- tion must be a linear combination of those of the first m equations. For- mally, there exist weights \u..., Xm such that

dgl dgm 3/(x*)

; (i.2i)

x,^ ( x . 1 + ... + x^ < x ., + ^!)=o. dxn dxn dxn

These X values are unique since we have assumed by the rank condition that G(x*), the coefficient matrix of (1.21), has full rank m. Equation (1.21) and the requirement that x* actually be on the constraint g(x*) = 0 are seen to be identical with the first-order conditions of (1.18).

Geometric interpretation. Consider the simplest case of the maximization of a function of two variables subject to one constraint. Find (x*9 x2) to maximize f(xux2) subject to g(xux2) = 0. This is illustrated in Figure 1.6. The thick curve is the constraint, while the thin curves are contour curves of the objective function; the arrow indicates the direction of in- crease of / . A point such as C is not feasible, while a point such as B is feasible but not optimal, since we can move along the constraint toward higher / values. At point A, however, any move along the constraint re- sults in lower / values and A is the optimal solution: it is at the tangency of one of the level curves of / with constraint g = 0. Recall that the slope of a level curve is given by dx2/dxx = -f\/f2 for / , and dx2/dxx = -g\/gi along g = 0; thus at a point of tangency such as A we have

1.2 Optimization under equality constraints 25

Figure 1.6

dx2 = /i = g\ dxx f2 g2

or

#1 # 2 *

If we define X as the above ratio, then

or fx + \gl = 0 and / 2 + X g 2 = 0. (1.22)

Equation (1.22) and the requirement that g(xx,x2) = 0 are the necessary conditions of (1.18) applied to this simple problem. From our derivation of this result, conditions (1.22) and more generally (1.18) are seen to re- flect the tangency of a level surface of the objective function with the intersection of surfaces representing the constraints.

Example 1.2.1. Maximize f(xux2) = In xx + In x2 subject to 2—x\—x2 = 0. From the obvious symmetry of the problem the solution must be Xj = 1, x2 = 1. Let us then illustrate the necessity of (1.18) at that maximum. With one constraint there is only one multiplier and the Lagrangean is

£(\uxux2) = \nxl + \nx2+\(2—Xi—x2).

The necessary conditions are

26 1 Static optimization

- = 2-X?-x! = 0,

- — = x f 1 - 2 X x 1 = 0,

— = X21-2\x2 = 0. OX 2

To solve this, pass the terms involving X to the right-hand side of the last two equations and divide through to get x2x^

l = xxx2 l\ hence, x\ — x\

and JC1 = JC2= 1, as predicted. Furthermore, substituting these values, we obtain X = 0.5.

Sign indeterminacy of the multipliers. By the very nature of problem (1.16) the signs of the multipliers cannot be ascertained. To see this, suppose that we had written the first constraint as — g1(x) = 0, quite an innocuous change. However, in the first-order conditions (1.18), whenever Xj ap- peared it now would be as -Xjg*. instead of X^]., / = 1,..., n9 nothing else having changed. Clearly, the solution to these equations would remain the same but for the sign of Xl5 which would be reversed. The reader is in- vited to rework Example 1.2.1 with the constraint written as x\+x\—2 — 0 and the Lagrangean as £(X, xu x2) = In Xj + ln x 2 + \[x?+x2 - 2]; the so- lution will be (—0.5,1,1). This peculiarity is of little consequence; con- straints must simply be written in a consistent way.

Remark. The first-order necessary conditions presented above apply equally well to a constrained minimum problem; to distinguish between the two, we must turn to second-order conditions.

1.2.2 The method of Lagrange: second-order conditions

Given a constrained local maximum x* and associated multipliers X, there exists a set of second-order necessary conditions that must hold at that point. There also exists a set of slightly stricter conditions which if satis- fied along with the first-order conditions at some (x*, X) ensures that this point is a local maximum. The situation is thus much the same as in the unconstrained case. These conditions involve the second-order deriva- tives of the Lagrangean £(X,x) = /(x) + X'-g(x). First we need to write down in detail the whole Hessian matrix of <£. It is important to keep precisely to the notation and the ordering of variables, since any change would usually alter the conditions. As before, let x be n x 1 and X be m x 1. The Hessian of <£ will be (ra + n) x(m + n), and since there are two sorts of variables we will often write this matrix down in partitioned form (check the Appendix for differentiation using matrices):

1.2 Optimization under equality constraints 27

B =

B =

..^.X X.'.J...!?.X*.'. <£xA' i <£xx'

0

i A gx' 1

r+fjCAjg^'i where B is the (AW+«) X (m+n) Hessian matrix of <C(X,x) and the four submatrices <CXx', "̂ Xx» ^xv>

a n d <Cxx'are of order mxm, mxn, nxm, and nxn, respectively. Matrix B can be written more precisely as

B =

0 0

0

*i, *l

ri.

» ' " . . . Q"g

Sxx &xn

am : f I y m \ a y . . . f i y m \ a y £*! : Jxxxx^ ^j-\

/ \ / S * 1 x 1 Jxxxn^ £>j = \ Kj&xxxn

Sxn \ / * , * „ + 2 / = i \%xxxn '" fxnxn+Aj = \^j8xnxn

or

mJ.~U.5- I G' ; L

where Gr,=

^rs ~

dg^- dxs

»

d2f dxrdx< 7 = 1

d2gJ

dxrdXr

(1.23)

Care must be taken to order the variables as shown, that is, Xlf...,Xm, xu..., xn since the following theorems are tailored to this format.

Theorem 1.2.2: necessity

(i) Let x* be a local maximum for problem (1.16) and let (x*, X*) sat- isfy (1.18). Then the matrix L in (1.23) is negative-semidefinite for all vectors z satisfying Gz = 0, where L and G are evaluated at (x*,X*).

(ii) If x* is a local minimum for problem (1.16), modify (i) to positive - semidefiniteness.

Theorem 1.2.3: sufficiency

(i) Let (x*, X*) satisfy the first-order condition (1.18) and in addition let L in (1.23) be negative-definite for all vectors z ^ 0 satisfying Gz = 0, where L and G are evaluated at (x*, X*); then x* is a local maximum for problem (1.16).

(ii) If we modify (i) to positive-definiteness we have sufficient condi- tions for a local minimum for problem (1.16).

28 1 Static optimization

One difficulty is that it is not straightforward to check the definiteness of a matrix under constraints. Fortunately there is a set of conditions equiv- alent to those of Theorem 1.2.3.

Theorem 1.2.4: sufficiency. Assume that the rank condition (rank G = m) is satisfied.

(i) Let (x*, X*) satisfy the first-order condition (1.18) and in addi- tion let the last (n — m) leading principal minors of B alternate in sign beginning with that of ( - l ) m + 1 , where B is evaluated at (x*, X*); then x* is a local maximum for problem (1.16). (This sign sequence can also be characterized by requiring the last leading principal minor, i.e., | B | , to have the sign of (—l)n; alternatively, each leading principal minor of order k x k must have the sign of ( - l ) ( * - m ) , A:=2m + l , 2 m + 2 , . . . , m + fl.)

(ii) Let (x*, X*) satisfy the first-order condition (1.18) and in addition let the last (n — m) leading principal minors of B be of the same sign as (—l)m, where B is evaluated at (x*, X*); then x* is a local minimum for problem (1.16).

The conditions of Theorem 1.2.4 are not necessary but can be made so with an additional restriction. The following result will be particularly useful in Section 1.3 (Takayama, 1985, p . 162).

Theorem 1.2.5. Suppose that B, the Hessian matrix of the Lagrangean, is nonsingular. (We say that we have a regular maximum or minimum.) Then the conditions of Theorem 1.2.4 are also necessary for a maximum (resp. a minimum) for problem (1.16).

In order to clarify those rather complicated requirements, we shall look at some examples. First we represent diagrammatically which leading prin- cipal minors we are concerned with:

Begin here; the first leading principal minor considered is of order (2m + l)x(2m + l).

B =

T m i t m i t

n — m i

in

0

G'

III II — III

1.2 Optimization under equality constraints 29

Examples 1.2.2

(i) Let n = 6 and m = 3; B is 9 x 9. The first leading principal minor to consider is 7 x 7; call it B7. For a maximum it must have the sign of (—1)4>0; Bs<0; B9 is the last one and has the sign of (—1)6>0. (Note that i?9is |B|.) In the case of a minimum all of these minors must have the sign of (—1)3<0: B7<0, Bs<0, B9<0.

(ii) Let n = 3 and m = 2; B is 5 x 5. The first leading principal minor to consider is 5 x 5 , thus simply the determinant of B which is required to be of the sign of (—l)3 < 0 for a maximum, whether we look at it as the first or the last one. For a minimum it must have the sign of ( —1) 2 >0.

Now let us consider some numerical examples.

Example 1.2.3. Find the maximum a n d / o r minimum of f(xl9x2,x3) = 2xxx2x3 subject to 3—x\—x2—x3 = 0. Form the Lagrangean

ob( A, X\, X2i X3) = 2X\X2X3 T A [ 3 —X\ —X2 —X3 J.

The first-order conditions are

d£ d\ = 3-xf-xj-xi = 0,

- — = 2*2*3 ~ 2V*i = 0, OX i

d£ — = 2xlx3-2\x2 = 0,

d£ dx*

= 2 J C 1 X 2 - 2 X X 3 = 0 .

These conditions are easily solved and have, among others, the two solu- tions

\ = x1 = x2 = x3 = l and \ = xl = x2= zx3 = —l.

The Hessian matrix of £(\,Xi,X29x3) is

0 -2x1 -2x2 -2x3

B = -2xx - 2 X 2 x 3 - 2 x 2 2x 3 - 2 \

—2A:3 2x 2 2xx

At the positive solution,

2*2

2xx - 2 X

30 1 Static optimization

B =

and

B,=

0 2 2 2

0 -2 -2

- 2 - 2

2 2

- 2 - 2

2

- 2 2

- 2 2

- 2 2

- 2

- 2 2 2

- 2

= 3

while £ 4 = |B| = -192 < 0 . Hence, the last ( 3 - 1 = 2) two leading princi- pal minors alternate in sign beginning with ( - 1 ) 2 > 0 (or alternatively ending with the sign of (—1)3<0); the positive solution is a local con- strained maximum.

At the negative solution,

B =

"0 2 2 2 2 - 2 2 - 2 2 2 - 2 - 2

2 - 2 - 2

2

and B3 = —32, B4= —192. The last two leading principal minors all have the sign of (—1)2<0; the negative solution is a local constrained min- imum.

There are six other solutions to the first-order conditions for the vec- tor (X, xu x2, x3); they are (1,1, - 1 , - 1 ) , ( - 1 , - 1 , 1 , 1 ) , (1, - 1 , 1 , - 1 ) , (1, —1, - 1 , 1 ) , (—1,1,1, —1), and (—1,1, —1,1). The reader is invited to check whether any one of these is a constrained maximum or minimum for this problem. (Hint: All points at which / is positive are maxima; all those at which / is negative are minima.)

Example 1.2.4. Here we deal with two constraints. Find xx, x2, and x3 that maximize f(xu x2, x3) = 4In xx + 2x 2 +8x 3 subject to 8—xx—x2—2x3 = 0 and 1 — 0.5*!—x3 = 0. The Lagrangean is

£{\u\2,xux2,x3) = 4lnxi + 2x2+8x3 + \i[8—Xi—x2—2x3]

+ X 2 [ l - 0 . 5 x 1 - x 3 ] .

The first-order conditions are

<£xj = 8— X\—x2—2x3 = 0,

c £ x 2 = l - 0 . 5 x 1 - x 3 = 0,

£ = 4 x f 1 - X 1 - 0 . 5 X 2 = 0,

1.2 Optimization under equality constraints 31

£* = 2 - X , = 0,

-2X,-X 2 = 0.

0 0

- 1 - 1 - 2

0 0

- 0 . 5 0

- 1

- 1 - 0 . 5 - 4 x f 2

0 0

- 1 0 0 0 0

- 2 - 1

0 0 0

The fourth equation yields Xj = 2 and the fifth X2 = 4; from the third equa- tion xx = 1, which substituted in the second equation yields JC3 = 0 . 5 ; fi- nally, the first equation gives x2 = 6. The solution is (2,4,1,6,0.5). The Hessian matrix of the Lagrangean is as follows (remember to differentiate the first-order conditions with respect to \ u X2, xu x2, x3 in that order):

B =

We need to look at the sign of the last (3 — 2 = 1) leading principal minor; it must be (— l ) 3 < 0 . |B| = — 4x{~2, and when B is evaluated at the solu- tion we have |B| = — 4 < 0 , a local constrained maximum indeed.

1.2.3 Some global results for equality- constrained problems

All the results of the preceding subsection were valid for local optima only; in economics we are more often concerned with global optima. As one would expect, some forms of concavity restrictions will be useful in securing global results.

Theorem 1.2.6. Let (X*,x*) be a solution to equation (1.18). If £(X*,x) is a concave (resp. convex) function of x, then x* is a global maximum (resp. minimum) for problem (1.16).

Proof. Concavity of <£(X*, x) in x along with the last n equations of (1.18) implies that x* is an unconstrained global maximum of £(X*,x); hence, £(X*,x*)>£(X*,x) vx, or/(x*) + X*'-g(x*)>/(x) + X*'-g(x) Vx. Clearly x* satisfies the constraints and g(x*) = 0. Therefore, /(x*) > f(\) + X*'-g(x) Vx and finally /(x*) > /(x) Vx for which g(x) = 0. •

An inconvenience here is that we have to solve the problem before we can ascertain the sort of optimum that is obtained. In some special cases we can do a little better.

Corollary 1.2.6'. Let /(x) be concave (resp. convex) and all gJ(\) be linear. Then a solution of (1.18) provides a global maximum (resp. mini- mum) for problem (1.16).

32 1 Static optimization

Corollary 1.2.6". Let f(x) and all gJ(x) be concave (resp. convex) func- tions. Assume either that f(x) is increasing in x and all gJ(x) are decreasing in x, or that / ( x ) is decreasing while all gJ(x) are increasing. If (X*,x*) solves (1.18) and all the multipliers are of the same sign, x* is a global maximum (resp. minimum) for problem (1.16). (If there is only one con- straint, the requirement on the sign of the multipliers can be dispensed with.)

To prove the second corollary simply note that the restrictions on the monotonicity of / and g and the sameness of sign of the multipliers imply by (1.18) that the multipliers will be nonnegative.

Example 1.2.5: derivation of cost functions. Let q = f(x) be the produc- tion function of a firm and w the vector of fixed input prices. In order to know the cost of production of q units of output the firm must solve the following problem, given w and q. Minimize w'-x subject to q—f(x) = 0.

The objective function is linear, hence convex, and increasing for posi- tive w; the constraint is convex and decreasing if we make the usual as- sumptions about the production function; hence, Corollary 1.2.6" applies. The first-order conditions are q—f(x) = 0 and w — X/X(x) = 0, the latter establishing the fact that X* will be positive and we can apply Theorem 1.2.6 to the Lagrangean £ ( \ * , x ) = w'-x + X*[#—/(x)], which is clearly convex in x under our assumptions. Solving will yield x*(q, w), and the cost function will be C(q, w) = w'-x*(<7, w). We now illustrate this with a numerical example.

Find the cost function associated with the production function q — (xx)

1/4(x2) l/2. This is clearly increasing in (xux2) and also concave ac-

cording to the result of (1.10). The Lagrangean for the minimum problem is

£(\,xux2) = wxxx + w2x2+\[q-(xx) 1/4(x2)V

2].

The first-order conditions are

< 7 - U i ) 1 / 4 U 2 ) 1 / 2 = 0,

Wi-kMxO-^ixjV^O,

w2-\\{xx)V\x2)-V 2 = 0.

The last two yield wx/w2 = x2/2xu and substitution into the first one gives (x1)

3/4 = q(2)-l/2{w2/wl) l/2; hence, xx = (2)-

2/3(q)4/3(w2/wx) 2/3

and x2- (2) l/3(q)4/3(wx/w2)

1/3. Total cost is wxxx + w2x2, and the cost function is

1.2 Optimization under equality constraints 33

C(q9 wl9 w2) = ((2)" 2/3 + (2)1/3)(g)4/3(w1)

1/3(w2) 2/3

= 3(2)"2/3(g)4/3(w1) 1/3(w2)

2/3, C(g9 wu w2) - 1.9(^)

4/3(w1) 1/3(w2)

2/3.

Note in passing that this is a convex function of q that will generate the usual increasing marginal cost feature. The value of the multiplier is X = (2)4/3(^)1/3(w1)

1/3(w2) 2/3.

Example 1.2.6: derivation of demand functions. Let U(x) be the utility function of an individual, p the vector of prices, and m the individual's income. This consumer seeks to maximize utility subject to the budget constraint, hence to choose x to maximize U(x) subject to p'-x = m.

If we assume that the utility function can be transformed into a concave function by a monotone-increasing transformation, we can take U(x) to be concave, and since the constraint is linear in x, Corollary 1.2.6' applies. The first-order conditions are m — p'« x = 0 and Ux(x) — Xp = 0. Taking the ratio of two of the first-order conditions establishes that C//(x)/L(/(x) = Pi/Pji the ratio of marginal utilities is equal to the corresponding price ratio. Solving will yield x*(m,p), which are the quantities of goods the individual is willing to buy at price p, with income m - hence the de- mand functions for this consumer. We now illustrate this with a numeri- cal example.

Derive the demand functions associated with the utility function

U(x)= S f t l n U f - 7 i ) , / = i

assuming /?,- > 0, 2 / ft = 1 (without loss), and m > p'-y = 2?= i A7/- The Lagrangean is

£(X,x)= S ftlnUy-TfJ + Xtm-p'-x). / = i

The first-order conditions are

m — p'-x = 0

0i = \pi(xi-yi)9 / = l , . . . , / i .

Summing the last n equations yields

1 = X(p'-x-p'-7) = X(m-pr-7),

and eliminating X gives (3i = [pi(xi — yi)]/(m — p''y) or finally X/(m,p) = 7/ + J3/[(AW — P'*y)]/Pi' These demand functions are known as the linear expenditure system.

34 1 Static optimization

1.2.4 The method of Lagrange: economic applications

We begin this section by providing an interpretation of the multipliers. Similar reasoning will be encountered throughout this book.

Economic interpretation of the multipliers. We first examine the simplest possible case of the efficient allocation of a resource between two indus- tries. Let qt = Fj(Xj) and pt be, respectively, the production function and output price in industry /, / = 1,2; xt is the amount of a resource used by industry /; the total amount of resource available to both industries is X. Assume that output prices are determined by the world market, and hence are fixed here. In order to allocate the resource efficiently, the central planner maximizes total revenue subject to the resource constraint:

Maximize R(xux2) = plFl(xl)+p2F2(x2)

subject to Xi+x2 = X.

Under the standard assumption of concavity of production functions, the first-order conditions will be necessary and sufficient for a maximum. Those conditions are

X-xf-x} = 0, AF{Uf)-X* = 0, (1.24)

P 2 *2(*2)-X* = 0,

where the primes denote derivatives, the asterisks denote optimality, and the Lagrangean is

£(\,xux2) = plFl(xl)+p2F2(x2) + \[X-xl-x2].

The last two conditions look remarkably like the conditions for profit maximization with X as the price of the resource input; indeed, the re- semblance is not coincidental, as we now demonstrate.

Suppose that there is an exogenous small change in the amount of re- source available: we now have X+ dX of it. How will the central planner allocate this extra amount, say dxx to industry 1 and dx2 to the other? We are dealing with arbitrarily small changes, and thus linear approximations through total differentials are acceptable. The change in the maximum revenue is

dR* = pxF[(x\)dxx+PlFi(xt)dXto

dR* = \*[dxl + dx2]9 by (1.24),

and since the resource constraint must be satisfied, we have dxx + dx2 = dX and finally

1.2 Optimization under equality constraints 35

dR*/dX=\*. (1.25)

This is a very important result. The total derivative of the maximum rev- enue with respect to the amount of resource available is equal to the op- timal value of the multiplier attached to that resource. Speaking loosely, one extra unit of the resource would allow the planner to generate $X* more in revenue; therefore, X* is what it is worth to him, and he would be willing to pay $X* to get that extra unit. We see that X* emerges as the marginal worth of the hitherto unpriced resource; a common name for it is the shadow price or the imputed value of the resource. Thus, the method of Lagrange generates as a by-product of the solution a shadow pricing system that operates as a real one, as we shall soon demonstrate. First we state this result in a more general form.

Theorem 1.2.7. Let (A*,xls,t,...,x"*) be the solution to the problem of finding (x1,..., x") to maximize ^(x 1 ,..., \n) subject to x1 -I h xn = X, where X is an ( m x l ) vector of fixed amounts of resources, and xl is the (mx 1) vector of inputs to the /th industry. Then

dR(xl*,...,x"*)/dXj = \*j, y = l,...,m. (1.26)

The proof is exactly the same as the one in the simplest case and is left as an exercise.

Let us continue with the single-resource, two-industry case. Consider a similar economy but a decentralized one in which each industry maxi- mizes its own profit regardless of what the other one does and without any knowledge of the resource constraint. Let the prices be pu p2 as be- fore plus X* for the resource. The conditions for profit maximization are

PlFi(Xl) = \* and p2F'2(x2) = \*.

Clearly, the (x*,x2) solution of (1.24) applies here as well. Therefore, if we were to use the shadow price of the resource as the actual price, optimizing competitive behavior by the industries would lead to the re- source being exactly exhausted without anyone paying any mind to it. This brings out the essential role of prices in decentralizing the decision- making process. Quite obviously, the result generalizes to many indus- tries and many resources. This is a simple instance of the "invisible hand" phenomenon. To gain yet more insight into the process let us consider a numerical example.

Example 1.2.7. Let Fl(xl) = 12(xl)V\ F2(x2) = {x2)^\ Pl = l, p2 = 3, and X=12. The central planner's problem is to maximize R(xx,x2) = \2(xx)

l/* + ?>(x2) l/3> subject to xl-\-x2 = 12. The Lagrangean is

36 1 Static optimization

£ ( X , x 1 , x 2 ) = 12(x1) 1/3 + 3(x2)

1/3 + X [ 7 2 - x 1 - x 2 ] ,

and the first-order conditions are

12-xl-x2 = 0, 4(xl)~ 2/3 = \ and ( J C 2 ) " 2 / 3 = X.

These yield xx = Sx2 and finally x* = 64, x2 = 8, and X* = 0.25. To understand the role of the multiplier, suppose now that each indus-

try maximizes profit while facing a price of w per unit for the resource. We have

(i) Maximize II j = 12(xj)1/3 - wxx\ hence, 4 ( x 1 )

_ 2 / 3 = w or xx = 8(w)~ 3 / 2.

(ii) Maximize n 2 = 3(x2) 1^3 — wx2\

hence, ( J C 2 ) ~ 2 / 3 = w or x2= (w)~ 3 / 2.

The total demand for the resource is thus

xl+x2 = 9(w)~ 3/2.

If the resource is competitively priced (e.g., it is owned by many small independent agents), demand at equilibrium will simply equal supply: 9(w)~3/2 = 72, or w = 0.25. Therefore, the competitive price of the re- source is the same as the value of the multiplier that we had found earlier in the efficient resource allocation problem.

These results are special cases of a very general and extremely useful result, which we now state and prove. The proof follows the lines of the interpretation of X but is more complicated; we shall use matrix nota- tion, and the order of matrices must be carefully monitored but may be skipped on a first reading. (Again, see the Appendix for the notation.) Primes denote transposition.

Theorem 1.2.8: the envelope theorem. Let (X*,x*) solve the following problem, for given p. Find x that maximizes u(x; p) subject to g(x; p) = 0, where X, x, and p are vectors. Then

du(x*;p) 3£(X*,x*;p) = = tor any ps element of p,

dps dPs where du/dps = du/dps + du/dx'-dx*/dps denotes the total derivative of u with respect to ps.

Note: This problem is the same as problem (1.16) with the addition of an ( 5 x 1 ) vector of parameters p.

1.2 Optimization under equality constraints 37

Proof. Let X and x be of order raxl and w x l , respectively. The La- grangean is £ ( A , x ; p ) = w(x;p) + g'(x;p)-A, and the first-order condi- tions are

g(x*;p) = 0, (1.27a)

M x * ; p ) + g'x(x*;p)'A* = 0. (1.27b)

Let the parameters change by dp; the variables will change by dx and the objective function by

du(x*; p) = wx,(x*; p) • dx + wp,(x*; p) - dp. (1.28)

For the constraint to still hold, the effects of changes in p and x must cancel out and we must have

gx,(x*;p).rfx + gp,(x*;p).tfp = 0 ( t h i s i s m x l ) . (1.29)

Transposing (1.27b), we have

M x * ; p ) = -A*'-gX'(x*;p),

and postmultiplying by dx we get

M x * ; P) 'dx = - A*'- gx,(x*; p) -dx = A*'- gp,(x*; p) • dp, by (1.29).

Substituting the above result in (1.28) we obtain

du(x*; p) = [ wp>(x*; p) + A*'- gP'(x*; p)] - dp,

rf«(x*;p) = £p,(A*,x*;p).rfp,

or

. , , , ! • d£(A*,x*;p) du(x*;p)= S T dps,

5=1 °PS

as claimed. •

This result establishes the importance of the method of Lagrange for equality-constrained problems, since this format enables us to gauge the sensitivity of the solution to exogenous parameter changes. It also ap- plies, however, to unconstrained problems in which we simply have

<fa(x*;p) = du(x*;p) dps dps

This case makes it easier to see the common sense behind the envelope result since optimizing in an unconstrained problem required ux(x*; p) = 0; this term has been "optimized out" of the total differential (1.28). In

38 1 Static optimization

the more complicated constrained case we need the multipliers to indi- cate how variables and constraints interact.

Maximum value functions: It is useful to think of the problem in Theorem 1.2.8 in the following way: We are to choose x, given the vec- tor of parameters p; hence, our optimal choice depends on p and may formally be denoted by x*(p). Clearly, in general, the optimal solution will not be unique and we cannot describe x*(p) as a function. However, the maximum value of the objective function that we obtain must neces- sarily be unique, and we can define

F(p)^w(x*(p);p) (1.31)

as a function of p. This is the maximum value function for the problem of Theorem 1.2.8. Minimum value functions could be similarly defined. They are all sometimes called simply value functions. The envelope theo- rem can then be stated as dV/dps = d£/dps. The study of maximum value functions has given rise to a branch of mathematical economics called duality theory, which is used extensively in microeconomic theory, in- ternational trade, and econometrics. A recent survey is that of Diewert (1982). It will sometimes be convenient in later chapters to use the concept of value functions. We shall see that it applies to a much broader class of problems than the one on which we defined it here.

We now illustrate the envelope theorem in a few instances.

Cost functions: Recall that in Section 1.2.3 we derived the cost function from the problem of minimizing w'-x subject to q—f{\) = 0. The minimized objective function (a minimum value function) is C(#,w) = w'-x*(#, w), from which the derivatives with respect to q and w are not obvious. However, from the Lagrangean

£(X,x;?,w) = w'.x*+X*[<7-/(x*)],

we see that

dC 3 £ ,* = — = X* and

dq

dC d£

dws " dws = x*

This establishes that X* is the marginal cost, while the first derivatives of C with respect to w are the input demand functions of the firm. The reader is invited to return to the numerical example of a cost function in Section 1.2.3 (Example 1.2.5) and verify the above results by differentiat- ing the value function C with respect to q, wl9 and w2.

Efficient allocation of resources: This is the problem of Theorem 1.2.7: find (x1, ...,x") to maximize ̂ (x1, ...,x") subject to x1 -I hx" = X,

1.2 Optimization under equality constraints 39

where x' and X are (rax 1) vectors. Forming the Lagrangean

<£(X,x1,...,xw;X) = /?(x1,...,xw) + X , - [ X - x 1 . . . - x " ] ,

we can apply the envelope theorem to obtain the result of Theorem 1.2.7 as a special case:

dR(xu,...,xn*)/dXj = d£(\*9x u,...9x

n*;X)/dXj = \*, or

dV(Xu...,Xm)/dXj = X}9 where

V(Xu...9Xm) = R(x l\...9x

n*)-

Welfare economics. The problem of allocating resources to production activities and of determining the outputs of goods to various consumers in a way that is somehow desirable is a (some say "the") central problem in economics. We can use the method of Lagrange to formalize this prob- lem and shed some light on the issues. First we must set down our nota- tion carefully. There are / individuals, G goods, and R resources. The resources are used as inputs to produce the goods which are allocated to individuals so as to maximize welfare.

Resource r is available in fixed amount Yr9 r=l,...,R. Fg(ylg9...9 yrg9..., yRg) is the production function of good g9 g = 1,..., G; the argu- ments are resource inputs. Ul{x[9...9x

l g9...9x

l G) is the utility function of

individual /, / = 1,..., / ; the arguments are amounts of goods consumed. Total consumption of each good is required to equal the output of it: Sf=i xlg = Fg9 g = 1,..., G. The welfare function to be maximized has as arguments the utility levels of individuals: W(Ul9..., U

l 9..., U

1). We now set up the problem. In order to lighten the cumbersome notation, we write only the "general" argument of any function; for instance, the pro- duction function of good g becomes Fg(...yrg...)9 and the utility func- tion of individual / becomes £/'(... xlg...). In this fashion the subscripts r and g and the superscript / easily identify resources, goods, and indi- viduals, respectively.

Find (...Xg...) and (...yrg...) that maximize

m...u!(...x'g...)...) subject to (1.32)

/ ^x'g = Fg(...yrg...)9 g = l,...,G, and

i = i

G

S yr8=Yr, r = i /?.

40 1 Static optimization

The last constraint indicates that the total amount of each resource used in all industries equals the amount available. We shall use irg as multi- pliers for the first set of constraints and Xr for the second set. The La- grangean is

£{*9\,x,y)=W(-U i(...x'g...)...)

+ 2 *g\Fg(...yrg...)- 2 *j]+ 2 v k - 2 j J - g=l L « = 1 J r=\ L g=\ J

Assuming that all functions are increasing and concave and all variables positively valued guarantees the optimality of the first-order conditions. We now derive these conditions for the four types of variables in the prob- lem: irg, Xr, x

l g, and yrg:

/ V ( . . . J V " ) - 2 * i = 0, g = l,...,G, (1.33) / = i

5 ^ - 2 ^ = 0, r = l /?, (1.34) 8 = 1

WiXUi-iCg^O, i = l , . . . , / , g = l , . . . , G , (1.35)

7rgxFr g-Xr = 0, g = l , . . . , G , r = l,...,tf, (1.36)

where ^ = aw/aty, C/£ = dU!/dx'g9 and Frg = dFg/dyrg. There are G+i? + IxG + GxR conditions altogether. The multipliers Xr and irg are the shadow prices of resource r and good g, respectively. The meanings of (1.33) and (1.34) are clear. To interpret (1.35) we first write it twice: for the same individual / and two different goods g and g':

WjUi=Tcg and W ^ = 7 i y .

Taking the ratio, we get

The ratio of the marginal utilities of any two goods is equal to the ratio of their shadow prices. We now write (1.35) for the same good but two different individuals,

WiUg=irg and WvU l g=icg9

which yields

WiU^Wi'uj;.

For any good, the marginal contribution of a unit of good g to wel- fare achieved through consumption by one individual is the same as that achieved through consumption by another individual. In other words,

1.2 Optimization under equality constraints 41

consumption is adjusted so that the marginal utilities of individuals are exactly balanced with their welfare weights.

We now turn to (1.36); again, writing it for the same good and two re- sources, we get

TrgFrg = \r and -KgFr>g=\r<.

The ratio is Frg/Fr>g = \r/\r>\ hence, the ratio of the marginal physical products of resources r and r' in any industry g is equal to the ratio of their shadow prices. Thus, the shadow prices play the same role here as do input prices for a profit-maximizing or cost-minimizing firm. Writing (1.36) for one resource but two different goods, we have

Trg'Frg> = \r and irgFrg = \r,

which yields Trg>Frg< = TrgFrg. The marginal value product of any resource (using shadow prices) is the same in all industries. If it were smaller in one industry than in another, it would be beneficial to shift some of that resource from the latter to the former.

The above welfare maximum is also known as a Pareto optimum, and it is a classical result of welfare economics that, under some restrictions, it can be supported by a competitive equilibrium. Though we will not go through the proof of such a result, we will nonetheless illustrate it in order to show that the Lagrangean format itself suggests such a relationship. The notation we shall use for the prices of goods and factors is that of the shadow prices identified earlier, irg and Xr for all g and r. For simplic- ity we also assume that production functions exhibit constant returns to scale, so that all firms earn zero profit, the G industries act like so many competitive firms and maximize profit, and the / individuals each own a share of all factors of production, which they sell to obtain the income to be spent on goods so as to maximize their utility. We then have for each firm g, g = 1,..., G: Choose [yrg] to maximize

R

*g'Fg(-»yrg-)- £ Kyrg- r=\

This yields Trg-Frg-\r = 0, r = l /?, g = l , . . . , G . (1.37)

The marginal value product of input r equals its price. Because firms earn zero profit consumers' incomes are just the value of their endowments of resources. Hence, for consumers we need only define their share of the resources (and not their ownership shares of industries). Suppose con- sumer / owns ylr of resource r, with (for feasibility)

2 t f = r r , r = l /?. (1.38) / = i

42 1 Static optimization

Then the income of the representative consumer / is

| X ^ ' - A ' - y ' r = l

and /'s problem is to choose [xlg] to maximize

£/<(...*'...)

subject to G

2 Trgx l g=\'-y

l.

The Lagrangean for this problem is

£(vi9x i) = Ui(...x'g...) + v

i\X-yi- 2 *8x'g

We require for each / = 1,..., /

A'-y'- S *gx'g = 0, (1.39)

C/j-^7rg = 0, g = l,...,G. (1.40)

Thus, given preferences and technologies, once a distribution of resources {yln} has been selected, equations (1.37) to (1.40) characterize demand and supply for goods and resources. A competitive equilibrium [xlg*9y*g] is said to exist at prices {n*, X*} if these equations are consistent, that is, if demand equals supply for both goods and resources without anyone try- ing to achieve these equalities. Specifically [y*g] from (1.37) are required to satisfy

(1.41)

(1.42)

F; = Fg(...y?g...).

Clearly, the existence of such an equilibrium is a substantial question. It occupied economists for many decades in one form or other. The simi- larity between this problem and the previous welfare optimum gives us a clue. Consider the welfare optimum solution {xlg,yrg, irg,\r] defined by (1.33)-(1.36) and try it as a competitive equilibrium. We can see that

G

S = l = Yr, r = l , . . ;R,

and IF*} from (1.37) and {xj,*} from (1.39)-(1.40) must satisfy

i = i

= F* 8=h- ..,G,

1.3 Comparative statics 43

(1.42), (1.41), and (1.37) are identical with (1.33), (1.34), and (1.36), re- spectively. There remain (1.39) and (1.40). Suppose that the distribution of resources [ylr] is such that (1.39) is satisfied for [x

l g, irg9 \r} of the wel-

fare optimum, so that each consumer can exactly afford the bundle allo- cated to her under the welfare optimum. Since this allocation satisfies (1.35), it will also satisfy (1.40) with vl= W{~x. Therefore, the particular welfare optimum (dependent on the welfare function), which we have assumed exists, provides us with a particular competitive equilibrium (de- pendent on the distribution of income), while the method of Lagrange provides a means to calculate it, at least in principle. This last applica- tion reinforces our earlier statement that this method of solving equality- constrained optimization problems provides insight into the structure of the problem.

The next section, although concerned with both constrained and uncon- strained problems, will provide more instances of the usefulness of our approach.

1.3 Comparative statics

In this section we attempt to answer some questions often asked of econ- omists: given some system in equilibrium, how will the variables respond to an exogenous change in some parameter? The question we shall answer is actually slightly different from this. We shall try to ascertain how the variable would have differed from what it is in this equilibrium had the parameter been slightly different. In other words, we do not indicate how the system responds to a change, since this would entail some dynamic movement from one state to another, but we compare two hypothetical static equilibria; this is why the method is called "comparative statics." We will often seem to forget this distinction, however, and speak loosely of response to change, for brevity.

It will be seen that much reliance is placed on second-order conditions. Therefore, the reactions of equilibrium values to outside changes depend crucially on which optimization problem the equilibrium was the outcome of. It is important to understand that this provides us with a framework in which to test the particular optimization hypothesis indirectly.

There are no general results - just a general method of doing compara- tive statics exercises. Before we embark on a general exposition, let us look at a simple case in detail. Recall the derivation of cost functions in Section 1.2.3; it is reproduced here in the case of three inputs. Minimize wlx1 + w2x2+w3x3 subject to q—f(xux29x3) = 0,

£(\9x) = wlxl + w2x2+w3x3 + \[q-f(xux2ix3)].

44 1 Static optimization

The first-order conditions are

q-f(xl9x29x3) = 09

wl-\fl(xl9x29x3) = 09 (1.43)

W 2 - X / 2 U i , X 2 , J f 3 ) = 0»

W3-\f3(Xl9X2,X3) = 0.

We have also seen that the optimal values of x, when seen as functions of q and w, are the demand functions for inputs by a firm producing q units of output. We now wish to examine how the demand for inputs reacts to price changes. Specifically we want to know the sign of dx3/dw39 say. In order to do this, we proceed much as we did when investigating the eco- nomic significance of multipliers. In reaction to a change in u>3, all vari- ables change. We are dealing with minute changes, so that linear approx- imations using total differentials evaluated at the original solution are appropriate. For any first-order equation, say is (v, p) = 0, involving some variables v and some changing parameter /?, we have after the change E(\9 p) + dE = 0, or E(\9 p) + Ey-d\+Ep-dp = 09 but since E(\9p) = 0the procedure simply amounts to setting the total differential of the equation to zero. We now proceed to do this for each equation in (1.43), keeping in mind that all variables (X, xu x2, x3) and the parameter w3 are changing; the other parameters remain fixed and have a zero differential. All deriva- tives are evaluated at the original equilibrium, and we skip all arguments for brevity:

-fi-dx1-f2-dx2-frdx3 = 09

-fl-d\-\fn-dxl-\fl2-dx2-\fl3-dx3 = 09

-frd\-\f2Vdxl-\f22-dx2-\f23-dx3 = 09

-frd\-\f3X-dxx-\f3rdx2-\f3ydx3 = -dw3.

We can divide both sides by dw3, let this differential tend to zero to ob- tain derivatives, and rewrite the system in matrix form to have

r o - / , -f2 -f3 —f\ - X / n — X/12 — X/13 ~fi ""X/2i — X/22 — X/23

[ _ — J3 ~ X/ 3 i — X / 3 2 — X/ 3 3

Equation (1.44) is the comparative statics equation for our problem. The alert reader will have recognized the matrix of coefficients as the Hesi- ian matrix of the Lagrangean, B - hence the matrix on which second-

r d\/dw3 dxx/dw3 dx2/dwi

1 ^ X 3 / ^ 3

0 0

0 - 1

(1.44)

1.3 Comparative statics 45

order conditions are based. The most convenient method of solution for dx3/dw3 is Cramer's rule; it yields

0 - / i -f2 ~f\ ~"Vll ~ \ / l 2 —fl — \Z*21 _ X/22

rfx3 rfw3

0

- / .

-fi

-fi 0

- / . -fi

-h

-fi - X / 1 1

- V 2 1 - X / 3 1

-fi

-x/„ - V 2 1 - x / 3 1

" / 2

- X / , 2

~^fll

~ \ ^ 3 2

~fl - X / , 2

— ^fll

- X / 3 2

0 0

0 - 1

T/T - X / , 3

—x/23

- V 3 3

(as before)

(1.45)

The reader will be pleased to know that we have no intention of expand- ing these determinants. Instead, we rely on the following crucial argu- ment: The input demand function x3(q, w) was not an arbitrary equa- tion. According to our model it was derived by a firm in the process of minimizing the total cost of producing output q. Therefore, the necessary conditions for a constrained minimum apply. Furthermore, we will as- sume that we have a regular minimum in the sense that the Hessian of <£ is nonsingular (without this assumption we could not solve the sys- tem (1.44)). Then according to Theorem 1.2.5 we can use the conditions of Theorem 1.2.4(ii) as necessary for a minimum: the leading principal minors of B are of the same sign (it does not matter which sign it is for our purpose here). Inspection of (1.45) reveals that the numerator is the third such minor, while the denominator is, of course, the fourth - hence, dx3/dw3 < 0. Therefore, the third input demand function is a decreasing function of its own price precisely because the demand function was de- rived through cost minimization. Had it been derived in some other way we would not have been able to use those second-order conditions, and we might not have been able to establish the result. In particular, in the rather incredible case in which a firm would have maximized cost, the sign of dx3/dw3 would be positive. Finally, note that we made no assump- tions on / and that our comparative statics results were derived from necessary conditions (under the regularity assumption) that follow from the existence of a solution to the problem. We must not make too much of this last remark, however, because some conditions are implicitly met by assuming the existence of a sensible solution.

We now describe the method on a general problem like that of (1.16) reproduced here using vector notation: Find x(n x 1) to maximize / ( x ; p) subject to g(x;p) = 0; g is ( r a x l ) and p is a (rxl) parameter vector. The Lagrangean is £(X, x; p) = / ( x ; p) + A'-g(x; p); the first-order condi- tions are

46 1 Static optimization

g(x;p) = 0 and / x ( x ; p ) + g ; ( x ; p ) - X = 0. (1.46)

If we assume the rank condition (as in (1.18)) and regularity (as in Theo- rem 1.2.5), the second-order necessary conditions are given in Theorem 1.2.4 and apply to the Hessian matrix of the Lagrangean, which is repro- duced here with the notation of (1.23):

B = 0 G

G' L is (n + m)x(n + m).

When taking the total differential of (1.46) with respect to X, x, and p, we always get

m[£M£]*-*

[i ^ H ! [dp] = 0. (1.47') or

Normally we simply write down the first part of (1.47) and derive the second part directly from the first-order conditions by differentiating them with respect to the parameters p. Equation (1.47) is the basic equation of comparative statics. Because we have assumed nonsingularity of B, there is always a unique solution to the problem. Sometimes the equation is solved at one go, but more often we are interested in isolating the influ- ence of one parameter on selected variables. Then, as in our introductory example, Cramer's rule is the best method of solution.

We must warn the reader that although the equation can be solved, it does not necessarily imply that we can answer our original question. In many instances the sign of some dxjdpj will be indeterminate. At times this indeterminacy can be resolved at the cost of some additional assump- tions; in other cases it is not possible to find economically sensible as- sumptions that will resolve it. We now illustrate the technique with a few examples.

Convex cost functions. Let us return to the problem of deriving a cost function. Minimize w'-x subject to q—f(x) = 0. We have <£(X,x; w, q) = w'-x + X[# — f(x)] and the first-order conditions

<7-/(x) = 0, (1.48)

w - X / x ( x ) = 0.

We want to investigate the effect of an increase in q, the target output, on the value of X, previously identified as the marginal cost of output. Using the above method we have

1.3 Comparative statics 47

-•&][*]-[> " o -/„. irrfx" where 0 on the right-hand side is n x 1.

The matrix of coefficients is the (n + l)x(n + l) Hessian of the La- grangean, and the right-hand-side vector is the negative of the partial derivative of (1.48) with respect to q. Solving by Cramer's rule, we have

-i -u o -x/„. dq

(1.50)

where A is the determinant of the matrix in (1.49). From Theorems 1.2.4 and 1.2.5 A has the sign of ( - 1 ) 1 < 0. However, we do not in general know the sign of the other determinant in (1.50). We need further assumptions. As- sume that / ( x ) is both increasing and concave; then from (1.48), X > 0, fxx> is negative-semidefinite and — X/xx> is positive-semidefinite. This implies

^ = A - > ( - l ) | - X / x x . | > 0 .

Therefore, an increasing concave function gives rise to the usual nonde- creasing marginal cost (recall that X is identified as the marginal cost by the envelope theorem); this is the same as d2C/dq2>0, and we have a cost function that is convex in the output q.

Compensated price changes. Consider a consumer maximizing utility under a budget constraint, with all prices and income fixed. The resulting consumption bundles treated as functions of prices and income are the demand functions. These are often called Marshallian demand functions, to be distinguished from Hicksian demand functions (which are derived from minimizing the cost of achieving a given level of utility, much as cost functions were derived subject to achieving a given level of output). Suppose now that the price of the last good, pn, rises. This will alter the relative prices and lead to some substitution among goods, but it will also induce a drop in the consumer's real income, since the consumer will no longer be able to afford the previous bundle of goods. Suppose that we want to isolate the substitution effect of the relative price changes. To achieve this we compensate the consumer for the drop in income: we add dy = xn dpn to the consumer's income, where xn is the quantity of good n previously purchased. This will, of course, modify our comparative stat- ics equation. We seek to maximize U(x) subject to y — p'-x = 0; we have <£(X,x) = U(x) + \[y — p ' - x ] . The necessary conditions are

j > - p ' - x = 0,

t / x ( x ) - X p = 0.

48 1 Static optimization

The compensated price change will affect the budget equation for which the total differential would normally be dy — p'-dx—xn- dpn = 0, but here reduces to — p'-dx = 0. Thus, the comparative statics equation is

(1.51) 0 j - p ' IT d\ 1 I 0

- P ! ^xx' J|_ dx J I 0

0 \dpn

Solving (1.51) for dxn/dpn by Cramer's rule involves substituting the right-hand-side column for the last column of the matrix and taking the ratio of that determinant over the determinant of the matrix of coeffi- cients. This yields dxn/dpn = X x (the ratio of the last two leading princi- pal minors of the Hessian of <£). If we assume U is increasing in x, X will be positive and the second-order conditions (Theorem 1.2.5) imply that the above ratio is negative; thus, dxn/dpn<0. This is one of the few re- sults of demand theory: a compensated increase in the price of a good decreases the demand for that good.

Although we have used the method of comparative statics in equality- constrained problems only, it can be applied equally well to unconstrained problems. The method remains the same - we totally differentiate the first-order conditions. To display the versatility of this technique we illus- trate it with a less conventional example.

Education and trade. Consider a small country initially populated by uneducated workers with a low level of productivity. It can bring in for- eign workers who are educated and have a high level of productivity, but this is costly in terms of foreign exchange. It can also use foreign workers to train local ones, thereby transforming the latter into high-productivity workers. For simplicity we assume that there are only two export indus- tries in the country; all other activities are taken as fixed and suppressed. The two export goods are traded at fixed world prices. The country wants to maximize net foreign exchange earnings while maintaining full employ- ment of educated and uneducated workers alike. Before we can charac- terize the optimal policy we must introduce some notation:

/ the number of local, initially uneducated workers (fixed), / the number of local workers that will be trained, T(l) the number of educated workers it takes to train / uneducated

ones,

1.3 Comparative statics 49

L the total number of immigrant (educated) workers, w the foreign exchange cost of an immigrant worker, /1? l2 the number of uneducated workers in industries 1 and 2,

respectively, LX,L2 the number of educated workers in industries 1 and 2,

respectively. The prices and the production functions are, in the obvious notation, Pu / i d ) . W i ) > Pi, fidi), and F2(L2).

Prices are expressed in terms of the foreign currency. We assume that all production functions are increasing and concave while T(l) is increasing and convex. Note that there is an implicit assumption that educated and uneducated workers work separately as the output of good / is given by fiOJ+FjiLi). The problem is thus to maximize

E = PdfiUO +Fl(Ll)] +p2[f2(l2)+F2(L2)] - wL,

where

/ = / - / ! - / 2 (1.52a)

and

L + l = Lx+L2+T{l) (1.52b)

are the full-employment conditions for uneducated and educated work- ers, respectively. Our model is necessarily static and as such has the usual shortcomings. Thus, the / workers being trained can simultaneously be- gin work as educated workers, whereas the T(l) teachers cannot take part in the production of goods. Substituting (1.52a) and (1.52b) we ob- tain the following: choose Lu L 2 , l\, and l2 to maximize

E = Pdfi(li)+Fl(Ll)]+p2[f2(l2)+F2(L2)]

-wm + Lz + li + k-ft-wTff-li-h).

It is easy to verify that this is a concave function of all variables and that the following conditions are optimal:

p1F{-w = 0,

p2Fi-w = 0, (1.53)

Pi/i-w+wr^o, p2f2-*+wT' = 0.

Assuming that a solution to (1.53) exists, the last two equations imply 1 — T' > 0, which means that it takes less than one full-time educated worker to train an additional local worker; the last condition is thus seen

50 1 Static optimization

to equate the marginal value product of an uneducated worker in indus- try 2 with the net benefit of training that worker (the saving of not taking another foreign worker, net of teaching cost). We want to examine the effects of increases in the domestic labor supply T and the cost of foreign workers w on all variables, including L and /. We begin with /. Taking the total differential of (1.53) yields the basic equation

P\F" 0

0 p2F2 n

0 0

0 pJi'-wT" -wT"

0 0 -wT" p2f2-wT"

dLx ~~dT dL2

dl\ dl dl2 dT

0

0

-wT

-wT

(1.54)

We shall depart from our usual method of solution by Cramer's rule, for three reasons: first we want to solve for all variables, second the matrix of coefficients is block diagonal and thus easy to invert, and third we can use the inverse again when investigating the effects of changes in w. The inverse of the matrix in (1.54) is now used to define the solution

PxF{'

0

0

1

0

PiFi

0

0

P2f2-wT" A

wT"

0

wT"

rfL,| | 1 ~w dL2 ~dT

dl I I " A A dh „ „ wT" p^i'-wT" dl

where

A = A f['p2fi - wT"(Pl f{'+p2fl) > 0.

This immediately yields

rfL, dL2 dl, -wT\p2f'j) ^ n dl2 dl dl dl A dl

Furthermore,

0

-wT"

•wT"

(1.55)

-wT"(Plf[') > 0 .

(1.56)

1.3 Comparative statics 51

dl , dT = 1-

from (1.52a) and

d l A -

d! = A

dlx dl2 dl dl

l[Pxfi'Pim>o

Finally, from (1.52b)

- = (- dl K

— > ( S )

(1.57)

= (-l + T')A-1[plf{'p2fl]<0. (1.58)

Equations (1.56)-(1.58) indicate that an increase in the domestic (uned- ucated) work force will not affect the number of educated workers in industry; it will increase the number of uneducated workers in each in- dustry and the number of local trainees, and this last effect will reduce the demand for foreigners to be used as teachers and hence will reduce immigration.

We now turn to the effects of an increase in the cost of educated immi- grants. We directly state the solution as in (1.55):

dLx dw dL2 dw

dw

dh dw

1

0

0

1

0

P2F2

0

0

0

Pifi-wT" A

wT"

0

0

wT" A

Pxfx-wT" A

X

1

1

1 - 7 "

l-T'

with A as in

dLx dw

dh dw

Therefore,

dl dw

(1.55). We have

- ! -o ~</MYr0'

A

, dL > 0 and — - :

dw

dL2 dw

<o,

_dLx dw

1

(PiFi

dJ2 = 0_ dw

dL2 t dw

-<0,

-T')Plf{' A

1 + 7") dl dw

< 0 .

< 0 .

52 1 Static optimization

We discover than an increase in the cost of educated immigrants decreases their number; this decrease is spread among industries but is compen- sated by an increase in the number of local workers trained. This in turn will decrease the number of local uneducated workers in both industries.

1.4 Optimization under inequality constraints: nonlinear programming

It is easy to think of instances in which equality constraints are too lim- iting: why insist that every last bit of a resource be used up when it ac- tually has a negative productivity? To overcome this objection we turn our attention to inequality-constrained problems; this has the advantage of allowing us formally to take into account the nonnegativity of most economic variables. The general format of the problem is to choose x to

Maximize f(x) (1.59)

subject to g ( x ) ^ 0 , x ^ O ,

where x is n X1 and g is m X1. Several preliminary remarks are in order:

(i) This is called a nonlinear programming problem; the theory was developed after that of linear programming (where / and g are linear) - hence the strange name.

(ii) The economic meaning of an inequality constraint on the amount of a resource available is that this particular resource is freely disposable; that is, there is no penalty for discarding it. This is not always reasonable.

(iii) We no longer have any restrictions placed on the number of con- straints relative to the number of variables (both cases m > n and m < n are admitted). At the optimum some constraints will hold as equalities; they are said to be binding constraints. Other con- straints will hold as strict inequalities; they are said to be non- binding or slack. The set defined by g(x) ^ 0, x ̂ 0, is called the feasible set and a point in it is called a feasible point.

(iv) It is essential, when formulating a problem, that the direction of the inequalities be carefully thought out. In many cases it will be some version of "demand cannot exceed supply."

Finally, note that we have not ruled out the usefulness of equality con- straints; for example, accounts must balance exactly.

We shall see that the method of solution is a modification of the La- grangean technique, but the necessary first-order conditions are now a mixture of equalities and inequalities, known as the Kuhn-Tucker condi- tions from the names of the original authors. Before proceeding to an

1.4 Optimization under inequality constraints 53

f(x)

f(x)

f(x)

x > 0 , f ' ( x ) = 0

(a)

f(x)

x = 0, f ' ( x ) < 0

(b)

7\

x = 0, f (x) = 0

(c)

x > 0 , f ( x ) > 0 (with upper bound)

(d)

Figure 1.7

exposition of the theory, we illustrate the central idea of maximization within bounds.

Let / ( x ) be a function of one variable to be maximized subject to x > 0. Whereas an outcome such as depicted in Figure 1.7a is possible, so is that of Figure 1.7b and even that of Figure 1.7c. The following condition covers all possible cases:

/ ' ( x ) < 0 and x > 0; if x < 0 then / ' ( x ) = 0, and if / ' ( x ) < 0 then x = 0.

(Note that f'(x) = 0 with x = 0 is admitted.) This can be expressed more succinctly by

/ ' ( x ) < 0 , * > 0 , x[f'(x)] = 0. (1.60)

Note that the possibility of a negative rate of change in / at the opti- mum is associated with the existence of the lower bound of zero; if there

54 1 Static optimization

x{ > 0 , f 1 ( x ) = x2 > 0 , f 2 ( x ) =

(a)

xx > 0 , f 1 ( x ) = 0 x2 = 0,f2(x) = 0

(c)

^ = 0 , f 1 ( x ) < 0 K2 > 0 , f 2 ( x ) = 0

(d)

Figure 1.8

were an upper bound, the derivative might be positive as depicted in Fig- ure 1.7d.

Condition (1.60) generalizes to functions of many variables. We can illustrate it for a function of two variables: to maximize f(xux2) subject to Xi > 0 and x2 > 0 we must set

fi(xux2)*09 *,>(), xi[fi(xux2)] = 0, i = l,2. (1.61)

There are altogether nine different possibilities; we have represented four of them in Figure 1.8. The reader is invited to construct the rest. The con- strained maximum characterized by (1.61) is called x, whereas the uncon- strained maximum occurs at point M. These two coincide in cases (a) and (c). In case (b) the small arrow represents the direction of increase of the function at x: it points toward larger values of the function and is per- pendicular to the tangent (the xY axis). This identifies /j(x) as zero and /2(x) as negative. Case (d) is similar to case (b).

1.4 Optimization under inequality constraints 55

W e n o w p r o c e e d t o s t a t e t h e K u h n - T u c k e r t h e o r e m . T h e regularity c o n d i t i o n r e q u i r e d m a y t a k e several f o r m s a n d c o r r e s p o n d s t o t h e r a n k c o n d i t i o n of T h e o r e m 1.2.1. W e shall discuss these at s o m e length after t h e m e c h a n i c s of t h e p r o b l e m h a v e b e e n d e s c r i b e d .

C o n s i d e r p r o b l e m (1.59). W e i n t r o d u c e m n e w variables similar t o m u l - tipliers b u t usually called dual variables, o n e for e a c h c o n s t r a i n t ; t h e y a r e d e n o t e d b y /iu..., \km. W e f o r m a n e w f u n c t i o n similar t o a L a g r a n g e a n :

m

, \Lm9X\9 ...9Xn ) = / ( * ! , . . . , • * „ ) + 2 HjgJ(xl9...9xn)9 (1.62)

7 = 1 or more compactly,

* ( | i , x ) = / ( x ) + M/-g(x). (1.62')

Remark. We must at the outset emphasize one aspect of this format: it is essential that the problem be written exactly like this. In particular, the constraints require g(x) =t0 and we add the products fijgJ(x) to the ob- jective function to form the </> function. The first-order conditions are stated here for this format and would be affected by any change in it. Care must therefore be taken to state the problem and to define the </> function exactly in this way. Because of this need for precision we will confine our attention to maximum problems; minimum problems can be handled by reversing the sign of the objective function.

Theorem 1.4.1: Kuhn-Tucker. Let x* be a solution to problem (1.59) and assume that a regularity condition applies (see Lemma 1.4.1). Then there must exist a set of values iil9...9fim such that

g ' ( x * ) > 0 , Hj^O, and fijgJ(x*) = 09 y = l , . . . , m , m

y;(x*)+s^^/(x*)<o, *r>o, (i.63) 7 = 1

r m ~\ */*• / i O O + 2 lljgf(x')\=0, 1 = 1,...,/!.

L 7 = 1 J

T h e s e a r e k n o w n as t h e K u h n - T u c k e r c o n d i t i o n s . T h e y c a n b e w r i t t e n m o r e c o m p a c t l y by using t h e </> f u n c t i o n of (1.62):

d<t> d<t> — > 0 , J K / > 0 , and py — = 0, y = l , . . . , m , dnj dnj

d</> d<f> - ^ < 0 , x f > 0 , and x*-^- = 0, / = l f . . . , / i , OX; dxt

or in v e c t o r n o t a t i o n ,

* „ i = 0 , fi^09 a n d AI'-</V = 0 ,

</>x=^0, x * ^ 0 , a n d x * / - 0 x = O.

(1.63')

(1.63")

56 1 Static optimization

The reason that only the inner product of p. and ^ is required to be zero (as opposed to each term as in (1.63')) is that the sign restrictions on \K and 0^ make each /*/•</>,,. term nonnegative. Hence, requiring each term to be zero is equivalent to requiring their sum to be zero. A similar argument applies to x*'-0x = O.

The inequalities and equalities that make up the Kuhn-Tucker condi- tions should be used as in the introductory example at the beginning of this section. Specifically, if /*y is positive, then g

y(x*) = 0; by contrast, if gJ(x*) is positive, then fij = 0. Similarly, if x*>0, then </>*,. = 0, and if <t>x.<0, then JC* = 0 . There are therefore many possible outcomes, and these conditions provide no a priori indication as to which variables will be positive and which will be zero. This difficulty cannot be completely resolved, and the derivation of the solution may require some economic intuition into the problem (or plain guesswork). We now illustrate the use of these conditions with some examples, assuming for now that the Kuhn-Tucker conditions yield a constrained maximum.

Example 1.4.1. F i n d x x a n d x 2 t h a t m a x i m i z e f ( x u x 2 ) = l n ^ ) + l n ( x 2 + 5) subject to

4 - * 1 - x 2 > 0 , X j > 0 , x 2 > 0 .

Form the <j> function </>(/*, xu x2) = ln(xj) + ln(x2 + 5) + ^[4—x{—x2], and the Kuhn-Tucker conditions are

</)Al = 4 - x 1 - x 2 > 0 , / * > 0 , n[4-xx-x2] = 0,

<t>Xx = Xi X-n<0, * ! > ( ) , x 1 [ x f

1 - ^ ] = 0,

^ 2 = ( ^ 2 + 5 ) - 1 - A t < 0 , x 2 > 0 , x2[(x2+5)-

l-fi] = 0.

We begin with a trial solution, with all variables positive; this results in three equations:

/*>0 implies 4-xx-x2 = 0,

Xi>0 implies x{~1 — fi = 0,

x2>0 implies (x2+5)~ l — fi = 0.

To solve, eliminate n to get Xi = x2+5, and use the constraint to obtain 4—x2—5—Jt2 = 0, or x2— —0.5. This is unacceptable, but it does give us a hint to set x2 = 0 at the outset. Leaving all other variables positive, we still have three equations, counting x2 = 0 as one of them. We obtain *i = 4, x2 = 0, and fi=\. We must check that ^ ^ O : </>A.2=5~

1- \ = —0.05 indeed. The solution is illustrated in Figure 1.9a, where the fea- sible area is hatched.

1.4 Optimization under inequality constraints 57

(b)

Figure 1.9

Example 1.4.2

Maximize / ( x 1 , x 2 ) = 6 x 1 - 2 ( x 1 ) 2 + 2 x 1 x 2 - - 2 ( j t 2 )

2

subject to x1 + 2 x 2 < 2 , 1 + J C 1 - ( X 2 ) 2 > 0 , * ! > ( ) , x 2 > 0 .

Form the </> function:

<t>(tiuti2^uX2) = 6xl-2(xl) 2^2xlX2-2(x2)

2

+ / * 1 [ 2 - x 1 - 2 x 2 ] + M 2 [ l + x 1 - ( * 2 ) 2 ] ,

0/il = 2 - x 1 - 2 x 2 > O , M l > 0 , ^ 1 [ 2 - x 1 - 2 x 2 ] = 0,

< / V 2 = l + * i - ( * 2 ) 2 ^ 0 , / * 2 > 0 , ^ 2 [ l + X ! - ( x 2 )

2 ] = 0,

0x1 = 6 - 4 x 1 + 2x2-jK1 + jii2<O, X ! > 0 , X i ^ ^ O ,

</>X2 = 2 x 1 - 4 x 2 - 2 / > t 1 - 2 ^ 2 x 2 < 0 , x 2 > 0 , x2<t>X2 = 0.

58 1 Static optimization

If we suppose that all variables are positive, we obtain four equations: <t>n = 0, <t>n = Q, </>*! = (), and </>*2 = 0. Solving the first two yields Xi = 0, x2=l (disregarding negative solutions); this is acceptable because it is possible that both xx = 0 and <j>Xx = 0. However, substituting these values in the last two equations yields /*i = 3, but \K2 — — 5, which is unaccept- able. Taking the hint, we set /*2 = 0, while all other variables are positive. This gives four equations: </>̂ = 0, ^2 = 0, 0 ^ = 0, and 0*2 = 0. Eliminat- ing /*! from the last two and using n2 = 0 yields 6 — 5xx + 4JC2 = 0, which with <^ = 0 is solved by xx —10/7, x2 = 2 / 7 ; substitution then yields ^ = 6/7. We must check that ^ 2 > 0 : <^2 = 1 0 / 7 - ( 2 / 7 )

2 + l = 2 . 3 5 > 0 . The solution is illustrated in Figure 1.9b, where M indicates the absolute max- imum of the objective function.

1.4.1 Regularity conditions (constraint qualifications)

The necessity of (1.63) for a maximum to (1.59) was originally proved by Kuhn and Tucker under an assumption termed the constraint qualifica- tion, which was designed to avoid cusps in the feasible set. Later others (notably K. Arrow, L. Hurwicz, and H. Uzawa) refined these conditions. Here we list some of these results and illustrate others without claiming completeness. For a detailed survey the reader is referred to Takayama (1985) or Mangasarian (1969).

Lemma 1.4.1. The regularity condition of Theorem 1.4.1 is satisfied if any one of the following conditions is satisfied:

(i) gJ(x) is linear, y = l , . . . , m . (ii) gJ(x) is concave and there exists x > 0 such that gj(x)>0, j —

l , . . . , m . (iii) The feasible set (g(x) ^ 0, x ^ 0) is convex and has a nonempty in-

terior, and gi(x*) ^ 0 if j is a binding constraint (i.e., gJ(x*) = 0). (iv) Renumber the constraints so that the first m' (<m) are binding.

The rank of the matrix [dgJ(x*)/dx], j = 1,..., m\ is equal to m'.

We recognize in (iv) the rank condition of Theorem 1.2.1, which, of course, here applies only to the binding constraints. Condition (ii) is known as Slater's condition. These conditions are not equivalent to one another. For instance if x* > 0, then (iv) guarantees the uniqueness of fi in Theo- rem 1.4.1, but other conditions do not, as we now illustrate.

Example 1.4.3: nonuniqueness of dual variables. Maximize ln( jq) + ln(x2) subject to 2Ar1H-x2<3, x 1 H - 2 x 2 < 3 , and xl-\-x2<2, with J C ^ O , A T 2 > 0 .

1.4 Optimization under inequality constraints

. / common tangent **^ - ^ to both constraints

(b)

Figure 1.10

As shown in Figure 1.10a the point (1,1) is optimal. Condition (i) in Lem- ma 1.4.1 applies, and there exist dual variables pi, fi2, and [i3 such that

</>*! = Xxx-2ixx-n2-M3^0, X ! > 0 , Jfi0Xl = O,

<t>x2 = X2 X~fli-2fl2-M3^0, X 2 > 0 , X2<l>X2 = 0,

where

0 = lnx 1 + l n x 2 + / A i [ 3 - 2 j c 1 - x 2 ] + / i 2 [ 3 - ^ i - 2 x 2 ] + /A3[2-x 1 -X2].

60 1 Static optimization

However, the /x's are not unique, since the rank of

- 2 - 1 - 1 - 1 - 2 - 1 gx =

is clearly less than 3. We can only give as a solution ^ = fi2 and fi3 = 1 - 3fi2 with the restrictions fiu /*2, ̂ ^ 0.

We will not state the Kuhn-Tucker constraint qualification because it is neither intuitively appealing nor easy to check; we will, however, give an example to illustrate why we want to eliminate cusps, because it is re- lated to the above rank condition (see Figure 1.10b).

Example 1.4.4: cusp

Maximize xx+x2

subject to

l + (xl) 0A-x2>0,

- 1 . 9 ( x 1 ) 3 + 5 . 7 ( x 1 )

2 - 5 . 8 x 1 + x 2 > 0 , xx7>, x 2 > 0 ,

ct> = xl^x2^ixl(l^(xl) 0A-x2)^^2(-1.9(xl)

3 + 5J(xl) 2-5.Sxl+x2).

The feasible region is defined by the intersection of the region above the curve, x2=1.9xi—5Jxi+5.$xu with the region below the curve, x2 = 1 — (*i) 0 1 ; this feasible region is hatched in Figure 1.10b and exhibits a cusp at point (1,2). Furthermore, this point dominates all other feasible points and is clearly optimal, since the objective function is increasing in all its arguments. We now show that conditions (1.63') are not satisfied at this constrained maximum. Suppose that conditions (1.63') hold; then, since xx > 0 and x2 > 0, we would have <l>x = 0 and <f>X2 = 0. However, these equations are

l + 0 . 1 / i 1 ( x 1 ) - 0 9 + / i 2 ( - 5 . 7 ( x 1 )

2 + 1 1 . 4 ^ - 5 . 8 ) = 0,

1 - ^ + ^2 = 0,

and when substituting xx = 1, x2 — 2 we have

-0.1/i 1 + 0.1/i 2 =l, (1.64)

/ * 1 ~ / * 2 = 1 >

for which no solution exists. It is easy to verify that the rank condition is not satisfied here. This is because of the occurrence of the cusp, in which both constraints are tangent to one another.

1.4 Optimization under inequality constraints 61

Remark. Example 1.4.4 gives us an opportunity to bring out one tech- nical detail which we have passed over. There would be no need for reg- ularity conditions if (1.62) and Theorem 1.4.1 had been stated in the fol- lowing form: Define </>0(/*0, ji,x) = /*0/(x) + j*'-g(x), and the theorem is modified to state that there exist /x0i fiu ...,nm, and so on. Clearly these ^'s can be scaled up arbitrarily. In nearly all cases fi0 ^ 0; hence, it is convenient to set J K 0 = 1 ,

a n d the theorem appears as originally stated. However, in a few pathological cases such as in Example 1.4.4, /*0 would be zero, as we now show, and the scaling is inappropriate. Let us rede- fine </>°as bi0—l)(Xi+x2) + (l>(ii,x). Then in order to get the new condi- tions, we need only attach a /*0 factor to the right-hand side of (1.64) and we obtain

- 0 . 1 / * ! + 0.1/̂ 2 = Mo>

This system now admits the solutions jtt0 = 0, ^1 = ^2- (There are many, because the rank condition is not satisfied.)

The same remark applies to Theorem 1.2.1, in which we could discard the rank condition if we had the Lagrangean <£°(X0, X, x) = X 0 /(x) + X'-g(x). We shall concentrate on regular problems, but the present remark should be kept in mind if a problem is encountered in which it is impossible to obtain a solution for the dual variables or multipliers.

We now turn our attention to a more restrictive class of problems for which we can obtain global results. This is not the only class for which we can obtain such results, but it is the simplest and most useful. For further results, the reader is referred to Mangasarian (1969) or Takayama (1985).

1.4.2 Concave programming

Definition 1.4.1. A regular concave programming problem satisfies the following:

(i) / and g are concave functions. (ii) There exists x ^ 0 such that g(x) ^ 0 and gJ(x) > 0 for all j con-

straints that are not linear. (This incorporates Slater's condition.)

Theorem 1.4.2. Assume that (1.59) is a regular concave programming problem. Then x* is a solution to that problem if and only if there exists fi such that (ji,x*) satisfy the conditions of Theorem 1.4.1 (Kuhn-Tucker).

62 1 Static optimization

g1(x) = 0

Figure 1.11

1.4.3 Geometric interpretation

In Figure 1.11a we have represented a concave programming problem with two variables and three constraints. The feasible set is hatched and the upper contour set of the objective function is lined with dots; the con- strained maximum is at x*. In the case depicted the first two constraints are binding, while the third one is slack; thus, we expect /*i>0, fi2>0,

1.4 Optimization under inequality constraints 63

and ^3 = 0. Some of the Kuhn-Tucker conditions hence require that at the optimum

or in gradient notation,

Vf=-HiVgl-H2Vgl-0Vgl

This last relationship indicates that the gradient of the objective function is equal to a weighted sum of the negative of the gradients of the con- straints where the weights are the dual variables: zero for slack constraints and positive for binding constraints. This is illustrated in Figure 1.11b, which is a blowup of 1.11a. It is interesting to compare Figure 1.11b with Figure 1.10, in which no values of the dual variables could be found in the ordinary way. In that case, the constraints are tangent at the opti- mum, their gradient vectors are colinear, and no weighted sum of them can duplicate the gradient of the objective function, which is on another line.

1.4.4 Derivation of the Kuhn-Tucker conditions

There are several proofs of the necessity of the Kuhn-Tucker conditions; these can be found, for instance, in either of the two references cited ear- lier, as well as in the original article. In order to emphasize similarities among optimization methods we derive them here using a modification of the method of Lagrange. This procedure will also highlight the links between the nonnegativity constraints on the x's and the inequality form of the Kuhn-Tucker conditions, as well as show the sign of the dual vari- ables to be a consequence of the inequality form of the constraints. Con- sider the problem of finding x that maximizes / ( x ) subject to

g y ( x ) > 0 , y = l , . . . , m ,

* / > 0 , 1 = 1,...,/!.

Suppose there is a maximum at x* at which the following constraints are binding:

gj(x*) = 0, j = h...,m'<m,

x* = 0, i = l,...,n'<n.

We can ignore the nonbinding constraints when characterizing the op- timum (i.e., deriving the necessary conditions) and form a Lagrangean

m' ri £ = / ( X ) + S M V ( X ) + 2 X , J C , .

7 = I ; = i

64 1 Static optimization

In the remainder of the derivation we shall omit asterisks a n d / o r argu- ments of functions when convenient in order to simplify the notation. The first-order conditions are

£ x , = x, = 0,

£My = g'(x) = 0, m'

7 = 1

1 = 1 . . .

j = h-

h=\,.

.,«',

..,m',

..,n.

(1.65a)

(1.65b)

(1.65c)

We now proceed to show that the multiplier associated with any binding restraint (either (1.65a) or (1.65b)) is positive or zero. Let us rewrite any restraint as

Rs(x)-as>0, s = l , . . . , ( r t ' + m ' ) ,

where a 5 is initially zero but we aim to increase it slightly. We associate with such a restraint a multiplier irs. (Naturally irs can be a /x or a X.) Consider a small increase in a 5 from 0 to e > 0. This will shrink the fea- sible set, since points x that satisfy e > Rs(x) > 0 are now excluded from it. This shrinking will result in a decrease (or no change) in the optimal value of the objective function, hence d / ( x * ) / d a 5 < 0 , all s. The Lagrangean in our modified notation is

n'+m'

£ = / < * ) + 2 *s-[Rs(x)-as]9 5 = 1

and by the envelope theorem

df(x*) d £ das dir*

= - T T 5 < 0 .

We have established that the multipliers associated with binding restraints are positive or zero. This is a consequence of the inequality form of the restraints, since this is what made it possible for some points previously included in the feasible set to become unfeasible when as increased.

We now revert to the original notation; we have established that

gJ(x) = 0, iij&O, j = l,...,m', (1.66)

Xj = 0, X / > 0 , / = 1 , . . . , / * ' .

We have ignored the slack restraints, but if we attach a zero-valued mul- tiplier to them, their inclusion in the Lagrangean will not alter the first- order conditions. Let

m n

* = / 0 0 + 2 M V ( X ) + 2 V X „ J=I (=i

1.4 Optimization under inequality constraints 65

where to each

gJ(x)>0 assign/>ty = 0, j = m'+l,...,m (1.67a)

and to each

xt>0 assignX/ = 0, J = / T + 1 , . . . , / I . (1.67b)

Taking into account (1.66) and (1.67), the first-order condition (1.65b) becomes

^ > 0 , ^ ( x ) > 0 , iijgJ(x) = 09 y = l , . . . , m . (1.68)

The first-order condition (1.65c) is still A ^ = 0 o r **A = 0 t>ut n o w *n"

eludes the slack restraints as well, with a zero multiplier: m

/ ^ + 2 ^ + x , = o , A = I /i.

However, if xh > 0, then \h = 0 and the condition is m

and if */, = 0, then \h > 0 from (1.66), and the condition is m

Summing up, the first-order condition (1.65c) can be expressed as

= 0 , / * = ! , . . .,A2. m

j = 1 (1.69)

Note that (1.69) also incorporates (1.65a) and x ̂ 0 . Equations (1.68) and (1.69) are recognized as the Kuhn-Tucker conditions of Theorem 1.4.1. The inequality form of (1.69) has been shown to be a consequence of the nonnegativity restriction on x, plus the asymmetric treatment of re- straints like xt > 0 and g

J(x) > 0, the former not being included in the </> function. Hence, if a variable were to be unrestricted in sign, we would use the familiar equality form of the first-order condition for this vari- able. Similarly, if a constraint is an equality, one cannot ascertain the sign of the multiplier associated with it. This observation can be for- malized.

Theorem 1.4.3: mixed problems. Consider the problem of maximizing / ( x , y) with respect to x and y subject to

66 1 Static optimization

g ( x , y ) 5 0 ,

h(x,y) = 0,

x ^ O .

Let

0(X, fi, x, y) = / ( x , y) + /*'• g(x, y) + A'- h(x, y),

where X and fi are vectors of appropriate orders. Assuming that the rank condition (on h and binding g constraints) is satisfied, the necessary con- ditions are

</>x = / x + ^ - g x + A ' - h x ^ 0 , x ^ O , x ' . 0 x = O,

y y (1.70) 4v = g ( x , y ) ^ 0 , ^ 0 , M / - ^ = 0 ,

</>x = h(x,y) = 0.

Unless h is made of linear functions, it is not possible to retain the concave programming format, since /zy(x,y) = 0 can be duplicated by hJ\x9 y) > 0 and —h

J(x, y) > 0, but both functions cannot be concave (un- less they are linear). When the use of equality constraints is necessary, care should be taken that the solution is indeed optimal; for instance, Theorem 1.2.6 may be used if applicable.

Example 1.4.5. Find (x, y) to maximize — (x)2—(y)2 + 20*—Ay subject to

-x-(y)2+9>09

( * ) 2 + O 0 2 - 2 6 = 0 and x > 0 ,

(t>(\,ti,x,y) = -x 2-y2+20x-4y + ti[-x-y

2+9] + \[x2+y2-26]y

c/)x=-2x+20-ti+2\x<0, x > 0 , x<j)x = 0,

<j>y=-2y-4-2iiy + 2\y = 0,

<t>,= -x-y2+9>0, / * > 0 , M</V = 0,

<l>x = x 2+y2-26 = 0.

We shall "guess" that the solution involves x > 0 and /* = 0 in order to save space. We then have the three equations

- 2 X + 2 0 + 2XJC = 0 ,

- 2 ^ - 4 + 2X^ = 0,

x2+y2-26 = 0.

1.5 Economic applications of nonlinear programming 67

The first two yield x = — 5y and, with the third, the two solutions (x, y) = ( 5 , - 1 ) and (—5,1). The latter is unacceptable, since x < 0. Using ( 5 , - 1 ) we obtain X= — 1. It remains to be checked that the solution satisfies the condition </>M > 0: </>M = — 5 —1 + 9 > 0 indeed. The rank condition requires that the vector of derivatives of the second constraint have rank 1, that is, not be nil, which is easily checked. The first constraint is slack and thus does not figure in this. As we previously noted we cannot cast this prob- lem into a concave programming format. However, we can use Theorem 1.2.6, which requires </>(A*, n*9X,y) to be a concave function of x and y. Weget(l)(-l,0,x,y) = -x2-y2+20x-4y-[x2+y2-26],v/hichiseas- ily seen to be concave and the solution (—1, 0, 5, —1) is optimal.

Remark. In some problems it may be convenient to use the equality form of first-order conditions relating to the choice of x. This is done by treat- ing the nonnegativity restrictions on x formally as constraints, each with a nonnegative multiplier \h. We then have conditions like (1.65c).

1.5 Economic applications of nonlinear programming

One advantage of nonlinear programming over the method of Lagrange is that it formally takes into account some restrictions which had pre- viously been ignored, although they were present in most cases: variables can now be restricted to nonnegative values, and resource constraints may indicate that there is no need to exhaust all of the resources avail- able. These features increase the richness of the economic interpretation of many problems, as we now illustrate with a simple case.

1.5.1 The pricing of resources and uneconomic activities

Let there be n possible activities. The level of the /th activity is denoted by xt and is constrained to be nonnegative, i = l,...,n. These activities use up resources; typically the vector of activities x uses up hj(\) of the yth resource, which is available in amount bj. Resources are freely dis- posable, so that no cost is incurred if some of each is unused and dis- carded. The yth resource constraint is therefore hj(x)<bj. Finally, the objective is to maximize net benefit B(xu..., xn). This can be formulated as nonlinear program, and we will assume B is concave, hj is convex, all y, as well as some regularity condition. Find xu...9xn that maximizes B(xu...9xn) subject to

hj{xu...ixn)<bj, y = l,...,ra,

x , > 0 , I = 1,...,/I,

68 1 Static optimization

m

<t>(p,x) = B(x) + S iij[bj-hJ(x)]-

The optimality conditions are m

<t>Xi = Bi- 2 iijhf<0; * , > ( ) ; ^ ^ . = 0, / = l , . . . , / i , (1.71a) y = i

</>,,.= Z ? y - / ^ ( x ) > 0 ; M y > 0 ; ^ ^ . = 0, y = l , . . . , m , (1.71b) where

£ , = 9B/3JC/ and A/ = 3 h j/dxt. First consider conditions (1.71b). The dual variable fij, like Lagrange mul- tipliers previously, is the shadow price of resource j , but the interpreta- tion is now richer. First of all, this shadow price is nonnegative, as we expect prices to be; if it happens to be positive, the resource will be exactly used up, since \ij >0 implies bj = h

J(x); if, however, the resource is not all used up, then hJ(x) < bj implies fij = 0. A resource still available in a positive amount at the optimum has a zero price; in other words, if a resource is not scarce, it is free. This sensible pricing mechanism is the consequence of formally taking into account the assumption of free dis- posal of resources.

We now turn to conditions (1.71a). Since hJ(x) is the amount of re- source j used by the vector of activities x, we interpret hj as the mar- ginal cost of activity / in terms of resource j ; more loosely, it is the extra amount of resource j used by the marginal unit of activity /. When this is multiplied by the shadow price of the resource, fij9 we have that cost in dollars (or whatever unit B is measured in). When these costs are summed for all resources, we have the overall marginal cost of activity /. The first part of (1.71a) then states that marginal benefit cannot exceed marginal cost for any activity. If marginal cost is greater than marginal benefit, the level of that activity is zero. Any activity carried out at a positive level has equal marginal benefit and marginal cost. The rationale behind this situation is that an excess of marginal cost over marginal benefit pro- vides an incentive to reduce that level of activity until a balance has been reached; however, for some activities this balance is not possible and even when the level of activity has been reduced to zero, marginal cost still exceeds marginal benefit: this is an uneconomic activity, given exist- ing resources and alternatives.

In order to gain yet more insight into the determination of shadow prices we now turn to a special case of the preceding problem in which there is a single linear constraint. We maximize B(xu...,xn) subject to

n

2 xt^b, x^O,

1.5 Economic applications of nonlinear programming 69

<t> = B(x) + ii

n

/ = i

</>Xi = Bj-ii<09 X/>0, Xi<t>x=0.

The second condition is the one that interests us most. It states that the shadow price is larger than or equal to the marginal benefit of each activ- ity; furthermore, it is equal to the marginal benefit of every activity that is carried out at a positive level. Those activities that are not carried out usually have a marginal benefit that is less than the shadow price. Thus, if we think of the optimization as an iterative process, we can imagine scanning all possible activities and allocating some of the resource to those activities with a high marginal benefit just as an auctioneer would allocate shares of the flow of some resource (e.g., water, oil) to the highest bidders. When the flow has been completely allocated, its price is the highest mar- ginal benefit obtainable from all bidders, the differential allocation of the resource itself smoothing out differences in the marginal benefits of the activities. This is what the shadow price is, the highest possible marginal return compatible with using up the resource. Any activity that cannot match this at any level of allocation simply misses out and is declared uneconomic.

We turn to a classic application of nonlinear programming. It has been selected because the inequality constraints are essential to the formula- tion of the problem.

1.5.2 Peakload policy

In many instances the supply of a product is limited by a capacity con- straint. When the product is required in different amounts for different periods, the problem of choosing the right capacity as well as the schedule of supplies arises. Typical examples are the pricing of public utilities such as electricity, water, and telephone services. One can also think of the problem of choosing the appropriate size of a tent for a traveling circus or choosing the size of a power station for a planning horizon over which demand conditions will vary.

There are T periods; xt is the supply in period t, X is the capacity, R(xu...,xT) is the revenue from sales in all periods (note that this form allows the demand in one period to be affected by demand in other pe- riods), C{xu...,xT) is the variable operating cost, and K(X) is the capi- tal cost of a plant of capacity X. We assume concavity of R and convexity of C and K. The problem is to find xu . . . , x r that maximize

b- s Xt i = \

70 1 Static optimization

R(xu...,xT)-C(xu...,xT)-K(X) subject to

X-xt>0, xt>0, ^ = 1,...,T. From

<Kp,x,X) = R(x)-C(x)-K(X)+ S VLtlX-x^ t = \

the necessary conditions are

<^ = M R , - M C , - ^ < 0 , *,>(), xt4>Xt = 0, t = l,...,T,

* = i

<^=x-x,>o, ^>o, <̂/>̂ =o, / = i,...,r,

where MRt = dR/dxt and MC, = 3C/dA:, are the marginal revenue and marginal operating cost of xt, respectively. If supply is less than capacity in some period t, then \kt = 0, and if the product is supplied at all in that period, marginal cost equals marginal revenue. It is possible that for some periods marginal revenue is always below marginal cost, in which case the supply is zero in these periods. There will, however, be periods in which the capacity constraint binds unless nothing at all is produced. In the case where the capacity constraint never binds, all \it are zero, imply- ing —K'< 0, which, assuming that K is an increasing function, must hold strictly, and this in turn implies X=0 (taking into account that (K')X must then be zero). There must therefore be some positive /i/s if any production occurs (X > 0). In those periods xt = X > 0 and the marginal revenue equals the marginal operating cost plus jxti which we interpret as a capacity surcharge. Furthermore, all the capacity surcharges add up to the marginal cost of capital needed to produce an extra unit of capacity K'. Note that we have assumed that the firm or public utility acts as a monopolist and maximizes profit, so that there is no implication of fair- ness in not using a capacity surcharge in offpeak periods; this is simply another instance of price discrimination. The monopolist makes a higher profit this way than if it charged a uniform price for all periods. Finally, note that we formulated the problem as if the monopolist chose the quan- tities, which is rather unreasonable. The actual problem is of peakload pricing, but if we let prices be the choice variables, we obtain the same overall results if we put reasonable restrictions on demand functions.

1.6 The special case of linear programming

The development of linear programming predates that of nonlinear pro- gramming, but the results are most easily obtained from concave program-

1.6 The special case of linear programming 71

ming theory. The great advantage of linear programming is that there exist very efficient solution algorithms suitable for very large problems; they are known as the simplex method or modifications of it. It is not our purpose here to present these algorithms but to make some analytical points. (For details of the simplex method see the classic Dorfman, Sam- uelson, and Solow, 1958, or Hadley, 1962.)

We define a linear programming problem that we call the primal as finding x to maximize p'-x subject to

A x ^ c , x ^ O ; A is mxn. (1.72)

The constraints and the objective function are linear. Therefore, using the concave programming results, we know that the Kuhn-Tucker condi- tions are necessary and sufficient for an optimum. We derive them in the usual way:

0(x,ji) = p'x + | * M c - A x ] , (1.73)

0M = c - A x ^ O , fi^O, / * ' - [ c - A x ] = 0, (1.74)

</>x = p - A > ^ 0 , x ^ O , x ' - [ p - A V ] = 0. (1.75)

Next consider another problem, called the dual and derived from the primal of (1.72) by interchanging the vector of coefficients p with the right-hand-side vector c, transposing matrix A , reversing the inequalities, and naming the variables for this problem fi. We then seek to find /i to minimize c'p subject to

A ' ^ p , fi^O. (1.76)

To obtain the optimality conditions for this problem, we change the sign of the objective function to have a maximum problem and obtain

* ( | i , x ) = - c / | i + x / - [ A V - p ] . (1.77)

The reason we have chosen to denote the multipliers by x will become obvious shortly. Bearing in mind that fi is the vector of maximizing vari- ables here and x the vector of multipliers, we obtain the following neces- sary and sufficient conditions:

* x = A V - p ^ 0 , x ^ O , x ' . [ A > - p ] = 0, (1.78)

¥ M = - c + A x ^ 0 , / i ^ O , n / . [ - c + Ax] = 0. (1.79)

If we remember that quadratic forms such as x'A'fi are scalar-valued, hence unaffected by transposition, it becomes clear that (1.78) and (1.75) are identical, as are (1.79) and (1.74). Solving (1.72) via the Kuhn-Tucker method uncovers a set of dual variables /*, which solves the dual problem (1.76). Conversely solving (1.76) uncovers the solution x to the primal

72 1 Static optimization

(1.72). Furthermore, the maximum value of the objective function of the primal (1.72a) is equal to the minimum value of the objective function of the dual (1.76a). Note that the name of dual variables adopted earlier for the multipliers of nonlinear programming problems comes from the dual of linear programming. It is also possible to define dual nonlinear programming problems, but this is not useful for our purposes here; for details one can consult Mangasarian (1969).

We now state and prove some important results.

Theorem 1.6.1. Let x * ^ 0 and ^ * ^ 0 be feasible vectors for the primal and the dual, respectively, and suppose that they yield the same value for their respective objective functions. Then x* and /i* are optimal solutions to the primal and the dual, respectively.

Proof. By assumption we have

A x * ^ c and A y ^ p .

Multiply the first by fi*' ̂ 0 and the second by x*' ̂ 0 to get

j * * ' c > y A x * and x * ' A y > x * ' p , or c>* > y ' A x * > p'x*.

With the assumption that c'fi* = p'x*, we have

c'gi* = p*Ax* and j**'Ax* = p'x*,

and (1.74), (1.75), (1.78), and (1.79) are satisfied, which proves the result. •

Lemma 1.6.1. x* and fi* are the solutions of the primal (1.72) and of the dual (1.76), respectively, if and only if they constitute a saddle point of the function <j> of (1.73); more precisely,

0(x, fi*) < 0(x*, A**) < <Mx*, ix) for all x ̂ 0, ji ̂ 0.

Proof. In Lemma 1.6.1, <t> is being maximized with respect to x ̂ 0, given fi*, and at the same time is being minimized with respect to /i ̂ 0, given x*. Maximization under nonnegativity restrictions means

0 x ( x V * ) ^ O , x * ^ 0 , x * ' . 0 x ( x * y ) = O,

or

p-Ay^o, x*^o, x*'-[p-Ay] = o, which is (1.75). Minimization under nonnegativity restrictions is much the same, but we require the derivatives to be positive or zero. It is easy to verify that this yields (1.74).

Conversely, if we have solutions (x*, /i*) to the primal and the dual, they satisfy (1.74) and (1.75) (or (1.78) and (1.79)), which implies that </> is

1.6 The special case of linear programming 73

minimized with respect to fi^O and maximized with respect to x^O; hence, (x*, fi*) is a saddle point of <£. •

We now turn to an economic interpretation of these linear programs. Let p be a price vector, x be the amount of goods produced, c be a vector of resources available, and A describe a linear technology. The primal (1.72) maximizes revenue subject to resource constraints. Conditions (1.74) have the usual interpretation for the pricing of scarce resources, where fi is the shadow price vector of the resources. The first part of (1.75) states that the price of a good cannot exceed the resource cost of it; furthermore, if cost exceeds price, that good is not produced and for every good pro- duced price equals cost. Note that this means that profit (per unit) is zero and that the first part of the condition requires profit to be nonpositive on any good. To investigate this last point further, consider the dual defined in (1.76). Clearly, it seeks a resource price vector that will minimize the total value of resources subject to the condition that profits on individual goods are nonpositive. The economic meaning of this problem may be sur- prising at first; however, we know that solving the primal is mathematically equivalent to solving the dual. Therefore, we must refine our interpre- tation of these two problems so that they are consistent with one another.

We wish to explain why the existence of an efficient allocation of re- sources is equivalent to the existence of a resource pricing system that rules out positive profits and minimizes aggregate resource cost. At the optimum the objective functions of the primal and dual are equal; hence, aggregate profit is zero. Furthermore, by (1.75) the profit made on each good is also zero, either because the unit profit is zero or because the good is not produced. This state of affairs resembles the outcome of competi- tive forces at work, and indeed such interpretation can be substantiated. Suppose the resources were held by agents with no market power and that the users of the resources and producers of the goods had no market power either. Then competition among resource owners would lead to price cutting, which would lower the value of resources as much as pos- sible - meaning not lower than the point where resource users would make positive profits; if positive profits were allowed, the resource users would again bid up the resource prices by competing among themselves for the resources. Clearly, the goods for which unit profit is negative would not be produced, and the resources that are in excess supply would not com- mand a positive price.

Thus, competitive pricing of resources in the way just described is equiv- alent to their efficient allocation. This became apparent through the con- nection between the dual and primal solutions.

The intricate dual relationship between allocation and pricing can be further highlighted by the use of the </> function and the saddle point of

74 1 Static optimization

Lemma 1.6.1. We know that x* maximizes </>(x, A**) = p'-x + j**'-c — ji*'-Ax over all x ̂ 0 given ji*. This is equivalent to maximizing aggregate prof- its, which are p'-x — j**'Ax (the maximum is zero). Also, p* minimizes </>(x*, |i) = p/-x*-h/i/-c —|i''Ax* over all j * ^ 0 , given x*. This is equivalent to minimizing the aggregate value of unused resources ji'-c — JA'AX*; the minimum is also zero, of course.

All of these results are valid in the special case of linear programming but are not generally valid in nonlinear programming. We can, however, duplicate any nonlinear programming solution with a linear program. Consider again the problem of choosing x to maximize /(x) subject to g(x) ^ 0, x ̂ 0, where x is n X1 and g is m x 1. We easily obtain the opti- mally conditions with </>(j*,x) = /(x) +j*'-g(x):

0 x (^,x*)=/ x (x*) + g'x(x*).M*^O, x * ^ 0 , x*'.0x = O, (1.80)

</>>*, **) = g(x*)^0, A<* = 0> /**'•</>„ = 0. (1.81)

Consider now the linear program to find x that maximizes

A'(x*)-x (1.82)

subject to

gx,(x*).(x-x*)^0, x^O, (1.83)

where the n x 1 vector /x(x*) and the m x n matrix gx(x*) are taken from (1.80) and (1.81). The optimality conditions are obtained from

^( M ,x)^/ x ,(x*).x + ^.gx,(x*)-(x-x*) as

*x = A(x*) + g x ( x * ) - ^ 0 , x ^ O , x'.*x = 0,

*,* = gx'(x*)-(x-x*)^0, ^ 0 , j i ' - ^ O .

It is easy to verify that p* and x* of (1.80) and (1.81) satisfy these condi- tions. The geometric interpretation of (1.82) is that level curves o f / a r e replaced by its tangent at x* and (1.83) substitutes the tangents to the con- straints at x* for the constraints themselves, as illustrated in Figure 1.12. This result reinforces the point that information such as rigid prices or fixed coefficient technology, although globally incorrect, may be suffi- cient to support an equilibrium.

Appendix

Functions and sets

A set is a collection of objects. The sets encountered here are often sub- sets of the ^-dimensional space of real numbers, denoted by Rn. If x is an

Appendix 75

Figure 1.12

element of the set X, we write xeX, which can be read as "x belongs to X." The symbol Vx means "for all elements x." A set X in Rn is bounded from above if there exists a vector b such that x < b, Vx e X. It is bounded from below if there exists a vector b such that b < x, Vx e X. If a set is bounded from above and from below, it is said to be bounded. A set is closed if and only if the limits of all converging sequences contained in the set are in the set. The complement of a closed set is an open set. Ex- amples of closed sets are closed intervals such as — 1 < x < 1, and examples of open sets are open intervals such as — 1 < x< 1. A set X is convex if and only if xxeX and x2eX imply xteX, where xt = txx + (1 — t)x2 and 0 < t < 1. We call xt a convex combination of xx and x2\ it is a weighted average of xx and x2 with nonnegative weights that sum to 1.

A real-valued function f defined on an ^-dimensional subset of Rn, say D, is a rule that associates with each vector in D a single real value. We may write f:D^>R or f(\) or f(xu...,xn). We may have multidimen- sional functions, say f:Rn^>Rm, but we still require that with each vector of Rn there be associated a single vector of Rm. A function / defined on a set D is said to be continuous at x° if, for each sequence x*,x2,... in D converging to x°, we have lim,^*, f(xn) =f(\imn^00 x

n) = / ( x ° ) . A func- tion is continuous on D if it is continuous at all points of D. A continuous function / defined on a set D that is closed and bounded has both a maxi- mum and a minimum in that set; that is, there exists a point xeD such that f(x) > f(x), VxeD, and there exists a point xeD such that /(x) > f(x), VxeZ>. The partial derivative of f(xl9 ...,xn) with respect to xh evaluated at x, is defined to be

76 1 Static optimization

v f(xl> •••>#/+€, "*iXn)—f(X\y . . . , Xn) lim .

If the limit exists, the function is differentiate with respect to xt at x. We say that f(xu ...,x„) is differentiable on D if it is differentiate with re- spect to xu...,*„ at all points of Z>. Partial derivatives are denoted by df/dXf (read "del / del x") or fx. The vector of partial derivatives is df/dx or fx. These are first-order partial derivatives, and clearly second-order partial derivatives can be similarly defined. The second-order partial de- rivative of / with respect to xt and Xj is denoted by d

2f/dXjdXj or fx x.. Whenever these derivatives are defined and continuous, it is always true that d2f/dx; dxj = d2f/dXj dxt: that is, the order of differentiation does not matter. Second-order partial derivatives are most conveniently arranged in a square matrix called the Hessian matrix, often denoted by H. Because of the above property, Hessian matrices are always symmetric.

Matrix notation

We use the following conventions: lowercase letters denote column vec- tors; row vectors are identified by a prime as transposes of column vec- tors. For instance, x is a column vector (nxl) and x' is a row vector (lxw). When differentiating a scalar-valued function with respect to a vector of variables, we assume that the derivatives are arranged exactly as the variables were. For instance, if / ( j q , . . . , xn) is a scalar-valued func- tion of n variables, then fx is the column vector of its first-order partial derivatives, while fx> would be the same derivatives but arranged as a row vector. Similarly, if we take the derivatives of a vector-valued func- tion, the same convention applies to each element of the function, and these vectors of derivatives are themselves arranged as the elements of the original function. To obtain the Hessian matrix of / ( x ) , we differ- entiate the first-order derivatives fx with respect to x', H =/xx<, and the matrix is of the same order as xx' (nxn). Suppose we have a function <£(A, x), where A is m x 1 and x is n x 1; then £Xx' is an ra x /? matrix of derivatives such as d2£/d\jdxi9 while <£xX' is its nxm transpose. The length (or Euclidean norm) of a vector x is |x| = (x'-x)1/2.

Matrices are sometimes used to define functional forms; an important instance is a quadratic form, /(x) = x'Ax = 2,-2/ OjjXjXj, where A is assumed to be symmetric. We have fx = 2Ax and fxx> = 2A. Another in- stance is the inner product f(x) = a'x = 2 / #/*,; this yields fx = a. Some matrices give rise to quadratic forms that have an invariant sign. A ma- trix is positive-definite (resp. negative-definite) if and only if x'Ax > 0, Vx ^ 0 (resp. x'Ax < 0, Vx ^ 0). Similarly, a matrix ispositive-semidefinite

Appendix 77

(resp. negative-semidefinite) if and only if x'Ax>0, Vx (resp. x'Ax<0, Vx). These matrices can be characterized by the sign of their character- istic roots - all positive for a positive-definite matrix, all negative for a negative-definite matrix, positive or zero for positive-semidefiniteness, and negative or zero for negative -semidefiniteness. There is a more prac- tical way, but we must first define some subdeterminants of square ma- trices. Consider an n x n matrix B. The determinant of the matrix formed by deleting the last n — r rows and the last n — r columns of B is called the rth leading principal minor; it is the determinant of an r x r matrix and is denoted by Br. There are n such minors inB: BuB2,...9Bn and JSW = |B|, while Bl = bn. The principal minors of B are obtained as the leading prin- cipal minors of any matrix obtained from B by a permutation of rows and columns. Alternatively, a principal minor of order r of B can be obtained by deleting any n — r pairs of rows and columns from B and taking the determinant of the remainder. A matrix is positive-definite if and only if its leading principal minors are all positive; it is negative- definite if and only if its leading principal minors alternate in (strict) signs beginning with negative (or (-l)rBr>0 r=\,...,n). Positive-semidefi- niteness requires all principal minors to be nonnegative, and negative- semidefiniteness requires all principal minors of order r to be such that ( - l K £ r > 0 , r = l /i.

Taylor's expansion and total differentials

Matrix notation is useful for expressing Taylor's expansion for functions of several variables up to the second degree. Let / have continuous fir st- and second-order derivatives. Then

/(x)=/(x*)+/ x ,(x*)-(x-x*)+i(x-x*r-/xx'(x*)-(x-x*) + /?

with R -» 0 as x -• x*; or in exact form,

/ ( x ) = / ( x * ) + ^ ( x * ) . ( x - x * ) + i ( x - x * r - / X X ' ( x , ) - ( x - x * ) , where

xt = tx + (l — t)x* for some f e [ 0 , 1 ] .

The total differential of a function df=fx>>dx = 2?=i fx.-dxh which ex- presses the change in / induced by small changes in x, can be viewed as a first-order Taylor's expansion with x close to x* and dx = (x —x*).

A useful device for representing functions of two variables is the con- cept of level curves (or contour curves). Let c be a value taken by the function f(xux2). Then f(xl9x2) = c implicitly defines a curve in the (xux2) plane along which / keeps the value c - it is called a level curve of

78 1 Static optimization

/ . To each feasible value c corresponds one level curve. The level curves corresponding to two distinct values of a function cannot intersect one another. An expression for the slope of a level curve can be derived by noting that as x{ and x2 move along such a curve, the value of / does not change; hence, df=fldxl+f2dx2 = 0 and dx2/dxl = -fl/f2. The total differential is also useful for obtaining an expression for total derivatives. Let the xt variables in f(xu...,xn) all depend on another variable, say t. Then we call df/dt the total derivative o f / w i t h respect to t. We can cal- culate it by "dividing" the total differential by dt:

*L-f **- V f (^L\ dt Jx" dt~ itiJxi\dt ) '

Homogeneous functions

A function /(x) is said to be homogeneous of degree h in x if and only if f(tx) = (t)hf(x)9 Vt > 0, Vx. Euler's theorem for homogeneous functions states that if /(x) is homogeneous of degree h in x, then

ixrfXi(x) = hf(x)9 Vx. / = I

It is also true that if /(x) is homogeneous of degree h in x, then fx.(x) is homogeneous of degree (h — 1) in x, j = 1,..., n. One of the implications of these results is that a function that is homogeneous of degree 1 must have a singular Hessian matrix - hence a zero determinant. The slope of the level curves of homogeneous functions of two variables are given by

dx2 = fi(xux2) = (xl) h~lfl(l9x2/xl) = / i ( l , * 2 / * i )

dxx f2(xux2) (xl) h~lf2(l9x2/xl) / 2 ( l , * 2 / * i ) '

Therefore, the slope of such level curves depends only on the ratio x2/xx and not on the value of the variables; hence, it is constant along a ray drawn from the origin across all level curves.

The implicit function theorem

Let F(y) be an m-dimensional function defined on an open set and y be n x 1 with n>m. Assume F to be continuously differentiable and the rank of the m x n matrix iy(y°) to be m at some point y°, where F(y°) = 0. Then for an arbitrarily chosen (y\9..*,ym) within a rectangular region around y° (yf—d ^ yt ^ y?+ 5, 6 > 0), there exists a unique set of values (ym+u . . . , yn) that depends on (yl9 . . . , ym) so that F(y) = 0. In other words, we can express the last n — m variables as a function of the first m variables in the neighborhood of y°.

Exercises 79

Exercises

1. Indicate which of the following functions are concave or convex in (xux2). Which are strictly concave or convex? Find the maximum or minimum if it exists.

f(xux2) = (xl) 2 + (x2)

2 + xlx2 + 10xl + 10x2, /(* 1 ,* 2 ) = 2 ( * 1 )

2 - ( * 2 ) 2 - 4 * 1 + 8*2,

f(xux2) = 2(xl) 2-Sxlx2 + S{x2)

2,

/(* 1 ,* 2 ) = 6(*1) 1/3(*2)

1/3-*1-*2, *!>(), * 2 > 0 , f{xux2) = \n(xl-l) + \n(x2-2)-2xl-3x2, f(xl,x2) = \n(xl + 3)-2x2, f(xux2)=xl(l-e-

x2/xi)9 Xl>0, * 2 > 0 .

2. There are two industries, hatching and laying, and two goods, eggs and chick- ens. The price of an egg is 1; the price of a chicken is p. The hatching industry produces yx chickens using xx eggs, which it buys from the laying industry. Its production function is yl = 4(*1)

1/2. The laying industry produces x2 eggs using y2 chickens, which it buys from the hatching industry. Its production function is *2 = (y2)

l/2. (We assume away any consideration of time lags, nonnegativity constraints, the indivisibility of live chickens or eggs, and claims to precedence by all parties.)

Express in terms of p the profit-maximizing quantities of chickens and eggs produced or used in each industry. Assuming that all eggs produced in the laying industry are used as inputs by the hatching industry, what is the value of pi What, then, is the net output of chickens available for consumption?

3. Use the method of Lagrange to find all constrained extrema of the following functions: (i) *1

2+*2 2 + 3*2 + 3*2*3 subject to *! + 2*2 + 3*3 = 1,

(ii) Inxx + Inx2 subject to (*1 + 1)(*2 + 1) = 4, (iii) 4*1 + 2 * 2 - 8 subject to (xl-2)

2 + (x2-l) 2 = S0,

(iv) (*! - 2)2 -h 3(JC2 — l) 2-h 2*3 - 2(*! - 4)(*2 + 1), subject to 3*! + * 2 + 5 = 0

and 4*!-I-2*3-1 = 0, (v) 2(*1)

1/2(*2) 1/2 subject to 2(*1)

3/2+16(*2) 3/2 = 32.

4. Find the vector x that maximizes £?= j ft ln(*j — 7,) subject to Sf= 1 A*/ = J7- It is possible to interpret the objective function as a utility function and the con- straint as a budget constraint: y is income, p is the price vector, and the posi- tive parameters fi and 7 characterize the individual's tastes. The solution, which is to express x in terms of y, p, 0, and 7, yields the demand functions.

5. A consumer has the utility function U= In C-I-In(24 — TV), where C is consump- tion and N is labor supply. Her budget constraint is pC = M+ wN9 where p is the price of the consumption good, w the wage rate, and M the consumer's non- wage income. (a) Formulate the problem of utility maximization subject to the budget con-

straint, and derive the first-order conditions, using the Lagrange multi- plier approach and ignoring the nonnegativity constraints.

80 1 Static optimization

(b) Find the demand function C = C*(/?, w, M) and the labor supply function N=N*(p, w9M) (i.e., express C and N in terms of/?, w, and M ) . Show that TV* and C* are homogeneous of degree zero in ( p , w , M ) .

(c) LetL^* = l n [ C * ( p , w , M ) ] + l n [ 2 4 - A ^ * ( / 7 , w , M ) ] . S h o w t h a t a ^ y a M > 0 and dU*/dp < 0. Show that U* is concave in M and convex in p. What is the relationship between dU*/dM and the Lagrange multiplier?

6. Two countries, 1 and 2, import a nonstorable resource from a third country in order to produce their domestic output. The production functions of the two countries are qx — 4 ( A : 1 )

1 / 2 and q2 = 2(x 2 ) 1 / 2 , where qt is output and xt input for

country /, / = 1,2. The amount of resource available is X= 500. (a) Calculate the allocation of the resource that would yield the largest total

output for the two countries under the constraint that all the resource is used up. Supposing that one unit of output is by definition worth $1, how much would the importing countries be willing to pay for an extra unit of resource (the shadow price of the resource)? Supposing that this shadow price is the actual price paid and that it costs the third country $0.1 to ex- tract the resource, calculate the profit made by each of the three countries.

(b) Suppose now that the third country unilaterally chooses the resource price; denote it by p. Country 1 maximizes profit, given p\ find its resource im- port as a function of p. Do the same for country 2. Use these results to express the third country's export revenue as a function of p. Recalling that it costs $0.1 to extract the resource, find the value of p that maximizes the third country's profit. How much of the resource is unused? Would the first two countries be able to bribe the third country into returning to full use of the resource?

7. Let the production function be q = x}/2x2 l/2 and the input prices wx and w2.

There is no fixed cost. Derive the total cost function and check its concavity properties. Suppose now that the production function is Q = f(x}/2x\/2), where / is a strictly increasing function. Use the transformation q = f~l(Q) = x\l2x\l2 and your earlier result to obtain the cost function.

8. In this exercise we must determine how many of two sorts of trees must be planted now with scarce labor and what are the optimal harvesting dates for the trees (harvesting is assumed not to require labor). The use of the land after harvesting is not considered here.

x number of small trees, g(x) labor used to plant x small trees, / harvesting date of small trees, X number of large trees, G(X) labor used to plant X large trees, T harvesting date of large trees, / ( / ) revenue at time t from the sale of each small tree at time /, F(T) revenue at time T from the sale of each large tree at time T, L fixed labor supply, available now, r fixed exponential rate of interest (the present value of %a received at

time 6 is ae~re).

Exercises 81

Both g(x) and G(X) are increasing and convex, while both f(t) and F(T) are increasing and concave. (a) Derive the first-order conditions for maximizing the present value of

total revenue from the sale of trees, subject to the labor constraint - x, X, t, and T are the choice variables. What is the economic meaning of these conditions?

(b) Assume that a regular maximum is obtained. Find the signs of dt/dL, dt/dr, dT/dL, and dT/dr. (Hint: Simplify the first-order conditions for t and T by canceling x, X, and the exponential terms before deriving comparative statics results.) Use these results to indicate what informa- tion is relevant to choosing harvesting dates and what information is relevant to choosing crop sizes. Finally, derive a numerical solution in the following case: f(t) = 2t1/2, F(T) = ST1/2, g(x) = 2x, G{X) = X\ L = 20, a n d r = 2%.

9. A firm produces one good in amount q for a price p. Its total cost of produc- tion is C(q, k), where k is the amount of capital available and paid for. C is increasing and convex in all its arguments. Derive the profit-maximizing con- ditions in the short run (A: is fixed) and the long run (A: is chosen). Derive an expression for the firm's changes in output in response to a price change, dq/dp, in the short run and in the long run. Determine their signs and com- pare their absolute values. Interpret your results.

10. A manager is responsible for n machines. In any given period, the probability that exactly k machines will break down is 7i>>0 (7r0 H-7TJ + 7r2 H 7r„ = 1), where k is any whole number between 0 and n: k— 0,1,2,..., n. The ick are "true" probabilities but are known only to the manager. The manager must report these probabilities to the control authority. In the absence of any re- ward or penalty, he might report some wrong probabilities P0,Pi, ..., Pn (PQ + Px+".+Pn = \).

The central authority devises the following scheme: if k machines break down, and the reported probability is Pk, the authority will pay the manager A+M In Pk dollars, where M and A are positive constants. The manager's expected reward is therefore

ER= S Trk(A+M\nPk). (1) A: = 0

The manager wants to choose P0, ...,P„ to maximize (1), subject to the con- straints that

P0 + Pi+-+PH = U ^ 0 . (2)

(a) Find the first-order conditions (ignore nonnegativity restrictions). (b) Are the second-order conditions satisfied? (c) Show that it is optimal for the manager to report the truth, i.e., Pk = irk.

11. Consider an economy with two industries (each producing one good) and one resource. The resource can be used directly as an input in each industry, or it can be used indirectly to develop a technology applicable to both industries.

82 1 Static optimization

The level of technology is denoted by K; the amount of resource used directly as an input in industry / is denoted by xh and the industries' production func- tions and prices are f\xh K) and ph i = 1,2. In order to produce the level of technology K, it is necessary to use H(K) units of the resource. We assume that fl and f2 are strictly increasing and strictly concave in all their arguments and that H(K) is strictly increasing and convex. The total amount of resource available is L, fixed. (a) In the first instance suppose that the level of technology K is fixed.

The amount of resource available for use as input is denoted by X= L-H(K). (i) Formulate the central planner's problem of maximizing total rev-

enue in both industries subject to the resource constraint (use X). Derive the first-order conditions and interpret them. Check that the second-order conditions are satisfied. Find the sign of the rate of change in x2 if X changes exogenously; explain your finding. Find the expression for the rates of change in xx and in x2 when K changes exogenously (X remains constant); interpret your results. Is it pos- sible that the improvement in technology leaves the production of all goods unchanged?

(ii) Let the price of the resource input be denoted by w. Formulate the twin problems of maximizing profit separately in industries 1 and 2, taking all prices as given. Derive the optimality conditions. Can you use the solution in (i) to identify the input price that would generate a total input demand by industries just equal to XI Is this an in- stance of the decentralizing role of prices?

(b) Let

fl(xliK) = l6V2(xl) l/2(K)l/\ p, = l,

f2(x2yK) = Sy[2(x2)V 2(Ky/\ p2 = 2,

L = 160, H(K) = 2K2.

(i) Let K= 1. Use the above data to solve the problem of (a)(i). (Hint: Make use of the symmetry of the example.)

(ii) Let K—\. Use the above data to derive total input demand as a function of w when firms act as in (a)(ii). Find the equilibrium value of w.

(c) In this section the level of technology K is to be optimally chosen. (Use L and H(K) in the constraint, not X.) (i) Formulate the central planner's problem as in (a)(i). Derive and in-

terpret the first-order conditions from the central planner's point of view.

(ii) Use the data of (b) - but not K= 1 - to solve the problem in (c)(i). (iii) Let the price of the resource be denoted by w and the price of tech-

nology (per unit) by r. In an attempt to decentralize the allocation of resources, let each industry / = 1,2 purchase x; of the resource

Exercises 83

for use as input and Kt of the technology for the common use; thus, each industry can use the level (Kx+K2) of technology, and the pro- duction function of industry / is f'(xi9Kx+K2), with KX+K2 = K. In addition there is now a need for a "technology industry" with output K at price r and cost function wH(K). Derive the profit- maximizing conditions for all three industries. Compare this solu- tion with that of (c)(i) and attempt to explain any discrepancy. Does the "invisible hand" work its magic here?

(iv) Solve the problem of (c)(iii) using the data in (b) - but not K=\. (Hint: First eliminate r and w from the optimality conditions.) Com- pare your results with those obtained in (c)(ii).

12. A monopolist wishes to maximize profit, but misjudges demand conditions. We denote by q the quantity of output, p(q) is the demand price, and C(q) is the cost function. The monopolist thinks otp(q) is the demand function, where a is a positive parameter. We shall call ap(q) the expected price. (a) Derive the first-order and second-order conditions for a maximum ex-

pected profit, 7re (ignore nonnegativity restrictions). Suppose the second- order condition holds strictly and find the expression for dq/dot and its sign.

(b) The actual profit 7ra is defined using the actual price p(q); how is it af- fected by otl Are your results sensible?

(c) The difference W= 7ra — 7re is called the unexpected windfall profit. How does its sign depend on a? Find the sign of dW/da; when a is less than 1, is it possible that lowering it further increases windfall profit? Can you rationalize this?

13. Derive the Kuhn-Tucker conditions for the following nonlinear programming problems. Verify that they all are regular concave programming problems and derive the solution. (Hint: First illustrate graphically and attempt to guess which variables are positive and which are zero in the optimal solution.)

(a) Maximize -8(xx) 2-10(x2)

2 +12x x x 2 -50*! + 80*2 xX > 0, x2 > 0

subject to xx+x2<l and 8(x1) 2 + (x2)

2<2.25.

(b) Maximize xx + 2x2-(xx) 2 + 3xxx2-3(x2)

2

x\>0,x2>Q

subjectto 2xx-\-x2<2 and — xx+x2< — 1.

(Hint: The optimal values of the dual variables may not be unique.)

(c) Maximize 6xx — 2(xx) 2 + 2xxx2 — 2(x2)

2

x\ > 0, x2 > 0

subjectto 3xx + 4x2 < 6 and — xx + 4(x2) 2< -\.

(d) Maximize 100 +In *! +In x2 j q > 0 , * 2 > 0

subjectto 9 8 - ( x 1 ) 2 - ( x 2 )

2 > 0 a n d 4 1 8 - ( X 1 ) 2 - 6 ( J C 2 ) 2 > 0 .

1 Static optimization

Two people live together. They have separate incomes. They buy some good (X) for their own consumption (e.g., food or clothing). Some other good (Z) is enjoyed by both in the sense that each person enjoys the purchases of both people (e.g., heating or home improvements). Both people are self- ish and maximize their own utility given their own budget constraint. Person / chooses Xi > 0 and zt > 0 to maximize U'(xh zx + z2) subject to pxt + irZi < yi9 i = l , 2 . Calculate the resulting equilibrium when Ul = In xx + ln(zx + z2)\ U

2 = lnjt2 + 21n(z1 + z 2 ) ; /? = 7r = l; yx = lO; J>2 = 2 0 . D O they both free-ride (i.e., buy no good Z)? Does one? Can you suggest an improvement in their living arrangement? Consider a profit-maximizing competitive firm that produces two goods with fixed amounts of land and capital; labor is also used as an input and is avail- able at the going wage rate:

Amounts of goods sold qx, q2at pricespx >0,p2>0, Labor used in the production of the goods xu x2 with wage rate w > 0, Capital used in the production of the goods ku k2\ total available K > 0, Land used in the production of the goods lu l2; total available L > 0. The production functions Fl(xl,kl,ll)9 F2(x29 k2, l2) are concave.

(a) Set up the problem, derive the optimality conditions, and give an eco- nomic interpretation of all Lagrange multipliers and of the optimality conditions.

(b) Give a numerical solution to the problem when

Pl=p2 = \9 w = 1 0 , # = 5 0 , L = 1 5 0 ,

qx = 110*! + 100A:! + 100/j - x\ - k\ - /, 2,

q2 = 310x2+300 A:2 + 300/2 - 5x2 2 - 5*f - 5/2

2.

What price would the firm pay for an extra unit of capital or land? The object of this exercise is to calculate the total cost of producing a specified amount Q of a final good. To produce the final good, some labor input lx is needed, as is some input x of an intermediate good. The production function is denoted by F(llfx) and is increasing and concave. To produce x units of the intermediate good, an input l2 of labor is needed, as is some input q of the final good. The production function f(l2,q)\s also increasing and concave. Note that the desired amount Q of the final good is the net output of that good; some of the gross output goes into producing x. (a) Write down for each of the two goods a constraint whereby demand can-

not exceed supply. Denote the wage rate by w and formulate the above minimum (labor) cost problem - no variable can be negative; Q is fixed. Derive the optimality conditions and interpret them.

(b) Give a numerical solution for the problem when w = 1, Q = 1, F(ll9x) = 2 ( / ! ) 1 / 2 + * - 4 , and / ( / 2 , q) = /2 + 4(^r)

1/2. What is total cost? How much would you pay for an extra unit of the intermediate good?

(c) Using the same production function derive the cost function for arbi- trary values w > 0 and Q > 0. (You might have to distinguish between Q < 2 a n d £ ) > 2 . )

Exercises 85

17. A firm supplies a nonstorable product during several periods; demand differs from period to period. The inputs are labor and capital; labor can be ad- justed freely from period to period but capital cannot. The firm chooses the capital and labor inputs to minimize total cost subject to meeting demand in all periods.

T number of periods (t = l,...,T), xt labor input in period t, xt>0, t = l,...,T9 k capital input, k > 0, r price of capital, "> w price of labor, > given positive parameters qt demand in period t, t = l9...,T,J f(xtik) amount produced in period /, t = l,...,T.

The function / incorporates a capacity constraint in the sense that, given k>0, there exists a positive value of xt9 say x(k), that maximizes f(xt,k). Set up the problem, derive the Kuhn-Tucker conditions, and interpret them. Show that it is optimal to have excess capacity in every period (i.e., xt <x(k) for all / ) , assuming the Kuhn-Tucker conditions are optimal. Solve the prob- lem when T= 2, ql = q2 = q, w = r = 1, and f(xt,k) = kxt — 0.5(xt)

2. 18. A consumer spends his income M on bread B, high-quality apples / / , and low-

quality apples L. His budget constraint is PHH+PLL + PBB<M. Assume thatPH = T+CH>0, PL = T+CL>0, and PB=CB>09 where CH, CLt and CB are unit costs of production and T is the transport cost per apple. Suppose that his utility function is U= U(H,L, B) (H> 0, L > 0, B> 0), where U is strictly concave, with continuous and positive first-order partial derivatives, and, for all positive Hand L, UH/UL>OL>\, where a is a constant. (a) Formulate the utility maximization problem using the Kuhn-Tucker

method. Derive the first-order conditions. Show that all income is spent. Show that both H and L are consumed in positive quantities only if (PH/PL) > a. If PH/PL ^ a» which sort of apple is not consumed?

(b) Let K = H/L. Suppose that for all L > 0 and H > 0, the ratio UH/UL de- pends on K alone, i.e., UH/UL = </>(#), and that <t>'(K) < 0. Show that if H > 0 and L > 0, then dK/dT > 0. What does this mean? (How does the transport cost affect the price differential?)

(c) Verify that if U(H9 L, B) = V(H, L) W(B) and V(H, L) is homogeneous, the assumption in (b) is satisfied.

19. A firm specializes in the processing of coconuts. The two main products are coprah and coconut oil. Further processing of the husks (using labor) yields floor matting and infant mattress filling. For simplicity, units are defined in such a way that one unit of raw coconuts can yield up to one unit each of coprah, oil, matting, and filling. The manager's problem is to decide how many units of coconuts to purchase and how much of each product to manufacture.

X quantity of raw coconuts purchased, qt quantity of product / manufactured, where / = 1, 2, 3, and 4 refer to

coprah, oil, matting, and filling, respectively, Rj(qi) revenue from the sale of qt units of product /,

86 1 Static optimization

c unit cost of buying coconuts and processing them for coprah and oil, a positive constant,

/(#3» QA) additional labor cost required to obtain q3 units of matting and q4 units of filling.

R, is increasing and concave, / = 1,..., 4, and / is increasing and convex.

(a) Set up the problem of maximizing total profit subject to the capacity constraints qt<X, i = 1,...,4. Derive the optimality conditions.

(b) Analyze the optimality conditions. In particular, determine whether all coconuts purchased are always processed to yield all four products. How is the cost of coconuts allocated among the final products?

(c) Solve this problem numerically in the following case:

Ri(Qi) = 4(Qi)l/\ R2(Q2) = Q2, #3(<73) = 0.5<73, *4(<74) = 0.5<74,

c = 2 , / ( ? 3 , ? 4 ) = 0.5(<73) 2 + 0.5(<74 + l ) 2 .

C H A P T E R 2

Ordinary differential equations

2.1 Introduction

The remainder of this book is devoted to the analysis of dynamic eco- nomic models. In these models time is an independent argument and the variables are all functions of time. The process of change in these vari- ables over time has to be described. If we let the time argument be a real number, the description of these processes will necessarily involve the de- rivatives of some functions with respect to time.

Notation. Let / denote the real-valued time argument and let the value of some variable (which depends on /) be denoted by x(t). Then the total derivative of x with respect to t is denoted by

—-—, or x(t), or x for simplicity. (2.1) at

In order to describe the process of change we must link the rate of change in x to the values of x and t; hence, we shall need equations of the type f(x(t)9x(t)9 0 = 0. This is an example of a differential equation. A solution to this equation is a function x*(t) such that x*(t) and x*(t) sat- isfy the preceding expression identically. An ordinary differential equation links a single independent variable, a function (or functions) of that vari- able, and its derivatives. In this book the independent variable will always be time. (The term "ordinary" is used to indicate that there is only one in- dependent variable. Otherwise, partial derivatives would appear and we would have a partial differential equation; these are not considered here.)

When the time variable is integer-valued we have equations of the type f(x(t), x(t — 1), t) = 0, say. This is an example of a difference equation. We shall encounter some difference equations in Chapters 4 and 5, but these will be simple enough to be solved by recursion. The interested reader is referred to Goldberg (1958) for a thorough introduction.

Example 2.1.1. Consider a net investment stream that decreases in value over time according to the formula / ( / ) = \00e~005t. Equating the value

87

88 2 Ordinary differential equations

of net investment to the rate of change in capital stock at each date t, we get

K(t) = ̂ - = W0e-005t, (2.2)

where K(t) is the value of capital stock at date t. This is a differential equation involving K(t) and t and can be solved by simple integration:

dK ^ d t = \l00e-005tdt,

K(t) = -^e-°™ + C, (2.3)

where C is an arbitrary constant. The solution of differential equations always involves some integration procedure, and as a result arbitrary con- stants appear. Expression (2.3) is known as the general solution to equa- tion (2.2) because of the presence of the undetermined constant C. In order to determine K(t) exactly, we need to know its exact value at one particular date: say, at date t = 0, we have K = 1,000, or more compactly, the initial condition is A (̂O) = 1,000. Substituting these values in (2.3) yields 1,000 = - 2 , 0 0 0 ( e ° ) + C or C = 3,000, thereby giving the particular solution

K(t) = 3,000-2,000e-°- 0 5 / . (2.4)

Had we had another initial condition, we would have had another partic- ular solution. Another feature of interest is that as time becomes large the solution in (2.4) converges to 3,000. In other examples the solution may become arbitrarily large and/or oscillate around some line.

We are now ready to state some formal results. More details can be obtained from many sources - for instance, Brauer and Nohel (1969), Coddington and Levinson (1955), or Pontryagin (1962).

2.2 Definitions and fundamental results

An ordinary differential equation of order m expresses a relation between a variable t, a function x of /, and the derivatives, up to the mth order of x with respect to /:

/ ( ^ , x , x ( 2 \ . . . , x ( w ) ) = 0, (2.5)

where x{l) is the /th-order derivative of x with respect to t, i = 2,..., m. Thus, the order of a differential equation is given by the highest-order derivative entering the equation.

2.2 Definitions and fundamental results 89

A differential equation of order m is said to be normal when it can be written as

x<m) = g(xim-l\...9x i2\*9x9t). (2.6)

Only normal differential equations will be dealt with here. If x( m ),x( m - 1 ), ...,x, and x are (nxl) vectors, (2.6) represents an n-

dimensional system of differential equations of order m. If t does not appear as a distinct argument of g9 equation (2.6) is said

to be autonomous. A solution to the differential equation (2.6) is a function x*(t) defined

over some domain D that admits derivatives up to the rath order, also defined on D9 such that this function and its derivatives satisfy (2.6) iden- tically when / is in D.

A general solution to (2.6) will contain m arbitrary constants, and a particular solution to (2.6) will contain no arbitrary constants. If (2.6) is an A2-dimensional system, we need m X n arbitrary constants for a gen- eral solution.

We call initial conditions for (2.6) the m conditions

x(t0) = x0, X(to)=*o, * (2)(>o)=42)> •••> x{m-l)(t0)=x

{ 0

m-l\ (2.7)

where x0, x0, xff\ ..., x^ m~l) are specified values. In the case of an n-

dimensional system each of these conditions is itself nxl. More generally, we call boundary conditions for equation (2.6) a set

of m conditions that may or may not involve the derivatives of x\ for instance,

x(tl)=xhx(t2)=x2,-..,x(tm) = xm (2.8)

could be used. In the case of an ^-dimensional system each of these con- ditions would be n x 1.

Theorem 2.2.1: existence and uniqueness. Consider the system of differ- ential equations in (2.6) and assume that g has continuous partial deriva- tives on XxD with respect to all its arguments. Then for each set of ini- tial conditions such as (2.7) that belongs to X9 and if t0 belongs to D, there exists a unique solution to system (2.6), valid for ted, where d is some subinterval of D.

A single equation of order m can always be reduced to a system of m first-order equations by the following very useful transformation:

yi=x,y2 = X9... ,ym-l=x {m-2\ym = x

{m-l).

90 2 Ordinary differential equations

Equation (2.6) is equivalent to the system

H=y*

i (2.9)

ym—i = ym >

^ « = ?(J'«Jffl-i h J i . 0 .

Therefore, a differential equation of order m can be treated as a special case of a system of m first-order differential equations when it is conve- nient to do so. Hereafter, we turn our attention to systems of first-order differential equations.

An equilibrium of the autonomous first-order system of differential equations

x = F(x), (2.10)

where x, x, and F are ^-dimensional vectors, is a point x such that F(x) = 0. (If the system were not autonomous the "equilibrium" would depend on t and would be much less amenable to analysis.)

An equilibrium point x of (2.10) is said to be stable if any solution <p(t) of (2.10) with initial condition x(t0) = x0, with x 0 "close" to x, remains in the neighborhood of x for all t > t0.

An equilibrium point x of (2.10) is said to be asymptotically stable if it is stable and if there exists a neighborhood of x such that for any solution <p(t) that begins in this neighborhood we have l i m ^ + 0 O <p(t) = x .

An equilibrium point is said to be unstable if it is not stable. Note that the concept of stability requires only that small perturbations of the equi- librium yield a solution that remains close to the equilibrium, whereas asymptotic stability requires that the solution eventually (in infinite time) return to the equilibrium.

An equilibrium point x is said to be globally asymptotically stable if the neighborhood of x in the definition of asymptotic stability is extended to the whole domain of definition of F .

For the remainder of this chapter we have two objectives: (i) to present techniques that will enable readers to solve some simple differential equa- tions such as the ones used in this volume (for this, readers should re- fresh their knowledge of the basic rules of integration; an appendix to this chapter is provided for that purpose); and (ii) to present an introduc- tion to the qualitative theory of differential equations. We do not aim for a thorough coverage of these topics; for this readers are referred to the works cited at the end of the preceding section.

2.3 First-order differential equations 91

2.3 First-order differential equations

First-order differential equations (FODE) are single equations with x, x, and t as the only arguments. We examine various simple types.

2.3.1 Linear FODE with constant coefficients

This type of equation has the form

x+ax = b9 (2.11)

where a and b are constants. We first consider the case a ̂ 0. To solve for x(t) we multiply both sides by ea\ and the left-hand side becomes the derivative of eatx(t). Hence, integrating both sides yields

Mf) eat + C9 where C is an arbitrary constant. Finally,

'b" x(t) = l - \ + Ce~

at (2.12)

is the general solution. C can be determined with the help of an initial condition. In the above derivation we multiplied both sides by eat; this term is called an integrating factor. From equation (2.11) the equilibrium is x = b/a, and from the general solution (2.12) it is globally asymptotic- ally stable if a > 0; it is unstable if a < 0. (If a = 0, the general solution is obviously x(t) = bt + C and is unbounded unless b = 0.)

In the case where b = 0 equation (2.11) reduces to

x + ax = 0. (2.13)

This equation is said to be homogeneous, since if x(t) is a solution, so is k-x(t), where k is any constant. The general solution to (2.13) is obtained by setting b = 0 in (2.12): x(t) = Ce~at. Note that the general solution to equation (2.11) (called a nonhomogeneous equation because of the pres- ence of b) is the sum of the general solution to the homogeneous equation (2.13) plus the equilibrium solution of (2.11), which, because it involves no arbitrary constant, is seen to be a particular solution to (2.11). We shall see that this result also applies to higher-dimensional linear systems.

2.3.2 Linear FODE with variable coefficients

The coefficients of this equation depend on /; it takes the form

x + a(t)x = b(t), (2.14)

92 2 Ordinary differential equations

where a(t) and b(t) are known functions of t. The appropriate integrat- ing factor is

7(0 = exp \a(t) dt

Multiplying both sides of (2.14) by I(t) yields

I(t)X + a(t)I{t)x = I(t)b(t), (2.15)

and we see that the left-hand side of (2.15) is the derivative of I(t)x. Hence, integrating (2.15) yields

I(t)x = \l(t)b(t)dt9

or

x(t) = [I(t)]-l\l(t)b(t)dt, (2.16)

where an arbitrary constant is implicitly included in the integral and can be determined with an initial condition.

Example 2.3.1. Consider the equation

x + 2tx = 5t.

Here #(0 = 2/and b(t) = 5t; hence, I(t) = el +k9 where £ is arbitrary. In- deed, we can set k = 0 without loss. By equation (2.16) the general solu- tion is

-.-'•[f.'+c

Suppose that the initial condition is x(t0)=x0; then the particular solu- tion can be determined by

* ( ' o ) = * o = f + C,e"('o)2> C = (* 0 -§)e'°, and 2

*(o = f+(xo-i)e~('2~'°2). Note that if x0 = §, the solution reduces to x(t) = § for all t. This identi- fies | as the equilibrium solution: if the trajectory begins there, it stays there forever. Note also that it can be obtained from the original differen- tial equation by setting x = 0. However, in most nonautonomous differ- ential equations, of which linear FODE with variable coefficients are a

2.3 First-order differential equations 93

special case, there is no fixed equilibrium point because of the presence of the t argument.

2.3.3 Nonlinear FODE

Nonlinear FODE cannot be put in the form of (2.14); in general they take the form f(x, t)x = g(x, t), where g is not linear in x. Nonlinear FODE are difficult to handle except for special cases, some of which are dis- cussed below.

Separable equations. A differential equation is separable if it can be writ- ten as

f(x)x = g(t). (2.17)

Integrating both sides with respect to t yields

\f(x)^dt = \g(t)dt, or \f(x)dx = \g(t)dt. (2.18)

This expression can be used to obtain the solution x(t), at least in im- plicit form.

Example 2.3.2

3x2x = 4t\

\3x2xdt=\3x2dx=\4t3dt, or x3 = / 4 + C,

x{t) = [tA + C]x/\

Bernoulli equations. An equation of the form

x + u(t)x = w(t)xn, (2.19)

where n is any real number other than 0 or 1, is called a Bernoulli equa- tion. It can be reduced to a FODE by a transformation of variables. Di- vide both sides by xn to get

x-"x + u(t)xx-" = w{t).

Define z = xl~n so that z = (1 — n)x~nx\ then the preceding equation be- comes

1 (1-A7)

which is a linear FODE in z. Once solved, x(t) = [z(0]1/(1_A2) is obtained.

z + u(t)z = w(t),

94 2 Ordinary differential equations

g(k)

Figure 2.1

2.3.4 Qualitative analysis on a phase diagram

While many nonlinear FODE cannot be solved analytically, the qualita- tive properties of their solutions can sometimes be described by a graphic device. Suppose the equation is x = g(x). We can plot the graph of x against x; this graph is called the phase line. We illustrate the usefulness of this approach in the following example.

Example 2.3.3. An economy produces an output q from its capital stock k\ the production function isq=f(k), where/is an increasing and strictly concave function, with/(0) = 0, lim* _ «,/'(£) = 0, and l\mk_+0f'(k) = b > 0. A constant fraction s of total output is saved and invested, so that gross investment is sf(k). The capital stock depreciates at the rate n > 0 (a discussion of depreciation is deferred until Chapter 3); therefore, the net rate of capital accumulation is

& = sf(k)-nkmg(k). (2.20)

According to the above assumptions we have g(0) = 0, g'(0)=sb — n9 lim^ _*oo g'(k) = — n, and g"(k) < 0. If we assume that sb — n>0, the graph of g(k) cuts the horizontal axis twice at k = 0 and, say, k = kx>0. This graph is drawn in Figure 2.1. The points k\ and 0 are equilibrium points since A: = 0 at these points; this could also be seen from equation (2.20).

2.4 Systems of linear FODE 95

The phase diagram of Figure 2.1 can be used to examine the stability of these equilibria. On the phase line we have added some arrows. For all values of k within the open interval ( 0 , ^ ) , g(k) is positive, indicating that k > 0 - hence, k must be rising; this is why the arrows in this interval point toward large k values. For values of k > kx, g(k) < 0, so that k de- creases. The arrows point toward the left, accordingly. We can see that the equilibrium at kx is asymptotically stable, since for any starting point around kx, k(t) will approach kx. The equilibrium at A: = 0 is unstable, because there is no neighborhood of 0 from which k(t) will tend to 0. (Indeed, if we rule out starting at k = 09 the equilibrium at kx is globally asymptotically stable.)

Had we assumed sb<n, the function g(k) would have been negative for all k > 0 and the unique equilibrium at 0 would have been globally asymptotically stable.

2.4 Systems of linear FODE with constant coefficients

Consider the ^-dimensional system of differential equations

x = Ax + b (2.21)

and the associated homogeneous system

x = Ax. (2.22)

The general solution of (2.22) is relatively easy to obtain, and a particular solution of (2.21) is its equilibrium point given by x = — A- 1b (defined if A is nonsingular). For this reason the following lemma is important.

Lemma 2.4.1. If xx(t) is a solution to (2.21) and x2(t) is a solution to (2.22), then xx(t) + x2(t) is also a solution to (2.21).

The proof is obvious and follows from the linearity of the two systems. Suppose now that we have the general solution to (2.22), x*(t) say, and a particular solution to (2.21) (e.g., the equilibrium x). Then x*(0 + x is a solution to (2.21) by Lemma 2.4.1, and since it contains n arbitrary con- stants it is the general solution to (2.21). Thus, the behavior of the solu- tion to (2.21) around its equilibrium point x = x is identical to that of the solution to (2.22) around its equilibrium point x = 0. For this reason we can restrict our attention to homogeneous systems when examining the qualitative behavior of solutions. In what follows we restrict our atten- tion to two-dimensional systems as a way of illustrating the types of tra- jectories that may emerge. Most results are given without proof; for a more detailed treatment the reader is referred to Brauer and Nohel (1969, sec. 2.8), for instance.

96 2 Ordinary differential equations

2.4.1 Algebraic solutions

We consider the homogeneous system

flu tfi2~ir*i _a2\ #22 J L*2

or more compactly,

x = Ax.

f-'i-f (2.23)

(2.23')

As in the single-equation case the solution will involve exponential func- tions of the type eXt. To see how these emerge, suppose a particular solu- tion is x = aeX/, where a is a vector of constants, not all zero. Deriving x and substituting in (2.23'), we obtain x = Xaex' = Aaex'. Hence, Xa = Aa or [A —XI]a = 0. For this system to have solutions other than a = 0, the matrix A —XI must be singular; that is,

| A - X I | = 0 ,

or in the case of system (2.23),

X2-trAX + |A| = 0, (2.24)

where trA = a n + tf22 and |A| = antf22-tf12tf21.

This equation is called the characteristic equation of A. Its roots \{ and X2 will serve to construct the general solution of (2.23) and will appear in terms such as eXl' and e*2'. Note that tr A = Xi + X2, while |A| = X1X2. Since equation (2.24) is quadratic, it is possible that its roots are conju- gate complex numbers such as a + i(3 and a —i(3, where / is the imagi- nary number defined by i2 = — 1 and a and (3 are real. In this case we shall transform the expressions to obtain real-valued solutions; these will in- volve trigonometric functions of (3t, and hence will generate oscillatory trajectories. For simplicity we assume hereafter that neither root is zero; this guarantees that A is nonsingular and that the origin is the unique equilibrium. For our purposes we can simplify the problem further. It can be shown that for any 2 x 2 real matrix A, there exists a real matrix T such that T_1AT = B, where B can take only the forms

«-K : (c) B =

X 0 1 X

(b) B =

(d) B =

"X 0" 0 X

a

>

81 (2.25)

2.4 Systems of linear FODE 97

where Xj and X2 are distinct real roots, X is a double root, and a ± i(3 are conjugate complex roots of both A and B. Then we can define by a linear transformation a new variable y such that x = Ty. Accordingly, x = Ty and (2.23') reduces to Ty = ATy, or y = T-1ATy, and finally

y = By. (2.26)

The solution to (2.26) differs from that of (2.23) only because it is dis- torted by the linear transformation x = Ty; they are qualitatively identi- cal. (One can be obtained from the other by rescaling and rotation of the axes.) In the following section we study the geometric properties of (2.26) keeping in mind that B can take only the forms described in (2.25).

2.4.2 Phase diagram representation of the solution

Recall that the origin is the only equilibrium point of (2.26) under our assumption that | A | ^ 0 (i.e., there are no zero roots). Throughout we denote the initial point by (y^y®) at f = 0 and assume that it is not the origin.

From a diagrammatic point of view the following classification into six cases is appropriate.

Case (a)

B = X 0 1

o xj* There is a single real root. The solution isyx(t) =y\e

Xt, y2(t) =j>2e X/. The

ratio yi(t)/yi(t) is constant over time, and the trajectories are rays through the origin. This is called a proper node. If X < 0, it is globally asymp- totically stable; if X > 0, it is unstable. The stable case is shown in Fig- ure 2.2a.

Case (b) 0 X2 Ho' and X!X2>0.

The roots are real, distinct, and of the same sign. The solution is yx(t) = y\eXl', y2(t) — y\e

Xlt. The trajectories will be stable if the roots are neg- ative, and unstable if they are positive. The slope of the trajectories is not constant, since y2(t)/yi(t) = (j>2/>>iV

(X2~Xl)/; thus, depending on the signs of j>2, J

7?* and X2 —Xj, we have different subcases; Figure 2.2b illus- trates 0 < Xj < X2, where the ratio goes to infinity. In any event the origin is called an improper node.

98 2 Ordinary differential equations

- « — • Y i

(a) X < 0

(c) X ! < 0 < X 2

(e) a < 0 , j 3 > 0

Figure 2.2

(b) Xj > X2 > 0

(d) X < 0

(f) a = 0 , / 3 < 0

B = and X!X2<0.

Case (c) \x 0" 0 X2

The roots are real, distinct, and of opposite signs (say, X! < 0 < X2). The algebraic solution is the same as in case (b) but, at / -• oo, yx(t) =y{°e

Xlt

2.4 Systems of linear FODE 99

goes to 0 and y2(t) =y2e X2t goes to ±oo depending on whether y2 is posi-

tive or negative, as long as y2^ 0. If y2 = 0, the trajectory is along the horizontal axis and is stable. For any other trajectory, y2(t) and also y2(t)/yx(t) go to infinity. The origin is called a saddle point and is de- picted in Figure 2.2c. (It is so named by analogy to the saddle-point con- cept encountered in Section 1.1.5, where it can be verified that the Hessian matrix has characteristic roots of opposite signs, just as the matrix of co- efficients does here.) Such points exhibit conditional stability in the sense that the solution is stable for some initial points ((yf, 0) here) and un- stable for others.

Case (d)

B = [ X ° U x

As in case (a) there is a single real root, but no real matrix T can trans- form A into a diagonal matrix and this variant is obtained. The solution includes another t term:

yi(t) =j>iV', y2(t) = (y$+y\°t)e Xt.

Suppose X<0; then both yx(t) and y2(t) go to the origin as t goes to infinity. If y^O, the ratio y2(t)/yl(t) = (y$/yl°) + t is not constant but increases to infinity with t, starting from negative values if y2 and y{° dif- fer in sign. The trajectories are as depicted in Figure 2.2d; the origin is called an improper node, as in case (b) and is stable since X < 0. When X > 0, the trajectories follow the same lines in the opposite direction and are unstable.

Case (e)

B = r a

, n n (2.27)

, and a * 0 , 0 * 0 .

The roots are the complex conjugates a + i(3 and a — iff with nonzero real part (a ^ 0). The solution is

yl(t) = e ai(y?cosfft+y$smfft)9

y2(t) = e at(y$cosfft-y\°smfft).

The expressions in parentheses in (2.27) have a periodicity of lir/ff, be- cause cos ff[t + 2-K/ff] = cos fft and sin ff[t + 27r/0] = sin fft. Furthermore, if a < 0 then the exponential term goes to zero and the trajectory spirals toward the origin; it is called a stable focus. If a > 0 then the trajectories form an unstable focus spiraling away from the origin. The stable case is depicted in Figure 2.2e when ff > 0, which induces a clockwise movement.

100 2 Ordinary differential equations

(If /3 were negative, the movement would be anticlockwise, but still stable as a < 0.)

Case (f) 0 0

B = -0 0

0*0.

This is much the same as case (e) except that the eat term is identically 1. Instead of spiraling toward or away from the origin, the trajectories are closed circles. This configuration is called a center and is illustrated in Figure 2.2f with 0 < 0, which results in an anticlockwise movement. Note that for a center the origin is stable but not asymptotically so.

Remark. Some important results emerge from the preceding discussion:

(i) A system such as (2.23) has a stable origin if and only if its char- acteristic roots have negative real parts.

(ii) A saddle point occurs if and only if the determinant of A is nega- tive.

(iii) A sufficient condition for instability is that tr A > 0.

2.5 Systems of two nonlinear FODE

In many interesting economic problems, the equations describing the evo- lution of a system are typically nonlinear. The behavior of the nonlinear system around an equilibrium point (if it exists) can be approximated by that of a linear system, as we now show.

2.5.1 Local characterization

Consider the nonlinear system

x,=f(xhx2)9 x2 = g(xux2)9

w h e r e / a n d g are assumed to have continuous second-order derivatives. We also assume that system (2.28) possesses an equilibrium point (xh x2), so that f(xh x2) = 0 and g(xu x2) = 0. For x close to x we can take a first- order approximation of / and g to obtain

/ U l , ^ 2 ) = / ( ^ 1 ^ 2 ) + / l ( ^ b ^ 2 ) U l - ^ l ) + / 2 ( ^ b ^ 2 ) ( ^ 2 - ^ 2 ) ,

S ( * b * 2 ) = S ( * b * 2 ) + £ l ( * l , * 2 ) ( * l - * l ) + g 2 ( * b * 2 ) ( * 2 - * 2 ) >

but since f(xux2) = 0 and g(xux2) = 0, if we denote / , , / 2 , gu and g2 evaluated at (xux2) by au, a12, a2h and a22, respectively, we can approx- imate (2.28) by

2.5 Systems of two nonlinear FODE 101

hUh1 ^lh-H (2.29) L*2j |_*21 ^22j L^2~^2j

and defining z, as the deviation JC/—J?/, we can write (2.29) as

f!'l = ffl" an]\H (2-30) [Z2J |_*21 ^ J L ^ J

which is exactly the form of (2.23) analyzed in the preceding section. The following theorem formalizes the similarities between (2.28) and (2.30).

Theorem 2.5.1. Assume ana22 — aua22^ 0. The qualitative behavior of the trajectories of the nonlinear system (2.28) in the neighborhood of the equilibrium point (xh x2) is the same as that of the trajectories of the cor- responding linear system (2.30) around the origin, with the exception that if the origin is a center, then (xux2) may be either a center or a focus.

The exception stated in the theorem can be explained as follows: the occurrence of a center hinges on the real parts of the roots of A in (2.30) being exactly zero; any perturbation gives rise to a focus. For more details see Lefschetz (1965) or Coddington and Levinson (1955, ch. 15).

2.5.2 Global characterization and phase diagrams

To determine the behavior of trajectories away from the equilibrium is a much more difficult task. There is no comprehensive general theory, and precise analysis often requires complicated topological arguments (e.g., Lefschetz, 1965). However, with sufficiently simple systems and with ap- propriate restrictions on the functions f(xhx2) and g(xux2), one can often obtain a good qualitative description of the global properties of the solution with the use of phase diagrams. Examples can be found in Hirsch and Smale (1974, ch. 12). Before we illustrate the technique we must de- scribe one type of configuration that is exhibited by nonlinear systems but that could not occur in linear ones: these are limit cycles. In such cases there exists a closed curve around the equilibrium point. Any start- ing point on that curve will remain on it indefinitely, but contrary to a "center" configuration this is the only such closed curve, or loop, in the neighborhood. This curve is called a limit cycle and may be asymptotically stable or unstable from within and/or outside the area it delineates in the sense that trajectories sufficiently close to the loop may either approach asymptotically or move away from it. Several types of limit cycles are de- picted in Figure 2.3, where the thicker curve is the limit cycle; (a) is asymp- totically stable, (d) is unstable, while (b) and (c) are conditionally stable. It is in general quite difficult to prove or disprove the existence of limit

102 2 Ordinary differential equations

(c) (d)

Figure 2.3

cycles. Necessary and sufficient conditions for their existence are avail- able (see, e.g., Coddington and Levinson, 1955, ch. 16). We now turn to some economic examples.

Example 2.5.1: pollution and growth. Let K denote capital stock and P the stock of pollution. Output is Y=Ka9 where 0 < a < 1, and savings are a constant proportion of output, so that the rate of growth of capital is

K = sKa-5K = K(sKa-l-8), (2.31)

where 6 > 0 is the rate of depreciation of capital. If a capital stock K gen- erates a flow of pollution K13, where 0 > 1 and the stock of pollution de- cays at rate y > 0, the net rate of change in the stock of pollution is

P = K^-yP. (2.32)

2.5 Systems of two nonlinear FODE 103

Figure 2.4

The initial conditions are K(0)=K0>:0 and P(0)=P0>:0. We wish to characterize the solution of the system (2.31)-(2.32) in the nonnegative orthant of the (K, P) plane.

Our first task is to determine the locus of points in the (K,P) plane along which P = 0. From (2.32) P = 0 if P = (l/y)K0. This curve is drawn in Figure 2.4; it is increasing and convex and goes through the origin. Next we turn to the K = 0 locus. From (2.31), K = 0 if K = 0 or if

K = (d/s)l/{a-l) = K*. (2.33)

The K = 0 locus thus consists of two vertical lines: K = 0 and K — K*. There are two points at which both K = 0 and P — 0. The first is the origin (0,0), and the second is at the intersection of the P = 0 locus and K = K*; hence, it is at (K*,P*), where P* = (\/y)(K*f. The locus P = 0 and the line K = K* (along which K = 0) define four regions, labeled I to IV, in the nonnegative orthant. These regions are called isosectors, and the signs of K and P are uniquely determined inside each isosector. To see this, note that by continuity of the expressions in (2.31) and (2.32) the sign of P changes when the trajectories cross the P = 0 locus and that of K changes across the K = K* line. Therefore, above the P = 0 locus, P>(l/y)K13

implies P < 0, but P > 0 below that locus. From (2.31) and (2.33) it is clear that if K>K*, then K<0, and if 0<K<K*9 then K>0. Therefore, in region I, K>0 and P < 0 . In this isosector we have drawn a horizontal

104 2 Ordinary differential equations

arrow pointing to the right (eastward) to indicate that K increases in this region; similarly, the downward vertical arrow (southward) signifies that P decreases in this region. All trajectories in region I point southeast; they can enter region IV only by crossing the P = 0 locus, and the slope of these trajectories is zero (horizontal) at the crossing, since it is given by dP/dK — P/k, and the numerator is zero on the P = 0 locus. Trajec- tories cannot enter region II from region I because when K approaches K*, K approaches zero and K* is not reached in finite time. We turn now to region III, where by examining (2.31) and (2.32) we can ascertain that P > 0 and K<09 indicated by arrows in Figure 2.4. Here again trajec- tories cannot move on to region IV but may move on to region II and have a horizontal slope when crossing the P = 0 locus into it. Similar con- siderations show that trajectories point southwest in region II and north- east in region IV and cannot leave any of those regions once in it. For this reason regions II and IV are called terminal isosectors or, more suc- cinctly, traps. It is clear that the K = K* line provides two routes of access to the equilibrium, from above and from below. There exist, however, many others. We have seen that any trajectory in region II (or IV) moves toward the equilibrium and that any trajectory in region III (or I) either goes to the equilibrium or moves on to region II (or IV). Therefore, all trajectories go to the equilibrium, and we expect to have a stable node (proper or improper) at (K*9 P*). The other equilibrium point, (0,0), is "conditionally stable": only trajectories with the initial condition # ( 0 ) = 0 will converge to it.

Our diagrammatic characterization of (AT*, P*) as a stable node can be confirmed locally by linearizing the differential equations (2.31) and (2.32) as we did in the general case in (2.30). We obtain

ffl _[asK a - l 0

- 7 K-K* P-P*

(2.34)

The matrix of coefficients in (2.34) must be evaluated at (K*,P*), and we have

K]-\ Ha-D 0 ~\[K-K* p\~[l3(8/s){0-l)/(a~1) -y J [ P-P*

The roots of the characteristic equation of (2.35),

X 2 - [ 5 ( a - l ) - 7 ] X - 7 5 ( a - l ) = 0,

(2.35)

are \i = 8(a — 1) and X2=— y,

both negative, confirming that we have a stable node (at least around (K*,P*)).

2.5 Systems of two nonlinear FODE 105

Let us consider a more complex example taken from the economics of fisheries.

Example 2.5.2: the fish and the fishermen. Suppose that the stock of fish present in some fishing grounds increases (or decreases) according to the process

R(t)=R(t)-(RV))2-x(t)9 (2.36)

where R(t) is the stock of fish and x(t) the total catch, both at date t. Let K(t) be the tonnage capacity of the fishing fleet. We assume that the relative skillfulness of fish and fishermen results in the following effec- tive catch:

x(t)=R(t)[K(t)]V2. (2.37)

We assume that there is free entry and exit into the fleet according to the flow of net returns and specifically that the proportional rate of change in fleet capacity is equal to a multiple of the difference between the average revenue per ton of capacity, px/K (where p is the price of fish), and the fleet average operating cost per ton, c:

K/K = m(px/K-c). (2.38)

We assume for simplicity of exposition that c = m=p = l, so that us- ing (2.36)-(2.38) we obtain, after eliminating x(t), the following two- dimensional, nonlinear system of differential equations (we skip the time argument):

K = K(RK-l/2-l), (2.39)

R = R(l-R-Kl/2). (2.40)

In order to study the global qualitative properties of the solutions of this system, we construct a phase diagram (Figure 2.5). First, we deter- mine the locus of points where R = 09 using (2.40). This consists of the K axis (along which R = 0) and the curve R = 1 - AT1/2 (defined in the non- negative orthant for K < 1). In the region to the right of this curve and above the horizontal axis, R>l-K1/2, so that R<0 by (2.40). To the left of the curve, R > 0. Second, we see from (2.39) that K — 0 along the vertical axis (K — 0) and the curve R — Kxl2. In the region above this curve K> K(Kl/2K~l/2-1) = 0, and in the region below it K<0. Since dR/dK = R/K, trajectories crossing the K locus have an infinite (verti- cal) slope at the crossing while trajectories crossing the R = 0 locus have a zero (horizontal) slope there. Equipped with these observations, we are able to delineate four isosectors in the (AT, R) plane and the general direc- tion of trajectories in each; these are indicated by right-angled arrows

106 2 Ordinary differential equations

K = 0

Figure 2.5

in Figure 2.5. Furthermore, the slopes of trajectories when crossing the boundaries of isosectors or touching the axes are also indicated. It is worth noting that this diagram has no "trap"; indeed, trajectories pass freely from one region to the next in a clockwise sequence. There are three equi- librium points: Ex at (0,0), E2 at (0,1), and E3 at ( | , \). The first equi- librium is conditionally stable: only trajectories starting at R = 0 converge to it. The second equilibrium is also conditionally stable, for trajectories starting at K = 0. The local stability of the third equilibrium can be deter- mined by linearizing the system of equations (2.39)-(2.40) around ( | , \) using the general formula (2.29). We obtain

[*]-[ \RK -1/2 _ !

SRK-"2 l-2R-Kl/2

and, evaluating the matrix of coefficients at (^, ^),

A =

(2.41)

(2.42)

t r A = — 1 and |A| = £; therefore, the characteristic equation of A is \ 2 + X + 5 = 0 and the roots are complex: 0.5(—1±/). The real parts are

2.5 Systems of two nonlinear FODE 107

negative, and the equilibrium point E3 is therefore a locally asymptotic- ally stable focus. This property could not be inferred from the phase dia- gram by examining the directions of the arrows, for these are consistent with a locally unstable focus or a center.

It is important to realize that we have not proved that any arbitrary trajectory with initial conditions (AT(0),/?(0))>(0,0) converges to the equilibrium point E3. We have only shown this to be true if ( # ( 0 ) , R(0)) are in the neighborhood of (£, \) (how close is unspecified). Therefore, the possibility of a limit cycle has not been ruled out. It is possible, how- ever, to do so here in a simple way. We need only consider trajectories within a square of side 1, having the origin as its southwest corner. Be- cause of the symmetry of the two curves K = 0 and R = 0 (R = Kl/2 and R = 1 — K1/2), starting from any point on one of these curves we can draw a rectangle with a corner on each of the curves (one such rectangle S is drawn in Figure 2.5). For a trajectory beginning at a corner of such a rec- tangle to return there, it would have to follow the sides, which conflicts with all trajectories pointing inward toward the equilibrium E3.

Example 2.5.3: competing species. Let X and Y denote the biomass of two species respectively. Assume that the two species compete for food, so that the rate of growth of each species is negatively related to the bio- mass of the other species. We postulate the functional forms

X/X=1-X-2Y, X = X(l-X-2Y)9 or (2.43)

Y/Y=\-Y-2X, Y=Y(1-Y-2X).

Thus, X = 0 if X = 0 or Y=(l-X)/2 and 7 = 0 if y = 0 o r Y=l-2X. These loci are drawn in Figure 2.6. It is easy to see that there are four equi- libria: (Xu Yx) = (0,0), (X2, Y2) = (1,0), (X39 Y3) = (0,1), and (XA, Y4) = ( j , j ) . The last one is a saddle point, the first one is an unstable node, and the other two are locally asymptotically stable nodes. The matrix of co- efficient of the linearized system is

X-1X~1Y ~2X n (2.44) A = -2Y 1-2Y-2X

The reader should check that A of (2.44), evaluated at (X4, y4), has a negative determinant, thus confirming that the interior equilibrium is a saddle point. The configuration of the other equilibria can be similarly checked.

2.5.3 A special case: Hamiltonian systems

Systems of nonlinear differential equations are often very difficult to solve. It is sometimes possible to do so by transforming a system into a higher-

108 2 Ordinary differential equations

X = 0

* V Y = O

Figure 2.6

order differential equation in one of the variables. In this section we con- centrate on a class of systems often encountered in optimal control theory.

Suppose that H(xhx2) is a twice-differentiable function and that we derive a system of two equations in the following way,

xl = dH(xl9x2)/dx2, x2=-dH(xux2)/dxu

(2.45)

where the time argument has been suppressed. If we differentiate the first equation totally with respect to time, we ob-

tain xi in terms of xu x2i xh and x2\x2 can be eliminated using the second equation; and finally an expression for x2 in terms of xx and xx can be ex- tracted from the first equation. This yields X\ in terms of xx and xx. We now proceed with some examples in which the solution is straightforward.

Example 2.5.4. Let H = (X\)ai(x1) €ll\ then the system of differential equa-

tions is

*i = a 2 < * i ) a , U 2 ) a 2 _

x2 = -ocx{xx)^-\x2r\

Using the above technique, we obtain

(2.46)

2.5 Systems of two nonlinear FODE 109

xi = ala2(xi) a>-](X2)ai-lxl + a2(a2-l)(xi)

ai(x2) ai-2X2

= a , « 2 ( J f I ) a ' - I ( X 2 ) a 2 - I J t i - a 1 « 2 ( a 2 - l ) J f i

2 a , " , * l a 2 " 2 .

and using x2 = [ a 2 ( * i ) a i ( * i ) - 1 ] 1 / < 1 _ 0 ' 2 ) , we obtain after simplification

xx = ^ ^ . (2.47) a2 xx

This is easily solved in two steps:

x1=oi_x1 xx a2 xx '

ln\xi\ = — ln\xA+A9 Oil

\xl\ = e A\xl\

ai/ai. (2.48)

We must now distinguish several branches of the solution:

* i > 0 , xx>0; thus, xx=xx i/oC2eA, and

0*^0*2, xl(t) = (Kt + B) 0i2/{a2-ai\

where K=(a2 — ax)e A/a2,

«i = «2» * i ( 0 = exp(eAt+B).

xx<0, xx>0; thus, xx = (-xx) ai/aie\ and

« i ^ a 2 , JC1(0 = - ( - ^ + ^ ) a 2 / ( a 2 " a , ) ,

where K= (a2 — ax)e A/a2,

ax = a2f xx(t) = -zxp(-e At + B).

Similarly, xx < 0, xx < 0 yields

a i * a 2 , ^ i ( 0 = - ( ^ + 5 ) a 2 / ( O 2 " a i ) ,

where AT = (0:2 — 0;!)eA/a2, ax = a2, xx(t) = -zxp(e

At + B).

xx > 0, xx < 0 yields «i * a2» JCi(0 = (-Jf/ + f l )

a 2 / ( a 2 " a , ) , where Ar = ( a 2 — ai)e

A/a2, a, = a 2 , j d ( 0 = exp(-e

At +B).

(2.49a)

(2.49b)

(2.49c)

(2.49d)

We can obtain the corresponding solution for x2 by using the first equa- tion of (2.46). For instance, if *,(*) = (Kt+B)a2^a2-ai\ w e have

xx(t)= ai KiKt+B)"^-^;

OL2-OL\

110 2 Ordinary differential equations

hence, after simplification,

- J ^ - L ) (^+fl)«>/(^-«.),

and so on.

Example 2.5.5. Let

/ / = 1 , a / 5 * l , 0 ;

then the system is

Using a similar technique, we quickly obtain

xx = (1 -ct2)(xx)^-\xx)^- 1)/{ai-X)\

, ( j t i ) l / ( « 2 - l ) N

* 1 - W i ( * i ) a i l - a 2

After integrating, we have l ( « 2 _ 1 ) / « 2

X\ = (2.50)

The general solution to this equation cannot be obtained, but we can give the solution in some special cases. As we shall see, it is in implicit form.

(i) Let ax = 0.5, a2 = - 0 . 5 ; then (2.50) become x{ = [A + ypc~x] 3.

Hence,

J (A + yIx[)**dt'

This can be integrated using substitution (y = A + yfx~x) to obtain

A(A + Jx\)-2-2(A + Jx-x)- x=B + t. (2.51)

(ii) Let ax = a2 = j ; then (2.50) becomes

J M ^ - ( * I ) 1 / 3 ] 2 = 1

or

[A2-2A(xx)^ + (xx) 2^]xx = h

A2xx-\A(xx) A/^\(xx)

s/?> = t^B. (2.52)

(iii) Let a1 = o:2 = 0.5; then (2.50) becomes Xj = [A- \[x[]~ x. Hence,

Axx-\(xx) vl = t+B. (2.53)

Appendix 111

As this last example shows, systems of nonlinear differential equations are often complicated to solve.

Appendix

Indefinite integrals

Given any continuous function / ( * ) , one can find a function F(x) such that its derivative is f(x). Such a function is called an antiderivative of f(x). For example, if f(x) = lOx, then 5x2 + 7 is one of its antiderivatives, and so is 5x2 — 3, or any function of the form 5x2 + C, where C is a con- stant. It is clear that all antiderivatives of a given function differ from one another only by a constant. For this reason, it is convenient to define the indefinite integral of a given function f(x) as the general form of its anti- derivatives; it always contains an arbitrary constant. The symbol for it is

\f(x)dx.

Indefinite integrals of some common functions:

\x"dx = (n + l)-lxn+l + C, n^-h (Al)

^x- ldx = \n\x\ + C, (A2)

\exdx = ex + C. (A3)

Properties of indefinite integrals:

l kf(x) dx = k\ f(x) dx for any constant k, (A4)

\(f(x) + g(x))dx = \f(x)dx + \g(x)dx. (A5)

Properties (A4) and (A5), together with the rules of integration by parts and by substitution (described below) are very useful for finding the in- definite integral of sums or products of functions.

Integration by parts:

\u'(x)v(x)dx = u(x)v{x)-\u(x)v'(x)dx. (A6)

Example, Find the indefinite integral of xex. Let u'(x) = ex and v(x)=x. Then u(x) = ex and v'(x) = 1. Using (A6), we obtain

112 2 Ordinary differential equations

[xexdx = exx-[exdx = exx-ex + C. (A7)

The reader should verify that the derivative of the right-hand side of (A7) is xex.

Integration by substitution:

\f(g(x))g'(x)dx=\f(u)du, where u = g(x). (A8)

Example. Find the indefinite integral of 2e{2x+5). Let u = 2*+ 5 =g(x) and f(u) = eu. Then f(g(x)) = e{2x+5) and g'(x) = 2. Applying formula (A8) yields

^2ei2x+5)dx = \eudu = eu + C = e{2x+5) + C.

It is easy to see that the derivative of the right-hand side of the preced- ing equation is 2e{2x+5).

Definite integrals

The definite integral

?b ' )dx (A9) fix) t

is the same as F(b)—F(a), where F(x) is any antiderivative of f(x). In the definite integral (A9), a and b are called the lower and upper limits of the integral. The variable x in (A9) is "mute"; that is, x can be replaced by any other symbol without affecting the value of (A9). For example,

\bf(x)dx=\bf(t)dt.

\j 2(10x)dx.

Example. Evaluate the following definite integral:

)<

SinceF(x) = 5x2 + C, F(2)-F(l) = 2 0 - 5 = 15. If F(x) is of the form u(x)v(x), then f(x) = u'(x)v(x) + u(x)v'(x),

and F(b)-F(a) = u(b)v(b)-u(a)v(a)

= ( {u'(x)v(x) + u(x)v'(x))dx. (A10)

Exercises 113

It follows from (A 10) that the definite integral counterpart of (A6) is

[bu'(x)v(x)dx = u(b)v(b)-u(a)v(a)-\bu(x)v'(x)dx. (All) Ja Ja

The definite integral counterpart of (A8) is

\b f(g(x))g'(x)dx= \8{b)f(u)du. (A12) Ja Jg(a)

The derivative of a definite integral with respect to the upper or lower limit of integration

If we define

I(a, b) S F(b) -F(a) = [" f(x) dx9 (A13) Ja

then the derivative 31/da can be shown to be equal to —f(a); similarly, dl/db=f(b).

The derivative of a definite integral with respect to a parameter

If f(x,s) is continuous in x and its partial derivative with respect to s is defined, then the definite integral

rb I(a9b,s)s\ f(x,s)dx

Ja

is a function of the parameter s; its partial derivative with respect to s is given by

dl (b df

as If a and b are themselves functions of 5, then the total derivative dl/ds

is dl _ dl da dl db dl ds da ds db ds ds

= -f(a,s)a'(s)+f(b,s)b'(s) + \b dfi*'s) dx. (A15) J a OS

Equation (A15) is called Leibniz's rule.

cbdf

'\.isdx- < A , 4 )

Exercises

1. In a certain economy, capital is accumulated according to the rule K(t) = sF(K(t),L(t)) — 5K(t), where s is the saving ratio, 0 < s < l , 5 is the rate of

114 2 Ordinary differential equations

depreciation, 6 > 0, and F is the production function. Let w > 0 denote the wage rate in a neighboring large country; assume that w is exogenous. This economy's labor force grows or declines according to the difference between the marginal product of labor and w: L(t) = L(t)[FL(K(t),L(t)) — w]. Assume that F(K(t),L(t)) = 4[K(t)L(t)]V4. (a) In the (L,K) space, find the locus of points K = 0; identify the regions

where K > 0 and K < 0, respectively. (b) Find the L = 0 locus and the regions where L > 0 and L < 0. (c) Is there an equilibrium (L\ K*) with K* > 0 and L* > 0? Is it stable? (d) Assume that w = 1, s = 0.1, 5 = 0.4 and determine A'* and L*. Linearize the

system around the equilibrium point, calculate the characteristic roots, and describe the behavior of the system around the equilibrium.

2. Let K(t) denote capital stock at time t and P(t) be the level of pollution. Out- put is Y(t) = (K(t))a/(1 + P(t)), where 0 < a < 1. Savings is a constant fraction of output, so that the rate of change in capital stock is K(t) = sY(t) — 5K{t), where 5 is the rate of depreciation and s is the saving ratio, 0 < s < 1. For sim- plicity assume that s = 26. The pollution level changes according to the formula P(t) = K(t)-P(t). (a) Find the loci P = 0 and K = 0 and identify the regions of the (P, K) plane

where P and K have definite signs. (b) Show that there exists an equilibrium, say (AT*, P*). Calculate these values

and show that they are positive. (c) Linearize the differential equations around the equilibrium and determine

whether the characteristic roots are real or complex if 6 = 0.4, s = 0.8, and a = 0.1. What are the local stability properties of this equilibrium?

3. Suppose that aggregate output adjusts according to the equation Y(t) = G(t) +A -sY(t), 0 < s < 1 and A > 0. (a) If government expenditure varies according to the rule G(t) = 0.5Y(t) —

G(t), determine the path of aggregate output. Is there an equilibrium? Is it stable? Construct a phase diagram in the {Y, G) space.

(b) If, instead, G(t) is determined by G(t) = G0 + a$ t 0(Y-Y(T))dT-t3Y(t),

where G 0 > 0 , a > 0 , (3>Q, and F > 0 , analyze the consequences of this new policy rule on the path of aggregate output. Construct a phase dia- gram in the (Y9 G) space. (Differentiate G(t) with respect to /.)

4. Let u(w) be the utility of wealth w. (a) Determine the functional form w(-) for which the coefficient of absolute

risk aversion is constant, i.e., — u"(w)/u'(w) = Ki a positive constant. (Hint: The basic variable here is wealth w, not time /.)

(b) Determine the functional form w(-) for which the coefficient of relative risk aversion is constant, i.e., —wu"(w)/u'(w) = R, a positive constant.

5. The rate of increase in the price of apples is proportional to excess demand P(t) = k(D(t) — S(t)). In each of the following two cases determine the general time path of price. Find the equilibrium price; what assumption must you make about the sign of k to ensure that the equilibrium is stable? (a) D(t)=A-BP(t),A>0, £ > 0 ,

S(t) = -M+NP(t), M > 0 , W > 0 .

Exercises 115

(b) D(t)=A-B[P(t)-l]9A>0, B>0, S«)=A + [P(t)-l]\

(Hint: Let Q(t) = P(t)-\.) Draw a phase diagram and examine the stability properties of the equilib-

rium when D(t) = e x p [ - P ( / ) ] and S(t) = [P(t)]\ 6. Consider the following predator-prey model:

x(t)=x(t)[A-By(t)-Mx(t)}> x(t)>0,

y(t)=y(t)[Cx(t)-D-Ny(t)], y(t)>0,

where A, B, C, D, M, and N are specified positive constants and we assume that £ > M < C A (a) Which variable represents the population of predators? Explain the mean-

ing of each equation. (b) Draw a phase diagram in the (x, y) space. Is there a positively valued equi-

librium? If so, linearize the system about the equilibrium point and deter- mine whether the characteristic roots have negative real parts. What can you infer about the stability of the equilibrium?

C H A P T E R 3

Introduction to dynamic optimization

This chapter is a very informal attempt to motivate the exposition of the dynamic optimization methods that take up the remainder of the book. We use a simple dynamic macroeconomic model to introduce several im- portant concepts through numerical examples. Let y, C, and / be aggre- gate income, consumption, and investment, respectively; then a simple macroeconomic model of income determination would be

C = cy, (3.1)

C+I=y. (3.2)

The first equation is a simple consumption function, while the second is the equilibrium condition. Given an exogenous value for /, the equilibrium values of y and C can be determined as long as y does not exceed the full employment level Y, which we now define. Full-employment income Y depends on the level of capital stock, s, through a production function

y<Y=f(s). (3.3)

Suppose that investment equals the full-employment level of savings, / = (l — c)Y; then the economy will be at full employment. To obtain a growth model we formally equate the net rate of change in capital stock to in- vestment

s = I. (3.4)

Then equations (3.1)-(3.4), with y replaced by Y, constitute a simple de- scriptive growth model under full employment, with all variables evalu- ated at the same time / and the time argument suppressed for simplicity of notation.

In such a model we no longer solve for the static equilibrium values but determine the functional form of the variables in terms of t. For this we reduce (3.1)-(3.4) to a differential equation in s: (3.1)-(3.3) yield / = Y-C = (l-c)Y=(l-c)f(s)9 and with (3.4)

s = (l-c)f(s). (3.5)

This equation can be solved for s9 provided that the functional form f(s) is specified and given some initial condition, 5(0) = s0, say.

117

118 3 Introduction to dynamic optimization

3.1 Optimal borrowing

In this section we entertain the possibility of borrowing to augment the initial stock of capital. Let the size of the loan be L, with continuously compounded interest at the rate r; the amount to be repaid at the ex- piry date of the loan, say T, is LerT. We use the production function Y= sa/(\ — a ) , 0 < a < 1, and the initial condition 5(0) = s0; letting the price of capital be 1 at any time, it is now possible to begin with s0+L units of capital; L is to be optimally chosen to maximize the amount of capital available at time T, after the loan has been repaid. The differential equa- tion (3.5) takes the form

$ = (l-c)sa/(l-*)9 (3.6)

which yields

\j(l-a)s- ads = \(l-c)dt,

sl-a = (\-c)t+A. (3.7)

If L is borrowed, the initial condition is

(s0+L) l-° = A. (3.8)

Hence,

s(t) = [(l-c)t + (s0+L) l-a]l/{l-a) (3.9)

is the solution for 0 < t < T. At time T the loan is repaid and what re- mains is

[(\-c)T^(s0+L) l-a]l/{l~a)-LerT. (3.10)

Maximizing (3.10) with respect to L yields the first-order condition

[(l-c)T+(s0+L) l-ar/{l-a)(s0+L)-

(X = erT,

and raising each side to the power (1 — a ) / a , we get

(l-c)T>(s0+L) a-l = erTil-a)/a-l,

and finally (l-c)T -p/d-")

L* = erT(\-cc)/a_l

s0. (3.11)

This is assumed to be a maximum. (The exact conditions to ensure this can be derived but are intricate.) Note that L could be negative. Substituting (3.11) into (3.10) yields the value of the remaining stock after repayment:

5*(r) = e r r [5 0 +[(i-c)r] 1 / ( 1 - a ) [e , " r ( 1 - a ) / a -i]- a / ( 1 - a ) ].

3.2 Fiscal policy 119

Figure 3.1

In order to obtain the solution for t > T, we must not simply deduct L*erT

from (3.9). Instead, we must use s*(T) as the new boundary condition and determine anew the constant A in (3.7). This implies that we jump onto a new trajectory at time T, as illustrated in Figure 3.1. This empha- sizes the importance of correctly specifying boundary conditions.

3.2 Fiscal policy

Here we add a government to the preceding model and give it the power to tax; the government's tax revenue is invested, so that it is a case of forced savings. The model is

C = c(l-0)Y,

G = 6Y,

Y=sa/(l-a),

C+I+G = Y,

5 = / + G ,

(3.12)

(3.13)

(3.14)

(3.15)

(3.16)

where 0 is the tax rate, (1 — 0)Y is disposable income, and G is govern- ment revenue, or public investment, and / is private investment. For ease of calculations we take a = 0.5 and obtain the differential equation

120 3 Introduction to dynamic optimization

s = 2sl/2(l-c + c$)9 (3.17)

which is solved to yield

s(t) = [(\-c + cd)t+s l 0

/2]\ (3.18)

where 5(0) = s0. Suppose that the government's objective is to maximize total consump-

tion over some horizon [ 0 , J T ] . Its problem is then to choose 0 so as to maximize

W=\TC(t)dt = \j T2c(l-0)[(l-c + cd)t+sl0

/2]dt

= c(l-d)[(l-c + c0)T2 + 2slo /2T]. (3.19)

The optimum value of 0 is found by setting dW/dd = 0, which yields

g = 1-M^I = l- 2^rl + 1. (3.20)

2cT 2c (It is easy to verify that d2W/d62 = -2c2T2 < 0.)

We now wish to illustrate the effect of the length of the time horizon (the duration of the political mandate of the government?) on the fiscal policy parameter 0. From (3.20) it is clear that 0 increases with T, to ap- proach the value of (c — 0.5)/c. For concreteness we set s0 = 1 and c = 0.75 and let T t a k e on various values. We have, from (3.20), 6 = (T-4)/3T, Y(t) = 2[(1 - c + cd)t + sl0

/2] = t(T- 2)/T+ 2, and s = (1 - c + cO)Y = Y(T-2)/2Tafter substitution.

If T= 5, we have 0 = ^ and Y(t) = 0.6/ + 2; government policy in- creases net investment. If T= 3, we have 0 = — \ and Y(t) = t/3 + 2; gov- ernment policy decreases net investment, but this is still positive since I+G = s = \Y. For a very long horizon, T-+ <x>,d^>\ and Y(t) becomes t + 2. This is the largest rate of tax and the fastest growth rate of income. These results are understandable. Savings decreases current consumption in favor of investment in capital, which will yield larger income, hence consumption, in the future. The more future periods are taken into ac- count, the stronger is the incentive to save. The length of the planning horizon can thus have a drastic effect on policies. Finally, if T= 1, we have 6 = — 1 and Y(t) = —t + 2. In this case the incentive to save is reversed so strongly that net investment is negative (s = — Y) and the growth rate of income is negative. This raises the issue of whether it is possible to use old capital stock to generate current consumption - the jargon for it is re- versibility of investment,

3.3 Suboptimal consumption path

We revert to a model without a government sector and choose f(s) = 4s; 5(0) = 1. Then equation (3.5) is

3.4 Discounting and depreciation 121

s = 4(l-c)s, s(0) = l. (3.21)

Our objective is to maximize utility over some horizon fT0 U(C(t)) dt. Tak- ing U(C(t)) = InC(t) and T=l, we must maximize

V=[l In C(t)dt (3.22) Jo

subject to (3.21), where C(t) = 4cs(t). Solving (3.21) yields s(t) = exp(4(l -c)t) and C(t) = Acexp(4(l -c)t).

When this is substituted in (3.22), we must choose c to maximize

V=[l[\n4c + 4(l-c)t]dt9 (3.23) Jo

K=ln4c + 2 ( l - c ) . (3.24)

The maximum is at c = 0.5, and we have s*(t) = e2t, C*(t) = 2e2t, and F* = ln(2) 4-1^1.69.

Let us now pause and reflect on the "optimality" of this procedure - in particular, the fact that the propensity to consume, c, was held constant throughout the horizon. Letting c vary over time could only improve the integral maximand, if it had any effect. In that sense the above solution is suboptimal. With a variable propensity to consume, our problem would be to choose c(t) to maximize

[l\n[4c(t)s(t)]dt (3.25) Jo

subject to

s(t) = 4(l-c(t))s(t) (3.26)

and boundary conditions on s, where c(t) is an unknown function of time. Clearly, then, it is not possible to solve equation (3.26), since we do not know the form of c(t) - this is precisely what we seek. The ele- mentary calculus techniques used in this chapter are of no help in solving the problem of (3.25)-(3.26). This is indeed a simple example of what we call a control problem; the solution of such problems is the subject of the next chapter.

3.4 Discounting and depreciation in continuous-time models

The concept of an interest rate is essential to dynamic economic models. It is well understood in a financial context. Denoting the rate of interest per period by /*, we can define the present value P of an amount of dollars A to be paid T periods hence as the number of dollars that, if deposited today, would grow into $̂ 4 if left to compound interest for Tperiods.

122 3 Introduction to dynamic optimization

Formally,

P(\ + r)T=A or P = A(l + r)-T. (3.27)

This assumes that interest is compounded each period, that is, reinvested after each period. If interest is in fact compounded, say, n times during a period, we then have rt!Tsubperiods, each bearing an interest rate of r/n. Then the present value of A is

P = A(l + r/n)-"T. (3.28)

As compounding takes place more and more frequently, n increases with- out bound, and recalling that \\mx _+O0(l+x~

l)x = e,we have \\mn/r ^QOP = l i m ^ ^ o o A[(l+r/n)"/r]~rT=Ae~rT. Therefore, as interest is continuously compounded, the present value formula becomes

P = Ae~rT, (3.29)

where r is the interest rate per period and T the number of periods. In continuous-time models, that is, when the time variable is real-valued,

equation (3.29) provides a convenient way of calculating present values. Note that using the exponential discounting formula does not presume that interest is actually compounded at every instant; we need only as- sume that the calculation can be made when T is a real number and that r is the effective interest rate calculated on the basis of continuous com- pounding. To see this, note that an interest rate / compounded once per period is equivalent to an interest rate r = ln(l + /) compounded continu- ously since it follows that er = l + i and Ae~rT=A(l + i)~T.

A positive real interest rate can be observed in most economies operat- ing near full employment; hence, its existence is not often an issue. Fur- thermore, we can give it a neat theoretical justification by the argument that production processes take time and that the use of a productive as- set - or money to buy it - for some length of time must attract an eco- nomic rent. Thus, if our objective is expressed in money terms, it seems appropriate to use a discount factor. For instance, if the flow of profit at time / is ir(c(t)91), where c is a policy variable, its present value would be e~rtTr(c(t), t) and the total present value of profits over the horizon [0, T], $T0e-

rtir(c(t),t)dL We use discounting for other criteria as well. For instance, if the ob-

jective is a level of utility, we often discount it using the same exponen- tial formula, but the rate of discount is now a subjective one reflecting the individual's or the planner's relative valuation of present over future enjoyment. For example, if the level of utility at instant / is given by u(c(t), t), we discount and aggregate this much as we would a monetary reward to obtain the criterion jje~ 8 t u(c(t) 9 t)dt. Here 5 is the subjective rate of discount. Such welfare or utility criteria are used throughout the

3.4 Discounting and depreciation 123

literature on dynamic economic optimization. There are several criticisms of this. The first, a moral one, is that it is wrong for the current genera- tion to attach less value to the enjoyment of future generations and that 6 should be zero. A second criticism is that of the summation of utility lev- els across time, which is implicit in the integral formulation pf the crite- rion. Whereas it is reasonable to assume that money flows cafa be added, there is less justification for adding flows of utility. One rem^rk^ble fea- ture of the exponential form of the discount factor is that if; is the only one that ensures that the planner will be able to formulate a plan that he will actually wish to follow, even if offered the opportunity to change it at a later date. This is the issue of consistency that will be briefly discussed in Chapter 4.

For now we turn to the model of the preceding section in order to illus- trate what our intuition tells us is true: the introduction of a positive rate of discount will lead to a relatively higher consumption early in the hori- zon and a relatively lower consumption later, reflecting the planner's "im- patience." Equation (3.23) is modified by the introduction of a rate of discount 5 > 0.

V=[\ln4c + 4(l-c)t]e-8tdt Jo

= l n 4 c | - | e - 6 ' | + 4 ( l - c ) ' -* 86 52

= (ln4c) 1-e' 4 ( l - c )

5 2 l-e"d(<5 + l) (3.30)

To maximize Kin (3.30), set

dV

dc \ 8 J ^ ( l - e - 6 ( S + l)) = 0,

from which

c = 8 ( 1 - * - )

4 ( l - e - 6 ( l + 6 ) ) ' (3.31)

One can show that l i m 6 ^ 0 + c = 0.5, but positive values of 6 will yield higher c values. As an illustration, 6 = 0.5 yields c = 0.5453 - hence C(t) — 2A72eim9t instead of C*(t) = 2e2t without discounting - and we verify that C(0) = 2.172 > C*(0), while C(l) = 13.39 < 14.78 = C*(l).

Depreciation

We have seen that if an amount of money P is invested with continuous compounding at the interest rate r, it will grow into Pert after t units of

124 3 Introduction to dynamic optimization

time. This would be the case for any stock of goods growing at a constant rate r\ if at time zero there is 5(0) of it, this will grow into s(t) = s(0)ert

after / periods. Clearly, if the stock decays or depreciates instead of grow- ing, the stock will decrease as s(t) = s(0)e~mt, where m > 0 is the rate of depreciation. This is the exponential decay typical of radioactive materi- als. We can express this phenomenon as a differential equation. The time derivative is

s(t) = s(0)(-me-mt) = -ms(0)e-mt = -ms(t)9 so that

s = -ms (3.32)

represents depreciation at the constant rate m, and

s = rs (3.33)

represents growth at the constant rate r. Equation (3.33) is a simple in- stance of equation (3.5); rs is seen as the new amount generated at each instant (the "interest").

In a more general fashion s could generate f(s) at each instant and also depreciate at the constant rate m\ this would yield

s=f(s)-ms. (3.34)

These and other forms will be used throughout this book to describe the evolution of dynamic systems.

Exercises

1. Reconsider the optimal borrowing problem of Section 3.1. Suppose that you now wish to choose both the amount of the loan L and the expiry date t to maximize the present value of the capital available after repayment (i.e., the expression in (3.10) multiplied by e~rT). Derive the first-order conditions. Ob- tain values for fand L when a. = 0.5, c = 0.75, s0 = 0, and r = 10%; verify that your answers satisfy equation (3.11). Does t depend on c?

2. A bottle of wine costs $3.00 now. Its future sale value at time / is given by V(t) = 3.00 + y[7. The storage cost per unit of time is $0.10, and the prevailing interest rate is 10%. You wish to choose a date for selling the bottle that maxi- mizes the present value of profit.

Calculate the total discounted value of storage cost from time 0 to time /. Express the present value of profit in terms of t. Find the optimal year of sale and the value of profits. Redo the calculations with interest rates of 5% and 20%. How does the rate of interest affect the optimal time of sale? Comment.

3. Reconsider the fiscal policy problem of Section 3.2. Find the form of Wwhen a = f» 5o = l> a n d c = 0.75. Determine the value of 6 that maximizes W and evaluate it for various values of T. Does it increase with T7? What is the lowest upper bound on 61

Exercises 125

4. Reconsider the consumption path problem of Section 3.3. Suppose now that the planner has for horizon the time interval [0,3]. He has more latitude in the choice of the propensity to consume in the sense that he can choose one value, cu during the time interval [0,1] and another value, c2, during the time interval [1,3]. Show that the general solution to s = 4(\ — c)s is s(t) = ,4exp(4(l — c)t), where A depends on initial conditions and is valid on inter- vals where c remains constant. Let Vx = \

l 0 In C(t) dt and V2 = if In C(t) dt. Cal-

culate Vx as a function of cx\ calculate s(l); use this initial condition to deter- mine C(t) over [1,3] and calculate V2. Find cx and c2 that maximize Vx + V2. Is the value of cx you obtain different from 0.5 as in the example of Section 3.3? Can you explain why?

Attempt now a more elaborate exercise in which the horizon [0,4] is split into four intervals of length 1. On each interval the propensity to consume is constant. The task is to choose cu c2, c3, and c4 to maximize the total utility (logarithm) of consumption on the whole interval. Let sx(t)9 ...,s4(0 be the time paths of capital over the four intervals. Determine their exact forms by using the boundary conditions. (The first one depends only on cx\ the last one depends on q, c2, c3, and c4.) Calculate Vt = J/'_,lnC(/)^, / = 1,2,3,4, and find the values of cXic2,c3, and c4 that maximize S?=i Vr Can you detect a pat- tern in the c values? Can you rationalize it?

5. Modify the basic growth model adding depreciation as s = I-ms. Using f(s)- 4s, express C(t) when the propensity to consume is constant. Find c that maxi- mizes V= {J, l n C ( 0 ^ - How is the value of consumption affected by the rate of depreciation?

CHAPTER 4

The maximum principle

In this chapter we present a first account of optimal control theory. The maximum principle is the central result of the theory. (It was originally developed by Pontryagin and his associates; see Pontryagin et al., 1962.) To help the reader become thoroughly acquainted with it, we proceed with the analysis of a simple case, without paying undue attention to some technical regularity conditions. (These and other matters will be dealt with in Chapter 6.)

4.1 A simple control problem

Consider a dynamic system - for instance, a moving spaceship or an econ- omy. Some variables can be identified that describe the state of the system: they are called state variables - for instance, the distance of the space- ship from earth or the stock of goods present in the economy. The rate of change over time in the value of a state variable may depend on the value of that variable, time itself, or some other variables, which can be con- trolled at any time by the operator of the system. These other variables are called control variables - for instance, the pitch of the motor or the flow of goods consumed at any instant. The equations describing the rate of change in the state variables are usually differential equations, as dis- cussed in Chapter 2. Once values are chosen for the control variables (at each date), the rates of change in the values of the state variables are thus determined at any time, and given the initial value for the state variables, so are all future values. For instance, the pitch of the spaceship engine determines its speed and hence its distance from earth once its initial posi- tion is known; the consumption path of the economy determines net in- vestment and hence capital stock accumulation over time. The object of controlling a system is usually to contribute to a given objective. For in- stance, the values of all the relevant variables determine the fuel con- sumption of the spaceship at any time, and the objective is to minimize total fuel consumption so that some destination is reached within a given time period. Similarly, the values of consumption, capital stock, and time may determine the welfare of the community at each instant, and the

127

128 4 The maximum principle

objective is to maximize total welfare over a fixed time horizoiv^iven specific values of the stock at the beginning and the end.

A salient feature of optimal control problems that emerges from the foregoing discussion is that it is necessary to choose a value for the con- trol variable (or variables) at each instant; when, as is usually the case, time is taken to be real-valued, there are infinitely many values of the control to be chosen. Another way of putting this is to say that we must find a functional form over some time interval, which the control variable is to follow. Thus, the problem appears to be far more difficult than those encountered in static optimization. Fortunately, the maximum principle provides a framework that makes these problems amenable to solution.

We now formally define a simple optimal control problem and state the maximum principle for it. For all t, find c(t) that maximizes

V=\Tv(s(t)9c(t),t)dt (4.1) Jo

subject to

s=f(s(t),c(t),t) (4.2) and

5(0) =s0, s(T)=sT, (4.3)

where s(t) is the state variable, s(t) is the rate of change of the state vari- able with respect to time, c(t) is the control variable, and t denotes the date; the interval [0, T] is the planning horizon, s0 and sT are the val- ues the state variable must take on at the boundaries. The values of T, s0, and sT are exogenously specified. (For simplicity we assume throughout this chapter that c(t) is unconstrained; this assumption will be relaxed in Chapter 6.)

If a functional form is chosen for c(t) over [0, T], the differential equa- tion (4.2) together with the boundary conditions (4.3) will determine s(t) uniquely over [0,T]; this in turn will yield a value for the integral K i n (4.1). The problem is to choose c(t) to yield the largest possible V. In this chapter we assume that an optimal control exists, is unique, and is differ- entiable with respect to time. This is a very restrictive assumption; see the final remark of Section 4.6 for details.

The necessary conditions that constitute the maximum principle are most conveniently stated after some auxiliary variables, akin to multi- pliers, have been introduced; with the state variable s(t) is associated an auxiliary variable called a costate variable denoted by ir(t). We define, at each instant, a new function called a Hamiltonian, similar to a La- grangean. The Hamiltonian for the problem defined in (4.1)-(4.3) is

H(s(t), c(t), TT(0, t) s V(s(t), c(t), t) + *(t)f(s(t), c(t), t). (4.4)

4.2 The maximum principle in discrete time 129

Theorem 4.1.1: the maximum principle. An optimal solution to the above problem is a triplet (s(t), c(t), ir(t)) and must satisfy the following con- ditions:

(i) c(t) maximizes H(s(t), c(t), ir(t), t), that is,

bH =0; (4.5) dc(t) and

(ii) the state and costate variables satisfy a pair of differential equa- tions,

m=jkr < 4 - 6 >

with boundary conditions as in (4.3).

Using the definition of the Hamiltonian in (4.4), equations (4.5)-(4.7) can be expanded as

dv df + ir(t)-^— = 09 (4.8) dc(t) dc(t)

Ht)=f(s(t),c(t),t), (4.9)

dv df ds(t) ds(t)

with 5(0) = s0 and s(T) = sT. Therefore, the optimal triplet is a solution of equations (4.8)-(4.10).

These consist of two differential equations and an algebraic equation, often called the first-order condition since it optimally selects the control. Before attempting to apply the maximum principle to a specific problem, we will show how easy it is to derive it in a discrete analog of the problem presented in this section.

4.2 Derivation of the maximum principle in discrete time

For a thorough grasp of important results such as the maximum principle, it is essential to work through and assimilate a heuristic proof. We present a proof for the continuous-time case in Section 4.6. For now we are con- tent to derive a proof in the simpler case where time is a discrete variable: the horizon consists of Tperiods, t = 1,2,..., T, instead of a continuous

130 4 The maximum principle

interval, as in the preceding section. Thus, it is a constrained maximum problem with a special recursive structure.

An optimal control problem in discrete time

Find c ( l ) , c ( 2 ) , ...9c(T) that maximize

V=Z v(s(t),c(t)) (4.11) t = \

subject to

s(t + \)-s(t)=f(s(t)9c(t))9 t = \929...9T9 (4.12)

5(1) = *!, s(T+l)=sT+l. (4.13) This is the discrete analog of the problem formulated in (4.1)-(4.3). The

difference equation (4.12) that describes how the state variable changes from one period to the next replaces the differential equation (4.2) that described the change at each instant. The independent time argument has been suppressed here to simplify the notation; its inclusion does not affect this derivation. The symbols s(t) and c(t) denote, respectively, the values of the state variable and control variable at the beginning of period t; thus, specifying a value for s(T+1) is the same as requiring the state variable to take on this value at the end of period T. We are free to choose any values for c(t) and s(t) to maximize (4.11) as long as the constraints (4.12) and (4.13) are satisfied. To this end we substitute (4.13) into (4.12) and assign a multiplier to each constraint. The Lagrangean of the problem is

L = v(suc(l)) + v(s(2)9c(2))+->+v(s(t),c(t))+-~

+ v(s(T-l),c(T-l)) + v(s(T),c(T))

+ x ( l ) [ 5 1 + / ( j b c ( l ) ) - 5 ( 2 ) ] + x ( 2 ) [ 5 ( 2 ) + / ( 5 ( 2 ) , c ( 2 ) ) - j ( 3 ) ]

+ .-+ic{t-l)[s(t-l)+f(s(t-l)9c(t-l))-s(t)]

+ *(t)[s(t)+f(s(t)9c(t))s(t + l)] + -

+ 7c(T-2)[s(T-2)+f(s(T-2)9c{T-2))-s(T-l)]

+ ic(T-l)[s(T-l)+f(s(T-l)9c(T-l))-s(T)]

+ w(T)[s(T)+f(s(T)9c(T))-sT+l].

The first-order necessary conditions are obtained by partially differenti- ating with respect to all free c(t), s(t)9 and ir(t):

*>c<n + ir(l)/c<i) = 0,

(4.14) Vc{T-l) + *(T-l)fC(T-\) = 0>

vc(T) + ir(T)fc{T) = 0;

dc(t)

(4.15)

4.2 The maximum principle in discrete time 131

« t ( 2 ) - » ( l ) + »(2) + x ( 2 ) / J ( 2 ) = 0,

vsil)—ir(t-l) + ir(t) + ir(t)fs(l) = 0,

i 7 J ( r _ 1 ) - T ( r - 2 ) + T ( 7 ' - l ) + i r ( r - l ) / , ( r _ , ) = 0,

5 , + / ( 5 „ C ( l ) ) - 5 ( 2 ) = 0 ,

5 ( 0 + / ( 5 ( 0 , c ( / » - s ( f +1) = 0, (4.16)

s(T)+f(s(T),c(T))-sT+l = 0.

These three sets of equations can be written more compactly as

«c(o + »(0/c(/) = 0, / = l , 2 7; (4.17)

*V)-irV-l) = -vs{l)—ir(t)fm, t = 2,3 T, (4.18)

s(t + l)-s(t)=f(s(t),c(t)), t = \,2,-..,T, (4.19) with

5(1) = 5, and 5 ( 7 + 1 ) =sT+i.

If we define a new function,

H(s(t), c(t), T ( 0 ) - y ( 5 ( 0 , c ( / » + T ( 0 / ( * ( 0 , c ( 0 ) , (4-20)

then these necessary conditions can be expressed as

dH = 0, t = \,2,...,T, (4.21)

T ( / ) - T < f - l ) = - - ^ - , / = 2 , 3 , . . . , r , (4.22) ds(t)

s(t + \)-s(t) = -^-, t = \,2,-,T. (4.23)

It is obvious that the expressions in (4.20)-(4.23) are the discrete counter- parts of the Hamiltonian and maximum principle of (4.4) and (4.5)-(4.7). It is instructive to apply our result to a simple example.

Example 4.2.1. Find c(t) that maximizes 3

F = 2 lnc(0 r = l

subject to

s(t + l)-s(t) = 0.ls(t)-c{t), t = 1,2,3,

5(1) = 1, 5(4) = 1.21.

132 4 The maximum principle

We form the Hamiltonian

H(s(t),c(t),ir(t)) = \nc(t) + Tr(t)[0.1s(t)-c(t)].

Then (4.21) yields

- L — x ( O = 0, f = l , 2 , 3 ; (4.24) c ( 0

(4.22) yields

7 r ( O - 7 r ( / - l ) = -0.l7r(O, f = 2 , 3 ; (4.25)

and (4.23) yields

s(t + l)-s(t) = 0.1s(t)-c(t)9 f = 1,2,3. (4.26)

We can use (4.24) to eliminate the costate variables to get

c(f) = U c ( f - l ) , f = 2 , 3 . (4.27)

Using (4.26), (4.27), and the initial condition, we can proceed recursively:

5(2) = ( l . l ) ( l ) - c ( l ) ,

5(3) = ( l . l ) 5 ( 2 ) - c ( 2 ) = 1 . 2 1 - ( l . l ) c ( l ) - ( l . l ) c ( l ) ,

5(4) = ( l . l ) s ( 3 ) - c ( 3 ) = 1.331 - 2 ( 1 . 2 1 ) c ( l ) - ( 1 . 2 1 ) c ( l ) ,

5(4) = 1.331-3(1.21)c(l).

In order to meet the terminal condition 5(4) = 1.21 we must therefore choose c(l) = 0.0333 Substituting it into the above equations we easily obtain all values of 5 ( 0 , c(t), and ir(t):

c(l) = 0.0333, c(2) = 0.0367, c(3) = 0.0403;

TT(1) = 30, TT(2) = 27.27, TT(3) = 24.79;

5(2) = 1.0667, 5(3) = 1.1367, 5(4) = 1.21.

It is interesting that in equation (4.28) we obtained the relationship be- tween the terminal value of the state variable and the initial value of the controls when an optimal path is followed. Hence, there is a different optimal path for each terminal condition. It is worth remarking that for some terminal conditions no feasible solution exists: since the control must remain positive, we get from (4.28) 5(4) < 1.331.

The procedure followed in solving this example by means of the dis- crete maximum principle involved several steps: (i) eliminating one of the variables (the costate in this case) by making use of the first-order condi- tion; (ii) solving two difference equations (one for the state and one for the control); (iir) making use of the boundary conditions to determine the initial value of the control; (iv) substituting this value in the solutions of

4.3 Numerical solution in continuous time 133

the difference equations and the first-order condition to obtain the opti- mal values of all variables.

A similar solution procedure will be followed when we apply the maxi- mum principle in continuous time in the next section.

4.3 Numerical solution of an optimal control problem in continuous time

In Section 3.3 we presented a simple dynamic choice model. In that prob- lem the optimal policy was determined by choosing the value of the con- sumption/output ratio: it was a parameter that remained constant for the whole horizon. Although it was pointed out that this was unduly restric- tive, more flexibility was shown to yield an insoluble problem.

In this section we reconsider this problem. It will be possible to choose a consumption/output ratio that varies optimally over time through the use of the maximum principle.

4.3.1 Optimal consumption

The problem, set up in an optimal control format, is to find c(t) that maximizes

V=[lln[c(t)4s(t)]dt (4.29) Jo

subject to

s(t) = 4s(t)(l-c(t)) (4.30) with

5(0) = 1, s ( l ) = e2. (4.31)

The reason (4.31) includes a terminal condition is to make this problem comparable with that of Section 3.3, in which this was the terminal value of the stock.

The Hamiltonian of the problem is

/ / ( 5 ( / ) , c ( 0 , 7 r ( 0 ) = l n 4 + l n c ( 0 + l n 5 ( 0 + 7 r ( 0 [ 4 5 ( 0 ( l - c ( 0 ) ] ,

and applying the maximum principle yields the following first-order con- dition and two differential equations:

dH 1 dc(t) c{t)

-4ir(t)s(t) = 0, (4.32)

* ^ - ^ - w r 4 i l - m ) T i t ) ' (4-33) s(t) = - ^ = 4s(t)(l-c(t)). (4.34)

134 4 The maximum principle

For notational simplicity we suppress the time argument while deriv- ing the solution. From (4.32), c = 1/(4^5), and substituting it in the other equations we obtain a pair of differential equations in -K and s:

7T = 4 7 T ( 1 — )

s \ 4TTSJ

and

These simplify to

7T = — 4 7 T ,

s = 4s-(l/ir).

The first differential equation immediately yields

7r(/) = 7r(0)e-4', (4.35)

which we substitute into the second equation,

s = 4s-e4t/Tr(0).

This is most easily solved by passing all s terms on the left-hand side and multiplying through by the integrating factor e~4t:

se-4t-4se-4t=-\/>ir(0).

The left-hand side is the derivative of se~4\ and integration yields the general solution

se-4t=-t/ir(0)+A.

We use (4.31) to determine 7r(0) and A:

5(0) = 1; hence, 1=^4;

5(1) = e2; hence, e " 2 = - l / 7 r 0 + l;

thus, 7r0 —1.156.

We have the solution

5 ( 0 = e 4 ' - 0 . 8 6 5 t e 4 ' . (4.36)

Substituting (4.35) and (4.36) into (4.32), we obtain

c(')=4^b*7- (4-37) Therefore, it is optimal for the consumption/output ratio to increase

in the manner described by (4.37). In order to show convincingly that this

4.3 Numerical solution in continuous time 135

solution is preferred to that of Section 3.3, we calculate the optimal value of V:

V= V \n(4c(t)s(t))dt = \l In -L- dt (by (4.32)) JO JO 7 r ( 0

= (1[ln(0.865) + 4 n ^ = Uln0.865+2f 2]J). Jo

Therefore, V—1.855, which is larger than V= 1.69 obtained previously. Before leaving this example it is useful to reflect on one hidden assump-

tion. Substituting t = 1 into (4.37) we find that at that time the consump- tion/output ratio is 1.662; therefore, for some time the amount consumed exceeds the amount produced, s is negative, and this implicitly assumes that it is possible to eat into the capital stock. Our purpose is not to argue for or against this assumption, as already mentioned in Section 3.2, but to note its importance. Note that s could become negative because the control was free of all constraints. Had we wished to restrict it to values between 0 and 1, we would have needed more general results; the con- strained control problem and other extensions are the subject of Chap- ter 6.

To conclude this section we shall solve a slightly more general version of this model.

4.3.2 Optimal consumption with discounting

Here the problem is to find c(t) that maximizes

V=[ e~8t In c(t)dt (4.38) Jo

subject to

s(t) = rs(t)-c(t), (4.39)

s(0) = s0, s(T) = sT. (4.40)

In this formulation c(t) denotes the consumption flow itself and not the consumption/output ratio. Moreover, the logarithmic utility function is discounted at the rate d>0. The rate of interest (r) and the boundary values of the state variable (s0 and sT) are exogenously specified.

The Hamiltonian of the problem is

H(s(t)9c(t)9t) = e- 8t\nc(t) + Tr(t)[rs(t)-c(t)].

Applying the maximum principle yields

dH _ „ 1 -dt = e-dl-—-<ir(t) = 0, (4.41)

dc(t) c(t)

136 4 The maximum principle

*{t) = ~~^(i)=~rir(t)' (4-42)

m = ^(t)=rS{t)~C(t)' (4A3)

When the time argument is omitted, these necessary conditions appear less cumbersome:

e~6' = 7rc,

7T = — T 7 T ,

s = rs — c.

It is obvious how to proceed: we can use the second equation to get the general solution for 7r, substitute it in the first equation to get c, and sub- stitute c into the last equation to get the general solution for s. The bound- ary conditions will determine the two constants of integration:

ir(t) = ir(0)e-rt, (4.44)

c(t) = [Tr(0)]-leir-d)t, (4.45)

s-sr = -[Tr(0)]-le{r-8)t,

se-rt-re-'ts = -[Tr(0)rle-8t,

se-rt = e-dt/6>ir(0)+A,

s(t) = e{r-8)t/8ir(0) +Aert. (4.46)

Substituting (4.40) into (4.46), we get

s(0) = s0; hence, s0 = A + - r - - — ;

e(r-6)T s(T) = sT; hence, sT = Ae

rT+ . O7T(0)

Solving these two equations yields

1 s0-sTe~ rT

TT(0) (l-e-6T)/69 (4.47)

Since c(t) has the sign of ir(t), which is the same as that of 7r(0), we must require that s0-sTe~

rT>09 or sT<s0e rT

9 for otherwise c(t) would be negative and the logarithm undefined. Upon reflection this is only the re- quirement that the final value of the stock not exceed the level to which the initial stock would have naturally grown had its growth not been cur- tailed by consumption.

4.4 Phase diagram analysis 137

From (4.45) we see that the flow of consumption varies exponentially; whether it increases or decreases depends on the relative values of r and 6. If the rate of interest r exceeds the consumer's own rate of discount 6, she tends to postpone consumption; thus, her consumption flow increases over time. From (4.48) it is obvious that the sign of A cannot be ascer- tained in general; hence, the path of s(t) may or may not be monotone. Given specific values for the parameters T, /*, 6, s0, and sT, the paths of all variables can be precisely determined. The reader is invited to verify that the derivative of c(0) with respect to 5 has the sign of [e~6T— (1 + 5T)] > 0, confirming that a higher subjective discount rate induces higher consump- tion at the beginning of the horizon, just as in the example of Section 3.4.

4.4 Phase diagram analysis of optimal control problems

In the simple type of optimal control problem that is the subject of this chapter, the maximum principle yields a first-order condition and a pair of differential equations. Nevertheless, these are often difficult to solve analytically. If all functional forms and relevant parameter values were specified, it would be possible to use numerical methods to derive the so- lution. This may be useful in physics or engineering; however, since our ultimate purpose in using control theory is to gain insight into the dy- namic behavior of economic models, we often deal with problems involv- ing unspecified functional forms. For example, often for the sake of gen- erality, we do not wish to specify a particular form of the utility function (e.g., logarithmic) or a particular form of the production function (e.g., Cobb-Douglas). In such cases the explicit solution of the differential equa- tions is impossible. The best we can hope for is a qualitative characteriza- tion of the optimal solution, as was often the case in static optimization problems in economics. This seems a formidable task. Fortunately, we have just the device needed: the representation of the solution on a phase diagram as described in Section 2.5. We will be able to partition the phase space into regions in which we know whether the variables increase or decrease over time. Further analysis will yield restrictions on the shape of the trajectories that are candidates for the optimal path. As with most qualitative tools this is not a perfect device. In many cases it will require some ingenuity to pinpoint exactly the optimal trajectory. Nonetheless, it provides a structure for detailed qualitative analysis.

Initially we shall illustrate the technique with a numerical example.

Example 4.4.1. Consider the problem of finding c(t) to maximize

V=[T e-° 05t [In c(t)]dt

Jo

138 4 The maximum principle

subject to

s(t) = 2[s(t)]05-c(t),

5(0) = s0, s(T)=sT.

We may interpret c(t) as the consumption flow and s(t) as the stock of capital; 2[s(t)]05 is the output produced with capital stock s(t)9 and there is no depreciation.

The Hamiltonian of the problem is

H(s(t), c(t), x(f), t) = e-°05t In c(t) + ir(t)[2(s(t))0'5 - c(t)],

and applying the maximum principle yields

dH

oc{t)

* ( » = - ^ T = - T ( » [ * ( / ) r 0 - 5 , ds(t)

air(/)

Skipping the time argument we must solve

e-0.05lc-l = ^

i = -Trs-0\

s = 2s05-c.

Let us use the first-order condition (4.49) to eliminate c pair of differential equations

* = - ™ - ° - 5 , s = 2s°-5-e-°05'ir-1.

(4.49)

(4.50)

(4.51)

and obtain the

(4.52)

(4.53)

These differential equations involve not only -K and s but also an inde- pendent exponential time trend. This makes it impossible to draw a phase diagram in the (s, T) space because the locus of points at which s = 0 is not well defined, since it depends on t. There is a way out of this difficulty and we shall present it later in this section.

For now let us look at an alternative pair of differential equations, one in 5* and one in c. Although we do not have a differential equation in c, we can obtain one by totally differentiating the first-order condition (4.49) with respect to time:

-0.05e-°05tc-1 -e-°-05tc-2c = TT

= -7r5-°-5 by (4.50) = _e-o.o5/c-i5-o.5 b y ( 4 # 4 9 ) .

4.4 Phase diagram analysis 139

Multiplying through by e005tc2 yields

- 0 . 0 5 c - c = -cs~0-5,

from which we obtain

c = c(s~05-0.05). (4.54)

Equation (4.54) is a differential equation in c involving c and s only; to- gether with (4.51) they can be analyzed with a phase diagram in the (s, c) space. The procedure used to derive (4.54) is used so often that it is worth spelling it out. First, we totally differentiated the first-order condition and obtained an equation involving a c term and a ic term. We eliminated the latter using the differential equation in IT. This in turn introduced a 7r term, which was eliminated by using the original first-order condition again. Finally, some algebraic manipulation yielded a simpler form. This procedure is often useful in solving simple problems, but it sometimes fails. In that event we will attempt to devise another way out of the dif- ficulty.

We follow the procedure outlined in Section 2.5.2. The main task is to use equations (4.51) and (4.54) to partition to (s, c) space into regions in which the respective signs of s and c are known. The first step is to ob- tain the loci of points where s = 0 and c = 0. The curve for s = 2s05 — c = 0 is the graph of the function c = 2s0'5. This is a concave and increasing function of s. The curve goes through the origin, where it has an infi- nitely large slope. The other equation, c = c(s~0'5 — 0.05) = 0, defines two straight lines: c = 0 and s~0-5 = 0.05, or s = 400. These "critical loci" are drawn in Figure 4.1. They define four regions (or isosectors), which are labeled I-IV. The expressions defining s and c are continuous in the posi- tive orthant (check (4.51) and (4.54)); therefore, s and c can change sign only when we cross over one of the above critical loci. In order to ascer- tain the sign of s and c in any one of the four regions, it is sufficient to evaluate these signs at an arbitrary point of the region.

In region I, c is less than 2s0-5; hence, s>0 by (4.51). Since c>0 and s<400, (4.54) implies c>0. In region II we still have c<2s0-5; hence, s > 0, and now that s > 400 we have c < 0. Region III is above the c = 2s0,5

curve, so that s < 0 and s > 400 implies c < 0. In region IV, s < 0 and c > 0. We could have come to this conclusion by noting that at any point below the c = 2s0'5 curve s > 0, and that c < 0 to the right of the s = 400 line. The signs of s and c are represented in each region of Figure 4.1 by two small perpendicular arrows. The horizontal arrow indicates an increase or a decrease in 5, while the vertical one refers to changes in c, since the s axis is horizontal and the c axis vertical. Since optimal trajectories must obey equations (4.51) and (4.54), they must follow the direction indicated by the pair of arrows in each region. A curve indicates the direction of

140 4 The maximum principle

400 c = 0

Figure 4.1

admissible trajectories in each region, with the arrows on the curve de- noting movement as time passes.

We must gather another piece of information before we can draw the general shape of trajectories. The slope of trajectories in the (s, c) space can be obtained from the relationship dc/ds = (dc/dt)(dt/ds) — c/s. Hence, when a trajectory goes through a locus where c = 0, it has slope zero, and when it goes through a locus where s = 0, it has an infinite slope. This in- formation, along with the direction of trajectories in each region, allows us to draw the shape of trajectories when crossing a critical locus from one region to the next; this is done in Figure 4.1 in all four cases. The gen- eral solution to (4.51) and (4.54) is a family of trajectories. The knowl- edge of the boundary conditions will determine the specific solution. We did not assign specific values to s0 and sT in order to discuss the influence of the boundary conditions on the choice of the optimal trajectory. To this end we represent a few possible trajectories in Figure 4.2. The posi- tions of trajectories relative to the equilibrium point E (c = 40, 5 = 400)

4.4 Phase diagram analysis 141

0 sT s0 400 sT

Figure 4.2

are of interest. In region I the trajectories go up and to the right; if one reaches the s = 0 locus, it turns left; if one reaches the c = 0 locus, it turns down. We know from the theory of differential equations that trajec- tories cover the whole space; therefore, there is one trajectory in region I that reaches the equilibrium point E. We refer to it as a stable path. Note that because s and c become arbitrarily small near E, it would take an in- finite amount of time to reach equilibrium.

Similar reasoning reveals the existence of a downward stable path in region III, while regions II and IV each possess an unstable path leading away from E, identifying E as a saddle point. We can check the saddle- point property locally by linearizing the system (4.51), (4.54) around E. The matrix of coefficients obtained is

f" 0.05 - 1 ] -0.0025 0 *

142 4 The maximum principle

which has a negative determinant corresponding to a saddle point, as noted at the end of Section 2.4. The arms of the saddle point and some other trajectories are drawn in Figure 4.2.

Suppose now that 5(0) =s0 and s(T) =sT, as indicated in Figure 4.2. Vertical dashed lines have been drawn through those values to emphasize the fact that although boundary values are specified for the state variable, all values for the control are to be optimally chosen. Thus, the optimal trajectory may begin anywhere along the s = s0 line and must end some- where on the s = sT line. The optimal path will be within regions I and IV. Indeed, there is no way to reach region III beginning at s0, and were the path to enter region II, it would never reach sT. The optimal path may be wholly in region IV, as are (i) and (ii), or begin in I, as does (iii). Since our objective is to maximize the integral of [lnc(t)]e~005t, it would seem that we should select the higher trajectory. However, to travel the length of a path such as (i) takes some fixed amount of time. Since the time hori- zon has been specified exogenously as [0, T], we must select the path that will go from s0 to sT in exactly T units of time. The further this path is from the s = 0 locus, the larger s is and the faster s changes. Hence, if T is small, the optimal path will be a high path, such as (i); the larger T9 the further down we go until we reach path (ii). This is, of all paths wholly within region IV, the one that takes the longest time to go from s0 to sT. If Tis still larger, we must select as optimal a path such as (iii) that begins in region I. No matter how large T is, we shall always be able to select an appropriate path, because we can choose one that goes arbitrarily close to E before turning left into region IV, and in the neighborhood of E, movement along the path would be very slow. The effect of Ton the choice of an optimal path can be explained in simple terms. Recall that c is con- sumption and s a stock of capital. You have been given T periods to eat into your capital from s0 to sT. The shorter the time you have, the higher the rate of consumption you will be able to afford. If you must plan for a long enough time, you will find it optimal to begin with a level of con- sumption low enough that you accumulate capital initially. In any event, consumption increases monotonically through time.

Other boundary conditions would yield different paths, based on anal- yses similar to the one just presented. To take one more instance, sup- pose that the initial stock is still s0 but the terminal stock must now be sT. We would follow a path such as (iv). The length of the planning horizon would determine exactly which one, since the higher the path, the closer it is to the s = 0 locus, around which s increases very slowly. Note that in this case s always increases throughout and c goes through a peak. Note also that if s0 and Tare small enough and sT large enough so that the differential equation s = 2s0-5 has no solution satisfying these boundary

4.4 Phase diagram analysis 143

conditions, then the control problem has no feasible solution, let alone an optimal one. If there is no way to satisfy the boundary requirements even by starving (c = 0), there can be no optimal way to solve the problem.

This completes the analysis of the (s,c) phase diagram for this prob- lem. In some cases it may be useful, necessary, or the only feasible choice to conduct the phase diagrammatic analysis in the (state, costate) plane. Recall that we attempted this task but encountered difficulties because of an independent time term in the costate differential equation. We now show how to deal with this problem by a change of variable. We had to deal with equations (4.52) and (4.53):

7r = -7T5-°-5, (4.52)

s = 2s°-5-e-°-05tir-\ (4.53) Define

t(t) = e005tir(t). (4.55)

Taking the time derivative, we have

i = 0.05e005tir + e°-05tir.

Substituting (4.52) and (4.55) into the above equation yields

rP = 0.05t-e°-05tTrs-°-5

= 0 . 0 5 ^ - ^ - ° ' 5 .

We now have a pair of differential equations in \j/ and s that are autono- mous, that is, free of an independent time argument:

^ = ^ ( 0 . 0 5 - 5 - ° 5 ) , (4.56)

5 = 2 s ° ' 5 - i / ' - 1 . (4.57)

There is a way to obtain these equations directly; it will be explained at the end of this section.

The phase diagram in the (s, 0) space can be constructed in the same manner as the preceding one. The critical loci are

^ = ^ ( 0 . 0 5 - 5 - ° 5 ) = 0, or ^ = 0 and 5 = 400,

5 = 2s°-5-i/'-1 = 0, or ^ = 0.5s-°-5.

The first locus is a pair of straight lines, while the second one is a rec- tangular hyperbola with both axes as asymptotes; these are drawn in Fig- ure 4.3. We shall again restrict our attention to the positive orthant since it is obvious from the optimality condition (4.49) that -K (hence \[/) cannot be negative.

From (4.57) we see that s < 0 below the hyperbola and s > 0 above it. Equation (4.56) implies that ^ < 0 to the left of s = 400 and \j/ > 0 to the

144 4 The maximum principle

0.025 h \

Figure 4.3

right. These signs together with the knowledge that the trajectories have a zero slope when crossing the \j/ = 0 locus and an infinite slope when cross- ing the s = 0 locus enable us to draw the general shape of the trajectories in Figure 4.3. There are again four regions in the phase space. We have numbered them in a way that is consistent with the numbering in Figure 4.2. In fact it is very important to grasp the correspondence between the two diagrams. The first-order condition (4.49) rewritten after the change of variable is c" 1 = \t. Therefore, whereas the horizontal axes in Figures 4.2 and 4.3 bear the same variable, the vertical axes bear variables that are the reciprocals of one another. Thus, region I of Figure 4.3, where s increases and ^ decreases, is indeed the mirror image of region I in Fig- ure 4.2, where both c and s increase. Similar correspondences apply be- tween other regions, as can readily be checked. With boundary values s0 and sT, a short planning horizon will lead to an optimal path such as (i). The most time-consuming monotone path is (ii). If more time is avail- able, a path such as (iii) would be selected. If boundary values were s0 and sT, a path such as (iv) would be chosen. Paths (i)-(iv) in Figure 4.3 correspond to like-numbered paths in Figure 4.2.

4.4 Phase diagram analysis 145

This completes our analysis by phase diagrams of Example 4.4.1. We have been able to obtain a fairly precise qualitative description of the opti- mal solution. Note that if a numerical solution were required, the insights gained from the phase diagram analysis would be very useful in tracking it down. Our next task in this section is to introduce a far more general growth model from which the previous examples were drawn. It will be shown that the phase diagram analysis is no more complicated in the gen- eral case than it was in the example. Hence, the technique is more power- ful than this example and those of Section 2.5 may have led us to believe.

Find c(t) to maximize

V=[Te~8tu(c(t))dt Jo

subject to

s(t) = F(s(t))-c(t),

5(0) = 5 0 , S(T)=ST,

where we assume w ' > 0 , w " < 0 , w'(0) = oo, F(0) = 0, F ' > 0 , F " < 0 , and F ' ( + o o ) < S < F ' ( 0 ) . The assumptions that the utility function u(-) and the production function F(-) are increasing and concave are standard ones. The additional assumption that marginal utility at the origin is in- finite guarantees that consumption is always positive along an optimal path. We shall see that the condition F'(+<x>) < 5 < F'(0) is necessary and sufficient to guarantee the existence of an equilibrium.

The Hamiltonian of the problem is

H(s, c, 7T, /) = e~btu(c) + TT[F(S) -C],

and applying the maximum principle yields

^ = e - V ( c ) - 7 r = 0, (4.58) oc

* = - ^ = -irF'(s)9 (4.59) OS

s = ^=F(s)-c. (4.60) OTT

To obtain a differential equation in c we totally differentiate (4.58) with respect to time:

-be-btu'(c) + e-btu"(c)c = TT

= —KF'(S) by (4.59)

= -e-btu\c)F'(s) by (4.58).

146 4 The maximum principle

Figure 4.4

Multiplying through by eht/u"(c) yields

'=W)[8-F'{S)]- (4-61) Equations (4.60) and (4.61) can be used to draw a phase diagram.

The 5 = 0 locus is c = F{s), an increasing concave curve going through the origin. The c = 0 locus is a straight line, s=s*, where s* is the number defined by F'(s*) = 5. The assumption F'(+oo)< 6 <F'(0) ensures that s* exists. These loci are drawn in Figure 4.4. The observations that s > 0 below the 5 = 0 locus and s < 0 above it, and that c is negative to the right of s* and positive to the left of it, enable us to draw in each region arrows indicating the directions followed by trajectories. Moreover, as the tra- jectories have slope zero and infinity, respectively, when crossing a c = 0 and an s = 0 locus, the general shapes can be drawn. We note the existence

4.4 Phase diagram analysis 147

of two stable paths in regions I and III and two unstable paths in regions II and IV. The equilibrium value of s, namely, s*, is determined by find- ing the point on the 5 = 0 locus that has slope 6. The similarities between Figures 4.4 and 4.2 are so obvious as to require no comments; indeed, we need not have drawn a new diagram, except that s* now replaces 400. We can prove the existence of a local saddle point by linearizing (4.60) and (4.61) around the equilibrium. The matrix of coefficients is

T 5 - 1 ] [-F"(s*)u'(c*)/u"(c*) oj*

Its determinant is negative, confirming that the equilibrium is a saddle point. We now simply derive the differential equations that we would need for a diagram in the (s, \p) space. Define

\ls(t) = e8tir(t). (4.62)

Differentiating (4.62) with respect to time yields

yP = 5edtir + e8tic = 5edtTr-edtirF'(s)

(by (4.59)). Hence.

^ ( 6 - F ' ( s ) ) . (4.63)

This equation is the general form corresponding to (4.56) and defines the \j/ = 0 locus as two straight lines of equations ^ = 0 and 5 = 5*; it can be used to draw a diagram in the (5, yp) space. Note, however, that the other equation, s = F(s) — c, includes c. In Example 4.4.1 we eliminated c with the aid of the first-order condition; this, in the present case, takes the form

w'(c) = 0, (4.64)

which is obtained from (4.58) and (4.62). We had assumed u"< 0; there- fore, u' is strictly decreasing and possesses an inverse function, which is also decreasing. Hence, the first-order condition can be expressed as

c = c(i/0, dc/d\/s<0, (4.65)

where c(-) is the inverse function of w'(-). Substitution into (4.60) yields

s = F(s)-c(i), (4.66)

which can be used in conjunction with (4.63) to draw the phase diagram in the (5, \j/) space. The derivation of the s = 0 locus is the only new as- pect of the procedure. Its equation is F(s) = c(^). Since c is the inverse of the function u\ applying u' to each side we get u'(F(s)) = u'(c(\//)) = xp. This gives \p as a function of s. Note that since u' is decreasing and F is

148 4 The maximum principle

i// = 0

Figure 4.5

increasing, ^ is here a decreasing function of s. Furthermore, we know that both u' and F a r e positively valued and defined for all positive values of their arguments. This implies that the graph of \[/ = u'(F(s)) is down- ward sloping and intersects the line s = s* but not the line \[/ = 0. Hence, this point of intersection is the unique equilibrium. Points above the graph of \l/ = u'(F(s)) are such that yp>u'(F(s))\ hence, c(\l/)<F(s) (because c(-) is a decreasing function of \j/)\ therefore, s > 0 at such points. Re- calling that F' is decreasing since F " < 0, (4.63) implies that \j/ > 0 at points to the right of the s = s* line. Figure 4.5 is the phase diagram. The only feature of the phase diagram in Figure 4.3 which is not necessarily pres- ent in Figure 4.5 is that the s = 0 locus need not be an asymptote to both axes. This would be guaranteed here by the special assumptions F(0) = 0, F(oo) = oo, w'(oo) = 0. To see this recall that the s = 0 locus can be repre- sented in the form ^ = u'(F(s))\ if s is close to zero, so is F ( s ) ; hence, w'(F(s)), which is i/s approaches infinity. Conversely, if 5 is arbitrarily

4.4 Phase diagram analysis 149

large, (4.66) and 5 = 0 imply that F(s) and c are too; then our special as- sumptions imply that u\ hence 0, approaches zero. Nonetheless, all the important features of the solution, given specific values for T, s0,

a n d ST> are common to the two problems, whether or not the above special as- sumptions are made. We have established our contention that phase dia- gram analysis is a powerful technique that can be used on models without specified functional forms.

This concludes our introduction to phase diagram analysis. Before we move on to the next section, it is convenient to include here an alterna- tive statement of the maximum principle that yields equations of the type (4.63) directly. This procedure applies to control problems in which the only independent time term appears in the discount factor in the maxi- mand; these are an important class of problems. They are called autono- mous problems, of which a simple prototype is as follows: find c(t) to maximize

V= [Te- 6tu(s(t),c(t))dt (4.67)

Jo subject to

s(t)=f(s(t)9c(t)) (4.68a) and

5(0) = 50, s(T)=sT. (4.68b)

The reason we call such problems "autonomous" is that they yield an autonomous system of differential equations (see Section 2.2) to charac- terize the solution, as we now demonstrate. Instead of introducing the costate variable ir(t) and the Hamiltonian

H(s(t)9 c(t), TT(0, t) = e- 8tu(s(t), c(t)) + ic(t)f(s(t), c(t))9

we introduce the costate variable \//(t) and the Hamiltonian

&(s(t), c(t), 0 ( 0 ) = u(s(t), c(t)) + +(t)f(s(t), c(t)).

It is still true that $(t) = e8tir(t) and consequently

H(s(t)9 c(t)9 i(t)) = e 8tH(s(t)9 c(t), TT(0, / ) .

The economic interpretation of the maximum principle is taken up in the next section, but it is already apparent that if H and \f/ reflect current val- ues, then / / a n d -K reflect discounted or present values (see Section 3.4). H and \p(t) are called the current value Hamiltonian and current value costate, respectively. The maximum principle, when applied to H of an autonomous problem, is stated as follows:

m 9u. + m^L=0, (4.69) dc(t) dc(t) r v dc(t)

150 4 The maximum principle

W) = irrL=f(s(t),c(t)), (4.70)

^ ) = " ^ + 8 ^ > = " ^ - ^ > ^ + W / >- (4'71)

ds(t) ds(t) ds(t) Condition (4.68b), of course, remains the relevant boundary condition. Special attention is drawn to the additional term 8\}/(t) in (4.71). One can easily check that (4.71) yields (4.63) of the preceding problem directly.

It is an instructive exercise to verify that (4.69)-(4.71) are equivalent to the equations obtained by applying the maximum principle to H in the way defined in Section 4.1. We display these equations here for easy reference:

dH *, du df = e-8t—— + ir(t)-^— = 0, (4.69') dc(t) dc(t) dc(t)

*M = TTZ=f(s(t)>c(t)), (4.70')

dH ,, du df

Since ir(t) = e~btyp(t), we see that (4.69) and (4.69') are equivalent. Equa- tions (4.70) and (4.70') are identical. To prove the equivalence of (4.71) and (4.71'), we totally differentiate the identity ir(t) = e~dt\//(t) to get

We substitute this result into (4.71') to obtain

ds(t) ds(t)

Passing the first term on to the right-hand side and multiplying by e5t

yields

Mt) = —-\P(t)-^— + d\l,(t)9 VK } ds(t) WK ds{t) WK h

which is (4.71). There are no independent time terms in (4.69)-(4.71); hence, phase dia-

grams obtained from these equations will not exhibit an s = 0 locus that shifts over time. It is therefore recommended that for autonomous prob- lems the maximum principle be stated using the current-value Hamilton- ian. The results presented in Section 4.1 nonetheless apply to all problems.

Remark. Before concluding this section we must comment on the spe- cial form of the discount factor used in problem (4.67); it is the usual

4.5 Economic interpretation 151

exponential discount factor introduced in Section 3.4. Had we instead used the more general formulation

maxf a(t)u(s(t),c(t))dt, Jo

we might have encountered a serious problem. Suppose that the individ- ual when allowed to recalculate her optimal policy at a later date, say 0 > 0, used the discount factor a(t — 0). She would do this if the discount factor reflected the weight attached to the utility flow at time t, not by virtue of its calendar time but merely because of its distance from the planning date (e.g., the case of "impatience" or "pure time preference"). Let (s*(t), c*(t)) be the plan that solves the maximization problem at time t0 = 0 subject to s(t) =f(s(t), c(t)) and s(0) = s0, s(T) = sT. At time 0, the individual would want to solve

maxf a(t-0)u(s(t),c(t))dt

subject to s(t) =f(s{t), c(t)) and s(0) = s*(0), s(T) = sT. It can be shown (see Strotz, 1955-6; Pollak, 1968) that the only form of the discount factor which ensures that the solution of this problem coincides with (s*(t),c*(t)) over [0,T] is the one adopted in (4.67), namely, a(t) = e~5t

(or equivalently a(t) — A1). The difficulty with any other form of discount factor is that the individual would persistently wish to change her opti- mal plan, thereby rendering planning rather meaningless. This problem is known as a problem of dynamic inconsistency; it is perhaps one of the reasons for the almost universal use of the exponential discount factor.

4.5 Economic interpretation of the maximum principle

In this section we show that the maximum principle can be given an ap- pealing economic interpretation that gives us further insight into the op- timality of dynamic choice. We deal with the general control problem in- troduced in Section 4.1. It is restated here:

K*(s0,sr,0, T) = max\ Tv(s(t)9c(t),t)dt9 (4.72)

c(t) J 0

subject to

s(t)=f(s(t),c(t),t),

s(0) = s0y s(T)=sT.

We shall refer to s as a stock of capital and to c as a flow of consump- tion; v is the instantaneous value function, V* the maximum value func- tion, and / the growth function of stock.

152 4 The maximum principle

The Hamiltonian of this problem is

H(s, c, x, t) = v(s, c, t) + TT/(5, C, t), (4.73)

where ir is the costate variable and again we omit the time argument. The maximum principle yields the following conditions:

(i) c*{t) maximizes H at each t; hence,

™ = vc+r% = 0. (4.74)

(ii) s*(t) and ir*(t) satisfy the pair of differential equations

dH ds

*'«) = -—= ^vs-T'fs, (4.75)

s*(t) = ̂ =f(s*,c*,t). (4.76)

Optimal values are denoted by an asterisk, and all functions are evaluated along the optimal path.

4.5.1 Costate variables as prices

We have defined the meaning of all variables but the costate, and now we turn to this important task. Let us evaluate the derivatives1 of the max- imum value function V* of (4.72) with respect to s0 and sT. Since s = f(s*, c*, t) at all time, it is true that

V*=\T[v(s*,c*,t) + Trf(s*,c*,t)-Trs*]dt Jo

for any arbitrary function ir(t). Note that the above expression is similar to a Lagrangean in a static problem; indeed, our argument here parallels the one used to interpret Lagrange multipliers. First we need to transform the above expression. We know from the method of integration by parts that $irs*dt = ics*-$'ks*dt. Therefore,

1 Here we assume that V* is a differentiable function of s0. In some abnormal cases, the derivative dV*/dsQ may not exist at some value of s0. For example, consider the problem

{ T s{t)dt (=s(T)-s(0)) subject to 5(0 = c(t)s(t), 1 < c(t) < 2,5(0) = 50, s(T) free, Tfixed. Without the need to use the maximum principle, it is easy to see that if s0 > 0, then c(t) = 2 and V* = s0(e

2T-1); if 50 < 0, then c(t) = 1 and V* = {e

T-1)50; if s0 = 0, then V* = 0. Hence, dV*/ds0 does not exist at s0 = 0.

4.5 Economic interpretation 153

V*=\T[v(s*,c*,t) + Trf(s*,c*,t) + ics*]dt-[Trs*]o Jo

= (T[H(s*9 c\ TT, t) + 7T5*] dt- TT(T)ST + 7r(0)s0. (4.77) Jo Jo

The derivative of V* with respect to s0 is

dV* CT[ ds* dc* d-K

ds0 J o [ ds0 ds0 ds0

/TT^ dt ds* ^d-k aso dso dso

where subscripts also denote partial derivatives. The value of s0 does not affect the independent time trend; thus, dt/ds0 = 0. Furthermore, since 7r is an arbitrarily chosen function of time, dir/ds0 = 0 and dir/dso = 0 also. Thus, (4.78) reduces to

rff + x(0), (4.78)

dV* r f , ^ . x * * / r r x d c * Jo L ds0

dt + ic(0). dso Jo L dsQ dso

This is true for any function ir(t); but suppose that we select the optimal path T*(t) obtained from the maximum principle equations (4.74)-(4.76). Then Hs + ir* = 0 and HC = 0, and we have

dV* -7— = TT*(0), (4.79) ds0

where 7r*(0) is the optimal value of the costate variable at time zero. An identical argument and almost identical calculations yield

dV* -7— = - i r m (4.80) asT

where TT*(T) is the optimal value of the costate variable at time T. The meaning of the results (4.79) and (4.80) is clear. A marginal increase

in the initial stock would contribute 7r*(0) per unit to the total value ob- tainable over the horizon. Hence, 7r*(0) is the worth, or imputed value, of one unit of initial stock. The meaning of this costate variable parallels that of the multipliers and dual variables encountered in static optimiza- tion. Similarly, requiring that one reach terminal time with more capital would deduct TT*(T) per unit from V*, thus identifying ir*(T) as the im- puted value of stock at time T. The difference in signs between (4.79) and (4.80) stems from the fact that we are endowed with s0 at time 0, whereas we must relinquish sT at time T.

These results can be generalized to interpret ir*(0) as the imputed value of stock at any instant 6 along the optimal path (see Leonard, 1987, for

154 4 The maximum principle

more details). It is necessary to alter the format of the control problem, because we cannot prove that dV*/ds*(6) = ir*(0) since s*(0) is not an ex- ogenously specified parameter but is optimally chosen. We must formal- ize the notion that at some time 0 in the interval (0, T), a small amount of capital stock is suddenly added to the existing stock. The rate of change of capital stock is now defined by

$=f(s,c9t) + a(t)9 (4.81)

where f 0 , 0<t<6,

a(t) = \ ae~\ 0 < / < 0 + e, [ o , 6 + e<t<T.

The number e is arbitrarily small and positive. The injection of capital takes place abruptly during the interval [0,0 + e); the smaller e, the more abrupt the injection. At the limit (e->0) it mimicks the addition of a units of capital at time 0, since

{ 0 + e f0 + e , , Q, a(t)dt=\j ae-

ldt = a[e-lt]ee +e = a.

The Hamiltonian become H(s, c, 7T, t, a) = v(s9 c, t) + T T ( / ( S , c,t) + a),

and (4.76) is replaced by (4.81). The same calculations as those used to interpret 7r*(0) and ir*(T) yield an equation similar to (4.78) but includ- ing a(t):

V*=[T[H(s*,c*,Tr,tia) + irs*]dt-ir(T)sT+ir(0)s0, Jo

where ir(t) is arbitrarily chosen. For e sufficiently small, the derivative of V* with respect to a will be an

adequate approximation of the rate of increase in V* per unit of increase in capital at time 6, which we shall as usual interpret as the worth of a unit of capital stock at that time. Recalling that dt/da, dir/da, and d-k/da. vanish as previously argued, we have

^ 1 - [ T \ H — H— ' — Hda]d da J o [ 5 da c da da a da J

Selecting the optimal trajectory ir*(t) and using (4.74), (4.75), and (4.81), this reduces to

dV* dt

ir*(t)e~ldt.

4.5 Economic interpretation 155

Letting n*(/) = j ir*(t) dt, we can calculate

• = 7 T * ( 0 ) . dV

lim — = lim

Therefore, we shall interpret the optimal value of the costate variable at any time 0 as the imputed value of a unit of stock at that time. Clearly, the costate variable is the dynamic analog of the static multipliers, and we shall again use the words "imputed value," "worth," "shadow price," or simply "value" or "price" interchangeably to refer to them. The units in which ir(t) is expressed are "value units" (as for V*) per unit of stock at that time. Now that the meaning of each variable is understood, we can turn to the maximum principle itself.

Recall that v(s, c, /) is some instantaneous value function and that f(s,c, t) describes the growth of s. The Hamiltonian function

H(S, C, 7T, t) = V(S, C, t) + 7T/(S, C, / )

is the sum of the instantaneous value plus the value of the instantaneous growth of s. One could call it a dynamic value function because it also takes into account the effect of current stock and control on the size and valuation of future stock.

Choosing c(t) to maximize the Hamiltonian at each instant of time takes into account the immediate effect of c(t) as well as its effect at all future dates. Therefore, maximizing H at each instant / yields a dynamic optimum at that time. The links between these optima are provided by the differential equations s = f(s, c, t) and ic = -dH/ds.

That is the general structure of the maximum principle. Let us now examine each of the equations (4.73)-(4.76) in detail.

4.5.2 The maximum principle as an economic program

First we will take the point of view of a central planner, alone responsible for solving the control problem. The Hamiltonian of (4.73) represents the nonmyopic value function at time /. When the control is chosen to maxi- mize that function, (4.74) must hold; it states

dv df

dc(t) dc(t) The first term is the marginal contribution of consumption to the instan- taneous value function, that is, to the current flow of value. The second term is the product of the value of stock at time t and the marginal effect of consumption on the rate of growth of stock at that time; hence, it is the marginal contribution of current consumption to future value, through its effect on the growth of stock. Together these two terms account for

156 4 The maximum principle

all marginal benefits (and/or costs) of consumption, and (4.74) is seen as the counterpart of the familiar first-order condition of a static model of choice.

Of the two differential equations the second one (s=f(s, c, t)) is al- ready understood. The first one, (4.75), requires some thought. It states

dv df

The first term is the marginal effect of stock on the instantaneous value function, that is, on the current flow of value. The second term is the product of the value of stock at time t by the marginal effect of the level of stock on its own rate of growth; hence, it reflects the marginal impact of current stock on future value. With regard to the term on the right- hand side, since f(t) is the rate of change in the value of stock, we can interpret — -k(t) as the rate at which the value of stock depreciates. Then the equation states that along the optimal path the value of a unit of stock should be depreciated at a rate equal to its net (or combined) marginal contribution: current (dv/ds(t)) and future (ir(t)df/ds(t)). This rate of depreciation may, of course, be negative, but then the "contribution" of stock would itself be a deduction. This interpretation is appropriate for a central planner who seeks the optimal pricing of stock over time.

Let us now imagine the reasoning of an individual agent to whom the price of stock is exogenously given. The Hamiltonian of (4.73) represents the net benefit accruing to the agent at instant t since it takes into account the current flow of value v and the value of the growth of stock at instant t. Hence, (4.74) states that at each instant the agent chooses the control variable so as to maximize net benefit. There is no need for the agent to take into account the whole planning problem. Indeed, we may view agents at different times as distinct individuals who simply maximize the Hamiltonian at one instant. Their knowledge of ir and their myopic opti- mizing behavior at each instant are sufficient to guide the economy along the optimal path.2 This interpretation brings to light once again the de- centralizing role of prices.

While the agent at each instant t need choose only the control, it is in- structive to reconsider equation (4.75) with the myopic agent interpreta- tion in mind. It can be written as dv/ds(t) + ir(t)df/ds(t) + ir(t) = 0. The

2 It may be worth noting an implicit restriction of this interpretation. Suppose that, for some time, f(s, c, /) > 0 while ir(t) < 0; then the stock grows at a positive rate but its value is negative (it is a "bad," or a nuisance, for whatever reason). If free disposal is a possi- bility, the agent should, of course, discard this spurt of unwanted growth. This is not allowed and the free-disposal assumption (often made in general equilibrium theory) does not apply to this problem. This remark also applies when the central planner interpreta- tion is adopted but is perhaps more natural in that context.

4.5 Economic interpretation 157

interpretation of the first two terms is similar to that of the central plan- ner: they account for the benefits (and/or costs) of holding the marginal unit of stock through its effect on current and future value. The last term, however, is now interpreted as the speculative gain made by the agent while holding a unit of stock at instant /, recalling that the value of stock is seen to vary exogenously by the agent. Setting this exhaustive sum of marginal benefits (and /or costs) equal to zero is necessary to determine the optimal number of units of stock to be held by the agent at instant t. Therefore, it is as if the agent were following an optimal investment plan at each instant.

Finally, we can show that if there are several economic agents who se- quentially share the responsibility for decision making, the optimal path will still be followed, provided that each agent respects appropriate bound- ary conditions on the levels of stock. Consider an agent responsible for the subhorizon [t0,ti], where 0 < / 0 ^ i < ^ and with boundary condi- tions s(t0)=s*(t0); £(/!)=£*(/,), where (s*(t),c*(t),w*(t)) denotes the optimal solution to the global problem (i.e., the one with horizon [0, T]). Thus, the boundary values of stock in this problem are on the optimal path of the global problem. We want to show that the optimal solutions of the two problems coincide over the subhorizon |70, t\\? Suppose that the optimal controls differ from one another in the two problems; since both of them take the stock from s*(t0) to 5

,*(/1), one of them must yield a higher value for the maximand, a contradiction (we assume uniqueness of the optimal solution). But if the controls are identical, the stock vari- ables that both satisfy the differential equation (4.76) and have identical endpoints must follow the same path. If we take the interpretation of eco- nomic agents for whom the price path is exogenously specified, nothing further need be said. If we choose to consider the agent as central planner for the interval [t0, tx], it is easy to see that the pair of differential equa- tions (4.75) and (4.76) along with two boundary conditions, s(t0) =s*(t0) and s(ti)=s*(ti), will yield the same solution in this subproblem as in the global problem once the controls are identically chosen. Thus, we can transform the aggregate global problem into a sequence of subproblems with appropriate boundary conditions. 3 Here we assume that the problem of inconsistent planning, referred to at the end of Sec-

tion 4.4, does not arise, either because v(s, c, t) takes the form e~btu{s, c) or because the independent time argument in v does not represent pure time preference and hence at time tQ the individual maximizes the integral

\T v{s{t\c{t),t)dt, J / 0

and not

{ T v(s{t),c{t),t-t0)dt. 'o

158 4 The maximum principle

4.5.3 Optimal growth

We now illustrate our economic interpretation of the maximum principle with a more specific problem. It is identical to the "general" growth mod- el presented at the end of Section 4.4 in all respects but one: the rate of change in stock is now

s = F(s(t))-ms(t)-c(t), m>0.

To begin with, we restate the problem and describe it in economic terms. Consider an economy with a single capital resource that can be used to produce a single good according to some technology and can also be con- sumed. If we denote the stock of capital at time / by s(t), gross produc- tion is F(s(t)), where F is the production function. This stock of capital depreciates naturally over time at the exponential rate m > 0 (see Sec- tion 3.4). Thus, if neither gross production nor consumption were to take place, the path of capital would be described by s(t) = —ms(t), whose solution is s(t) =s0e~

mt, where s0 is the specified value of 5(0). However, as we have seen, production does take place, and the stock of capital is also depleted by a flow of consumption c(t) at time /. Thus, the rate of change of capital stock is described by

Ht)=F(s(t))-ms(t)-c(t).

Suppose that a central planner is in charge of charting the path of the economy over some time horizon [0,T]. He knows that at time 0 there are s0 units of capital available and that he must reach time T with sT units of the stock. Meanwhile, he has chosen as his objective the maxi- mization of the total discounted utility over the planning horizon. At any instant t, utility depends on the flow of consumption c(t) only; hence, it can be formulated as u(c(t)). This implicitly assumes that utility at any instant is not directly dependent of the stock of capital at that time (no Scrooge element). The planner attaches less utility to future consumption than to current consumption, and this is reflected by a subjective expo- nential rate of discount 5 > 0.

This means that the discounted utility of consuming c at time t is given by e~btu(c). In a continuous-time model the sum of utilities takes the form of an integral, and the planner's objective is to maximize

V=[Te-dtu(c(t))dt, Jo

subject to

s(t)=F(s(t))-ms(t)-c(t), s(0) = s0i s(T)=sT.

4.5 Economic interpretation 159

We assume that u' > 0, u"< 0, u'(0) = oo, F(0) = 0, F' > 0, F"< 0, and F'(oo) < 8 + m <F'(0). These assumptions are identical with those made in Example 4.4.1, save for the last one, which now incorporates the rate of depreciation m as well as the rate of discount. Its purpose is again to guarantee the existence of an equilibrium. The production and utility functions are strictly concave and increasing. This implies that more con- sumption (resp. capital) yields more utility (resp. output) but does so at a decreasing marginal rate. A zero capital stock yields no output. Marginal utility becomes infinite as consumption approaches zero.

The Hamiltonian of this problem is

H(s,c,Tr,t) = e-dtu(c(t)) + Tr(t)[F(s(t))-ms(t)-c(t)],

and the necessary conditions are (skipping arguments)

— = e-8tu'(c)-<jr = 0, (4.82) dc

ic = - ^ = -Tr[F'(s)-m], (4.83) OS

dH s = — =F(s)-ms-c. (4.84)

ir(t) is the present value of capital at instant t. (Note that value is mea- sured in units of utility.) To reach a dynamic maximum the Hamiltonian is maximized at each instant; the influence of the consumption flow on the growth of capital is taken into account. This maximization is charac- terized by (4.82), which equates at each instant the discounted marginal utility of consumption to the present value of capital. This reflects the fact that consumption is simply subtracted from the growth in capital stock. Equation (4.84) reflects the optimality of the path of capital. The net mar- ginal physical product is the gross marginal physical product less the rate of depreciation. Hence, equation (4.83) prescribes that the rate of change in the value of stock plus its net marginal value product sum to zero. This sum represents the net marginal gain of a would-be investor holding s units of capital and facing the price path 7r. If it were to be negative or positive, the investor would be holding too much or too little capital, re- spectively. We can derive further insights into the nature of the optimal solution by examining the consumption path. To this end we totally dif- ferentiate (4.82) with respect to time and use it with (4.83) to obtain c:

-6e-8tu'(c) + e-8tu"(c)c = ic

= Tr[m-F'(s)]

= e-btu'(c)[m-F'(s)]i

160 4 The maximum principle

Figure 4.6

c = ^[(F'(s)-m)-S]. (4.85)

Equation (4.85) indicates that the optimal flow of consumption will in- crease (resp. decrease) if and only if the net marginal product of capital is larger (resp. smaller) than the subjective rate of discount. It pays to have a higher future consumption relative to present consumption if the cost of postponing it is smaller than the gains it provides through its marginal effect on net capital return, where the cost of postponing consumption evaluated at the margin is indicated by the discount rate 6.

To conclude this section we now briefly present a phase diagram analy- sis of this problem in the (s, c) plane, using equations (4.84) and (4.85). Figure 4.6 is the phase diagram; it is similar to Figure 4.4 except that the 5 = 0 locus is now c = F(s) — ms and thus may have a maximum. (It does if F'(co)<m\ this is the case depicted in Figure 4.6.) The construction of the diagram is left to the reader as an exercise. Three optimal paths are

4.6 Necessity and sufficiency 161

illustrated for various lengths of the planning horizon T. If Tis very small, a high path such as (i) is chosen; for larger T, (ii) may be optimal. Along both these paths capital decreases monotonically. If T is even larger, we may, as in case (iii), go through an initial phase when capital stock is increasing. This diagram will be referred to in Section 6.4.

4.6 Necessity and sufficiency of the maximum principle

In this section we provide a heuristic derivation of the maximum principle as a set of necessary conditions for optimality, and we also define a class of problems for which the maximum principle is sufficient for optimality.

Consider the control problem stated at the beginning of Section 4.5 with the Hamiltonian given by (4.73). Denote the optimal solution by (s*(t)9 c*(t)9 TT*(0), and let / / * = v(s*(t), c*(t)9 t) + ir*(t)f(s*(t)9 c*(t), t). We assume for simplicity that all variables are continuously differentiable functions of time. Given the equation s(t) =f(s(t), c(t)9 t)9 consider the class of functions c(t) that drives s(0) = s0 to s(T) = sT. This class of functions is denoted by A. Clearly, c*(t) is a member of A. Given any arbitrary function a(t)9 let us construct a family of functions

4 a(t9 e), where e is a parameter such that a(t90) = a(t) and c*(t) + ea(t9 e) is a member of A. Suppose now that we deviate from the optimal trajectory by using c(t) = c*(t) + ea(t, e). The resulting state variable path (through (4.76)) is denoted by s(t9 e) since it depends on the value of e; it obeys the original boundary conditions. Since c*(t) is the optimal control, Kmust be maximized when e = 0. We first manipulate the expression for V and then evaluate its derivative with respect to e when e = 0:

V=[ v(s(t,e),c* + ea(t9e)9t)dt Jo

= ^[v(s(t9e)9c m + ea(t9e)9t) + T(t)f(s(t9€)9c* + ea(t9e)9t)

-T(t)S(t9e)]dt9

for any arbitrary function Tr(t). Proceeding, let

H(t9e) = v(s(t9e)9c* + ea(t9e)9t) + ic(t)f(s(t9e)9c* + ea(t9e)9t) and

V=\T[H(t9e)-Tr{t)s(t,e)]dt Jo

= \T[H(t9e) + ir(t)s(t9e)]dt-Tr(T)sT+ir(0)s0. Jo

4 The existence of such families of functions and their differentiability with respect to e are simply assumed here. This assumption would, of course, be innocuous if sT were free, and not exogenously specified.

162 4 The maximum principle

Then

de e = 0 - I . TYdH o [ dc e = 0

.«,.>+(£ + *W e = 0

ds(t,e) de

dt.

It is necessary for a maximum that the derivative dV/de vanish when eval- uated at e = 0. This must be true for any arbitrarily chosen pair of func- tions cx(t) and ir(t). Therefore, it must be true, in particular, for a ir(t) function satisfying the differential equation 7r(t) + (dH/ds)\e=0 = 09 and for this particular choice of the function ir(t) we have, for any arbitrarily chosen a(t),

dV_

de

TdH _ f 6ti

e = 0 Jo dc a(t)dt = 0.

This implies that (dH/dc)\e==0 vanishes at each instant, for otherwise it would be possible to find a function a(t) such that the above integral does not vanish. Therefore, collecting the results, the necessary conditions for optimality are

dH „ dH ^ — = 0 , 7T+ — = 0 , dc as

and, of course,

s=f(s,c,t),

with s(T) =sT and 5"(0) =s0. We recognize here the maximum principle stated in more detail in equations (4.74)-(4.76).

To obtain sufficient conditions consider now a class of optimal control problems for which the Hamiltonian H(s, c, 7r, t) is concave in (5, c). We will shortly state conditions guaranteeing this. For simplicity of notation, we will suppress all arguments of functions, and let an asterisk denote optimality when superscripted to variables or functions; the absence of an asterisk denotes any other feasible solution (in particular, s satisfies the boundary conditions). We manipulate the expression for Kin a man- ner similar to that used to derive the maximum principle:

V*-V=[T(v*-v)dt Jo

= [T[(H*-Tr*s*)-(H-Tr*s)]dt Jo

= [ (H* + ir*s*)-(H+<k*s)dt (by integration by parts) Jo

= [T[H*-H+<k*(s*-s)]dt Jo

4.6 Necessity and sufficiency 163

>[T[(s*-s)Hs* + (c*-c)H?+ic*(s*-s)]dt Jo

(by concavity of the Hamiltonian)

= f T[(s*-s)(H? + ic*) + (c*-c)H?]dt Jo

= 0

if the starred solution satisfies the maximum principle. It follows that if H is concave in (s, c), then any solution satisfying the maximum principle yields a value V* at least as high as the value V yielded by any feasible solution. Note that if H is strictly concave in (s, c), then V* > F a n d the optimal solution must be unique. We now state the result formally.

Theorem 4.6.1: sufficiency. Consider the problem defined in (4.1)-(4.3). Suppose that the maximum principle of (4.5)-(4.7) yields the solution (**(/), c*(t), TT*(0). If the Hamiltonian of (4.4), with ir(t) = ir*(t)9 is con- cave (resp. strictly concave) in (s, c) jointly, then (s*(t), c*(t), ir*(t)) is an optimal solution (resp. the only optimal solution) to the above problem.

This sufficiency result depends on the concavity of / / , which may be difficult to ascertain without obtaining an explicit solution. This is why we now provide a set of conditions that ensure the concavity of H.

Theorem 4.6.2. If v is concave in (s, c), and either IT > 0 and / is concave in (s, c) or 7r < 0 and / is convex in (s, c), then H is concave in (5, c) and the necessary conditions provided by the maximum principle are also suffi- cient for an optimal solution. Furthermore, it can be shown (see Leonard, 1981) that if vs > 0 (resp. vs < 0) along the optimal path then -K > 0 (resp. 7T<0).

In some cases where Theorems 4.61 and 4.62 fail, there is yet another re- sult that might apply. First we must define a new concept and restate the maximum principle.

Definition 4.6.1. For the problem of (4.1)-(4.3), let

H°(s(t), TT(0, t) s max H(s(t), c(t), TT(0, 0 , (4.86) c{t)

where H(s(t), c(/), ir(t), t) is as in (4.4). We call H° the maximized Ham- iltonian.

Note that (4.86) implicitly defines c as a function of s, 7r, and t - through equation (4.5).

We can now restate the maximum principle.

164 4 The maximum principle

Theorem 4.6.3: necessity. Consider the problem of (4.1)-(4.3) and let (c, s, 7r) be an optimal solution; then c satisfies (4.5) and ir and s are solu- tions to

ic(t) = -dH0/ds(t) and s(t) = dH0/dir(t)9 (4.87)

where H° is defined in (4.86).

Theorem 4.6.4: sufficiency. Suppose (s*(t),c*(t),ir*(t)) is a solution to (4.3), (4.5), and (4.87). If the maximized Hamiltonian of (4.86) is concave (resp. strictly concave) in s(t) when ir(t) = ir*(t), then (s*(t), c*(t), ir*(t)) constitutes an optimal solution (resp. the unique optimal solution) to the problem of (4.1)-(4.3).

The above statement of the maximum principle using H° in Theorem 4.6.3 is equivalent to (4.5)-(4.7), and the two may be used interchange- ably. Note that attention must still be paid to the properties of H in terms of c (given s and ic) to ensure that (4.86) indeed defines a maximum. The proof proceeds along the same lines as that of Theorem 4.6.1, leaving out terms specifically referring to c and using H° instead of H. We now illus- trate Theorems 4.6.3 and 4.6.4 with some examples.

Example 4.6.1. We want to maximize \^Ac(s)^2e~btdt subject to s = — (c)4, and s0 and sT are exogenously specified and positive. The Hamil- tonian H = 4c(s)l/2e~8t — 7r(c)4 is clearly not concave in (c,s) jointly. However, it is concave in c alone (assuming 7r>0), and the first-order condition, dH/dc = 0, yields a maximum at

c = (s)l/6(ir)l/3e-dt/\ (4.88)

[From this we can show T > 0: s must be positive or the maximand is un- defined and a negative value of c is never chosen, since it makes both s and the maximand negative; hence, TT must be positive.] We find the max- imized Hamiltonian by substituting the optimal c value into H:

/ / 0 = 4 ( 5 ) 1 / 6 ( 7 r ) - 1 / 3 c - ^ 3 ( 5 ) 1 / 2 ^ - ^ - 7 r ( ^ 4 / 6 ( 7 r ) - 4 / 3 e - 4 ^ 3

= 3e- 4 5 '/ 3 (7r)- 1 / 3 (s) 2 / 3 ,

which is evidently strictly concave in s. The alternative form of the maximum principle is

TT = -dH°/ds = -2e-46t/3(Tr)-l/3(s)-l/\ (4.89a)

s = dH°/dir = -e-48t/3(ir)-4/3(s)2/3, (4.89b)

with c given by equation (4.88). The reader is invited to verify that sub- stituting dH/dc = 0 (i.e., (4.88)) into s = dH/dir and ir = -dH/ds yields equations (4.89).

Exercises 165

Example 4.6.2. Here we illustrate the remark made after Theorem 4.6.4. Consider a stock whose growth is enhanced by its own size and the ef- fort made to tend it. Utility depends on the size of the stock and is nega- tively related to effort. We maximize \l(s)l/2(c)~le~6tdt subject to s = (s) 1 / 2 c, and s0 and sT are exogenously specified. The Hamiltonian is H = (s)l/2(c)-le-8t + irc(s)l/2. The first-order condition dH/dc = 0 yields (c)~2 = ire6t and H° = 2e~5t/2(ir)l/2sl/2, which is clearly concave in s. However, the first-order condition did not select a maximum over c but indeed a global minimum since the Hamiltonian H is clearly convex in c. In fact, this Hamiltonian has no maximum when no (positive) lower and upper bounds have been imposed on c from the outset.

Remark. Before leaving this section, recall that we have assumed through- out that the variables are continuously differentiable functions of time. It is an attractive feature of optimal control theory that it can deal with a wider class of solutions. All that is needed is that the control variable be piecewise-continuous. This means that it is acceptable to have a control function c(t) that exhibits some jump discontinuity at a finite number of points. It follows that s and -k themselves are piecewise-continuous while s and 7r are continuous and piecewise-differentiable. A more formal state- ment of the maximum principle is postponed until Chapter 6, and the special features of problems exhibiting such discontinuities are the sub- ject of Chapter 8.

Exercises

1. Applying the maximum principle, find c(t) to maximize V— \TQ\n(c(t))dt sub- ject to s(t) = — c(t), 5(0) = So, s(T) = sT, where T, s0, and sT are specified pos- itive constants, with sT<s0. Show that the optimal control is constant over the horizon and that the optimal path of the state variable is a linear function of time.

2. Repeat exercise 1 with \TQ(c(t)) adt as the maximand (0 < a < 1). Is the optimal

control different in this case? Can you show that c would be constant with \TQU(c(t))dt as the maximand, where U is any strictly increasing and strictly concave function?

3. Applying the maximum principle, find c(t) to maximize V=$le~t\n(c(t))dt subject to s(t) = — c(t), s(0) = s0, s(T) = sT, where T, s0, and sT are specified positive constants, with sT<s0. (Hint: Obtain the general solution to the co- state differential equation, substitute for the control variable in the state vari- able differential equation, and use the boundary conditions to determine the constants of integration.) Is the control constant over time? Contrast this with the result of exercise 1.

4. Replace the maximand in exercise 3 by JJ"e"~'(c(/))afifr and repeat the exercise (0 < a < 1). Does the optimal control depend on the exact form of the maxi- mand? Contrast this with the results of exercise 2.

166 4 The maximum principle

5. Reconsider the control problem of exercise 1 now with the modified state differ- ential equation s(t) = —s(t) — c(t). Solve this problem. What restriction must be placed on T, s0, and sT to ensure that the optimal control is positive? (With- out this a solution cannot exist.)

6. Consider a modified version of the problem of exercise 5. Find c(t) to maximize V=\TQ[(c(t)y

+y(\ + y)]dt subject to s(t) = -s(t)-c(t), s(T) = sT, s(0) = s0, where —y, T, s0, and sT are specified positive constants. Apply the maximum principle and solve the problem.

7. In this exercise you will formulate a control problem and apply the maximum principle. It concerns the optimal management of a mineral spring. You must pay particular attention to the units in which the variables are expressed. Your aim is to maximize the total present value of profits over the planning hori- zon. The spring flows at a constant rate. Outflow may be sold immediately or stocked in a natural reservoir at no cost; however, the reservoir leaks and a constant proportion of current stocks is lost per unit of time. We count vol- umes of water in megaliters (L) and time in days (d). The following notation is used (units are indicated in parentheses):

c(t) quantity sold at time t (L/d), P(c(t)) profit, at time t, from the sale of c(t) units at time t ($/d), R constant rate of outflow of the spring (L/d), s(t) level of stocks at time t (L), a constant proportion of stocks that leaks at time t (d~l), e~ht discount factor, i.e., the present value of $1 at time t (a pure number). R, 6, and a are fixed positive constants. The time horizon [0, T] is also fixed, as are the boundary conditions s(0) =s0 and s(T) =sT.

Express the rate of change in the levels of stocks, s(t) (in L/d) in terms of the other variables. Express the present value of profits over [0, T], an integral (in $). Set up the control problem. Which is the state variable and which is the control? Let the costate variable be ir{t). Write down the Hamiltonian and apply the maximum principle (you will have three equations).

8. Consider the mineral spring problem of exercise 7 with the following data: P(c) = 10c-c2, R = 15, a = 1, 6 = 0.5. Show that the application of the maxi- mum principle yields 10-2c = 7rc'/2, T = T, and s = 15 —s — c. Solve the differ- ential equation in -K - you will need to use an arbitrary constant of integration, say A. Obtain the optimal path for c - it depends on A. Use this to solve the differential equation in s - you will need another constant of integration, say B. Use the initial condition 5(0) = 50 to eliminate B.

Now let T= In 2 and s(T) = 30; determine the constant A and the exact paths of c, 5, and IT. Is c constant over time?

This time let T= In 5 and s(T) = 0; determine A and the exact paths of c, s, and 7r; calculate c(0) and c(T). Can you plot a rough path in the (c,s) plane?

9. Repeat exercise 8 with the following data: P(c) = 5 0 c - c 2 , R = 15, a = 1, 6 = 0.5, s(0) = 30. Show that the maximum principle yields 50 —2c = 7rc//2, -k = ir, and s = 15 — s — c. Determine the exact paths of c, s, and ir for the two terminal conditions (T= In 2, s(T) = 10) and (T= In 5, s(T) = 0); plot the latter solution in the (c,s) plane.

Exercises 167

Suppose now that you want to determine the value of T, say T, that would lead to c(T) = 15 with boundary condition s(T) = 0. Determine the approxi- mate value of T.

Now choose a value T larger than T; use the boundary condition s(T) = 0 to determine the constant of integration. Show that there is a problem with this outcome. (Hint: Calculate s(t) for / very close to f, but smaller.)

10. Consider the problem of choosing c(t) to maximize

v = [ v 4 [ c ( ; ) ] ( l + a ) U

Jo [ 1 + a J subject to s(t) = rs(t) — c(t) with s(0)=s 0 and s(T)=sT, where 7", 6, —a, r, 50, and 5 r are specified positive constants. Apply the maximum principle. Obtain a differential equation for c and solve it. Use the result to solve the differential equation in s. Use the boundary conditions on s to determine the constants of integration. Show that if sT = s0, c(t) is always positive.

11. Use the problem of exercise 1 to derive autonomous differential equations for c and s in which c and s depend only on the values of c and s. Draw a phase diagram in the (c,s) plane for arbitrary values of T, s0i and sT (s0>sT>0). How does the optimal trajectory change if a larger rvalue is selected? Show that the results are qualitatively identical if an arbitrary U function is selected as in exercise 2.

12. Use the problem of exercise 3 to derive autonomous differential equations for c and s as in exercise 11 and draw a phase diagram in the positive quadrant of the (c,s) plane. Show that the optimal trajectory is a straight line of slope 1. How does the trajectory change when a larger T value is selected? Repeat the exercise with the problem of exercise 4. Show that the optimal trajectory has slope (\ — a)~l. Does the exact value of a change the general shape of the optimal trajectory?

13. Consider the problem of exercise 5. Apply the maximum principle and derive autonomous differential equations for the state and the control variable (i.e., c and s in terms of c and s only). Show that the loci of c = 0 and s = 0 are out of bounds if c and s are positive. Draw the phase diagram. Derive autono- mous differential equations for the state and costate (TT) variables. Show that the loci of s = 0 and -k = 0 are out of bounds if c and s are positive. Modify now the state equation to s(t)=s(t) — c(t). Draw the phase diagrams in the (c, s) plane and the (IT, s) plane. Show that c always increases over time but s is not always monotone. Let s0 = sT>0, and draw an optimal trajectory for various values of T. Can the value of T have a qualitative effect on the tra- jectory of si

14. Repeat exercise 13 for the problem of exercise 6. 15. For the problem of exercise 10 apply the maximum principle to the current-

value Hamiltonian (denote the costate variable by \p). Obtain a system of autonomous differential equations for c and s and draw a phase diagram in the (c,s) plane. Do the same for \p and s. Attempt the same exercise with the maximand \T0e~

btu(c(t))dt, where u' > 0, u"< 0. Do you get qualitatively dis- tinct phase diagrams when using u(c) instead of (c)l+a/(l + a)l

168 4 The maximum principle

16. Draw the phase diagrams in the (c,s) plane for exercises 8 and 9. In exercise 7, assume that P(c) reaches a maximum at c* > 0. First assume c* > R and draw the phase diagram in the (c,s) space; then assume c*<R and repeat.

17. Consider the problem of choosing c(t) to maximize V= jj\n(c(t))dt subject to 5(0 = \0s(t)-0A(s(t))2-c(t) with5(0) = s0, s(T) = sT, where T, 50, and sT are specified positive constants. Use the maximum principle to derive a sys- tem of autonomous equations for c and s; draw the phase diagram in the pos- itive quadrant of the (c,s) plane. Show that there is a steady state at (c = 250, s = 50). Linearize the differential equations at this point and formally prove that it is a saddle point. Make sure that the trajectories in your diagram con- firm this. Carry out a similar exercise for the state-costate pair. After the dia- grams have been completed, attempt to pinpoint the optimal trajectory for selected allocations of T, 50, and sT as in Section 4.4.

18. Repeat exercise 17 when discounting and depreciation are included, name- ly, choose c(t) to maximize V= fte^lnicifydt subject to s(t) = 105(0 - 0.1 (5(0)2 — 25(0 — c(t). Do the phase diagrams differ qualitatively from those of exercise 17?

19. Suppose that the problems of exercises 1 and 3 are concerned with the optimal consumption of a nonrenewable resource where the utility of consumption is the logarithm of consumption; in exercise 3 utility is discounted. What does the costate variable represent? What does the maximum principle indicate? Contrast the two problems. In exercise 5 a new feature was added: this is now a naturally decaying resource. What does this imply for the costate variable and the consumption path?

20. Repeat exercise 19 for the problems of exercises 2,4, and 6 (1 -I- 7 is equivalent to a).

21. Consider the mineral spring problem of exercise 7. Give an economic inter- pretation of the costate variable. Discuss the conditions obtained from the maximum principle and comment on the trajectories identified in exercise 16. What does c* represent?

22. You are exploiting a renewable resource; its stock at time / is 5(0; its natural rate of growth is g(s(t)); the flow of harvesting is denoted by c(t) (this is sub- tracted from 5(0). Furthermore, you can boost the growth of the resource by spending %x(t)\ this adds b(x(t)) to 5(0- The revenue from c(t) units har- vested at time / is R(c(t)) and the cost x(t) is subtracted from it. The rate of interest is 5. Formulate the problem of maximizing the present value of profits from time 0 to time T, subject to the growth equation. The values of 5 at times 0 and T are specified. Assume that R, g, and b are concave. Show that the maximum principle yields necessary and sufficient conditions for a maximum. Obtain these; interpret the costate variable and all the optimality conditions.

C H A P T E R 5

The calculus of variations and dynamic programming

Any introductory treatment of optimal control theory would be incomplete without explicit mention of its predecessor, the calculus of variations, and the parallel development of dynamic programming. The calculus of vari- ations owes much to the eighteenth-century mathematician Euler, but many developments and refinements were made in the following centu- ries. Optimal control theory, developed by Pontryagin and his co-workers in the late 1950s, may be regarded as a generalization of the calculus of variations: not only is its field of applicability broadened, but the general problem is approached from a fresh and more insightful viewpoint.

Dynamic programming was developed by Bellman, also in the late 1950s. It was designed primarily to deal with optimization problems in discrete time, but Bellman's famous "principle of optimality" also applies to continuous-time problems, where the Hamilton-Jacobi-Bellman equa- tion plays a crucial role.

In this chapter, we examine the connection of optimal control theory with the calculus of variations and dynamic programming. We illustrate how all three approaches lead to the same solution and comment on their relative usefulness in analytical economics.

5.1 The calculus of variations

In Chapter 4, we studied the problem of finding (s(t), c(t)) that maximizes

\Tv(s(t),c(t),t)dt (5.1) Jo

subject to

Xt)=f(s(t)tc(t),t), (5.2) s(0) = s09 (5.3) s(T) = sT. (5.4)

Let us assume that (5.2) can be inverted to yield

c(t) = <Hs(t),s(t),t)- (5.5)

Substitute (5.5) into v(s, c, t) to obtain

169

170 5 Calculus of variations and dynamic programming

v(s(t), c(t), t) = v(s(t), 0 ( 5 ( 0 , HO, 0, t). (5.6)

The right-hand side of (5.6) is a function of s(t), s(t), and t. We give this function a name, F(s(t)9s(t),t). Problem (5.1) becomes: find s(t) and hence s(t) that maximize

\TF(s(t),S(t),t)dt (5.7) Jo

subject to

s(0) = s0, (5.8)

s(T) = sT. (5.9)

This problem is written in the format of the calculus of variations, in which the choice variable is the rate of change of the state variable. We as- sume that Fpossesses continuous second-order partial derivatives. In opti- mal control problems we require only that the function s(t) be piecewise- differentiable (i.e., it is allowed to have kinks at isolated values of / ) ; this means that s(t) is piecewise-continuous (it may have jump discontinuities at isolated values of t). In the calculus of variations, at least in its early development, attention is restricted to the class of functions s(t) that have continuous second-order derivatives for all / in [0, T], This class of func- tions is denoted by C 2 [0, T]. In the remainder of this section we present a necessary condition for a maximum and show how it can be used to derive the solution. We first state an important result.

Theorem 5.1.1: Euler's equation. Assume that there exists a function s*(t) in C 2 [0, T] that maximizes (5.7), that is,

\TF(s*(t),&*(t)9t)dtz>\ TF(s(t)9Ht),t)dt (5.10)

Jo Jo

for all s(t) in C 2 [0, T] subject to (5.8) and (5.9). Then s*(t) must satisfy the equation

Fs-^(F,) = 0. (5.11)

Equation (5.11) can be written more fully as

ds ds Fs = FsS-r+Fss — +Fst (Euler's equation). (5.12)

at at

Remark. Note that ds/dt is the second derivative of s(t), making (5.12) a second-order differential equation. Equation (5.11) assumes the existence of this derivative.

5.1 The calculus of variations 171

We now proceed to prove the necessity of (5.11) by means of a pertur- bation argument. Any function s(t) satisfying (5.8) and (5.9) must satisfy the condition

s(t) = s*(t) + eg(t) (5.13)

for some e > 0 and some g(t) in C2[0, T] with the properties g(0) = 0 = g(T) (otherwise conditions (5.8) and (5.9) would not be satisfied). Let us define the difference between the right-hand side and the left-hand side of (5.10)asZ>(e):

D(e) = \TF(s* + eg(t),s* + 6gV),t)dt-\TF(s*,s*J)dt. (5.14) Jo Jo

By definition D(e) is nonpositive and attains its maximum at e = 0 (since when e = 0, both integrals are identical). Since D(e) is continuously differ- entiate, the fact that it attains a maximum at e = 0 implies that D'(0) = 0. Let us use (5.14) to evaluate this derivative:

D'(e) = \T[Fsg(t)+FsgV)]dt. (5.15) Jo

Evaluated at e = 0, and set to zero, (5.15) gives

-\TFs(s*,s\t)g(t)dt = \ TFAs*,s*,t)gV)dt. (5.16)

Jo Jo

Integrating by parts the right-hand side of (5.16),

\TF,gV)dt = [F,g{t)]\ -\7 g{t)df^ dt. (5.17) Jo |0 Jo at

Since g(T) = g(0) = 0, equations (5.17) and (5.16) yield

F^)g(t)dt = 0. (5.18) «'.-£ Equation (5.18) must hold for any function g(t) (subject only to g(0) = g(T) = 0). This is possible only if

dt s

where the derivatives are evaluated at (s*(t),s*(t), t). This completes the proof of the necessity of Euler's equation.

Our main reasons for not devoting a great deal of space to the calculus of variations are, first, that optimal control theory is capable of dealing with a wider class of problems and, second, that the necessary conditions obtained from the calculus of variations contain no new information,

172 5 Calculus of variations and dynamic programming

since they can be readily derived from the maximum principle. This we now proceed to demonstrate.

Problem (5.7) can be transformed into the standard form of the con- trol problem (4.1). Let c(t) be a control variable, and let

Ht) = c(t).

Then (5.7) becomes

Maximize [ F(s(t),c(t),t)dt Jo

subject to

s = c9 5(0) = j 0 , s(T) = sT.

Applying the necessary conditions (4.8)-(4.10) to this problem, we have

F C + TT = 0 , (5.19) s = c, (5.20) TT = - / V (5.21)

Since it has been assumed that s(t) is twice differentiate, we can differ- entiate (5.19) with respect to time:

- * ( 0 = ^ F C . (5.22)

Euler's equation (5.11) can thus be obtained from (5.21) and (5.22). (Re- call that Fc is the same as F^.)

Example 5.1.1. To illustrate the use of Euler's equation, let us return to problem (4.38) of Chapter 4. In order to formulate this problem in the calculus of variations format, we use (4.39) to eliminate c(t) from the maximand in (4.38). We need to find s(t) and hence s(t) that maximize

[Te-bt\n(rs-s)dt (5.23) Jo

subject to

s(0)=s0 and s(T)=sT. (5.24)

Applying Euler's equation to (5.23),

e-&tr(rs-$rl = —(-e-8t(rs-syl)

= 8e-8t(rs-s)-l + e-6t(rs-s)-2(rs-s).

This yields a second-order linear differential equation,

5.2 Dynamic programming in discrete time 173

s-(2r-b)s + (r-b)rs = 0. (5.25)

We can transform this equation into a system of two first-order linear dif- ferential equations by defining z = s; hence, z = s can be obtained from (5.25) and we have

The characteristic roots of this matrix are r and (r — b). Therefore, the solution is

s(t)=Aert+Be{r-6)t9 (5.26)

where A and B are constants to be determined with the help of (5.24):

s0 = A+B,

sT = Ae rT+Be{r-d)T.

Solving for A and B,

A = (sTe- rT-s0e-

bT)/(\ -e~bT\ (5.27)

B = (s0-sTe- rT)/(\-e-dT). (5.28)

It is easily seen that equations (5.26), (5.27), and (5.28) are equivalent to (4.46), (4.47), and (4.48).

This concludes our brief introduction to the calculus of variations. (For a thorough treatment of economic growth using the calculus of variations see Hadley and Kemp, 1971.) We will not discuss this topic again, because it is more economical to approach continuous-time optimization prob- lems using optimal control theory. The latter is a more unified, more ele- gant, and more systematic body of knowledge that contains all of the results of the calculus of variations as special cases.

5.2 Dynamic programming: discrete-time, finite-horizon problems

In Section 4.2, we introduced an optimization problem in discrete time and stated the discrete-time maximum principle. An alternative method of solving this type of problem is the dynamic programming approach, which successfully exploits the recursive nature of problem (4.11). To ex- plain this, we first rewrite (4.11) and (4.12) in a more convenient form for our present purpose. The problem is to find c ( l ) , c ( 2 ) , . . . , c ( T ) that maximize

174 5 Calculus of variations and dynamic programming

V=2 vt(s(t),c(t)) (5.29) t = \

subject to

s(t + l) = ht(s(t),c(t)), t = l,2,...,T, (5.30)

5(1) = 5!, S(T+1)=S, (5.31)

where sx and s are fixed exogenously. The usual dynamic programming terminology is as follows: In (5.29),

vt(s(t), c(t)) is the net benefit at time /. Equation (5.30) is called the tran- sition equation, and hf(s(t),c(t)) is called the transition function at /. The subscript t in vt and ht indicates that these functions may depend on t.

Problem (5.29), subject to (5.30) and (5.31), is a discrete version of a control problem and as such has the two fundamental properties of sep- arability and additivity over time periods. More precisely,

(i) for any t, the functions vt and ht depend on t and on the current values of the state and control variables, but not on their past or future values;

(ii) the maximand V is the sum of the net benefit functions.

Using these two properties, Bellman (1957) enunciates an important theorem about the nature of any optimal solution of problem (5.29). This theorem is known as the principle ofoptimality. Roughly speaking, it says that an optimal policy has the property that at any stage /, the remaining decisions c*(t), c*(t + l),..., c*(T) must be optimal with regard to the cur- rent state s*(t), which results from the initial state s{ and the earlier deci- sions c*(l), ...,c*(/ —1). This property is obviously sufficient for optimal- ity since we require it to hold for all t: when we put t = 1, we have the definitions of an optimal policy. Furthermore, the property is also neces- sary, since any deviation from the optimal policy, even in the last period, is clearly suboptimal. It was left to Bellman's genius to transform this rather trite, nearly tautological observation into an efficient method of solution. We now state the result formally.

Theorem 5.2.1: principle of optimality. c*(l), c*(2),..., c*(T) is an opti- mal solution to the problem (5.29)-(5.31) if and only if c*(0,c*(/ + l), ..., c*(T) solve the following problem for / = 1,..., T:

T

Maximize / ? , = £ VT(S(T),C(T)) (5.32) T = t

subject to

5(7+1) = hT(s(T),C(T)), T = t,t + l,t + 2,...,T, (5.33)

5.2 Dynamic programming in discrete time 175

s(t) = s*(t), (5.34a)

s ( r + l ) = 5. (5.34b)

This theorem implies that the values of c*(t), c*(t + 1 ) , . . . , c*(T) can be determined from a set of equations (namely, the necessary conditions of problem (5.32)) that do not contain past values of the control and state variables: the knowledge of c * ( ^ - l ) , c * ( ^ - 2 ) , . . . , c * ( l ) , 5 * ( / - l ) , 5 * ( / - 2 ) , and so on is irrelevant, and only / and s*(t) matter. This follows the fun- damental observation made earlier that the value of the state variables at some time summarizes all the relevant information about the system at that time.

Clearly, the principle of optimality relies on properties (i) and (ii) stated earlier. This principle does not apply to problems that cannot be put in the form (5.32), as the following example illustrates:

Maximize [c(l)c(2)] 1 / 2 + [c(2)c(3)]1 / 2 + [c(3)]1 / 2

subject to

s(t + l) = s(t)-c(t), / = 1,2,3,

s(l) =Si fixed, 5(4) =54 fixed.

A little reflection will establish that in this example, for given s*(2), we cannot determine c*(2) and c*(3) if we do not know c*(l) and Sj; in other words, c*(l), c*(2), c*(3), s*(2), and s*(3) must be determined simul- taneously.

We can prove the principle of optimality formally by establishing a con- tradiction. Thus, let c*(l),c*(2), . . . , c * ( / - l ) , c * ( 0 , . . . , c * ( r ) be an opti- mal solution to (5.29) and suppose that c*(t),c*(t + l), . . . , c * ( r ) did not yield a maximum for (5.32) for given s*(t). Let c(t),c(t + l),c(t + 2),..., c(T) be an optimal solution of (5.32). Then the path c*(l), c*(2), . . . , c * ( / - l ) , c ( 0 , ...,c(T) would yield a higher value V for (5.29) than the path c * ( l ) , c * ( 2 ) , . . . , c * ( / - l ) , c * ( / ) , . . . , c * ( r ) . This would contradict the hypothesis that the latter is an optimal solution of (5.29). This completes the proof of the principle of optimality.

Bellman's principle of optimality gives rise to an important equation called the functional recurrence equation, which is the key to the dynamic programming method of solution. Let Vt(s(t)) denote the maximum value of Rt in (5.32) for given s(t). We call this the return function. The prin- ciple of optimality implies that

Vt(s(t)) = max[vt(s(t),c(t)) + Vt+l(s(t + D)], (5.35a) c(t)

subject to

176 5 Calculus of variations and dynamic programming

s(f + l) = /i,(s(/),c(0), (5.35b)

5(0 given and 5(7>1) =5. (5.35c)

Combining (5.35a) and (5.35b), we get

K/(5(0) = max{t;,(5(0,c(0) + K/+1[/i/(5(0,c(0)]}. (5.36) c(t)

Equation (5.36) is Bellman's functional recurrence equation. It is called a functional equation because it implicitly defines the functional form of Vt(s(t))y which is the unknown entity. It provides the basis for an efficient method of solution called backward induction, which we explain below.

Example 5.2.1. We first illustrate how Bellman's equation works by using it to solve the control problem of Section 4.2. The problem is to find c(t), t = 1,2,3, to maximize

21nc(0

subject to

5(f + l) = l . l 5 ( 0 - c ( 0 ,

5(1) = 1, 5(4) = 1.21.

Thus, in dynamic programming notation we have

y,(5(0,c(0) = hic(0,

ht(s(t),c(t)) = 1.15(0 - c < 0 , 3

F,(5(0) = max 2 Inc(r), C(T) T = t

with 5 ( r + l ) = 1 . 1 5 ( T ) - C ( T ) , 5(0 given, and 5(4) = 1.21. Bellman's equation states

K,(5(0) = max[lnc(0 + K/+1(5(/ + 1))]. c(t)

We begin with the last period, when we simply have

K3(53) = maxlnc(3) c(3)

subject to 5(4) = l.l5(3)-c(3), 5(3) given, and 5(4) = 1.21. Since there is only one point in the choice set, we must have

c*(3) = l.l5(3)-1.21.

Hence, K3(5(3)) = ln[l.l5(3)-1.21].

5.2 Dynamic programming in discrete time 177

We now deal with the next period down the line:

K2(s(2)) = max[lnc(2) + K3(s(3))] c(2)

subject to

5(3) = 1.15(2)-c(2), 5(2) given.

Substituting for V3, we have

K2(5(2)) = max[lnc(2) + ln(l.l5(3)-1.21)] c(2)

= max[lnc(2) + l n ( 1 . 2 l 5 ( 2 ) - l . l c ( 2 ) - 1 . 2 1 ) ] . c(2)

The first-order condition yields

c*(2) = 0 . 5 5 ( 5 ( 2 ) - l ) ,

and substituting we have

F2(5(2)) = ln(0.33275) + 2 ln(5(2) - 1 ) .

Next,

K1(5(l)) = max[lnc(l) + F2(5(2))] c(l)

subject to

5(2) = l . l s ( l ) - c ( l ) , 5(1) given.

Substituting for F2(5(2)) and 5(2) yields

K1(51) = max[lnc(l) + ln(0.33275) + 2 1 n ( l . l 5 ( l ) - c ( l ) - l ) ] . c(l)

The optimality condition yields

C*(l) = ( l . l 5 ( l ) - l ) / 3 .

Using 5(1) = 1 and the expressions obtained for c*(l), c*(2), c*(3), and the transition equation, we work forward in time to obtain the optimal solution

c*(l) = 0.0333,

5*(2) = 1.1-0.0333 = 1.0667,

c*(2) = 0.55 (1.0667 - 1 ) = 0.0367,

5*(3) = 1.1(1.0667) - 0 . 0 3 6 7 = 1.1367,

c*(3) = 1.1(1.1367)-1.21 =0.0403.

Thus, we see that the backward induction method consists of solving the last period first, taking as given the value of the state variable and

178 5 Calculus of variations and dynamic programming

working backward until the first period, when we actually know the value of the state variable. We then obtain the optimal solution by retracing our steps.

We now give a more formal account of the procedure. At time T, for given s(T), we choose c*(T) that solves the problem facing the planner when there is only "one period to go":

max RT = vT(s(T), c(T)) (5.37) c(T)

subject to

s(T+l) = hT(s(T)9c(T)),

s(T) given, s(T+l)=s fixed.

This problem yields c*(T) as a function of s(T) (we suppress mention of its dependence on s):

c*(T)=gT(s(T)).

By definition, VT(s(T)) is the optimal value of RT for given s(T). There- fore,

VT(s(T)) = vT[s(T),gT(s(T))]. (5.38)

Working backward, at time T—\ we seek c*(T— 1) that solves the prob- lem facing the planner when there are "two periods to go":

Maximize RT_X = vT_x(s(T-1),c(T-1)) + VT(s(T))

= vT.x(s(T-1), c(T-1)) + vT[s(T), gT(s(T))] (5.39) subject to

s(T) = hT_x(s{T-\\ c(T-1)), (5.40)

s(T— 1) given.

This gives c*(T— 1) as a function of s(T— 1):

c*(T-l) = gT_l(s(T-l)). (5.41)

The optimal value of RT_i is obtained by substituting (5.40) and (5.41) into (5.39):

VT-l(s(T-l)) = vT_l[s(T-l)9gT_l(s(T-l))]

+ vT{hT-l[s(T-l),gT-l(s(T-l))]9

gTlhT-iMT-lhgT^MT-l))]]}, (5.42)

which is a composite of known functions and has s(T— 1) for sole ar- gument.

Proceeding in this way, we are faced, at each date / > 1, with the simple problem of finding c*(t) that maximizes

5.2 Dynamic programming in discrete time 179

Rt = vt(s(t),c(t)) + Vt + l(s(t + l)) (5.43)

subject to

s(t + l) = ht(s(t),c(t)), (5.44)

s(t) given.

The solution is expressed as

c*(t) = gt(s(t)). (5.45)

The process is repeated until we reach t = 1, and the problem is then to find c*(l) that maximizes

Rl = vl(s{l),c(\)) + V2(s(2))

subject to

s(2) = hl(s(l),c(l)), (5.46)

s(l) given (=sY).

This problem yields c*(l). The optimal value s*(2) is then computed using (5.46). Next (5.45) is used to obtain c*(2), which is in turn substituted into (5.44) to find s*(3). Again, (5.45) is applied to obtain c*(3), and so on.

We now apply the backward solution method to an economics problem.

Example 5.2.2: consumption policy. Let s(t) denote an individual's stock of wealth at time /, and c(t) the individual's consumption in period t. Let r be the rate of interest and let /3 = 1 + r. Then

s(t + l) = 0[s(t)-c(t)]. (5.47)

The individual wishes to find the time path c(t), t = 1,2,..., T, that maxi- mizes the sum of discounted utilities:

T

Maximize £ alu(c(t)) (5.48) t = \

subject to (5.47) and the boundary conditions

5(1) =sx fixed, s ( 7 > l ) = 0.

The term a1 is called the discount factor. We assume that 0 < « < 1 . The function u(c) is assumed to take the form

u(c) = [K/(l-y)]cl-> (K>0, 7 > 0 , 7 * 1 ) . (5.49)

Obviously, if we multiply the right-hand side of (5.49) by a positive con- stant, or add any constant to it, the solution c*(0, t = 1,2,..., T, will re- main unchanged.

180 5 Calculus of variations and dynamic programming

We proceed by applying the general approach described by (5.36)-(5.46) to the present example. For simplicity, we will write

A=K/(l-y).

Solving backward, we first determine c*(T) that maximizes

RT = * TA[c(T)]l-T

subject to

s(T+l) = 0[s(T)-c(T)],

5(r+l) = 0, s(T) given.

This yields

c*(T)=s(T). (5.50)

Hence,

VT(s(T)) = a TA[s(T)]l-y.

Next we find c*(T—l) that maximizes

RT_x = cL T-xA[c(T-\)]x-*+aTA[s(T)]1-^ (5.51)

subject to

s(T) = /3[s(T-l)-c(T-l)], (5.52)

s(T— 1) given.

Use (5.52) to substitute for s(T) in (5.51), differentiate the resulting ex- pression with respect to c(T— 1), and equate the derivative to zero:

Hence,

Let

Then

c ( r - l ) "

a/3l_1,=

•y = ap l-Y[s(T-i)-

s(T-l)-c(T-iy c(T-l)

- c ( r - i ) ] - *

T

>

c(T-\)

rf=(a/3'-1')1/1'. (5.53)

c*(T-l) = s(T-l)/(l+d). (5.54)

Substitute this into (5.51) to obtain the optimal value of RT-\-

Kr_,(j(r-1)) = ct T-xA[s(T- \)}x-\\+d)\ (5.55)

From (5.55) we obtain the expression for RT-2,

5.2 Dynamic programming in discrete time 181

RT_2 = a T-2A[c(T-2))l-T+aT-{A[s(T-l)]l-y{l + d)\

which must be maximized subject to

s(T-\) = (3[s(T-2)-c(T-2)], s(T-2) given.

This yields

c*(T-2)=s(T-2)/(l + d + d2). (5.56) Hence,

VT-2(s(T-2)) = a T-2A[s(T-2)]l-y{\ + d + d2)\ (5.57)

From (5.54)-(5.57) is it natural to guess that for any integer / (/ = 1,2,..., T— 1), the functions c*(T—i) and VT_i are of the form

c * ( r - / ) = 5 ( r - / ) / ( i - h ^ + r f 2 + - - - + r f / ) , (5.58)

VT_i(s(T-i)) = a T-iA[s(T-i)]l-'Y(l + d + d2+---+di)\ (5.59)

To show that our guess is correct, the method of proof by induction may be used. This method consists of showing that if (5.58) and (5.59) are correct for / = ra, they are also correct for / = ra + l. We leave this task to the reader.

It is appropriate at this stage to make some comments on the dynamic programming approach to control problems in discrete time. When we consider equations (5.50), (5.54), (5.56), or the general case (5.58), we see that our method of solution consists of devising a strategy or policy rule, given an arbitrary state of the system at each date. When we work back to initial time, for which we actually know the state of the system, we can begin to derive the particular optimal solution corresponding to this ini- tial condition. The solution as described by (5.58) is known as a closed- loop control because the optimal value of the control variable during pe- riod (T—i) is given as a function of the state variable at the beginning of that period. This is in contrast to an open-loop control, in which the solution is given as a function of time only. Typically dynamic program- ming yields a closed-loop solution, whereas the maximum principle yields an open-loop solution (e.g., see equation (4.45)). The reader must be warned that closed-loop solutions, such as (5.58), are not readily ob- tained for more complicated problems. The strength of dynamic pro- gramming in discrete time lies mainly in its applicability to numerical problems, and its usefulness in analytical economics is rather limited.

As we shall see in the next section, dynamic programming compares even less favorably with optimal control when time is represented as a continuous (i.e., real-valued) variable.

182 5 Calculus of variations and dynamic programming

5.3 Dynamic programming in continuous time

Bellman's principle of optimality, stated in Section 5.2, is clearly applic- able to the continuous-time version of problem (5.29). In this section we show that the continuous-time counterpart of the functional recurrence equation (5.36) yields a useful equation called the Hamilton-Jacobi-Bell- man equation, which provides an alternative method to optimal control theory for solving continuous-time control. In addition, the Hamilton- Jacobi-Bellman equation can be used to derive heuristically the basic ver- sion of the maximum principle as presented in Chapter 4. We must hasten to add that for more complicated control problems the techniques pro- vided by optimal control theory are more powerful; this is why we will concentrate on optimal control theory from Chapter 6 onward.

Since we will differentiate our equations with respect to time, it is use- ful to write our net benefit function asv(s(t),c(t),t) and our return func- tion as V(s(t), t). By definition,

V(s(t), t) = max j T V ( S ( T ) , C(T), T) dr (5.60)

subject to

s(t) given, s(T) = sT given,

S(T)=AS(T),C(T),T).

It follows from (5.60) and from the principle of optimality that

U + At K(s(0,0 = max j

/ + 'i;(s(T),c(7),T)tfT + K(s(/ + A 0 , ' + A0 (5.61)

For a sufficiently small At, we can rewrite (5.61) as

V(s{t),t) = max[v(s(t),c(t),t)At + V(s(t + At)9t + At)] c(t)

+ 0(At), (5.62)

where O(At) is the sum of higher-order terms in At and has the property

hm = 0. A / - 0 At

Now, assuming that V is continuously differentiate, we can write

V(s(t + At),t + At) = V(s(t),t) + VsAs + VtAt + 0(At)

= V(s(t),t) + [Vss + Vt]At + 0(At).

Substitute the above equation into (5.62):

5.3 Dynamic programming in continuous time 183

V(s{t)9t) = max[v(s(t)9c(t)9t)At + V(s(t)9t) CU) + Vsf(s(t)9c(t)9t)At + VtAt] + 0(At).

Cancel V(s(t), t) on both sides, divide the resulting equation by At, and take the limit At -> 0:

0 = max[v(s(t)9c(t)9t) + Vsf(s(t)Mt)J) + Vt]. (5.63) c(t)

Equation (5.63) is called the Hamilton-Jacobi-Bellman equation. If we define the Hamiltonian

H(s, c, 7T, t) = v(s, c, t) + *•/($, c, t)9 (5.64) where

x = K5(5,0, (5.65)

then (5.63) can be written as

- F , = max//. (5.66) c(/)

Equation (5.63), or (5.66), is a partial differential equation because it in- volves the partial derivatives of V with respect to s and t. In general, this type of equation is difficult to solve, even for very simple v and / functions.

In order to give the reader a better appreciation of the Hamilton- Jacobi-Bellman equation, we illustrate it with a simplified version of problem (5.23), with S = r = 0. Find c(t) that maximizes

( r i n ( c ( 0 ) * (5.67) Jo

subject to

S(t) = -c(t)9 J(0) = 50, s(T) = sT.

V is defined by

V(st91) = max f rin(c(r)) dr (5.68)

C(T) *t

subject to

S(T) = -C(T)9 s(t)=st given, s(T) = sT.

The Hamilton-Jacobi-Bellman equation takes the form

0 = max[ln(c(0) + (KJ)(-c(0) + K/]. (5.69) c(t)

The first-order condition yields c~l = VS9 and substituting in equation (5.69), we have

0 = - l n ( K 5 ) - l + K,. (5.70)

184 5 Calculus of variations and dynamic programming

This is a partial differential equation the solution of which would yield the function V(si91) and thus Vs and c. Unfortunately, the analytical solution of partial differential equations is a formidable problem, and we will not attempt to solve (5.70). We wish, however, to convince the reader that the solution obtained through dynamic programming is the same as that ob- tained from the maximum principle. To this end we derive V(st, t) directly from its definition in (5.68) using optimal control techniques and verify that the solution satisfies (5.70).

Form the Hamiltonian H(s, c, 7r) = l n c — ire and apply the maximum principle to obtain c _ 1 = 7r, TT = 0 , and s = — c; hence, C(T)=A, S(T) = B—AT, and the boundary conditions st = B —At and sT = B—AT deter- mine the constants

A = (st-sT)/(T-t)f

B = (stT-sTt)/(T-t).

Consequently, the return function is

V(s,91) = j r[ln(5, -sT) -\n(T- /)] dr

= (T-t)[ln(st-sT)-ln(T-t)]. (5.71)

It is easy to verify that

Vs = (T-t)/(st-sT)9 Vt = l-ln(st-sT) + ln(T-t)

do satisfy the partial differential equation (5.70). This completes our exposition of dynamic programming. Although not

well suited to the analytical approach we wish to follow in this book, it is a very useful technique. Interested readers may consult Bellman and Dreyfus (1962).

Exercises

1. Use Euler's equation to solve the following calculus of variations problems with boundary conditions: (a) Maximize Jj - (s)2dt, 5(0) = 0, s(l) = 5, (b) Maximize Ji-(fo-(5)2)flfr, 5(0) = 0, 5(1) = 2, (c) Maximize ]l0(4s-(s)

2-(s)2)e-°Atdt, 5(0) = 0,5(1) = 4. 2. Let k be capital stock, f(k) output, and c consumption. Let U(c) be the utility

function, where c=f(k) — k. Use Euler's equation to find the necessary condi- tions for the following problem: maximize j ^ U(f(k) — k)dt subject to &(0) = k0, k(T) = kT. Show that the resulting second-order differential equation in k can be transformed into a pair of first-order differential equations in c and k.

Exercises 185

3. Show that if the integrand in equation (5.7) does not have t as a separate argu- ment, then Euler's equation implies F-sF^ = Af where A is an arbitrary con- stant. (Hint: Take the time derivative of equation (5.7), divide by s, and com- pare it with Euler's equation.)

4. Derive the Hamilton-Jacobi-Bellman equation for the following problem: Maximize \TQ\n(c(t))dt subject to s(t) = rs(t) — c(t), s"(0) = s0, s(T) = sT and show that the following return function satisfies that equation:

V(sl,t) = {T-t)\n( S,e~"-*e~'T^ + t(Ti-t*).

Use optimal control theory to obtain this return function. (Hint: Use C(T) = Berr in s(T) = rs(r)-c(T); hence, s(r)e~rT = A-BT. Then solve for A and B using s(t) = st and s(T)=sT.)

5. Consider the problem of maximizing \TQ\n(c(t))e~ 8t dt (5 > 0) subject to s(t) =

rs(t) — c(t), s(0) = s0, and s(T) = sT. Show that the following return function satisfies the Hamilton-Jacobi-Bellman equation,

V(st, t) = 5- l(e-8t-e-8T)\n(5B)

+ 5-l(r-d)(te-bt-Te-6T) + 82(r-5)(e-8t-e-8T),

where B = e-r{t + T)(ertsT-e rTst)/(e-

8T-e-8t).

CHAPTER 6

The general constrained control problem

In Chapter 4, we studied the problem of finding a function c(t) that maximizes

V=\Tv(s(t),c(t),t)dt Jo

subject to

& = f(s(t),c(t),t),

s(0) = s0, s(T) = sT.

We now wish to consider a more general control problem involving many state variables and many control variables. We also wish to intro- duce constraints on the values that the control variables may take on at any point of time and also constraints on their overall paths from time 0 to time T. For ease of exposition, we retain the assumption that initial time and terminal time are exogenously specified, as are the initial and terminal values of the state variables; these conditions will be relaxed in Chapter 7.

6.1 The set of admissible controls

In many economic problems, the set of values that the control variables may take on is restricted. For example, consumption cannot be greater than output, and the rate of extraction of a natural resource may not ex- ceed a certain upper bound specified by environmental control laws.

In order to be more specific let us consider a version of the optimal con- trol problem studied in Chapter 4. Let s(t), c(t), and I(t) denote re- spectively, the stock of capital, the flow of consumption, and the flow of gross investment at time t. Let m > 0 be the rate of natural depreciation of capital. Then the rate of change in capital stock is

s(t) = Ht)-ms(t). (6.1)

We assume that output at time t is given by eytF(s(t))9 where y is the rate of technical progress and F the production function. At any time t, the following constraints are imposed on the control variables /(/) and c(t):

187

188 6 The general constrained control problem

\V7//fy>>^ A, V////////>^ x Y////////7)\. o. *

Ke** ?K A- fyfo

x Figure 6.1

I{t) + c(t)<e^F(s(t)), (6-2)

7 ( 0 * 0 , (6.3)

c{t)izc. (6.4)

Condition (6.2) states that the sum of consumption and gross investment cannot exceed total output at any instant. Condition (6.3) means that gross investment cannot be negative (negative gross investment, which was allowed in Chapter 4, means that capital stock can be consumed). Condition (6.4) stipulates a lower bound on the rate of consumption. The set of admissible controls is therefore the striped area in Figure 6.1. Note that the northeast frontier of the set moves with time and depends on the value of the state variable s.

Since the set of admissible controls depends on t and s(t), we shall denote it by the symbol W(s(t), t). Formally,

W(s(t)9t) = {(I,c):I>:09 c>c, I+c<e ytF(s(t))}.

It is important to note that (6.2) is a constraint on the values of I(t) and c(t) for given values of t and s{t), and not a constraint on the value of the state variable s(t). (It is, of course, possible to specify constraints on the values of the state variables alone, e.g., s(t) < s or s(t) > £ ; how- ever, these constraints give rise to much more complicated necessary con- ditions, since they also imply a constraint on s(t) when s(t) = s or s(t) = s; for this reason we will not deal with constraints on state variables in this chapter and defer this topic until Chapter 10.)

6.1 The set of admissible controls 189

We are now ready to state the constrained problem that is the subject of this chapter in its most general form.

The constrained control problem

Assume that there are r control variables, n state variables, and m con- straints on the control variables. Suppose that the first m' constraints are inequality constraints and that the remaining m — m' constraints are equal- ity constraints. Then the set of admissible controls can be represented by m' inequalities and m — m' equations.

For notational convenience, we now adopt the convention of writing s(t) and c(t) for the vectors of state variables and control variables, where *(t) = [si(t),s2(t)9...,sm(t)]' a n d c(t) = [cl(t)9c2{t)9...9cr(t)]'. W e c a n then write the constraints as

gi(s(t)9c(t)9t)>09 1 = 1 , 2 , . . . , m ' ; (6.5a)

gJ(s(t)9c(t)9t) = 09 y = m ' + l , . . . , m . (6.5b)

The set of admissible controls is

W(s(t)9t)^[e(t)\g i(s(t)9e(t)9t)^09i = l929...9m';

gJ{s(t),c(t)9t) = 09 j = m'+l9...9m). (6.6)

Since we are dealing with constraints on control variables, it is impor- tant to note the requirement that each constraint contain at least one con- trol variable. In addition, since we will have to find values of the control variables that maximize a Hamiltonian function subject to the constraints (6.5a) and (6.5b), we must require that these constraints satisfy one of the "constraint qualifications" discussed in Chapter 1 (see Lemma 1.4.1). The most convenient constraint qualification is the rank condition, which we restate here.

The rank condition. If the vector c(t) = (cx(t)9... 9 cr(t))' satisfies con- straints (6.5a)-(6.5b) for given values of sx(t)9 ...9sn(t) and t9 and if p of these constraints are satisfied with equality (p>m — m'9 because m — m' is the number of equality constraints), then it is required that the matrix (of order p x r) of partial derivatives of these p constraints with respect to the control variables be of rank p.

This constraint qualification implies in particular that while the num- ber of constraints (m) may exceed the number of control variables (r)9 we require that the number of active constraints (p) not be greater than the number of control variables. In what follows we shall assume that the rank condition is satisfied.

190 6 The general constrained control problem

There is an additional requirement, or rather a lack of one, on the con- trol variables. Heretofore we have been implicitly assuming that the vari- ables were differentiable functions of time. We know that if the variables were simply continuous, we would have to deal with left-hand-side and right-hand-side derivatives, and this would somewhat complicate the al- gebra. However, control theory actually encompasses far more general problems. It is capable, and this is one of its great achievements, of deal- ing with control variables that have a finite number of discontinuities of the "jump" kind. Formally we say that a variable is apiecewise-continuous function of time if and only if it is continuous almost everywhere, that is, anywhere but at a finite number of points, where it may exhibit jump dis- continuities. It would seem rather queer to complicate things so much were it not for the fact that even in some simple problems, no solution would exist if jump discontinuities were disallowed. To prove our point it suffices to consider a trivial example.

Find c(t) that maximizes

[ 2 - [ 5 < f ) - l ] 2 * Jo

with s(t) = c(t) and a double restriction on the control 0 < c(t) < 1; 5(0) = 0, 5(2) = 1.

Clearly, to minimize the squared deviation of s from 1 (given that 5(2) must equal 1) we must increase s at the highest speed, c = 1, until the goal is reached, 5 = 1, and then leave 5 at 1 by setting c = 0. Therefore, the opti- mal solution involves setting c = 1 from time 0 to time 1 and henceforth setting c = 0. The optimal control has a jump discontinuity at time 1.

Unless we wish to disregard such problems, we must impose only the re- quirement of piecewise continuity on control variables. All results in this chapter formally mention this qualification, but we will not study examples of discontinuous controls until Chapter 8. We will, however, encounter optimal controls that are continuous but not everywhere differentiable.

In the remainder of this chapter we shall see that the maximum prin- ciple for the constrained problem essentially involves working with a Lagrangean-type expression that incorporates the constraints (with mul- tipliers) as well as the Hamiltonian, rather than the Hamiltonian alone. Before we proceed, we must deal separately with a seemingly different kind of constraint.

6.2 Integral constraints

In addition to constraints on control variables at any instant, we may wish to allow for another kind of constraint that imposes a restriction on

6.2 Integral constraints 191

the overall path of the variables. These will naturally take the form of integral constraints. For example, Robinson Crusoe may wish to choose the time paths of the control variables xu x2, cu c2, and / (respectively his own labor, Man Friday's labor, his consumption, Man Friday's con- sumption, and gross investment) so as to maximize his own utility,

Vi=\T VtiXtithCiV), s(t),t)dtf (6.7) Jo

subject to the constraint that Man Friday's utility be equal to a given constant F,

\Tv2(x2(t), c2(t), s(t), t) dt = V, (6.8) Jo

and other constraints, such as

s(t) = IU)-ms(t),

cx{t) + c2(t) + I{t)<F(xx{t),x2(t),s(t))>

5(0) = s0, s(T) = sT.

On reflection, an equality integral constraint such as (6.8) presents no new problem, for it can be replaced by a new differential equation, with two boundary conditions, for a newly defined state variable k(t). Thus, (6.8) can be replaced by

k(t) = v2(x2(t)9 c2(t), s(t), t)9 (6.8a)

£(0) = 0, (6.8b)

k{T)=V. (6.8c)

To verify that (6.8a)-(6.8c) are equivalent to (6.8), we note that (6.8a) can be integrated to give

k(t) = V2(X2(T), C2(T), S(T), T) dr, Jo

and hence k(T) is the left-hand side of (6.8). Similarly, any inequality integral constraint (e.g., (6.8) with = V re-

placed by > V) can be replaced by a differential equation together with two boundary conditions: an equality boundary condition and an inequal- ity one (&(0) = 0, k(T) > V). Although we have not dealt with inequality boundary conditions yet, they will be examined in the next chapter.

In what follows, we shall assume that all integral constraints have been transformed into differential equations with boundary conditions. This simplifies the exposition without loss of generality.

192 6 The general constrained control problem

6.3 The maximum principle with equality constraints only

In this section we state the maximum principle for the case in which all con- straints are equality constraints, and work out a simple example. We con- sider the following problem: find piecewise-continuous functions c(t) = [Ci(t)>c2(t),...,cr(t)Y that maximize

V=\Tv(s(t)Mt),t)dt (6.9) Jo

subject to n differential equations,

*/(0 = / W ) , c ( / U ) , i = l,2,...,/i, (6.10a)

m equality constraints,

gj(s(t),c(t),t) = 0, 7 = 1,2,...,m, (6.10b)

and 2/7 boundary conditions,

Si(0) = si0, si(T) = siT, i = l , 2 , . . . , / i . (6.10c)

In this problem, both the initial time and the terminal time are fixed. The boundary values si0 and siT, / = 1,...,/!, are also exogenously speci- fied. The functions v, / ' , gj are assumed to possess continuous second- order partial derivatives.

As one would expect, the necessary conditions for this problem are quite similar to the ones presented in Chapter 4. There are, however, cer- tain differences. First, since there are n state variables, we must have n costate variables: Tri(t),ic2(t),...,Trn(t). The Hamiltonian is therefore

ms(t)9c(t),*(t),t) = v(s(t)Mt),t)+ S M O / ' ' ( s ( 0 , c ( f U ) . (6-H) /=I

Second, at any given time t and for given values of s^t) and ?r/(0> * = 1,..., n, the control variables cx(t),..., cr(t) must maximize the value of the Hamiltonian subject to the constraint that the values taken by the controls belong to the set of admissible controls W(s, t) as defined by (6.6), with m' = 0. This means that one must introduce a multiplier \j(t) for each constraint gJ9 form the Lagrangean function,

m

£ = ms(t),c(t),*(t),t)+ 2 X y (0* y (s(0,c(fU), (6.12)

y' = i

find the derivatives d£/dciy i = 1,..., r, and equate them to zero. Third, the differential equations for the costate variables are now

^ ( 0 = - ^ — - r

6.3 Maximum principle with equality constraints 193

(and not just —dH/ds^t) as before), so as to take into account the effects of present changes in the state variables on the future set of admissible values of the control variables represented by the constraints gl9 g

2 9..., g

m

included in the Lagrangean function. For future reference, we state the necessary conditions that constitute

the maximum principle in a more formal and complete manner.

Theorem 6.3.1: necessary conditions for the equality-constrained prob- lem. Let c*(t) = [c*(t)9 ..., c*(0]' be an optimal solution to the control problem (6.9)-(6.10) and s*(t) = [s*(t),...9s%(t)]' be the corresponding time path of the state variables. Then there exist costate variables ir(t) = [iriit),..., Tn(t)]' and Lagrange multipliers \(t) = [\\(t)f..., \m(t)Y such that:

(i) At any time t9 for given vectors s*(0 and ir(t)9 the control vari- able vector c*(t) maximizes the Hamiltonian

H(s*(t)9 c(t)9 *(t), t) « v(s*(t)9 c(f),1)

+ 2 > / ( 0 / ' ( s * ( 0 , c ( 0 , 0 (6.13) i = i

subject to the condition that c(/) belongs to the set of admissible controls defined by (6.10b).

(ii) The costate variables 7rz(0, / = 1,2,...,«, are continuous func- tions of t and have piecewise-continuous derivatives (with re- spect to time) that satisfy the condition

*'<*> = - 7 T 7 ^ ; 1 = 1,2,...,/!, (6.14) dSi(t)

where the asterisk indicates that the partial derivatives of £ from (6.12) are evaluated at (s*(t)9c*(t))'9

(iii) Equation (6.10a) holds, or from the definition of the Lagrangean,

d-Ki(t)

It is also required that s*(t) satisfy the boundary conditions (6.10c):

5/(0) = 5/0, si(T) = siT9 / = l,2,...,/i. (6.15b)

(iv) The Lagrange multipliers X/(0, / = 1, 2, . . . , m, are piecewise- continuous on 0<t<T and are continuous whenever c*(t) is continuous. (6.16)

(v) The Lagrangean <£(s*(0, c*(0, *(t), MO, t) is a continuous func- tion of time on 0 < t < T. On each interval of continuity of c*(/), the Lagrangean is differentiable totally with respect to t9 and

194 6 The general constrained control problem

d£* d£*

-dT=-ar- (6-17) where d£*/dt is the partial derivative of £(s*(t), c*(0, ?r(0, A(0, t) with respect to the last argument. (Condition (6.17) can be derived from (6.13)-(6.17); see Section 6.5.)

Remark (a). Condition (6.13) can be stated formally as

H(s*(t)9 c*tf), * ( 0 , /) * H(s*(t), c ( 0 , * ( 0 , 0 , (6.18)

for all c ( 0 satisfying g y ( s * ( 0 , c ( 0 , 0 = 0, y = l , 2 , . . . , m , or since we as- sumed that the rank condition is satisfied,

dJu* = 0, / = 1 , 2 , . . . , r , (6.18')

where the asterisk indicates that the partial derivatives of (6.12) are evalu- ated at (s*(t),c*(t)).

Remark (b). We should, strictly speaking, include an additional constant 7r0 in our statement of the necessary conditions, where ir0 is the constant multiplier associated with the integrand of the objective function. In other words, we should have defined the Hamiltonian as

H=ic0v(s(t),c(t),t)+ S 7 r , ( 0 / W ) , c U U ) .

If 7r0^ 0, we may set 7r0= 1 because the equations for .itj(t) are linear in (7r0,7TJ,..., 7r„). Only in pathological cases does 7r0 equal zero; we will not be concerned with these cases. The reader may consult Long and Vousden (1977, pp. 16-17) or Athans and Falb (1966, p . 291) for further discussions.

We now show how the necessary conditions can be used to solve a con- trol problem involving one state variable, two control variables, and an equality constraint on the control variables.

Example 6.3.1: extraction of an exhaustible resource. Let s(t) denote the stock of an exhaustible resource and x(t) the rate of extraction from that stock, so that

s(t) = -x(t). (6.19)

The output of the finished good is a function of both the rate of extrac- tion and the stock size s(t). This output is denoted by F(s(t),x(t)). It is assumed that the function F is positive and increasing in each argument

6.3 Maximum principle with equality constraints 195

and that output is zero if x(t) is zero. This may reflect the fact that higher- yielding ore is obtained when s is relatively large. Let c(t) denote the flow of consumption of the finished good, and u(c(t)) the utility function with the usual properties (w'(0) = oo, w'(c)>0, w"(c)<0). We also as- sume that the output of the finished good cannot be stored. Therefore, consumption must equal output:

F(s(t),x(t))-c(t) = 0. (6.20)

Assuming for simplicity that the rate of discount is zero, the problem is to find the time paths of the control variables c(t) and x(t) that maximize

r=[Tu(c(t))dt (6.21) Jo r Jo

subject to (6.19), (6.20), and the boundary conditions s(0) = 50, s(T) = sT (<s0).

(The assumptions that w'(0) = oo and F(s, 0) = 0 guarantee that both c(t) and x(t) are positive along an optimal path; therefore, we do not have to specify the constraints c(t) > 0 and x(t) > 0.)1

Let 7r(0 be the costate variable. The Hamiltonian for this problem is

H(s(t),c(t),x(t),Tr(t)) = u(c(t))-Tr(t)x(t),

and the Lagrangean is

£ = u(c(t))-ir(t)x(t) + \(t)[F{s(t),x(t))-c(t)].

The maximum principle yields the following conditions:

d£ dc(t)

= u'(c(t))-\(t) = 0, (6.22)

= -*(t) + \(t)Fx = 0, (6.23) dx(t)

ds(t) * ( 0 = - ^ 7 7 7 = - M O F J , (6.24)

*(0=-7^- = -x(t). (6.25) OTr(t)

We now attempt to interpret these necessary conditions. From (6.22) we see that X is the imputed value of the finished good; it is positive since

1 In this simple example we could obviously use (6.20) to eliminate c and reduce the prob- lem to an unconstrained one with a single control. Our aim, however, is to introduce the reader to the use of the maximum principle in a constrained problem.

196 6 The general constrained control problem

u'(c) > 0. It follows from this and (6.23)-(6.24) that the value of the re- source, 7r, is positive and is decreasing over time (recall that F is increas- ing in both arguments). Equation (6.24) says that the rate of depreciation of the value of the resource, — TT, is equal to the marginal product of the resource in the production of the finished good multiplied by the value of the finished good. Thus, our interpretation in Section 4.5 carries over to problems with constraints on the control variables. In equation (6.23), TT is to be interpreted as the cost of a marginal increase in x; the other term is the marginal value of the contribution of x to the finished good.

We now proceed to show how an explicit solution may be obtained when both the utility function and the production function are specified. To this end we assume that

u(c) = (l/y)cy and F(s,x) = s"xf*, (6.26)

where 7 < 1 , a > 0 , / 3 > 0 , and a + /3<\. Using (6.26), conditions (6.20) and (6.22)-(6.25) take the special form

c = sax^ (6.20')

ci~l = \9 (6.22')

ir = \(3saxV-\ (6.23')

7r=-\asa-lx^ (6.24')

s = -x. (6.25')

Taking the ratio of (6.24') and (6.23') yields

*/ic=-(a/P)(x/s) = (*/P)(&/s). (6.27)

Substituting (6.20') and (6.22') into (6.23') to eliminate X, we obtain

T = 0S°^X^'1. (6.28)

Differentiating (6.28) with respect to time yields, after simplification (us- ing (6.28) itself),

ir/ir = ays~ls + (i3y—l)x~lx.

Since x=—s and x= — s, we have

ir/ir = ay(s/s) + ( j S 7 - 1 )<*/*). (6.29)

Together (6.27) and (6.29) yield

(S/s)(a0-l-ay) = (py-l)(s/S),

and finally, provided that (07 — 1)5*0 so that we can divide both sides by it,

6.3 Maximum principle with equality constraints 197

5/&=-(*/0)(&/s). (6.30)

This is a second-order differential equation, but it is so simple that it can be solved directly by repeated integration. Recall that \(ii/u)dt = ln| u| + C, and integrate both sides of (6.30) to obtain

In|*| = -(a/j8)ln|s|+i4.

Since s < 0 and s > 0, this is equivalent to

\n(-s) = -(a/p)\ns+A.

Applying the exponential operator to each side, we have

s=-eAs-a/fi or s{sa/l3) = -eA,

which can be integrated again to yield

[(3/(a + l3)]s{a+0)/V= -teA + B.

Recalling that eA>0 and (3/(a + (3) > 0 , we can rewrite the preceding equation as

where E and G are arbitrary constants, E > 0. Finally,

s(t) = (-Et + G)m<x+^. (6.31)

The constants of integrations E and G can be determined using the bound- ary conditions

s(O) = Gma+0) = so, or G = s ( o

a+0W,

s(T) = (-ET+ Gf/(a+0) = sT,

-ET+G = s(T a+m,

-ET=s(T a+0W-s(o

a+m<O.

It remains to find the optimal paths x*(t) and c*(t). From (6.31),

x*(t) = -s*=(E(3/(a + P))(-Et + G)-a/{a+l3)>0, (6.32)

c*(t) = (s*)a(x*)l3=[Ep/(a + 0)]l3 = const. (6.33)

This completes our solution of the problem, given that the utility function and the production function take the special forms described by (6.26).

In economic theory, it is desirable to find out the extent to which a par- ticular result (e.g., constant consumption) depends on the special func- tional forms assumed. The remainder of this section is devoted to this task.

198 6 The general constrained control problem

First note that the constants E and G are independent of 7; hence, none of the optimal paths of x, S, and c depend on this parameter. There- fore, the particular value of 7 does not affect the "primal" part of the solution. (It does affect IT and X, however.)

Furthermore, suppose that for some utility function u we have found an optimal path with the property that c is constant. Then consider the same problem, with u replaced by another form u, where «(c) still satis- fies the property that marginal utility is positive and decreasing. Equa- tion (6.22') is altered. Let us look at the previous optimal (asterisked) solution as a candidate solution. Equations (6.20') and (6.24') remain sat- isfied. Use (6.22) to define the constant X = «'(c*) and similarly scale IT to obtain it — 7r*X/X*. Equations (6.23') and (6.24') are automatically satis- fied because of their linear homogeneity and because of the constancy of both X* and X. We have thus found the "new" optimal solution, with **, s*, and c* being the same as before.

Since we have already found an optimal consumption path for u = (l/y)cy, this procedure always yields the solution. Sufficiency is guaran- teed because u is concave. We can state our result more formally: in Ex- ample 6.3.1, the consumption and extraction policies are independent of the utility function. The constancy of the consumption path relies heav- ily on the Cobb-Douglas form of the production function. Indeed, any homogeneous and concave production function with a constant elasticity of substitution different from unity (hence, not a Cobb-Douglas pro- duction function) will yield a nonconstant consumption path. In fact, a slightly stronger assertion can be proved: within the class of homothetic and concave production functions F(s,x), the optimal consumption path is a constant c* only if the production function exhibits a unitary elastic- ity of substitution in some neighborhood of the set [(s, x): F(s, x) = c*, s0<s <sT}. The proof of these two assertions is the subject of exercises at the end of this chapter.

6.4 The maximum principle with inequality constraints

We now turn our attention to the case in which the set of admissible con- trols can be described by m inequality constraints:

gj(si(t),s2(t),...,sn(t),c1(t),...,cr(t),t)>0, y = l , 2 , . . . , m ,

where each gj function is continuously differentiate in all arguments. Our control problem is to find the vector of controls c(t) that maximizes

V=[Tv(s(t),c(t),t)dt (6.34) Jo

subject to

6.4 Maximum principle with inequality constraints 199

si(t) = f(s(t)9c(t)9t)9 I = 1 , 2 , . . . , / I , (6.35)

gJ(s(t)9c(t),t)*0, y = l , 2 , . . . ,m9 (6.36)

S/(0) = s/ 0, Si(T) = siT9 / = l , 2 , . . . , / i . (6.37)

The necessary conditions for this problem are identical to those stated in Section 6.3 (Theorem 6.3.1), except that the set of admissible controls is now

W(sm(t)9t)m[c(t)\gJ(s*(t)9c(t)9t)^09j = l9...9m) (6.38)

and the Lagrange multipliers X1,...,XW are nonnegative. Thus, the fol- lowing familiar complementary slackness conditions hold:

\j(t) > 0, g'"(s*(0, c*(0, t) > 0, and \j(t)gJ(s*(t)9c*(t)9t) = 09 y = l , . . . , m .

(The reader may refer to Section 1.4, where the complementary slackness conditions are explained in detail.)2

For problems with a sufficiently simple inequality constraint and one control, it is sometimes possible to obtain a solution by using both the method of Chapter 4 when the constraint is not binding and the method of Section 6.3 when the constraint binds. The first example of this section falls in that category; subsequently, we present a more complex example.

Example 6.4.1: optimal consumption with irreversible investment. In Sections 4.4 and 4.5, we studied a problem of optimal consumption. We did not impose the restriction that consumption be no greater than out- put. Thus, along paths such as (i) in Figures 4.2, 4.3, and 4.6, consump- tion may exceed output, implying that the capital stock can be consumed directly (or converted into the consumption good without cost). In this section we wish to investigate the effect of the constraint that consump- tion not exceed output:

c(t)*F(s(t)).

We also adopt the assumption made in Section 4.5, that capital depre- ciates at the rate m9 so that

s(t) = F(s(t))-c(t)-ms(t). 2 For simplicity of notation, we shall adopt the convention of excluding from the La-

grangean any nonnegativity constraints on the control variables. As a consequence if the constraints contain inequalities such as Cj(t) >0, the first-order condition (6.18') must be modified to

as for a Kuhn-Tucker problem.

200 6 The general constrained control problem

We retain the assumption that instantaneous utility is a function of consumption and that this utility flow is discounted at a rate 5 > 0, with u'(c) > 0, w'(0) = oo, u"(c) < 0; we also assume F ( 0 ) = 0, F'(s) > 0, and F f ( 5 ) < 0 .

In addition, we shall assume as in Section 4.5 that F ' ( 0 ) > 6 + m > F'(oo) so as to ensure the existence of a steady-state equilibrium point. As before, the assumption w'(0) = oo ensures that c*(t) is positive along an optimal path. We could have replaced this assumption by the require- ment that c(t) > c, where c is some prespecified nonnegative lower bound on consumption. We refrained from doing so in order to keep our first example as simple as possible.

Our problem is to find c*(t) that maximizes

V=\Tu(c(t))e-6tdt (6.39) Jo

subject to

s(t) = F(s(t))-c(t)-ms(t),

F(s(t))-c(t)>:09

s(0) = s0, s(T) = sT.

The Hamiltonian is

H(s9c9Tr,t) = e- 8tu(c(t)) + Tr(t)[F(s(t))-c(t)-ms(t)),

and the Lagrangean is

£(S9 C, 7T, X, t) = H(S9 C, 7T, t) + \(t)[F(s(t)) ~C(t)].

The optimal solution must satisfy the following conditions (the asterisk is omitted where no ambiguity arises):

(i) c*(t) maximizes H(s, c, 7r, t) subject to the constraint that F{s(t)) — c(t) > 0. In terms of the Lagrangean, this means

^ = e -btu'(c(t)) - T(t) - \(t) = 0 (6.40) ac

with

X ( / ) ^ 0 , F(s(t))-c(t)*09 \(t)[F(s(t))-c(t)] = 0. (6.41)

(ii) * = - ^ = -[ic(t) + \(t)]F'(s(t)) + mic(t). (6.42) OS

(iii) &=^ = F(s(t))-c(t)-ms(t). (6.43) OIT

6.4 Maximum principle with inequality constraints 201

Recall that in Chapter 4 we constructed phase diagrams in the (s, c) space after deriving a differential equation for the control variable (equa- tions (4.61) or (4.85)). Unfortunately, that method will not work here because it would introduce a X term that cannot be eliminated with (6.41). The way to deal with such problems is in general to construct a diagram in the (state, costate) space; this is done in the next example for a more complicated case. Here, however, there is a single control variable and the inequality constraint is a simple bound on the control. This makes it possible to proceed directly to a phase diagram in the (s, c) space. If the bound is inactive, we can use the same method as in Chapter 4; if the control is on the boundary, we make use of that information to obtain the solution. In what follows we apply this simple idea to the present example.

Let us consider the two cases separately: case A, in which the bound is active, and case B, in which it is not. In case B, the results are the same as in Section 4.5. Using equations (4.84) and (4.85),

s = F(s) — ms — c, c=-[u'(c)/u"(c)][F'(s)-m-6]9

the phase diagram is easy to draw. This is done in Figure 6.2. (It is similar to Figure 4.6, "capped" by c = F(s).) Note that these phase lines are valid only when c < F(s). When X > 0, the equality holds and we are in case A, to which we now turn.

In case A the bound is active. This means that c = F(s)\ hence, s = —ms and both c and s are negative. Therefore, starting from any point on the graph of c = F(s) the optimal trajectory moves in the southwest direc- tion. Let s denote the value of s at the equilibrium. If s < s, this trajectory moves down along the graph of c = F(s) and cannot stray away from it, since in the proximity of this graph, c must increase; this is displayed in Figure 6.2. If s>s9 the optimal path follows the c = F(s) graph down- ward; it may, however, leave this graph to enter region B, in which the constraint ceases to be active.

The choice of the optimal policy depends on the values of s0, sT, and T. We now have a good opportunity to enhance our understanding of dynamic optimality by comparing optimal trajectories with and without the constraint, for various specified values of the parameters. This is done in Figure 6.3, which is to be compared with Figure 4.6. We take it that the two problems are identical save for the constraint c<F(s)\ in par- ticular, in each case the values of s0, sT, and T are common to the two problems.

The case in which trajectory (i) of Figure (4.6) was optimal may now have no feasible solution. Indeed, if T is small enough, there may not be

202 6 The general constrained control problem

Figure 6.2

enough time to go from s0 to sT with the maximum consumption allowed c = F(s).

When time is more plentiful but the optimal trajectory remains in re- gion I with consumption rising all the while (path (ii)), we may now en- counter the upper bound. If we were to follow the unconstrained path (ii) initially, we could not reduce the stock to sT because when the con- straint binds later on, the stock diminishes more slowly than along the unconstrained path. Therefore, we must start at a higher level of con- sumption initially. This results in the constrained optimal path labeled (ii'c). These paths are plotted against time in Figure 6.4, where (iic) repre- sents the (unfeasible) path that coincides with (ii) before reaching the bound. The kinks on consumption paths (iic) and (ii'c) occur at the time the constraint begins to bind.

If the unconstrained path begins in region I and eventually crosses the curve c = F(s), as (iii) does, we must now choose the slower trajectory (iii'c). An interesting feature is that we now actually begin with a lower

6.4 Maximum principle with inequality constraints

ci

c = 0

(ii)

c = F(s)

III

r (iic) s = 0

IV (iii)j'

(iiic)f

0 Sn

Figure 6.3

F(sT)

T r time

Figure 6.4

204 6 The general constrained control problem

Figure 6.5

consumption. This serves to build up the stock of capital so that there is a much higher level of consumption in the intermediate period before consumption begins to decline. The imposition of the constraint thus has rather far-reaching effects on the optimal path. Whereas consumption was previously rising monotonically with time, it now may go through a peak. The planner compensates for lower consumption levels when the constraint is active with higher levels of consumption before that time; in order to reach these high levels at an earlier time, it must start with lower consumption at the beginning of the horizon in order to accumu- late capital.

Before we leave this example let us mention that it is also possible to construct a phase diagram in the {s, \j/) space, where \[/ is the current-value costate variable. The main obstacle is that it must be shown that the con- trol and the multiplier can be expressed in terms of s and yp only, so that the (^,5*) system is autonomous. The reader will be guided through this task as an exercise at the end of this chapter. For now, let us simply dis- play the result in Figure 6.5, which is comparable to Figure 4.3. There

6.4 Maximum principle with inequality constraints 205

exists a unique equilibrium point ( 5 , $ ) , and it has the familiar saddle- point property. The differences between Figures 4.3 and 6.5 are now illus- trated. Take, for example, the path MNP in Figure 6.5 and compare it with the path starting at the same point M in Figure 4.3. The two paths are denoted by (iic) and (ii), respectively, where the subscript c stands for "constrained." Clearly, both paths are identical along MN. However, once path (iic) enters region A the capital stock will fall at the rate ms, because consumption just equals output in that region. Along path (ii) consumption is increasing over time; hence, it exceeds output once the trajectory passes point N. Thus, (iic) takes a longer time (say, T'>T) to reach the prescribed final stock sT. Therefore, for a common fixed time horizon T, if path (ii) is optimal for the unconstrained case, then path (iic) is not optimal for the constrained case, because it does not satisfy the boundary condition s(T) = sT. The optimal path must therefore start with a higher rate of consumption; path (ii'c) is one such path. These are the same consumption profiles illustrated in Figure 6.4.

Example 6.4.2: the mushroom grower's problem. In this example we model the husbanding of a resource, the growth of which can be con- trolled by both harvesting and cultivating. Let s(t) denote the stock of mushrooms at time t. In the absence of harvesting for consumption or cultivation, this stock is assumed to grow at the rate s = \ns(t). Let c(t) denote consumption and x(t) denote cultivation effort. We assume

s = lns(t)-c(t)+F(x(t)9s(t)),

where F(x, s) takes the special form F(x, s) = s[l — e~x/s]. Note that this F function is nonnegatively valued, strictly increasing, homogeneous of degree 1, and concave in (x, s) when x > 0 , s > 0 . We assume that har- vesting for consumption cannot exceed an upper bound, which depends on current stock s and the (fixed) quantity of harvesting equipment E\ for simplicity we take the constraint to be c < sE, where E=l. Consump- tion and cultivation effort must be nonnegative. The present value of in- stantaneous utility is assumed to be v(c, x, t) = (ln(c) —x)e~rt9 where r > 0 is the discount rate. Recapitulating, the mushroom grower's problem is to find c and x that maximize

V=[T[\nc-x]e-rtdt (6.44) Jo

subject to

s = lns-c+s[l-e-x/s], (6.45)

c < 5 , (6.46) c>0, x > 0 , (6.47)

206 6 The general constrained control problem

s(0) = s0, s(T) = sT. (6.48)

For concreteness we assume that r<e~l; other cases will be discussed briefly. The current-value Hamiltonian and Lagrangean are, respectively,

H=lnc-x+\ls[lns-c+s(l-e-x/s)], (6.49)

£ = H+\[s-c]. (6.50)

Applying Theorem 6.3.1, modified as per equation (6.38'), we obtain the necessary conditions:

M = c - i _ ^ _ x < 0 , c > 0 , c ( c - J - ^ - X ) = 0, (6.51)

oc

M = - l + ^ - * A ^ 0 , x > 0 > x ( - l - h ^ - ^ / 5 ) = 0, (6.52)

^ - = s - c > 0 , X > 0 , \(s-c) = 0, (6.53) oX

as \ s J (6.54)

s=^- = \ns-c+s(\-e-xh- (6.55) oy/

Because there are two control variables constrained by inequalities, it is not possible to construct a two-dimensional phase diagram in the (state, control) space, and so we shall solve this problem in the (s, \p) space.

First note that from (6.51), c is always positive as l i m c _ 1 = +oo when c-> 0 + . We can therefore use instead

c-l = f + \. (6.51')

We shall divide the nonnegative orthant in (s9 yp) into regions according to whether or not the constraints on the controls ( c < s and A : > 0 ) bind. These and other features will be added one by one to make up Figure 6.6. The reader is invited to duplicate this building process on a separate graph:

(a) We define region A by X = 0. Therefore, by (6.53), s > c, and by (6.51'), c=\p~x\ hence, s>\p~l, or \l/>s~x characterizes region A in Figure 6.6.

(b) Define region B by X > 0. Therefore, s = c by (6.53), and (6.51') yields s _ 1 — \j/ > 0, or \p < s _ 1 , which characterizes region B.

(c) In region C, x> 0, and by (6.52), \[/ = ex/s> 1; thus, \p > 1 charac- terizes region C.

6.4 Maximum principle with inequality constraints 207

Figure 6.6

(d) In region D, x = 0, and by (6.52) ^ < e~0/s= 1; thus, 0 < 1 char- acterizes region D.

The economic significance of these regions is of interest. When the shadow price \p is high relative to the cost of effort, (0 > 1), it pays to cul- tivate the mushrooms (x>0); when x// is low, no cultivation takes place. The extent to which mushrooms are harvested for consumption depends on two factors: the price yp of the mushroom stock and the availability of mushrooms. If ^ is high and/or the available stock s is high, the inequal- ity \p > s _ 1 will tend to hold; in this case mushrooms are not harvested at the maximum level ( c < s ) , although the size of the harvest may be very great if s is large. A low price and/or a meager stock of mushrooms will tend to imply \p < s _ 1 and s = c.

Clearly, the rectangular hyperbola ^ = s _ 1 separating regions A and B, and the straight line \p = 1 separating regions C and D, together delineate four sectors (AC\C, BOD, etc.). The derivation of the ^ = 0 locus and of the s = 0 locus must proceed sector by sector.

208 6 The general constrained control problem

Derivation of the \j/ = 0 locus

(i) In sector AHC the characteristics from (a) and (c) above are combined: X = 0, ex/s—yp. Substituting these in (6.54) we obtain

^ = ^ ( r - 5 - 1 - l + 0 - 1 + ^ - 1 l n ^ ) . (6.56)

Thus, \j/ = 0 if and only if

g ( 5 , ^ ) = r - 5 - 1 - l + ^ - 1 + iA"1ln^ = 0, (6.57)

since yp = 0 does not belong to this sector. Equation (6.57) yields d\l//ds = —gs/gxj/=\l/

2/(s2ln\l/)>0; hence, g(s, yp) = 0 defines yp as a strictly increasing function of s. Its graph passes through the point (s = r~l, \p = l); as s - ^ + o o , \p^(\p)~, where \f>\ is obtained by writing (6.57) as s — \p/[(l + ln yp) — yp(l — r ) ] , and set- ting the denominator to zero. (A rough graph of (1 + In yp) against (1 — r)yp will convince the reader that \£>1 is unique.) Further- more, dyp/ds = +oo at (5 = r - 1 , ^ = 1). Since dg/ds* = s ~2 > 0, we have i/' > 0 to the right of the g(s, yp) = 0 curve and \p < 0 to the left of it.

(ii) In sector A H D we have, from (a) and (d) above, X = 0 and x = 0; hence, (6.54) becomes

\l/=\ls(r-s-1). (6.58)

Therefore, ^ = 0 along the vertical line s = r~l, yp >0 to the right of it, and yp < 0 to the left.

(iii) In sector BOD, gathering the information from (b) and (d), we have x = 0, c = s, and X > 0 ; (6.51') yields \ = s~1 — yp>0 and (6.54) becomes

\p = \l,(r + l-s-l)-s-1. (6.59)

Therefore, \j/ = 0 along the curve yp = ((r + \)s — l)~l, which inter- sects the boundary curve yp = s~x at s = r~l and goes to zero as s goes to infinity. It is easy to verify that the yp = 0 locus remains below the boundary curve as s increases. Again yp > 0 to the right of the curve as dyp/ds > 0 in (6.59).

(iv) In sector BO C, yp = ex/s and X > 0; hence,

yp = ^(r- 5 - 1 - l + ^ - 1 - h ^ - 1 l n ^ ) - X = ^ ( 5 , ^ ) - X . (6.60)

We know from calculations in (i) that g(s, yp) < 0 to the left of the g(s, \p) = 0 graph. Therefore, yp < 0 everywhere in sector BDC.

We now turn our attention to the locus of s = 0:

6.4 Maximum principle with inequality constraints 209

(i') In sector A f! C, X = 0 and x > 0; hence, c=\p~l and\// = ex/s> 1; these yield with (6.55)

5 = l n 5 - ^ - 1 + 5 ( l - ^ - 1 ) = </>(5,̂ ). (6.61)

The 5 = 0 locus is described by ^ = ( l + s ) / ( s + l n s ) . This curve passes through (s = e, \p = l) and as s-*s+, ip-* H-oo, where s — 0.567 is defined by s + l n s = 0. Since yp is positive, s + l n s is also positive; also, since \p>l, we have l n s < l ; therefore, d\l//ds = (Ins — 2 — s ~l)/(s + I n s ) 2 < 0, and the s = 0 locus is negatively sloped. Finally, note that in this sector î = ( l + s ) / ( s + l n s ) > s _ 1

as s 2 > l n s and the s = 0 locus lies above the \p = s~l curve; it begins at (s = e, \p = 1) and goes up to infinity as s approaches s from above. In Figure 6.6 we have represented this locus under our assumption that r < e~l. Since d<j>/ds > 0, as rf/ > 1 in this sec- tor, we see that s > 0 to the right of the locus.

(ii') In sector ADD, x = 0 and X = 0; hence, c = ^ - 1 and (6.55) be- comes

s = \ns-^-\ (6.62)

The 5 = 0 locus is \[/ = (Ins)" 1 , which passes through (s = e, \[/ = 1) and goes to zero as s goes to infinity. The curve ^ = ( l n 5 ) _ 1 is above the boundary curve \l/ = s~l, since s > l n 5 . The signs of s are clear.

(iii') In sector BHD, x = 0 and c = s; hence,

5 = l n 5 - s < 0 . (6.63)

(ivr) In sector B D C, c = s and \J/ = ex/s and

s = \ns-s\l/-l<0, (6.64)

since \p > 1 and s _ 1 > yp imply In 5 < 0.

The diagram in Figure 6.6 is now complete, and we see that there exists a unique equilibrium in the ADD sector. (Had we assumed that e~l< r < 1 or 1 < r, we would have obtained in either case an equilibrium in the A fl C sector. The reader is invited to rework the derivation of the \j/ = 0 locus in these cases; the 5 = 0 locus is unaffected.) By inspection we see that the equilibrium is a saddle point, but we can confirm it locally by lin- earizing the system (6.54)-(6.55) around the equilibrium point (s* = r - 1 , >P*= (Ins*)"1). We obtain from (6.62) and (6.58)

5 = lns — i/'-1,

i = \p(r-s~l),

210 6 The general constrained control problem

and

\H U w 2 ° J The determinant of this matrix is negative, which characterizes a saddle point.

Note that at the equilibrium, the derivative of the natural growth rate of mushrooms with respect to s ((d/ds)ln(s) = s~l) equals the rate of discount, r. This characterization has its parallel in the optimal growth model in which F'(s) — m = 8 at equilibrium.

A variety of possible patterns emerges, depending on the values T, s0, and sT. We have not drawn these so as not to complicate the diagram, but the reader can easily do this. With s0<r~

l, sT>r~ l, and T large

enough, the optimal path would go from sector A n C into sector A C\ £>, above the s locus, and perhaps back into sector A H C, below the \J/ = 0 locus. Along this trajectory, x would go from positive, to zero, to posi- tive again, while the upper bound on c would never be reached. In another instance, with 5"0and s r b o t h below r"

1 , the optimal path may again enter sector AnD from sector ADC above the s = 0 locus, but then cross this locus to proceed to the BC\D sector, where the constraint c < s binds. Many other scenarios may be envisaged, and it is important to keep track of the behavior of the control variables relative to the constraints on them.

It is obvious that we could not have (autonomous) phase diagrams in either the (s, c) or the (s, x) space. We can, of course, construct a phase diagram in (s, c) when x = 0 and a diagram in (s,x) when c = s (these correspond to regions D and B, respectively), but we cannot construct a two-dimensional diagram when c<s and x>0; this is sector AC\C. Therefore, in models such as the one treated in this example, with more than one control and inequality constraints on them that depend on the state variables, there is no alternative but to work in the (state, costate) space, although the phase diagrams we just mentioned may be of some interest.

6.5 Necessity and sufficiency theorems: the case with inequality and equality constraints

Many optimal control problems in economics involve both equality and inequality constraints. This type of problem creates no new difficulties. The necessary conditions are a straightforward generalization of those stated in the preceding sections. We shall state them here for easy refer- ence. A sufficiency theorem is also stated and proved for this more gen- eral case.

s — s* (6.65)

6.5 Necessity and sufficiency theorems 211

The general constrained control problem with fixed endpoints consists of finding the optimal path c*(/) for the control variables so as to maximize

V=[Tv(s(t),c(t),t)dt (6.66) Jo

subject to n differential equations,

*/(0 = / ' W O , c ( 0 , 0 , 1 = 1,2,...,/!, (6.67)

m' inequality constraints,

g'"(s(O,c(f),O^0, y = l,2,...,m', (6.68)

m — m' equality constraints,

g*(s(O,c(/),f) = 0, * = m ' + l , . . . , m , (6.69)

and In boundary conditions,

Si(0) = si09 Sj{t) = siT9 / = l,2,...,/i. (6.70)

The time horizon T and the initial and terminal values of the state vari- ables are exogenously specified.

As discussed in Section 6.1, we shall assume that constraints (6.68) and (6.69) satisfy the rank condition. We shall use the notation W(s(t), t) to denote the set of values of control variables that satisfy (6.68) and (6.69) simultaneously.

For problem (6.66)-(6.70) we define a Hamiltonian,

H(s(t),c(t),ic(t),t) = v(s(t),c(t)9t) + 2 7 r ^ / ) / ' ( s ( 0 , c ( f U ) , (6.71) / = i

and a Lagrangean,

£(s(f),c(f), * ( » , MO, t) = H(s(t)9c(t)9 w(t)91) m

+ 2\j(t)gj(s(t)9c(t)9t)9 (6.72)

where \i(t),..., \m(t) are multipliers. Theorem 6.5.1: necessity. Let c*(/) be an optimal solution to the con- strained problem (6.66)-(6.70) and s*(t) be the corresponding time path of the state variables. Then there exist costate variables v(t) and (assum- ing the rank condition is satisfied) multipliers X(/) such that:

(i) At any time t, for given vectors s*(/) and v(t)9 the control vari- able vector c*(0 maximizes the Hamiltonian (6.71) subject to the condition that c(t) belong to the set of admissible controls

212 6 The general constrained control problem

defined by (6.68) and (6.69). In view of the rank condition this implies that there exist multipliers X(t) such that

^ - = 0, i = l,2,...,r, (6.73) OC;

\ y ( 0 > 0 , gl(s*(t),c*(t)9t)7>09 \j(t)gHs*(t),c*(t)9t) = 0, y = l,2,...,m', (6.74a)

gk(s*(t)9c*(t)9t) = 09 k=m'+\9...9m9 (6.74b)

where the asterisk on <£ indicates that the derivatives are evaluated at (s*(0»c*(/)). The multipliers \(t) are piecewise-continuous and continuous on each point of continuity of c*(t).

(ii) The costate variables 7T/(/)> / = 1,2, ...,#, are continuous and have piecewise-continuous derivatives satisfying

*'•<'> = -7TT77' / = l , 2 , . . . , / i . (6.75) OSi(t)

ftp* (iii) sHt)=^-r = fl(s*(t)9c*(t)9t)9 i = l,2,...,/i. (6.76)

(iv) The Lagrangean £(s*(0, c*(0, * ( 0 , x < 0 , 0 = 0 ( 0 is a contin- uous function of t. On each interval of continuity of c*(t), <t>(t) is differentiable and

0 ' ( , ) . ^ = ^ . (6.77) dt dt

(v) The boundary conditions (6.70) must be satisfied.

A heuristic proof of Theorem 6.5.1 using the approach adopted in Sec- tion 4.5 is possible but slightly more involved. It is left to the interested reader. The necessity of (6.77) follows from (6.73)-(6.76). To see this, differentiate <£ totally with respect to t:

d£* _ d£* dsf _ d£* dcf _ d£* d^ _, d£* d\t d£* = y L-L. y L + y L_|_ y L J

dt * dst dt * dct dt * d<Kt dt * Ski dt dt '

But d£*/dCi = 0 by (6.73) and (d£*/dsi)it=-(d£*/dvi)i:i by (6.75) and (6.76). It remains to show that (d£*/d\i)(d\i/dt) = 0. If

g'(s*(O,c*(O,X) = 0,

then d£*/d\t = g''(s*(0, c*(t)9 A) = 0; if g'(s*(0, c*(0,0 > 0, then X,(/) = 0 and hence d\j/dt = 0. This completes our proof of the necessity of (6.77). •

6.5 Necessity and sufficiency theorems 213

We now show that if the Lagrangean is concave in the variables (s, c), then the necessary conditions just stated are also sufficient. The proof is much the same as that offered in Section 4.6. Let (s*(t),c*(t)) be a pro- gram that satisfies all the necessary conditions and let (TT*U), X*(t)) be the associated costate variables and Lagrange multipliers. The asterisk for 7T and X was suppressed in our statement of the necessary conditions so as to simplify the notation; it is reintroduced here, because we wish to emphasize that the function H, described for any feasible program ( s ( 0 , c ( 0 ) , is defined using the same values (ir*(t), \*{t)) that were found in the optimal program. Thus,

/ / * m v(s*(t), c * ( 0 , 0 + 2 *?(')/'"(s*tt), c*(0, t) s v* + 7r*.f*, (6.78)

£* = H*+ 2 X}(t)gJ(s*(t), c*(f), t) = i;*+ **.f *+ X*-g*, (6.79)

H= v(s(t), c ( » , 0 + 2 * ? ( 0 / ' ( s ( 0 , c ( 0 , t) a t; + x*.f, (6.80)

and

£ = / / + 2 X * ( 0 ^ y ( s ( 0 , c ( 0 , 0 = ^ + 7r*.f+X*-g, (6.81)

where (s, c) refer to any program satisfying (6.67)-(6.70) on [0, T]. Thus, <£* is the function <£ evaluated at (s, c) = (s*, c*). For simplicity, the nota- tion 7rM denotes the inner product of the vector ir* = (71-*, -K\, ..., TT*) and the vector f = (f\ f 2 , . . . , / " ) , where / ' ' stands for / ' ( s ( 0 , c ( 0 , 0 - Simi- larly, the symbol (c*—c)»d£*/dc denotes the inner product of the vector

(c*-c)m(cW)-cl(t),...9c%t)-cn(t)) and the vector (d£/dcu...9d£/dcn), where the derivatives are evaluated

a t ( s * ( 0 , c * ( 0 , 7 r * ( 0 , X * ( 0 , 0 .

Theorem 6.5.2: sufficiency. Let (s*(t),c*(t)) satisfy the conditions of Theorem 6.5.1 and assume that the Lagrangean (6.81) is concave in (s, c); then (s*(0, c*(0) is an optimal path for the problem (6.66)-(6.70). If £ is strictly concave, (s*(t),c*(t)) is the unique optimal solution.

Proof. This is a straightforward generalization of the proof offered in Section 4.6:

V*-V=[T(v*-v)dt Jo

= [T[(H*-<K**S*)-(H-**•*)] dt Jo

= [\(H*+ * * * s * ) - ( / / + ?r*-s)] dt + [ S * ( 0 ) * T T * ( 0 ) - s * ( 7 > * * ( r ) ] Jo - [S(0)«TT*(0) -S(T)*TT*(T)] (by integration by parts)

214 6 The general constrained control problem

= [T[(H*+ir**s*)-(H+ic**s)]dt Jo

(because s(0) = s*(0) = s0 and s(T) = s*(T) = sT)

>[T[(H*+\*.g*)-(H+\*.g) + ic**(s*-s)]dt Jo

(since \*-gj = 0 and \*gj > 0)

= [T[(£*-£) + <k*.(s*-s)]dt Jo

- J o l ( C * " C ) # ^ + (S*"S)#^ + ^ # ( S * ~ S ) l ^ (by the concavity of the Lagrangean)

= 0 (because the asterisked solution satisfies the maximum principle).

If the Lagrangean is strictly concave in (s,c), then V*> Kand the opti- mal solution is unique. •

Corollary 6.5.1. The concavity of the Lagrangean is ensured if the fol- lowing conditions are met:

(i) v is concave in (s,c); (ii) each term 7r*(/)/'(s, c, /) is concave in (s, c); thus, i f / ' ( s , c, t) is

concave (resp. convex), the condition is satisfied, provided that 7r?(O^0(resp. <0);3

(iii) each of the m' inequality constraints gJ(s, c,t)>0 is concave in (s, c) (recall that \*(/)> 0 for these inequality constraints);

(iv) each of the m — m' equality constraints gk(s,c,t) = 0 has the property that \*k(t)g

k(s,c,t) is concave in (s,c); thus, if gk is concave (resp. convex), the condition is satisfied provided that \X(0 :> 0 (resp. <0).

Just as we did at the end of Chapter 4, we now use the maximized Ham- iltonian to restate the maximum principle and present a more general suf- ficiency theorem that places restrictions on the maximized Hamiltonian.

Definition 6.5.1. For the problem (6.66)-(6.70) we define the maximized Hamiltonian as H°(s(t), Tr(t),t) = maxc{t)H(s(t)9c(t),Tr(t),t) subject to (6.68)-(6.69), where H is as in (6.71) and c(t) is piecewise-continuous.

This is a formal definition; some necessary conditions for c(t) to be opti- mally chosen are given in (6.73)-(6.74); presumably these conditions can

3 For conditions ensuring that Tct(t) > 0 , see Leonard (1981).

6.5 Necessity and sufficiency theorems 215

be used to obtain c(t) in terms of s(t)9 ir(t), and t, and this can be sub- stituted in the Hamiltonian.

The maximum principle can be restated.

Theorem 6.5.3: necessity. Let c*(0 be an optimal solution to the prob- lem (6.66)-(6.70) and s*(/) be the corresponding path of the state vari- ables. Then there exist costate variables ir(t) such that

(i) / / ° ( s * ( 0 , TT(0, t) ss H(s*(t), c*(0, * ( 0 , t)

= m a x / / ( s * ( 0 , c ( 0 , *(t), t) (6.82a)

subject to (6.68)-(6.69), where H is as in (6.71) and c(/) is piece- wise-continuous;

(ii) the costate variables -KX(t), i = 1,..., n, have piecewise-continuous derivatives satisfying

. a//V(o,7r(o,o . TT/(0 = ^—T-T , i = l , . . . , / i ; (6.82b)

(m) sf(t) = — — , Z = 1 , . . . , H ; (6.82c)

(iv) the maximized Hamiltonian H°(s*(t), ir(t)91) = <j>(t) is a contin- uous function of t. On each interval of continuity of c*(t), <j>(t) is differentiable and

* ' ( , ) . ^ = ^ £ ; (6.82d) a/ at

(v) the boundary conditions (6.70) are satisfied. (6.82e)

Remark. The main differences in the forms of Theorem 6.5.1 and Theo- rem 6.5.3 occur between conditions (6.75)-(6.76) and conditions (6.82b)- (6.82c). It is instructive to verify that these are indeed equivalent to one another. The alert reader will notice the strong similarities between this argument and the proof of Theorem 1.2.8. Indeed, this is but another instance of the envelope theorem, where the derivatives of the maximum value function H° are equal to those of the Lagrangean <£.

Let us define a function c = 0°(s, TT, t) by conditions (6.73) and (6.74); that is, 0° represents the optimal value of c, given s, 7r, and t, and tak- ing into account the constraints (6.68) and (6.69). Then / / ° ( s , ir, t) = H(s,0°9Tr,t) and (6.82b) can be expressed as

^ -dH° - d / / ( s , 0 ° , 7 r , O d($0Y 7 - d / / ( s , 0 ° , 7 r , / )

ds ds ds \ dc

dH i d(S 0Y » dgHs,9°,t) . , _ _

'-to^-ar-g^—rc— by(6J3)-

216 6 The general constrained control problem

) . .

All the constraints for which gJ > 0 have Xy = 0 and can be ignored. For those that hold as equalities, the controls have been chosen, for given s and 7r, such that gJ remains zero. Hence, for given TT, dgJ = 0 implies

dgJ(s,0°,t) , d(0°Y /BgJ(*,0°,t) ds ds \ dc

and we have

._-dH(s,0°,ir9t) g X dg j(*,0°,t)

9s j=1 J ds

which is (6.75). Briefly, we do the same for (6.76), ,_dH^_dH_ d(0°y dH _dH « d(fY_^gj _dH

dir dir 3TT 3C dir y = j J dir dc dir '

since d(60)' dgJ(*,0°,t)

dir dc = 0 (wheng' = 0).

Example 6.5.1. We now return to the mushroom grower's problem (Ex- ample 6.4.2) and derive equations (6.56) and (6.58)-(6.64) using the max- imized Hamiltonian. Note that the controls are (c,x) and the current- value costate is xf/. The multipliers will be (X,/*) because we need to at- tach a multiplier ji > 0 to the constraint x > 0 and the Lagrangean of (6.50) is replaced by £ = H+\(s — c) + iix; this is done so that (6.73) represents the first-order conditions accurately. We know c>0 because lnc is in the maximand; hence, (6.51) and (6.52) become c~l — \J/ + \ = 0 and — l + \I/e~x/s+ii = 0. From this we obtain

c = (^ + X)-1 and x = s l n ( 0 / ( l - j i ) ) . (6.83)

These equations do not correspond to the 0° that we introduced on our remark on Theorem 6.5.3 because they still contain the multipliers X and t̂. (It is not possible to eliminate the multipliers until we know which con-

straints bind.) We say that equation (6.83) defines (c,x) = 0, say. Substi- tuting 0 into H of (6.49) we obtain

^ = ^ ( 5 + l n 5 ) - 5 A - ^ + l n ^ i ^ - i - l n ( ^ + X ) .

In this problem we cannot obtain H° as a single expression, but we must distinguish cases A, B, C, and D as in Section 6.4. We briefly summarize their relevant properties (in terms of the multipliers):

X = 0, X > 0, and s = c or s = (^ + X)"1, „ g 4 )

li>0andx = 0 or \p=l-fi.

6.5 Necessity and sufficiency theorems 217

In each case, AC, AD, BD, and BC, when we know which constraints bind, we can calculate H° and then use (6.82b) and (6.82c) to obtain it and s. We now do this.

AC: / / ° = - ( l + 5)(l + l n ^ ) + ^ + ln5); hence, iA = r ^ - a / / ° / a 5 = ^ [ r - l - 5 - 1 + ^ - 1 + ^ - 1 l n ^ ] , which is (6.56); s = dH°/d\l/ = -(l+s)\l,-l+s + lns, which is (6.61);

AD: H°=-\n\P + \ls\ns-l; \j/ = \l/(r—s~l) and s = Ins — 0 " 1 , which are (6.58) and (6.62);

BD: / / ° = (l + i / 0 1 n s - ^ ; \l/ = \//(r + l — s~l) and s = \ns—s, which are (6.59) and (6.63);

BC: J f ° = ( l + 0 ) l n s - s ( l + lntf); 5 = Ins—s\l/~l, which is (6.66); ^ = ^ ( r - s " 1 - l + ^~ 1 + i / ' " 1 l n ^ ) - s " 1 + ^, which is (6.60) with X = s - 1 — \j/ from case B.

It is important to realize that obtaining -k and s directly from H (or H) and then eliminating X and fi with (6.84) according to each case would not yield the correct results. In each case the correct expression for H° must be obtained before -k and s are derived. The reader is invited to verify this.

Remark. It is always possible, and sometimes convenient, to use an ap- proach intermediate between the one illustrated here and the method used in Section 6.4. For instance, we can use (6.83) to eliminate c and x from <£ = / / + X ( s — c) + fix; we obtain an "optimized Lagrangean":

Thus,

and

£ = ^ ( s + l n s ) + s ( ^ - l ) A + l n - ^ - J - h s X - l n ( ^ + X ) - l .

^ = ̂ - ^ = 0 ^ - l - s - 1 + ( l - / . ) ^ - 1 l n ^ + l - / i + xY

s= —- = s + lns + (ii — l)- d^ ^ 't 0 + X'

which are valid everywhere and do not explicitly contain the controls. Using (6.84) it is then easy to specialize the two preceding differential equations to obtain (6.56) and (6.58)-(6.64).

Theorem 6.5.4: sufficiency. Let (s*(t)9c*(t)) satisfy the conditions of Theorem 6.5.3 and assume that the maximized Hamiltonian H° of (6.84) is concave in s. Then (s*(t),c*(t)) is an optimal path for the problem (6.66)-(6.70); if H° is strictly concave, it is the unique optimal solution.

218 6 The general constrained control problem

Note once again that attention must be paid to the properties of H and g\ j = 1,..., m, in terms of c so that condition (6.82a) is satisfied. Con- cavity of H°, by itself, does not guarantee optimality.

6.6 Concluding notes

In this chapter we have explored the techniques of solving control problems involving equality and inequality constraints. The reader will have noted that sometimes it requires a certain ingenuity to obtain the optimal solu- tion from the necessary conditions, which are themselves easy to derive.

We have been assuming that the terminal values of the state variables are fixed. We have done this to keep the exposition as similar as possible to the introductory account given in Chapter 4. The time has now come to relax this assumption. This is the main purpose of the next chapter.

Exercises

1. Consider the following modified version of the mushroom grower's problem (Example 6.4.2). Let the utility function and the natural growth function be 2 ln(c+1) — x and ln(s +1), respectively; all other specifications are unchanged. Derive the necessary conditions and solve the problem using the phase diagram method. Note that there will be a region in which the optimal consumption is zero.

2. Modify the mushroom grower's problem (Example 6.4.2) by assuming that the utility function depends on both c and s, and there is no possibility of cultiva- tion (x = 0). We choose u(c,s) = \n(cs) and 5 = 5(1— s) — c. The constraint s — c > 0 still applies. Construct the phase diagram in the (state, costate) space and show that the equilibrium is a saddle point.

3. Generalize the mushroom grower's problem (Example 6.4.2) by not assuming specific functional forms. Restrict instead the functions as follows: assume for the utility function u'(c) > 0, u"(c) < 0, w'(0) = +<x>, and for the natural growth function G(s) 3s > s > 0 such that G(0) = G(s) = 0, G'(s) = 0, G'(0) > r, and G"(s) < 0. The constraint s — c > 0 still applies, but we omit the possibility of cultivation (x = 0). Derive and interpret the necessary conditions. Construct the state-costate phase diagram and verify that the equilibrium is a saddle point.

4. At the end of Example 6.3.1, it was claimed that if the production function is homogeneous and has a constant elasticity of substitution, then the optimal consumption path is constant only if the elasticity of substitution is equal to 1. Now prove this result, using the following steps: (a) Write the production function in the form F(s, x) = zh, where z = f(s, x)

is homogeneous of degree 1. Substitute this in (6.23), take the time deriva- tive, and use (6.24) to establish that fxxx+fxss = —fs. (First note that the constancy of c implies that of F, z, and X.)

(b) Differentiate the identity c = zh with respect to time and deduce that c is constant if and only if fxx+fss = 0.

Exercises 219

(c) Apply Euler's theorem to / and fx to show that

fxx+fss = f and fxxx+fxss = 0.

Use this and the results of (a) and (b) to show that along the optimal path with constant consumption, 1 = (fxsf)/(fxfs) = o. {Hint: Recall that s = —x; use the results (a) and (b) to solve for this x term; use Euler's theorem results to solve for x also.)

5. For Example 6.4.1, we asserted without proof that Figure 6.5 is the phase dia- gram in the (i/s s) space. Construct this diagram using the necessary conditions derived from the current-value Lagrangean.

6. There is a resource that is freely available in unlimited amounts; however, some equipment is required to harvest it, and the equipment is costly to build. The problem is to find the optimal building and harvesting policy that maximizes the total present value of profit over the horizon [0,T], The following nota- tion is used: x(t) is the flow of resource harvested at date t; b(t) the flow of equipment built at date t; E(t) the stock of equipment available at date t\ R(x(t)) the revenue, at date t, from selling x(t) units of resource at date t; C(b(t)) the cost, at date t, of building b(t) units of equipment at date /; and 8 the positive rate of discount.

We assume that R' > 0, R" < 0, C" > 0, C" > 0, and there exists a positive value E such that R'(E) = SC'(O). The constraints are that x(t) > 0, b(t) > 0 at all times, and if the stock of equipment is E(t) at date t> no more than E(t) units of the resource may be harvested (i.e., x(t) <E(t))\ T, E(0), and E(T) are specified positive constants. Set up the problem in optimal control format, apply the maximum principle, and show that E(t) = x(t) > 0 at all times. Draw a phase diagram in the (<p, E) space, where <p is the current-value costate. Iden- tify the region where b = 0. Describe in words the optimal policy for selected values of T, E(0)9 and E(T).

7. Consider the operation of a commercial fishing fleet. The amount of fish caught depends only on the size of the fleet. The fleet deteriorates in constant propor- tion to its size, but this can be counteracted by boat building, which is costly. Profit is simply revenue from the amount of fish caught minus the cost of boat building. At instant /, s(t) is the size of the fleet with which a quantity F(s(t)) of fish is caught. The fleet deteriorates at the proportional rate m\ x(t) is the flow of boat building and C(x(t)) is its cost; fish sell at price p per unit and the discount rate on profit is b. The planning horizon is [0, T] and the size of the fleet at time zero is s0. We assume that all of Ty 5, m, p, and s0 are specified positive constants, F is positively valued, strictly increasing, and concave with F'(<x>) = 0, while C is strictly positively valued, strictly increasing, and convex with C"(0) > 0; A: is restricted to nonnegative values (boats cannot be taken apart and sold). We also assume that C"(0) <pF'(0)/(m + 8). (a) Suppose that you have leased this fleet for T periods and that the contract

specifies the size of the fleet when it must be returned, say sT > 0. Formu- late the problem of maximizing the total present value of profit over the horizon [0, T] subject to the above restrictions. Apply the maximum prin- ciple and give an economic interpretation of all variables and conditions.

6 The general constrained control problem

Draw a phase diagram in the (x9s) space. (It is advisable to deal sepa- rately with x > 0 and x = 0). Identify the intercepts of the x = 0 locus with the axes (x and s, say). As a first step you might want to use the following numerical example: p = l, 5 = 0.1, AW = 0.4, F(s) = 2(s + l)1/2 — 2, and C(x) = 0.5(x+1)2 — 0.5. Give a complete account of the optimal policy for selected values of r a n d sT (some above s, some below). Can you tell whether x is ever zero for any length of time? Draw a phase diagram in the (<p, s) space where <p is the current-value co- state. Identify the region where x = 0 and describe the optimal policy for selected values of T and sT.

8. Yabbies are an Australian variety of small freshwater crayfish, prized as a deli- cacy by connoisseurs of crustaceans. Consider a pond containing a yabby pop- ulation of size s(t); its natural growth rate is f{s(t)), but this can be reduced by c(t), the catch of yabbies at time t (they are very easy to catch); this catch can be sold and provides a revenue R(c(t)) at time t. The exponential rate of discount for revenue is 5. Both the revenue function and the growth rate func- tion are strictly concave and have a global maximum; more precisely,

3c, c, with c > c >0, R'(c) = 0, R(0) = R(c) = 0, and R"(c) < 0, all c.

35,J, with s >s > 0, f(s) = 0, / ( 0 ) = f(s) = 0, and f"(s) < 0, all s;

/ ' ( 0 ) > 5 .

Our aim is to find the catching policy c(t)>0 over some specified horizon [0,T] that maximizes the present value of revenue flows subject to s(t) = f(s(t)) — c(t) and s(0) = s0. For the time being we ignore constraints on s(T). First draw a rough graph of R and / a c c o r d i n g to the above assumptions. Show that there exists 5, with 0 < s < s and f'(s) = 5. (a) Carefully derive the conditions necessarily obeyed by an optimal policy

and interpret them. (b) Draw a phase diagram in the (c,s) space under the special assumption

f(s) < c. How many equilibrium points are there? Which one is preferred? Draw a phase diagram in the (<p, s) space under the above assumption - <p is the current-value costate; identify the region where c = 0. Choose arbi- trary positive values for T, s0i and sT and show that they determine the exact optimal path.

(c) Redo part (b) with R(c) = 0 . 5 c ( l - c ) , f(s) = s(l-s), and r = 0.2. 9. Reconsider exercise 8 without the special assumption f(s) < c.

(a) Redo part (b) of exercise 8 with the special assumption f(s) < c<f(s). How many equilibrium points are there?

(b) Redo part (b) with the special assumption c<f(s). (c) Examine the stability of the various equilibria. If T were very large, can

you guess which one would be the optimal path? Can you identify a crucial value of SQ that determines whether c(t) = c will be the optimal solution W? Why is this solution desirable? While c(t) = c, how much would one pay for more yabbies to stock the pond?

220

(b)

(c)

C H A P T E R 7

Endpoint constraints and transversality conditions

In Chapters 4 and 6 we assumed that the time horizon T, the initial values of state variables, si0, and their terminal values siT were exogenously spec- ified. Obviously these are very restrictive assumptions. In many economic problems we want to allow some of these values to be determined endog- enously (subject to constraints). For example, the optimal consumption problem (Example 6.4.1) may be modified to allow the planner to select the value of the terminal stock s(T), subject only to some constraint such that s(T) may not be less than a certain lower bound s, or even to select the economy's doomsday T after which all activities cease. Obviously, when s(T) or T is not fixed, we need additional necessary conditions to determine the new unknown (s*(T) or T*)m, these conditions are called the transversality conditions.

We shall look at various cases, beginning with the simplest. Section 7.8 contains a general statement synthesizing the various transversality con- ditions. Because there are many kinds of boundary conditions, there are also many kinds of transversality conditions. This array of special cases sometimes appears formidable to students of optimal control theory. For this reason a summary table is provided in Section 7.10. The table lists various features of control problems, and for each one gives the associ- ated transversality condition. If a problem has several of these features, all corresponding transversality conditions apply.

Each of the following sections presents one type of problem and derives the associated transversality condition. The bulk of each section is de- voted to the solution of an example of that type so as to illustrate the im- portance of the transversality condition for the determination of the opti- mal path.

We shall deal with various modified versions of the problem of Section 6.5, which we reproduce here for convenience.

Maximize V=[ v(s(t),c(t),t)dt (7.1a) Jo

subject to

4 / = / ' " ( s ( 0 , c ( 0 , 0 , / = l,2,...,/i, (7.1b) gJ(s(t)9c(t),t)>:0, y = l,2,...,/w', (7.1c)

221

222 7 Endpoints and transversality conditions

g*(s(f),c(O,O = 0, k = m'+l,...,m, (7 Ad)

Si(0) = si0, / = l , 2 , . . . , / z , (7.1e)

Si(T) = siT, I = 1 , 2 , . . . , / I , (7.1f)

where 5 / 0 , s,T, and T are exogenously given. Because the siT values are fixed at the outset, such problems are called fixed-endpoint problems.

7.1 Free-endpoint problems

In this section we modify (7.If) by allowing some s^T) to be free. For concreteness, let

5/(7) free, / = l , 2 , . . . , / z ' , (7.2a)

Sj(T) = sjT given, j = n'+l9 ...,/!. (7.2b)

Intuitively, since there are now n' more objects of choice, as reflected in (7.2a), we must obtain n' additional necessary conditions to help de- termine the optimal value of each Sj(T), i = 1,2,...,/*'. These additional necessary conditions are the transversality conditions for this problem.

Transversality conditions for free endpoint

Theorem 7.1.1. If problem (7.1) is modified as in (7.2), the additional necessary conditions, called the transversality conditions, are

7Tf(D=0, 1 = 1,2,...,/!'. (7.3)

Proof For simplicity we shall offer only a heuristic proof of the necessity of (7.3). For any given s r , let V*(sT) denote the optimal value of the inte- gral (7.1a). The optimal choice of the free siT values must maximize the function V*(sT). In other words, s*T must be chosen such that

dV(sT)/dsiT = 0, i = l , 2 , . . . , / ! ' . (7.4)

But in Chapter 4 (see equation (4.80)) we have shown that

dV(sT)/dsiT=-Tr(T). (7.5)

This equation and (7.4) imply (7.3). •

The transversality condition (7.3) has an economic meaning: since the planner attaches no value to the terminal stock and is not constrained to meet a certain target siT, the stock should be used until its marginal contribution is zero at the end of the planning horizon. We now illustrate the use of the transversality condition by considering a variation of Ex- ample 6.4.1.

7.1 Free-endpoint problems 223

Example 7.1.1. Our problem is to find c(t) and sT that maximize

V=[Tu(c(t))e-6tdt (7.6) Jo

subject to

Ht)=F(s(t))-c(t)-ms(t), (7.7)

F ( s ( O ) - c ( / ) > 0 , (7.8)

5(0) = 50 fixed, (7.9)

5 ( D = s r f r e e . (7.10)

We note that sT is free in the sense that it is not exogenously specified, but the planner's freedom to choose sT is in fact limited because (7.7)- (7.9) imply a certain lower bound on sT: since sT must obey the law of motion (7.7) and F(s(t)) — c(t) > 0, the state variable cannot fall at a rate exceeding ms(t). The smallest feasible value of sT is therefore s(0)e~

mT. However, this restriction on the choice of sT need not be stated as a sepa- rate constraint, because it has been reflected in conditions (7.7)-(7.9).

The necessary conditions for problem (7.6) are (6.40)-(6.43) and the transversality condition

ir{T) = 0. (7.11)

Recall that the current-value shadow price was defined as

W) = ic(t)e*t;

the transversality condition in terms of the current-value shadow price is

0 ( r ) = O. (7.11')

We shall use (7.11') and Figure 6.5 to characterize the solution of the free- endpoint problem (7.6).

Figure 7.1 is a reproduction of Figure 6.5. Suppose path PP' is opti- mal for the fixed-endpoint problem, with s(T) = Q. For the free-endpoint problem, path PP' remains feasible but no longer optimal, because at time T i t reaches the point P'; hence, \I/(T) is positive, violating the trans- versality condition (7.11'). For \I/(T) to be zero, the optimal path must end at a point on the line ^ = 0, that is, on the horizontal axis. Paths such as GG' and EE' satisfy this condition. The time t\ at which path EE' cuts the vertical line QP' is less than T because we know that it takes exactly T units of time for path PP' to travel from P to P' and that the capital stock decumulates more quickly the farther down the trajectory is in the diagram (recall that to lower values of ^ correspond higher values of c). Similarly, the time t2 at which path GG' cuts the line QP' is less than T but greater than t\. Which one of the paths EE' and GG' is optimal

224 7 Endpoints and transversality conditions

Figure 7.1

depends on the length of the time horizon, T. Given T, there is exactly one optimal path. Say that GG' is the optimal path; then the optimal value of the stock at terminal time is read at s*(T). Note also that since the opti- mal path lies below path PP\ the consumption c*(t) along the optimal path is higher than the consumption along path PP'.

In order to reinforce the reader's grasp of the solution of control prob- lems with a transversality condition, we now solve another simple free- endpoint problem, this time without the use of a phase diagram.

Example 7.1.2. An economy produces a consumption good. Output of the consumption good, c(/), is a function of effort, n(t):

c(t) = an(t), a>0. (7.12)

Consumption adds to the stock of pollution, p(t):

p(t) = (c(t))2-mp(t), (7.13)

where m > 0 is the natural rate of decay of the pollution stock.

7.1 Free-endpoint problems 225

Utility is a function of consumption, effort, and pollution:

U=\nc-(3n2-bp, (7.14)

where j8 > 0 and b > 0. The planner's problem is to find c(t)9 n(t), and pT that maximize

V= [T (In c-/3n2-bp)e-5tdt (7.15) Jo

subject to (7.12), (7.13), and the boundary conditions

P(0)=p0 given, (7.16)

p(T)=pT free. (7.17)

Substitute (7.12) into (7.15) to eliminate n\ the Hamiltonian is

H= (In c-\c2-bp)e-dt+Tr(c2-mp),

where \ = l3/a2>0 and 7r is the costate variable. The necessary condi- tions are

^ = (--2\c\e-5t + 2Trc = 0, (7.18)

ir = -— = be-8t + mir, (7.19) dp

dH ~ p = — = c2-mp, (7.20)

and the transversality condition

7r(r) = 0. (7.21)

(The boundary condition p(0) =p0 must, of course, be satisfied.) From (7.18), the optimal consumption at any time is given by

c = (2\-2<ice8trl/2. (7.22)

(We shall show that w is nonpositive along an optimal path.) Substitute (7.22) into (7.20):

p = (2\-2ire8trl-mp. (7.23)

The pair of fifst-order differential equations (7.19) and (7.23) together with the two boundary conditions (7.16) and (7.21) yield unique time paths for ir(t) and p(t). This can be done by first integrating (7.19):

ir(t)=Aemt-(b/(5 + m))e-dt, (7.24)

where A is a constant of integration that can be determined by using the transversality condition (7.21):

226 7 Endpoints and transversality conditions

0 = TT(T) = AemT- (b/(d + m))e-bT. Hence,

A = (b/(5 + m))e-{d+m)T.

Therefore,

x(Oe6'M6/(fi + /w))(e~ ( m + 6 ) ( r ~' ) -l), (7.25)

which is negative for all t<T. (This negativity could have been obtained using the result of Leonard, 1981.) Thus, the transversality condition en- ables us to get the exact form of ir(t). The rest of the solution follows.

The optimal consumption path is obtained by substituting (7.25) into (7.22):

c(t) = [2\ + (2b/(8 + m))(l-ei8+m)i'-T))rl/2.

It is easy to verify that c(t) > 0. The optimal time path of pollution can be obtained from (7.23). First,

rearrange (7.23):

(2\-2Tre8trl=p + mp.

Multiply both sides by emt:

emt(2\-2Tredtrl = emt(p + mp) = —(p(t)emt). (7.26) at

Integration from 0 to t and using r as the variable time yield

p(t)emt-p(0) = [t(2\-2>ir(T)e8TrlemTdT. Jo

It is clear that p(t) is not necessarily a monotone function of time. As an exercise, the reader may construct the phase diagram in the (/?, \p) space where ^ is the current-value shadow price:

rfr(t) = ic(t)e6'.

It should be noted that \//(t) is negative along an optimal path; see (7.25). This makes economic sense: since the stock of pollution is a "bad," its marginal value is negative for all t < T.

7.2 Problems with free endpoint and a scrap value function

In the preceding section as well as in all problems heretofore considered, it was assumed that the planner derives no benefit from the stock of capital left over at the terminal time T. We now consider a more general problem in which the planner attaches a value to what is left over at T. Thus, we must deal with problem (7.1) modified by adding a "scrap value function" </>(sr, T) to the integral of (7.1a). For convenience it is restated in full here.

7.2 Free endpoint and a scrap value function 227

Our problem is to find the vector of control c(t) and the terminal stock sT that maximize

W=\Tv(s(t)Mt)9t)dt + 4>(sT,T) (7.27) Jo

subject to Sj(t)=f(s(t),c(t),t), i = \,2,...,n,

gj(s(t),c(t),t)>0, y = 1,2,...,/»',

gk(s(t),c(t),t) = 0, k = m'+l,...,m,

Si(0) = 5,o given, i = l,2, . . . , « ,

5,(7) free, / = 1,2,...,«',

5,(7) = 5,T given, i = n'+l,...,n.

(7.28a)

(7.28b)

(7.28b')

(7.28c)

(7.28d)

(7.28e)

Notice that in (7.27) the function </>(sr, T) is added to the integral. A possible economic interpretation is that this function represents the max- imum value of an integral of future utility flow starting from time 7" with an initial capital stock s r , in just the same way as V*(s0,sT) represents the value of the integral expression in (7.27) for given initial stock s 0 and terminal stock sT.

The fact that the planner attaches some value to the terminal stock has a bearing on the transversality conditions.

Transversality conditions for free endpoint with scrap value

Theorem 7.2.1. For problem (7.27)-(7.28) the following conditions are necessary:

MT)=d(t>{*T'T), / = l , 2 , . . . , / i ' . (7.29) dsiT

Condition (7.29) makes economic sense: it equates the marginal benefit of an increase in siT (through its contribution to the scrap value) to the marginal cost of such an increase measured over the whole horizon and represented by irL(T) (see equation (4.80)).

Proof We now provide a heuristic proof that relies on solving problem (7.27)-(7.28) using a two-step procedure. For simplicity we assume that V*(s0,sT) is differentiate, although this assumption is not needed for (7.29).

In the first step, we arbitrarily fix the terminal stock at some value sT and find c(/) to maximize

V=\Tv(s(t)9c(t),t)dt (7.30) Jo

228 7 Endpoints and transversality conditions

subject to (7.28a)-(7.28c) and the fixed-endpoint condition

s{T) = sT fixed.

Let V* be the maximized value of V; then V* depends on s 0 and sT. The second step consists of finding the optimal sf that maximizes

m*o, *T) = *~(s 0 i s r ) + 0 ( s r , T). (7.31)

The necessary conditions for sf to be the solution of problem (7.31) are

dw « - * * • r — = 0 , 1 = 1,2,...,/!',

3s l T

which are equivalent to

dK* _ 30

Equations (4.80) and (7.32) imply the transversality conditions (7.29). •

Example 7.2.1. In order to illustrate the use of condition (7.29), let us return to the pollution control problem of Example 7.1.2 and modify it by adding the scrap value function

4>(PT, T) = -ypTe- 8T,

where y > 0 is a constant. The scrap value function has a negative deriva- tive with respect to pT because pollution is a "bad."

Our problem is to choose c(t) and pT that maximize

W= \T(\nc-\c2-bp)e-5tdt-ypTe- dT

Jo subject to

p = c2-mp, (7.33)

P(0)=p0 given, (7.34)

p(T)=pT free. (7.35)

The necesary conditions are (7.18)-(7.20), (7.34), and the transversality condition

v(T) = -ye-6Tt. (7.36)

Condition (7.36) is used to determine the constant of integration A in (7.24):

-ye~bT= AemT- (b/(S + m))e-8T.

This yields

7.3 Lower bound constraints on endpoint 229

\ r d+mj

and with (7.24)

e*'*(t) = (-y + ^-)e-(^HT-t) __ (*_\. w V 6 + mJ \d + mj Hence, from (1.22),

c(o=r2x^2f- 7 +-^-V" ( 6 + m ) ( r " o +f l b

b + m) \b + m

-1/2

It follows that the larger is 7, the smaller is the consumption flow. This makes economic sense: if it is more costly to dispose of the stock of "bad" at the terminal time T9 then the consumption flow, which adds to pollu- tion, must be reduced.

Remark. If the control problem is autonomous (in the sense that the only independent time term is e~bt appearing as a multiplicative factor in the integrand) and if the scrap value function takes the form (t>(sT, T) = e~bTh(sT), then the transversality condition can be represented in a phase diagram, thus providing useful qualitative information about the opti- mal path. For example, if problem (7.6) is modified so that the objective function is

W= Vu(c(t))e~bldt + e-dTh(sT), Jo

where h'(sT) > 0, h"(sT) < 0, then the transversality condition can be rep- resented by the curve

+(T) = h'(sT).

This curve is depicted in Figure 7.1. Any optimal path starting from s(0) = s0 must end at a point on this curve, and not on the horizontal axis as when there was no scrap value.

7.3 Lower bound constraints on endpoint

Quite often the terminal value of a state variable may be constrained to be not less than a prespecified constant. For example, a sand-mining firm may be required to leave a stock of sand, s(T), not less than some lower bound sL. Clearly, in many economic problems we wish to impose non- negativity constraints on terminal capital stocks.

In this section we consider the problem of finding sT and c(t) that solve problem (7.1) with (7.If) modified as

230 7 Endpoints and transversality conditions

Si(T) > siL given, i = 1,2,..., /T. (7.37)

The rest of (7.1f) remains unchanged.

Transversality conditions for lower bound constraints on endpoint

Theorem 7.3.1. For problem (7.1) modified as by (7.37), the following conditions are necessary:

T r ^ m O , Si(T)-siL^09 Vi(T)[Si{T)-siL] = 0, I = 1 , 2 , . . . , / I ' . (7.38)

Condition (7.38) is essentially the same as the complementary slackness condition in static optimization and has the same economic interpreta- tion: if a stock is not used up to its maximum extent (i.e., if the lower bound constraint is not binding), then its price must be zero.

To provide a heuristic proof of the necessity of (7.38), we use a by now familiar argument that relies on the two-step procedure. In the first step, we consider the solution of problem (7.1) with Si(T) = siT (fixed) for all /. In the second step we find the "best" sT, that is, the value sf that maxi- mizes V*(sT) subject to the constraint

s / T - s / L > 0 , / = l,2,...,/i'.

The necessary condition characterizing the best sT is

dV* /dV*\ -^-^0, s * r - s / L > 0 , (-—\(s*T-siL) = 09 i = l f 2,...,/!'. (7.39)

This condition and (4.80) imply the transversality condition (7.38). •

Example 7.3.1. Let us illustrate the use of condition (7.38) on a simple example. A firm has a fixed time interval [0, T] over which it can extract an exhaustible resource. Let s'(O) denote the initial stock of resource and c(t) the rate of extraction, so that

s(t) = -c(t). (7.40)

The profit flow derived from the extraction and sale of the i ̂ source good is v(c(t)). The firm's objective is to find the time path c(t) that maximizes

\T v(c(t))e~rt dt (7.41) Jo

subject to (7.40) and

7.3 Lower bound constraints on endpoint 231

c(t)>0, (7.42)

5(0) = s0 fixed, (7.43)

s(T)>0. (7.44)

We note that in this case the lower bound sL is zero. We do not need the constraint s(t) > 0 for all t in [0, T], because conditions (7.42) and (7.44), together with (7.40), ensure that s(t) is always nonnegative.

The Hamiltonian for this problem is

H=v(c)e-rt-7rc, (7.45)

c(t) is required to be nonnegative, and the necessary conditions are

t)hf — = v'(c*)e-ri- 7T < 0, (7.45a) oc

(v'(c*)e-rt- TT)C* = 0, c* > 0, (7.45b)

TT = - ^ = 0 , (7.45c) OS

s = ^ = -c\ (7.45d)

s(0) = s09 (7.45e)

7 r ( r ) > 0 , s f > 0 , T(T)S} = 0. (7.45f)

To sharpen our results, we assume that the profit function v(c) has the following properties:

i/(0) > 0 , (7.46a)

v"(c) < 0 for all c > 0, (7.46b)

v'(cM) = 0 for some cM > 0. (7.46c)

Condition (7.46a) ensures that extraction is profitable at some positive rate of extraction, condition (7.46b) ensures that for any given ir{t) there exists a unique c*(t) that maximizes the Hamiltonian, and condition (7.46c) im- plies that it never pays to extract at a rate c(t) exceeding cM because mar- ginal profit is negative beyond cM.

From (7.45c) and (7.45f), ir(t) is a nonnegative constant, say K:

7r(O = 7 r ( 0 ) = / i : > 0 .

The constancy of -K and condition (7.45a) imply that the discounted mar- ginal profit must be a constant during any time interval of positive ex- traction. (This result is known as Hotelling's rule, in honor of Hotelling's contribution to the theory of the mine.)

232 7 Endpoints and transversality conditions

There are two possible cases: K > 0 and K = 0. We shall show that K = 0 only if the time horizon is so short that the total cumulative extraction that yields the highest profit cannot exceed the initial stock. In this case the firm can extract the resource at the myopic profit-maximizing level cM for all t in [0, T] and still have s(T) > 0, thereby justifying the use of the myopic rule. Under these circumstances, any addition to the stock will be of no value to the firm. (Recall that the firm has a fixed time horizon T\ after T, it no longer has the right to extract.) We now seek to confirm our intuitive reasoning by a simple manipulation of the necessary conditions (including the transversality condition). Let us define

f=s0/cM. (7.47)

If the fixed time horizon T is equal to t, then extraction at the constant rate cM will just exhaust the resource stock at time f9 because

sT-s0=\ T*(t)dt = -cMT$ (7.48)

Jo so that if r = T, then sT = 0. If T is less than T, then extraction at the con- stant rate cM will leave some positive stock at time T, as can be seen from (7.47) and (7.48). So if T< ?, the optimal extraction policy is c*(t) = cM for all t. The necessary conditions (7.45a)-(7.45f) will be satisfied with 7r(0 = 0. Sufficiency is satisfied because of the concavity of the Hamil- tonian in the control and state variables.

We now show that if T> T9 then K must be positive. Suppose K = 0 when T> T. Then from (7.45a) and (7.45b), c\t) = cu for all t in [0, T], but this is not possible, because at the rate of extraction cM, the resource stock is exhausted at t< T. Thus, K must be positive when T > Tand the transversality condition (7.45f) implies s*(T) = 0.

In the remainder of this section we examine the time profile of the opti- mal extraction path when T> f. Recall that the transversality condition and K > 0 imply s*(T) = 0. We derive some properties of the optimal path in this case.

We first show that c(t) must be monotone decreasing. Consider any pair (ta, tb) with ta < tb and c(ta) > 0, c(tb) > 0; then from (7.45a)

v'(c(ta))e- r'< = v'(c(tb))e-

rti> = K. (7.49)

Since ta<tb9 this implies v'{c{ta)) < v'(c(tb)) and hence c(ta) >c(tb). Next we show that if c ever becomes zero, it will remain zero until the

end of the horizon. Again consider ta < tb with c(ta) = 0. We will establish that if c(tb) > 0, we would have a violation of the necessary conditions. Suppose c(tb)>0; then by (7.45a)

K=v'(c(tb))e- rtb>v'(0)e-rt°. (7.50)

7.3 Lower bound constraints on endpoint 233

Hence, v'(c(tb)) > u'(0)e

r{tb-^ > v'(0)9 (7.51)

but this is impossible because v'(0) > v'(c) for all c>0. Finally, we show that the optimal path is continuous. This is done in

several stages. We can rule out jumps from zero to positive with the pre- ceding result. We use the strict concavity of v to show that it is never opti- mal to have any other jump discontinuities in c(t). Suppose there is one at t\ < T. Let c(t^) and c{t+) denote the left-hand and right-hand limits of c(t). (Recall that c(t) is only required to be piecewise-continuous.) If both limits are positive, then from (7.45a) and the piecewise continuity of c(t), there exists e > 0 such that for all positive 5 < e,

v'(c(tl + 5))e- r{ti+8) = v'(c(tl-d))e-

r^-d) = K. (7.52)

Taking the limit as 5 tends to zero,

v'(c(tt)) = vf{c{tD).

The strict concavity of v implies that the preceding equation is satisfied if and only if c(t+) = c(tf). Alternatively, suppose that c(^+) = 0, so that (7.52) no longer holds. Then for tx < T, we have

v'(c(tx + b))e- r{t^b)<K=v'(c(tx-b))e-

r{ti-b\ (7.53)

Again taking the limit as b -> 0,

but this inequality cannot be satisfied for c(t\) = 0 and c(t^) > 0, because v'(c) is strictly decreasing.

To sum up, c(t) is continuous everywhere, decreasing while positive, and if it reaches zero it will remain at that level.

Let us see how the optimal path varies as T takes on values larger than T. The path is determined by

v'(c(0)) = K= v'(c(t))e~rt (7.54)

as long as c(t) remains positive. Thus, c(t) remains below cM for all t and any small increase in T above T results in a small decrease in c for all t. The exact path is determined once the value c(0) is chosen and it is done so as to exhaust s0 since the transversality condition requires s*(T) = 0. This does not mean that s(t) necessarily remains strictly positive for all t < T: it may be optimal to exhaust the resource stock at some time tE < T, provided that Tis large enough (in relation to r and s0) and provided that i/(0) is finite, as we now demonstrate.

Let us try to understand why this may occur. Given a finite supply of the resource, we tend to accumulate more benefit if we reduce the flow of

234 7 Endpoints and transversality conditions

the resource to a trickle but for a longer period; this is due to the strict concavity of the net benefit function v(c). This phenomenon, however, must be balanced by the preference for earlier benefit due to the positive discount rate. For this reason there may exist a time tE<Tsuch that c becomes nil at that date and according to (7.54)

v'(c(Q)) = v'(0)e-r\ (7.55)

tE = r- l[\nv'(0)-lnv'(c(0))]9 (7.56)

where c(0) is to be determined. If v'(0) = a>, there is no such time, for T is finite. But if v'(0) is finite, there is a time tE at which c becomes zero. To determine tE and c(0), note that the choice of c(0) determines c(t) through (7.56) and that we must have

\'Ec(t)dt = s0. (7.57) Jo

We must choose c(0) such that the path c(t) determined by (7.54) satisfies (7.57) subject to (7.56).

Let us apply our results to a special case, where v(c) is quadratic:

v(c) = ac-(bc2/2), a > 0 , 6 > 0 ,

v'{c) = a — be.

The function v(c) attains its maximum at c M :

cM = a/b.

The time horizon T, defined by (7.47), is in this case

T=bs0/a.

The time tE can also be determined. Note that (7.55) yields

a-bc(0) = ae-rt* (7.55')

and (7.54) yields

c(t) = (a/b)-b-{[a-bc(0)]ert. (7.54')

Thus, the integral (7.57) gives

(a/b)tE-[a-bc(0)](e rtz-l)/br = s0. (7.58)

Substitute (7.55') into (7.58) to obtain

(a/br)[e-rt*-l + rtE] = s09 (7.58')

and for each s0 a unique tE can be determined, because the expression in- side the square brackets is a strictly increasing function of tE. Having de- termined tE, we can use (7.55') to solve for the optimal c(0), if the time

7.4 Lower bound constraints with a scrap value 235

horizon T exceeds or equals tE. It is clear that both c(0) and tE are in- creasing functions of the initial stock size.

Finally, if the time horizon T is shorter than tE but longer than T, then using the results that c{t) is positive for all t < Tand the stock is exhausted in this case, we can compute c(0) from (7.54') and the exhaustion con- dition:

^c(t)dt = ~T-(erT-l)[a-bc(0)]^; = So.

Therefore,

c(0) = (a/b) + [(a/b)T-s0][r/(e rT-l)]9

which is positive because (a/b)T> (a/b)f= s0. The reader should be able to draw a phase diagram in the space (s, \l/)>

where \p is the current-value shadow price,

We also invite the reader to verify that all the qualitative results obtained so far for the resource extraction problem, with v(c) satisfying (7.49a)- (7.49c), remain valid when the discount rate is not a constant, that is, when we replace the discount factor e~rt by a general function a(t) > 0, where a ' ( / ) < 0 .

7.4 Problems with lower bound constraints on endpoint and a scrap value function

In this section we consider a slightly more general class of problems that combines the features of problems in Sections 7.2 and 7.3: we retain the lower bound constraints on endpoint as in (7.37) of Section 7.3 and mod- ify the objective function to allow for a scrap value function; thus, the objective function is as in (7.27) of Section 7.2. Both these equations are reproduced below and renumbered for clarity. It will come as no surprise that the transversality condition is a combination of those of the preced- ing two sections.

Transversality conditions for problems with lower bound constraints on endpoint and scrap value function

Theorem 7.4.1. For problem (7.1) modified by (7.59a) and (7.59b), the transversality conditions (7.60) are necessary:

s,T>s/ Lgiven, sjT free, / = !, 2,...,«', j = / I ' + 1 , . . . , / I , (7.59a)

236 7 Endpoints and transversality conditions

W=\Tv(s(t)Mt),t)dt + 4>(sT,T). (7.59b) Jo

Equation (7.59b) replaces (7.1a) as the objective function. Then, for / = 1,2,...,/!',

d<t> / d</> \

M r ) ~ ^ 0 , 5 / T - 5 / L > 0 , U ( r ) - - ^ - j ( 5 / T - 5 / L ) = 0 (7.60a)

and fory = n'+l, . . . , « ,

irj(T) = -^-. (7.60b) dsjT

Equations (7.60) are the transversality conditions for this problem. The reader is invited to give a heuristic proof of the necessity of this con- dition by using the familiar two-step procedure, the second step being the maximization of

W(sT) = V*(sT) + <l>(sT,T)

subject to

siT>siL, Syr free, / = 1,...,/?', j = AZ'+1, ..., n.

Example 7.4.1. We now illustrate the use of (7.60) on a model of opti- mal saving.

An individual has the utility function

u(c) = (l-y)-{cl-y+Ai 7 > 0 , ,4 = const,

where c(t) denotes consumption at time t. Her stock of financial assets is s(t), which yields the interest income I3s(t)9 where (3 > 0 is the interest rate. Her wage income is an exogenous flow and is denoted by w(t). The difference between total income and consumption is her net saving, which is the net addition to her stock of financial assets:

S(t) = Ps(t) + w(t)-c(t). (7.61)

The initial stock of financial assets is 5(0), exogenously given. The termi- nal stock, 5 T , is bequeathed to her son on her retirement. The individual's valuation of this bequest is

<t>(sT,T) = e- 8TmsT, m > 0 , (7.62)

where 5 > 0 is the rate of discount. The individual is free to choose sT, but there is a lower bound constraint

sT>sL.

7.4 Lower bound constraints with a scrap value 237

(sL is exogenously given; if sL = 0, the constraint means that the individ- ual cannot bequeath a negative stock of financial assets, i.e., debts.)

The time horizon Tis fixed. The individual's problem is to find c(t) and sT that maximize

[TlA + (l-y)-l[c(t)]l-y}e-6tdt + e-8TmsT (7.63) Jo

subject to (7.61) and

s(0) = s0 given,

s(T) > sL, sL given; we assume sL < s0.

Notice that we did not write down the constraint c(t) > 0 because the util- ity function we have chosen has the property u'(0) = oo and this ensures that c(t) is strictly positive along an optimal path.

The Hamiltonian for problem (7.63) is

H=[A + (l-y)-lcl-y]e-dt + >K((3s + w-c)9

and the necessary conditions are

^ = c - ^ - 6 ' - 7 r = 0, (7.64)

dc 7r = - ^ = -/37r, (7.65)

ds

s = —- = l3s + w-c, (7.66)

s(0) = *<>. (7.67)

From (7.60) we obtain the transversality condition

TT(T) - me-hT> 0, sT -sL > 0, (ir(T) - me- 8T)(sT-sL) = 0. (7.68)

Condition (7.64) says that the discounted marginal utility of consump- tion is to be equated with the discounted shadow price of the stock of fi- nancial assets. This condition and (7.68) imply that if the constraint sT > sL is not binding, then for an optimum the discounted marginal utility of consumption at T must be equated with the discounted marginal val- uation of the bequest. However, if m is "very small," then this equality cannot be satisfied and the individual values consumption more than be- quest and would have wished to erode further her financial assets but the constraint sT > sL prevents her from doing so. We now proceed to verify our intuitive reasoning.

238 7 Endpoints and transversality conditions

From (7.65), we have

7r(0 = 7r(0)e-^. (7.69)

Conditions (7.64) and (7.69) yield

[c(t)]-ye~8t = ir(t) = Tr(0)e-Pt = [c(0)]-ye~(3t. (7.70)

Hence,

c(/) = c ( 0 ) e x p [ ( / 3 - S ) / / 7 ] , (7.71)

where exp[ j>] denotes ey for any y. Substitute (7.71) into (7.66):

s-/3s = w(t)-c(0)exv[((3-d)t/y].

Multiply both sides by exp[—fit] and integrate from 0 to T:

exp[-(3T]s(T) = s(0) + Y

- c ( 0 » J r i - e x p [ ( g - 8 " ^ ) 7 1 ] , (7.72) 'S + 0 7 -/H

y[ y \y where Y is the present value of the stream of wage income

Y=[ w(t)exp[-l3t]dt. Jo

The right-hand side of (7.72) is a decreasing function of c(0), because if 5 + fiy — (3 is positive (negative) then the last exponential is smaller (larger) than unity. Therefore, the terminal stock s(T) is smaller the larger c(0) is. Furthermore, if both sides of (7.72) are multiplied by exp[/3r] we have

s(T) = (5(0) + Y)efiT- c(O)ae0Tf {1.12')

where

a = [y/id + l3y-(3)]{l-Qxp[(i3-5-(3y)T/y]}>0. (7.73)

Since we have assumed 5(0) > 5L, the first term on the right-hand side of (7.72') exceeds 5L and if c(0) is sufficiently large, then s(T) = 5L; we de- note this particular value of c(0) by c L (0):

cL(0) = (5(0) + Y- e-V TsL)/a. (1.1 A)

Using (7.70) we obtain the value of TT(T) that corresponds to cL(0):

7 r L ( n = [ c L ( 0 ) ] - ^ - ^ .

If TTL(T) > me~ 8T, then the transversality condition (7.68) is satisfied, with

5T = 5L, for the consumption path starting with c(0) = cL(0), and therefore this path is the optimal path. (Recall that by construction the path (7.71) with the initial condition given by (7.74) satisfies all the necessary condi- tions; sufficiency follows from the usual concavity property.) In contrast,

7.4 Lower bound constraints with a scrap value 239

if TTL(T) < me~, then the consumption path starting with cL(0) is not op- timal because the transversality condition is violated. In economic terms, this is the case in which the marginal value of bequest, ra, is so high that if the stock of financial assets were driven to sL at time T, the mar- ginal value of consumption would be lower than m. What is the optimal consumption path in this case? Since the optimal terminal stock must be strictly greater than sL, the transversality condition (7.68) implies

ir(T) = me-6T.

The optimal value c*(0) can then be calculated from (7.70):

[c*(0)]-ye-(3T=me-8T

and c*(t) = m-l/yexp[(l3-d)y-\t-T)].

This completes our analysis of the determination of the optimal path. We now draw the reader's attention to some interesting features of the solution. First, the lower bound constraint on sT is binding if and only if

[(s(0) + Y-sLe-V T)/a]-y>mew-8)T.

Therefore, if 5(0) or Y is sufficiently large, given sL, the constraint will not bind. In retrospect, this result is intuitively obvious.

Second, from (7.71) c(t) > 0 if and only if the rate of interest exceeds the rate of discount, that is, (3 > 6, This result is plausible: if the rate of interest is sufficiently high, the individual will have an incentive to save more during the earlier part of the planning horizon. This result relies on the assumption that marginal utility of consumption at any time depends only on c(t).

Third, using (7.71) we can calculate for any given sT the present value of the utility flow:

V{s0ysT) = \ T{A + {\-y)-x[c(t)]l-y}e-btdt

Jo

+ y ( l - e x p [ - S r ] ] , (7.75)

where from (7.72)

c(0) = (s(0) + Y-e-l}TsT)/a

and a is given by (7.73). The derivative of Fwith respect to s T is

L y \

240 7 Endpoints and transversality conditions

dV dc(0) 7 f -^- = [c(0)]-y * ' J R 1-exp ds T ds T 6 + 187-18 ( y \

= - [ c ( 0 ) ] " ^ - ^ .

This result and (7.70) imply that

(7.76)

which serves to verify in this instance our general proposition about the meaning of the shadow price. Similarly,

dV - _ = [ C ( 0 ) ] ^ = 7 T ( 0 ) . os0

7.5 Free-terminal-time problems without a scrap value function

Thus far we have always assumed that the terminal time T is fixed. In many economic problems it makes sense to allow the planner to choose T. For example, mining firms are often free to discontinue exploiting a site before every grain of ore is extracted, purely for economic reasons. The optimal terminal time will be denoted by T*. To determine the addi- tional unknown T*, we need another necessary condition.

Additional transversality condition for free-terminal-time problems without a scrap value function

Theorem 7.5.1. For problems without a scrap value function such as problem (7.1) or modified as in Sections 7.1 or 7.3, the additional trans- versality condition when terminal time is free is

H(s*(T*), c*(T*)9 ir(T*)9 T*) = 0, (7.77)

provided that T* is finite.

We now proceed to offer a heuristic proof of Theorem 7.5.1. Let us first consider the case in which the terminal stocks must take on fixed values as in problem (7.1):

5;(T*) = bh bt fixed, / = 1,2,..., n. (7.78)

We then have the following free-terminal-time, fixed-endpoint problem: find c(t) and T t h a t maximize

VF=\ Tv(s(t),c(t),t)dt (7.79)

Jo

7.5 Free terminal time without a scrap value 241

subject to

si(t)=f i(s(t),c(t)9t), (7.80)

Si(0) = si0 fixed, (7.81a)

Sj(T) = bi fixed. (7.81b)

Notice that we use b{ rather than siT in order to emphasize that when we vary T9 the terminal stocks must remain fixed at the value bi9i = \9 2, ..., n. The subscript F in (7.79) indicates that this is a free-terminal-time problem.

For any fixed T, define

V(b9T) = V(bl9b29...9bn9T) = max\ Tv(s(t)9c(t)9t)dt (7.82)

subject to (7.80) and (7.81). Using the argument that we developed in Sec- tion 4.5 we have for an arbitrary function ir(t)

V(b9T)=\ T[H(s*9c*9ir9t) + <k.s*]dt-Tr(T).b + Tr(0)*s0. (7.83)

Jo Differentiate (7.83) with respect to T:

^ = H(s*(T)9 c*(D, *(T), T) + 7r(7>s*(r)

ds* dc*l d (Hs + *). — + (Hc).— \dt- — (7r(T)b). (7.84) + r

Jo Since the integral term is zero when v(t) is the optimal path of the co- state variables and since s*(T) = b by (7.81), we have

dV(b, T)

dT = H(s*(T)9c*(T)9ir(T)9T). (7.85)

Equation (7.85) says that when we lengthen the time horizon of problem (7.82) by a small increment AT, the value of the integral of the fixed-time problem (7.82) will increase or decrease according to whether the Hamil- tonian, evaluated at T, is positive or negative. The best time horizon T* is that value of T t h a t maximizes K(b, T); thus, the derivative (7.85), eval- uated at T\ is necessarily zero provided that T* is finite. This establishes the necessity of (7.77). If (7.85) were positive for all T, the optimal time horizon would be infinite.

A moment's reflection will convince the reader that (7.77) is also neces- sary if the 6/'s are not fixed but are to be chosen optimally (possibly sub- ject to some constraints). This is because for any optimally chosen set of values (b*9 b\>..., b„) condition (7.77) is necessary with respect to the choice of T\ given that

s!{T*) = bf.

242 7 Endpoints and transversality conditions

In order to offer an economic interpretation of the transversality con- dition (7.77), we begin with a concrete example. Consider a resource ex- traction problem (similar to Example 7.3.1) with fixed endpoint s(T) = b (fixed). For given T, the firm finds c{t) that maximizes

Tv(c(t))e~rtdt o

subject to

Ht) = -c(t),

s(0) = s0,

s(T) = b.

Now consider an increase in Tby an amount A r s o that the terminal con- straint would be

s(T+AT) = b.

The gain to the firm would be the flow of profit from T to AT". This is approximately

v(c(T))e~rTAT.

But the stock at T would have to be larger:

rT+AT s(T) = b-\j s(t)dt = b + (AT)c(T).

In other words, the cumulative extraction from 0 to T would have fallen by the amount (AT)c(T). The shadow price of the resource being ir(T), the opportunity cost of this amount of stock is therefore ir(T)(AT)c(T). If T is optimally chosen, the marginal gain must equal the marginal loss:

v(c(T))e~rTAT= ir(T)(AT)c(T). (7.85')

Dividing both sides of (7.85') by AT, and taking the limit A!T-> 0, we have

v(c(T))e-rT-ir(T)c(T) = 0.

But this is precisely the Hamiltonian at time T. The marginal gain of increasing 7Ms v(s(T), c(T), T)ATfor more gen-

eral models, and the marginal loss is -ir(T)ATf(s(T), c(T), T). The opti- mal choice of Tmust equate marginal gain with marginal loss.

Example 7.5.1. We now illustrate the use of the transversality condition (7.77) in the determination of the optimal terminal time T*. Let us return to problem (7.63) and make some modifications: instead of a fixed time horizon T we now allow T to be optimally chosen; we also assume that

7.5 Free terminal time without a scrap value 243

dH dc

x =

s =

= c ye

8H _ ds ~

dH _ dir

-*'_

= 0,

c,

-TT = 0,

ra = 0, w(t) = 0, 0 = 0 for simplicity. We may interpret our problem as that of a resource-extracting firm, which derives a profit flow

u(c(t))=A + (l-y)-l[c(t))l-\ 1-T>0,

where c(t) is the rate of extraction and A < 0 is a fixed rent per unit of time that must be paid as long as the firm remains in business. The as- sumption 1 — 7 > 0 is made so that we can identify the other term as a positive revenue. The firm must determine the closing-down date T* and the extraction path c*(t) for t in [0, T*]. The initial stock is given and the terminal stock must be at least as large as sL > 0.

The Hamiltonian for this problem is

H=[A + (l-y)-lcl-'Y]e-dt-Trc. (7.86)

The necessary conditions are

(7.87a)

(7.87b)

(7.87c)

and the transversality conditions are

7 r ( D > 0 , s T - s L > 0 , 7 r ( D ( 5 T - 5 L ) = 0, (7.87d)

H(T) = [A + (l-y)-lc(T)l-y]e-8T-ir(T)c(T) = 0. (7.87e)

We have pointed out in the preceding section that c*(t) is strictly positive because w'(0) = <x>. Hence ir(t) is positive. Condition (7.87d) then implies that Sj = sL.

Turning to condition (7.87e), we can see that this condition and (7.87a) yield a precise characterization of the optimal terminal rate of extraction:

[A + {l-y)-lc(T)1-^/c(T) = [c(T)]-i; (7.88)

that is, the choice of c(T) must equate average profit with marginal profit. Since A < 0 and (1 - 7) > 0, this equation yields a unique value c*(T) > 0:

c*(T) = [-A(\-y)/y]W-y\ (7.89)

This is illustrated in Figure 7.2, which shows that the average profit is maximized at c*(T). This determines the optimal terminal rate of extrac- tion, but by itself does not determine the terminal time. To determine T* we must combine (7.89) with other necessary conditions. From (7.87)

c*(T) = (s(0)-sL)Wy)[e dT/y-irl. (7.90)

244 7 Endpoints and transversality conditions

Figure 7.2

We obtained this by noting that [c*(t)]~ye

[Tc*(t)dt = s(0)-sL.

-«/ — [c*(T)rye-dJ and

We obtain T* by equating the right-hand side of (7.89) with that of (7.90):

e 5 7 , / ^ l = ( 5 ( 0 ) - 5 L ) ( 6 / 7 ) [ ( - > l ) ( l - 7 ) / 7 ] " 1 / ( 1 " 7 ) . (7.91)

Thus, T* is an increasing function of 5(0) and a decreasing function of the fixed cost B (where B = — A > 0).

The alert reader will have noticed that the assumption A < 0 is essen- tial, given that 1 - y > 0; for if both A and 1 — y are positive, then for any fixed value of T the derivative dV/dT(=H(T)) is positive, indicating that the value of the integral is monotone increasing in Tand that the optimal time horizon is infinite. This can be verified directly from (7.75) by dif- ferentiating its right-hand side with respect to T.

Thus, problems with free terminal time have the special feature that the optimal path itself depends on the sign of the integrand. For instance, negatively valued utility functions are acceptable for fixed T but yield T* = 0if Tis free.

7.6 Free-terminal-time problems with a scrap value function

In free-terminal-time problems with a scrap value function, equation (7.77) is no longer the correct condition.

7.6 Free terminal time with a scrap value 245

Additional transversality condition for free-terminat-time problems with a scrap value function

Theorem 7.6.1. For free-terminal-time problems with a scrap value func- tion such as problem (7.27), the following transversality condition is also necessary:

d<Mb, T*) //(s*(r*), c*(r*), ir(r*), T*) + ^ = 0, (7.92)

where </>(b, T) is the scrap value function. This applies whether or not endpoint values of the state variables are constrained.

To see the necessity of (7.92) recall that the choice of Tmust maximize

K(b,r)+0(b,r),

where F(b, T) is defined by (7.82). The optimal T* must satisfy dV dct> „ dT dT

provided that T* is finite. But from (7.85) dV/dT is H(T) and (7.92) follows.

Example 7.6.1. Let us apply (7.92) to a version of the resource extraction problem considered in Section 7.5. Find the extraction path c*(t) and the closing-down time T* that maximize

[T[A + (l-y)-lcl-y]e-dtdt + mbe-dT, (7.93) Jo

where b is terminal stock, m > 0, A < 0, (1 — 7) > 0. The maximization is subject to

s(t) = -c(t), (7.94a) 5(0) = s0 fixed, (7.94b) s(T) = b, b fixed. (7.94c)

The Hamiltonian is (7.86), the necessary conditions are (7.87a)-(7.87c), and because s(T) = b (b fixed) the only transversality condition is, by Theorem 7.6.1,

[A + (\-y)-xc(T)l-')<]e-bT->K{T)c(T)-bmbe-bT-0. (7.95)

Use (7.87a) to simplify (7,95):

[-dmb+A + (l-y)-lc(T)l-^]/c(T)^[c(T)]-\ (7.96)

This solves for c*(T) uniquely:

c*(T) = [(8mb-A)(l-y)y-l]l/{l-y). (7.97)

246 7 Endpoints and transversality conditions

profit

^ profit — 5mb

/ / ° * ( T )

A | /

A — 6mbf

Figure 7.3

This condition and (7.90), with b = sL,

c*(T) = (s(0)-b)(6/y)[e6T^-l]-\ (7.98)

determine the optimal time T* uniquely:

r * = ( l / a ) l n [ l + ( 5 0 - Z 7 ) ( 5 / 7 ) [ ( 6 m 6 - ^ ) ( l - 7 ) / 7 ] - 1 / ( 1 - 7 ) ) . (7.99)

Equation (7.96) resembles (7.88) but for the presence of the scrap value term. It is illustrated in Figure 7.3, which shows that the value of c*(T) has been increased by the introduction of the scrap value. The terminal time T* is smaller, the larger is the scrap value parameter m\ see (7.99). Equation (7.95) gives us a concrete example that is helpful for interpret- ing the transversality condition. The first term consists of the additional discounted profit flow per unit of time that would be obtained if T* were increased. The third term is the discounted interest cost: if the firm were to delay the closing time, it would have to delay the receipt of the scrap value, and the opportunity cost of this delay is the interest income fore- gone. The second term is the opportunity cost of the extra amount of stock needed, as we argued earlier (see (7.85')).

7.7 Other transversality conditions 247

Finally, let us consider the case in which the terminal stock b is free, subject only to a lower bound constraint:

s(T) = b>sL, sL fixed. (7.100)

In this case, in addition to the transversality condition (7.95) we have a second transversality condition:

Tr(T)-me-8T>0, 6 - 5 L > 0 , (ir(T)-me- 8T)(b-sL) = 0. (7.101)

(This condition is identical to (7.68).) There are two possibilities: (i) b* = sL and (ii) b* > sL. Let us try the first

solution. Substitute sL for b in (7.97) to obtain a specific value of c*(T), and denote this by cl(T). Equation (7.87a) implies that ir(T) must take the corresponding value

*L(T) = [ct(T)]-'Ve- dT. (7.102)

If [cl(T)]~y>:m, then the transversality condition (7.101) is satisfied and therefore b* = sL is the optimal solution. If [cl(T)]~

y < m, then (7.101) is violated, indicating that b*>sL is optimal. Now since the lower bound constraint is not binding, we have ir(T) = me~8T, implying [c(T)]~y = m. This and (7.97) yield an equation that determines b* uniquely:

m-l/y=[(5mb*-A)(l-y)y-l]l/il-y\

The optimal time T* can in either case be determined using (7.99).

7.7 Other transversality conditions

In Sections 7.1-7.6 we derived transversality conditions corresponding to various endpoint or terminal-time specifications that are frequently en- countered in economic problems. There are, of course, many other, less common specifications. For example, the initial condition 5/(0) = si0 (fixed) may be modified to 5/(0) = at (#/ free). Similarly, the initial time condition t0 = 0 has so far been taken for granted, but there is no reason why t0 it- self cannot be an object of choice. More generally, we may allow equality and inequality constraints involving the initial time t0, the terminal time T, the initial values of the state variables 5 / 0 , and their terminal values siT.

The reader is invited to use the approach adopted in Sections 7.1-7.6 to derive the following transversality conditions for the various cases.

(a) Free initial time:

H(t0) = 0. (7.103)

(b) Free initial conditions: With 5/(0) = #, free, the transversality condition is

248 7 Endpoints and transversality conditions

7T/(0) = 0. (7.104)

(c) Free initial conditions, with initial purchase cost: If the initial capital stock Sj(t0) can be chosen (i.e., it is "free" in the sense that it is not fixed) but at a cost that increases with its size, the objective function is

[ % ( s , c , O * + 0 ( b , r ) - 0 ( a , f o ) , (7.105)

where 0(a, /0) is the cost of purchasing the stocks s(/0) = a. If t0 is fixed and at is free, the transversality condition is

*i(to) = -^-. (7.106) oat

If t0 is also free, we have an additional transversality condition:

H(t0) = ~ . (7.107) dt0

It becomes apparent that it would be useful to have a general formula from which the various transversality conditions can be derived. Fortu- nately, a theorem by Hestenes (1966) provides us with exactly what we need. The following section is devoted to this theorem.

7.8 A general formula for transversality conditions

In order to obtain a general formula for transversality conditions, we as- sume (without loss of generality) that the following variables are poten- tially free, subject to some constraints:

the initial time t0, the terminal time T, the initial values of the state variables Sj(t0), i = 1,2,..., n, the terminal values of the state variables 5/(7"), / = 1,2,..., n.

For notational convenience, we write

t0 = a0, (7.108a)

Si(t0) = ah / = 1,2,..„/i, (7.108b)

T=b0, (7.108c)

St(T) = bh i = 1,2,..., n. (7.108d)

Let (a,b) = ( t f 0 , « i » - - . , a n , b 0 , b u . . . , b n ) . We assume that the constraints on (a, b) can be expressed in the form of E equality constraints plus / in- equality constraints:

7.8 General formula for transversality conditions 249

hJ(*,b) = 0, y = l , 2 , . . . , £ ,

/*'(a, b ) > 0 , i = E+l,E+2,...,E+I.

All the functions hj and hl are assumed to be differentiate. In most prac- tical cases they take very simple forms. For example, if t0 is fixed at 0, then one of the h\ say hl, takes the form

tf0-0 = 0. (7.109)

If t0 is free, but is constrained to lie in the interval [ 2 , 8 ] , say, then we can incorporate this constraint in the form of two /*'-type inequalities:

tf0-2>0, (7.110a)

8 - t f 0 > 0 . (7.110b)

Similarly, if sx(T) is fixed at sh then one of the /* y,s, say h3, takes the form

bx-s^O. (7.111)

If s2(T) must be at least as large as a certain lower bound S2L> w e write as

one of the hl%

b2s2L>0. (7.112)

If s4(T) must be at least twice as great as s3(T), we write

6 4 - 2 6 3 > 0 . (7.113)

Our general control problem can now be formulated as follows: find the time path of the control variables c ( 0 and the values (a,b) to maximize

W=[T v(s(t),c(t),t)dt + G(*,b) (7.114)

subject to

s , ( 0 = / ' ( s ( 0 , c ( 0 , 0 , i = l , 2 , . . . , / ! , (7.115a)

gJ(s(t),c(t),t)*0, y = l , 2 , . . . , m ' , (7.115b)

g * ( s ( 0 , c ( 0 , 0 = 0, A: = m ' + l , . . . , m , (7.115c)

Si(t0) = ah / = l , 2 , . . . , / i , (7.115d)

Si(T) = bh / = 1,2 /i, (7.115e)

'o = *o. (7.H5f)

T=b0, (7.115g)

A'"(a,b) = 0, j = l,2,...,E9 (7.115h)

/ * ' ( a , b ) > 0 , / = £ + l , . . . , £ + 7 . (7.115i)

Notice that the function G(a,b) in (7.114) generalizes the concepts of scrap value function and initial cost function. It might take a simple form such as

250 7 Endpoints and transversality conditions

mbxe-^o-qaie-6"*,

which has the interpretation of a scrap value function minus an initial purchase cost function, where 8 is the rate of discount, m the disposal value per unit, and q the purchase price per unit.

We are now ready to state the theorem on transversality conditions.

Theorem 7.8.1. If problem (7.114) has an optimal solution, then this so- lution must satisfy the necessary conditions (6.73)-(6.77) and the follow- ing transversality conditions (where all derivatives and all functions are evaluated at s*, c*, a*, b*):

dG I±E dhk H(t0)-~ 2 j i * — = 0, (7.116a)

°<*o k = \ °a0 dG !+E dhk

*&*) + — + S Mit-5—= 0, * (7.116b) dOi k = i da{

where / = 1,2,...,«;

dG r+E dhk / / ( r ) + ^ 7 T + 2 M*7jj- = 0, (7.116c)

dG !+E dhk

dbj k=i dbi

where / = 1, 2, . . . , n, and where \kk are multipliers with the following properties:

(i) for k = 1,2,..., E, \kk are constants and

/ * V , b * ) = 0; (7.116e)

(ii) for k = E+l,...,E+I, ^ are constants and

fik > 0, A*(a*, b*) > 0, nkh k(**9 b*) = 0. (7.116f)

Remark. One should distinguish X(/), piecewise-continuous multipliers associated with (7.115b)-(7.115c) and included in the Hamiltonian, from H, constant multipliers associated with (7.115h)-(7.115i) and not included in the Hamiltonian.

We will not offer a proof of the theorem. The reader is invited to con- struct a heuristic proof along the lines we used to prove the various trans- versality conditions in Sections 7.1-7.6. The reader should also check that the conditions stated in those sections are special cases of (7.116a)-(7.116f).

We should warn the reader that Theorem 7.8.1 has not been stated in the most rigorous language. For example, we should have included a shadow

7.9 Sufficiency theorems 251

price 7r0 > 0, but we have chosen to set TT0 = 1 even4hough income abnor- mal cases 7r0 = 0. The reader should recall a similar remark in Section 6.3, where we referred to other sources for further discussion. For a precise statement and a formal proof of Theorem 7.8.1, the reader may consult Hestenes (1966, theorem 11.1); see also Long and Vousden (1977, pp. 14- 19) for discussion.

7.9 Sufficiency theorems

In Chapter 6, we offered a sufficiency theorem for the case of fixed end- point and fixed time horizon. That theorem is not applicable to the vari- ous cases considered in the present chapter. We now offer a number of sufficiency theorems which ensure that, under certain conditions, any so- lution that satisfies the necessary conditions (including the transversality conditions) is an optimal solution.

It turns out that if both r a n d t0 are fixed and finite, then the required sufficiency theorem is only a simple adaptation of the one offered in Chap- ter 6. Matters are not so simple when either t0 or T(or both) are free. For the first case, we offer the following theorem.

Theorem 7.9.1: sufficiency. Let both t0 and T b e fixed and finite. With- out loss, we set t0 = 0. Let (s*(0, c*(/)) with initial and terminal endpoints (tff, flrj, . . . , a*9 bj, b$, . . . , b*) be a path satisfying (7.115a)-(7.115i), let ir*(t), X*(t) be costate variables and Lagrange multipliers associated with (7.115a)-(7.115c), and let /i* be constant multipliers associated with end- point conditions (7.115h)-(7.115i). Then the necessary conditions stated in Theorem 7.8.1, that is, conditions (6.73)-(6.77) and (7.116a)-(7.116f), are also sufficient for a global maximum provided that the following con- ditions are satisfied:

(i) the Lagrangean <£ = i> + i r M + A**g is concave in the variables (s,c);

(ii) the function G(a,b) is concave in the variables (aha2,...,an,bh b2,...,bn);

(iii) the functions tithk(a, b), k = 1,2, ...,E+I, are concave in (au a2, ...,an,bhb2,...,bn).

Proof. Let a = (au a2,..., an) and b = (bu b2,..., bn) so that G and h become G ( a 0 , a , Z?0,b) and h ( a 0 , a , 6 0 , b ) . Note that both a0 and b0 are fixed (t0 and T being fixed). Let G* and h* denote the functions G and h evaluated at (a*,b*). The notation TTM, X*»g, and j**»h denotes the inner products as explained in Section 6.5. Then, recalling the proof of Theo- rem 6.5.2,

252 7 Endpoints and transversality conditions

W*-W=V*-V+G*-G

> ( [(jy* + * * . s * ) - ( / f + x * . s ) ] * + x*(0)-(g'-i) Jo

dG* dG* -7r*(r).(b*-b) + ( a * - a ) . — - + (b*-b)« 3a

v" -' db (by the concavity of G)

= [ ( f f * + i * . s * ) - ( f f + f % s ) ] * - ^ - (I*-1) Jo da - ^ 4 ^ - ( b * - b ) (by (7,116b) and (7.116d))

do

>[T[(H*+***s*)-(H+ir*»s)]dt-fi**(h*-h) Jo

(by the concavity of p**h)

> [T[(H* + ir^s*)-(H+**'s)]dt Jo

(because fi*»h* = 0 and fi**h > 0) > [T[(£*-£) + ir*.(s*-s)]dt. (7.117)

Jo The last integral is nonnegative by the concavity of <£; see the proof of Theorem 6.5,2, •

Remark. Condition (i), that the Lagrangean is concave in (s,c), can be replaced by a weaker condition. Let us define the function

//°(s, w, /) = max / / ( s , c, w, t), c

where c belongs to the set of admissible controls defined by equation (6.6). We can replace (i) by

(i') H°(s% IT*, t) is concave in s.

To see that (i') implies that (7.117) is nonnegative, note that

£* = H0(s*9**9t)**H°*9 £ < / / ° ( S , T T V ) .

Hence,

£ * - < £ > H°(S\ 7T*, t) -H°(S, 7T*, t)

> (s*-s)-(d//0*/ds) (by the concavity of H°) = - ( s * — s)»7r*

(because dH°*/ds = d£*/ds, as a result of the envelope theorem). Theorem 7.9.1 applies to the case in which both t0 and Tare fixed and

finite. Consider now the case in which T is free. Then to ensure that T*

7.11 Control parameters 253

is a maximizing choice (and not a minimizing one) we must add to the list (i), (ii), (iii) of Theorem 7.9.1 a fourth condition:

(iv) The function W= V+ G(a, b) is concave in b0 (recall that b0 = T),

Condition (iv) is rather stringent because V itself is a function of T. Take the simplest case, in which b0 does not appear in h or in G. Then (iv) is satisfied if and only if H(T) (= 3V/dT) is a decreasing function of T. This means that H(T) must be positive for all T<T* and negative for all T>T*, where T* is the optimal terminal time. It is very difficult to check that these conditions are satisfied. (The reader may consult Seier- stad, 1984, for further discussion.)

7.10 A summary table of common transversality conditions

Although Theorem 7.8.1 provides a very useful and general formula for obtaining transversality conditions for a variety of cases, for some pur- poses it may be more convenient to have a simple table summarizing these conditions for common cases.

For convenience, we restate problem (7.114) here, in a somewhat less general form. Find c(t) and, depending on the cases, t0, T, st(T), Sj(T), S/c(to)> sh(*o) f ° r some or all /, j9 k, or h, so as to maximize

W=\Tv{8(t)Mt),t)dt + 4>(s(T),T)-0(*(to)ftoh (7.118)

where <t> is the scrap value function and 0 the initial cost function. The maximization is subject to (7.115a)-(7.115c) and other conditions as stated in the table.

Table 7.1 lists six main features, (A1)-(A6). If a problem has several of these main features, all of the corresponding transversality conditions apply. Categories (B1)-(B6) generalize (A1)-(A6) to the case with a scrap value function and an initial value function.

7.11 Control parameters

In some economic problems the restrictions imposed on the actions of the central planner take on the peculiar form of requiring some controls to remain constant over the whole horizon; we label these control parame- ters. In Section 1.5 we discussed optimal peakload policies whereby the plant capacity was to be selected by the firm but had to remain fixed dur- ing the whole program; we will shortly reformulate this problem in an optimal control format with the capacity as a control parameter. Another example would be to include in an exhaustible resource problem the pur- chase of the mine itself with the quality of the ore treated as a control

Table 7.1. Common transversality conditions

st(T) free

(1) (2)

Free terminal time T (3)

sk(t0) free (4)

(A) No scrap value, ^ {T) = 0 no initial cost

(B) With scrap value0 -Kt{T) = and initial cost*

d(f> dSi(T)

TTJ(T)>0 H(T*) = 0

Trj(T)[sj(T)-sjL] = 0

d<j) dsj(T)

d<j>

> 0

dsj(T)}

x[sj(T)-sjL] = 0

d<f> H{T*)+-^=0 *k(to) =

36

dsk(t0)

a<f>(s(T),T). bd(s(t0),t0).

7.11 Control parameters 255

parameter: once chosen, it would remain constant throughout exploita- tion and influence the efficiency of extraction; it would also figure in the purchase price of the mine. We shall see that some of the transversality conditions encountered earlier in this chapter can be seen as special cases of the conditions characterizing the choice of control parameters when these parameters are the terminal value of state variables or terminal time, for instance. We shall also see that control parameters can sometimes be treated as an additional state variable, say /3(t), with the equation of mo- tion $(t) = 0 and free initial and terminal values.

We first state the main result. Consider again problem (7.1) with the addition of a vector of (constant) control parameters j8=[j3i, ...,j3/?]'« Choose c(0 and 0 to maximize

V=\T v(s(t),c{t),P,t)dt-K(p) (7.119a)

subject to

* / ( O = / W > , c ( O , 0 , O , I = 1,...,/I, (7.H9b) gJ(s(t),c(t),p,t)>0, y = l,...,m', (7.119c)

^(s(O,c(O,/M) = 0, j = m'+l,...,m, (7.119d)

si(t0)=A (!(P), Si(T)=AT(fi), I = 1,...,/I, (7.119e)

t0 = a 0(p)9 T=a

T(fi). (7.119f)

Theorem 7.11.1. In problem (7.119) the optimal choice of the vector of control parameters # implies the following necessary conditions - in addi- tion to those of Theorem 6.5.1:

da0 " 3A°i 0Pr / = i dpr

S{T)°4-i„iT)^ m ,t, >s,

" ^ • M * . , . , *, (7,20) dPr J / 0 dpr

__dK t (T d£(t)

where

£(t) = v(s(t)Mt), fi,t)+i TiW'WUV), P, t)

+ £ Xy(0* y(s(0,c(0,|8,0.

y = i

A heuristic derivation of (7.120) is most easily obtained by transforming V in the manner of Section 7.5, where for arbitrary (3 and ir(t) we have

K<0)= 0 £(0+2 */(')*/(') \dt

- S *i(T)A[(p)+i Ti(t0)AHfi)-K(P).

256 7 Endpoints and transversality conditions

Using Leibniz's rule (Appendix to Chapter 2) we can calculate dV/d(3r and set it to zero:

= £(T)-rz--£Vo)— + 3ft 3ft

- I S *iVo)Si(t0) I d/3r ;T[d£(t) , £ (d£(t)

2 *,{T)Si(T) < = i aft

; = 1

dt

T*< dK aft

• = o ,

where the first three lines come from Leibniz's rule. When we use the op- timal Tj(t) trajectories, this expression reduces to (7.120) with the usual simplifications.

Remark. We have proceeded under the implicit assumption that the choice of each ft is unrestricted. If restrictions are imposed, the usual modifi- cations are required. For instance, if we impose ft > 0, the above equal- ity must be changed to d K / 3 f t < 0 with ft>0 and ft(dK/dft) = 0. This would result in a "larger than or equal t o " sign (>) in (7.120).

Although these calculations appear complicated, they take on a much more familiar aspect in some special cases.

Special case 1. The control parameter vector does not appear as a distinct argument in any of the functions v, / ' , or gJ\ Then we have to maximize

[ v(s,c9t)dt-K(fi)

subject to

s , = / ' ( s , c , 0 , / = l , . . . , / i ,

g y ( s , c , 0 ^ 0 , y" = l , . . . , m ' ,

gJ(s,c, t) = 0, j = m ' + 1 , . . . , ra,

*,(/()) =v4?(/3), Si(T)=Aj(p), I = 1 , . . . , / I ,

t0 = a°(P), T=a T(p).

This is easily recognized as a problem similar to the one of Section 7.8, where a general formula for transversality conditions was derived. Indeed,

7.11 Control parameters 257

if we take 0 = (a, b) of (7.115) with K(p) = - G(a, b), A?(P) = ah Aj(ff) = bh a°(j3) = a0, and a

T(p) = b0, we have here problem (7.115) but without (7.115h) and (7.1151). The reader is invited to use (7.120) to obtain (7.116).

Special case 2. The control parameter does not appear in the initial and terminal conditions imposed on Sj(t0), Sj(T), t0, or T, / = 1,...,«. The problem then reduces to (7.119a)-(7.119d). We can obtain the necessary conditions (7.120) by introducing R new state variables Pr{t), A* = 1,..., /?, each with the equation of motion

$r(*) = 0 (7.121)

and boundary conditions

(3r(t0) = ft free and pr(T) free. (7.122)

Let the corresponding costate variables be n r ( 0 . The new Lagrangean is n R m

£ ( 0 = y ( s , c , j 8 , 0 + S * / / / ( s > C 0 , O + Z n r x O + S \jg J{s,c,p,t),

/ = 1 r = \ j = \

which is the same as in Theorem 7.11.1. The only new equations are (r = 1,...,/?)

flr = - ^ , (7.123)

and the transversality conditions at initial and terminal time are

n'<'o> = | £ (7.124a) OPr

because Pr(t0) = Pr is free but appears in the initial cost K(13), by conven- tion

n r ( r ) = 0 (7.124b)

because fir(T) is free. Together they imply

CT d£(t) j ^ „ t x CT d£ , dK ^ J/0 d0r h0 d(3r d(3r

which is (7.120) in this special case. (Note that we could have equally well taken the convention that &r(T) = (3r appears in K while j3r(/0) does not, but although the same result follows, this appears very artificial.)

We now offer an example of the second special case.

Example 7.11.1: peakload policy in continuous time. We follow the no- tation of Section 1.5: ^denotes plant capacity, x(t) is the supply at time /, Rt(xt) and Ct(xt) are the instantaneous revenue and cost functions,

258 7 Endpoints and transversality conditions

respectively, K(X) is the cost of capital at time 0, and 6 is the discount rate. The control parameter X is dealt with by introducing a state vari- able X(t). We must maximize

\T[Rt(x{t))-Ct(x(t))]e- btdt-K(X)

Jo

subject to

^ ( 0 = 0, ^ ( 0 ) = ^ free, X(T) free, 0<x{t)<X(t), £(t) = [Rt(x(t))^Ct(x(t))]e-

dt + U(t)xO + \(t)[X(t)-x(t)].

The necessary conditions are

^ J | = [MR/WO)-MC/W/))]r 6/-X(0<0,

dx(t)

fl(0 = -X(/), Il(0)=K'(X), 11(7) = 0,

X(t) -x(t) > 0, \(t) > 0, \(t)[X(t) -x(t)] = 0.

From the differential equation in n we obtain

II(T)-11(0) = -\T\(t)dt, Jo

or 0 = -K'(X) + [T\(t)dt,

Jo

which we recognize as a special case of (7.120). The economic interpreta- tion of the other conditions is as in Section 1.5. Note that the function K(X) might also be interpreted as the initial cost net of the discounted scrap value of X after T periods of use.

Example 7.11.2: optimal choice of the quality and size of a mine. Sup- pose that you can buy a mine with known reserves, ft say, and a known quality of ore, ft say. If the rate of extraction is x(t), this ore will provide c(t) = f(P2)x(t) units of output, which is nonstorable and yields a net benefit v(c(t)); the discount rate is 5 > 0 . Let the purchase price of the mine be p(@2) P

e r u n ^ °f reserves. After eliminating c(t) the problem is to choose ft, ft, and x(t) to maximize

\Tv(fW2)x(t))e- 8tdt-pW2)^

Jo

subject to

s(t) = -x(t)9 5(0) = ft, 5(7") = 0. (7.125)

Exercises 259

The function v is defined on positive c values only, and v and / are in- creasing functions with enough concavity to yield a concave Hamiltonian (e.g., v(c) = l n c and / s t r i c t l y concave). The Hamiltonian is

H(t) = v(fW2)x)e- dt-irx9

and the necessary conditions are

v'(fW2)x)f(P2)e- 5' = ic9 (7.126)

TT = 0 , (7.127)

-*l°)-£r = —»a-Ptf2>h' o r *(O)=P(02), (7.128)

p W f t = [ V(/(0 2 )x)/'(0 2 )jre- 6 '^. (7.129)

Jo From (7.126) v'(f(P2)x)e-

6t = Tc/f(P2), a constant, and (7.129) can be in- tegrated to yield, with (7.125),

p'W2)Pi = v f(fW2)x)rW2)e~

8t \Txdt Jo

T ;f'W2)Wi-0] Mi) PW2) fWi)

and finally we obtain

p'(P2) f'Wi)

f'(P2)Pi by (7.128), (7.130)

(7.131) P(P2) fW2) '

which defines the optimal quality of the ore, say jSJ, which we will assume is uniquely defined. Once this is known (7.126) yields

V(fWi)x)f(Pi)e-*'=p(Pi),

from which x*(t) can be calculated. Finally, I3* = ^x*(t)dt indicates the optimal size of the mine. Note that (7.131) requires that the proportional rate of gain in productivity, f'(&2)/f(&2), equal the proportional rate of increase in price, p'(&2)/p(@2).

This concludes our presentation of control parameters. For further de- tails see Long and Vousden (1977) and Hestenes (1966).

Exercises

1. (Free endpoint) Find the time path of the control variable c(t) and the terminal value of the state variable, 5(1), that maximize the integral of net benefit from

260 7 Endpoints and transversality conditions

time 0 to time 1. Maximize \l0[as(t)-{i(c(t)) 2]dt subject to s(t) = c(/), s(0) =

0, ^(l) free, where a and 0 are specified positive constants. 2. (Free endpoint with scrap value)

(a) Derive the necessary conditions for the following problem:

Vxm max \ Tl\n(c(t))dt + <t>(s(Tx\T,)

c ( / ) , 5 ( 7 j ) J 0

subject to s(t) = rs(t) — c(t)9 s(0) = s0, s(T{) free, where

<A(5(ro,ro = (r2-roin[5(r1)e- rrV(^-^i)]+^((7,2)2-(^i)2)A

and 50, 7„ and 72 are specified positive constants, TX<T2. (b) Show that 5*(r1) = 5 0 (r 2 -r 1 )e

r r V7 , 2 . (c) Show that over the interval [0,71] the solution to the problem is identical

to the solution to the following problem:

F2 = max( 2\n(c(t))dt

c{t) J o

subject to s(t) = rs(t)-c(t), s(0) = s0, ands(72) = 0. Show that K, = K2. Can you guess the exact form of

rr2 K12 = max \n(c(t))dt

c(t) J 7 i

subject to s(t) = rs(t) - c(t), s(Tx) = s*(7,), and s(T2) = 0? Verify that your guess is correct.

3. In a study of the "political business cycle," Nordhaus assumes that the political party in power seeks to maximize its popularity index V before the next elec- tion takes place, at date 7. Maximize V— \\v{u{t),p(t))e*1dt subject to s(t) = P(P(t) —s(t)), s(0) = s0, and p(t) — m — nu(t) + bs(t), where s0, [i, ]8, m, n, and b are specified positive constants and b < 1. The control is the rate of unem- ployment u(t); p(t) is the rate of inflation; p. is the fixed rate of decay of voters' memories, and the state variable s(t) represents the expected rate of inflation. We take v(u,p) = 5 — u2—Kp, where 5 and K are positive constants. Note that s(T) is free. (a) Eliminate p(t), apply the maximum principle, and solve explicitly for

u(t). Show that the optimal rate of unemployment is

u*(t) = -B/A + [Kn/2 + B/A]eA«-T\

where A = @(\-b)-fi and B = (fi-p)Kn/2. Prove that u*(t) is a mono- tone-decreasing function of time.

(b) Construct a phase diagram in the (w, s) space for each of the three cases: , 4 > 0 a n d £ < 0 M < 0 a n d £ < 0 ; . 4 < 0 a n d £ > 0 . (Hint: Show that the transversality condition implies u(T) = Kn/2.)

4. (Free endpoint, free time) Modify the problem of exercise 3 by relaxing the as- sumption that 7 is exogenously fixed. Suppose that the government can choose 7within some upper and lower bounds: TL<T<TM. What are the additional transversality conditions associated with the optimal choice of 7 (to be denoted

Exercises 261

by T*)l Show that if TL< T*< TM, the transversality conditions uniquely de- termine s(T*).

5. The following problem is sometimes called doomsday or the gold miner prob- lem. We prefer the fable of an economist marooned on a desert island with no hope of rescue, but in possession of a supply of cans of macaroni and cheese. After assuming that he has a can opener, we can address the following ques- tion: what is the rate of consumption that will maximize the economist's total utility before food runs out and the inevitable occurs? It is supposed that he must maintain at least some minimum intake c, at which his utility level is zero. Formally, he must choose c(t) and T to maximize \lu(c(t))e~btdt subject to s(t) = —c(t), c(t)> c; s0 and 6 are specified positive constants; u is strictly in- creasing and concave, and u(c) = 0. (a) Apply the maximum principle, not omitting the transversality conditions

relating to the choice of r a n d s(T). (b) Show that the consumption at time Tmust exceed c. Use the transversality

condition and the first-order condition to characterize the optimal value c*{T); depict it on a graph of u(c). Show that s*(T) = 0. On a (c, s) phase diagram can you now represent the optimal path? Does c(t) ever equal c? Why does this make sense? Is the optimal value of T finite? How would a zero discount rate (6 = 0) affect your findings?

6. Consider now a more pleasant variation on the theme of exercise 5. Suppose that you have come in possession of some amount of capital K0 at time 0. You may keep it as long as you wish, say time T9 when you must give back an amount at least as large as KT, a specified positive constant. You have no other means of support. All other assumptions are the same as in exercise 5 except that there is a return on capital; thus, K(t) = rK(t) — c(t), where r is a specified positive constant and K(T)>KT. In addition we assume rK0>c>rKT.

Specify the problem and apply the maximum principle. Show by contradic- tion that c*(T) > c; use this result to derive an equation that defines c*(T); il- lustrate this on the graph of u(c). Draw phase diagrams in the (c,K) plane, distinguishing various cases according to the relative values of 5 and r. Give a description of the optimal policies. Does c(t) always exceed c? Show that even when 8 < r the path of AT is a monotone function of time (because at any in- stant, terminal time is freely chosen).

7. You have taken control of an established private club, which has a number of faithful old members. You are considering advertising as a means of increasing revenue. This brings in a flow of new but ephemeral visitors at the time of ad- vertising. Unfortunately, it also drives away some old members, thereby dimin- ishing their numbers permanently. There is also a cost to advertising. You want to maximize the total present value of net revenue over some specified planning horizon, at which date the club must close down; the number of members at that date is of no consequence.

Let s(t) be the stock of old members at time t9 x(t) be the flow of advertising that generates ax(t) ephemeral visitors at time t. The total number of custom- ers at time t is z(t) = s(t) + ax(t). The rate of change in the stock of members is s(t) = —yx(t). The revenue function R(z(t)) is increasing and concave; the

262 7 Endpoints and transversality conditions

unit cost of advertising is c, and we assume that there exists a positive value z such that aR'(z) = c. You must choose x(t) > 0 to maximize the total present value of profits (the discount rate is 5) subject to 5(0) = 50, s(T) > 0 , and the above restrictions. 50, T, a, 7 and c are specified positive constants.

Formulate the problem (eliminate z). Apply the maximum principle and in- terpret the necessary conditions. Are they also sufficient for an optimum? Draw phase diagrams in the (</>,5) space where 4> is the cur rent-value costate - dis- tinguish between ad — 7 > 0 and ad — 7 < 0. Identify the regions where x> 0 and x = 0, respectively. Describe in words the optimal policy for various assign- ments of 7", 50, and z. What is the interpretation of zl When is it optimal never to advertise?

8. Reconsider the commercial fishing fleet problem in exercise 7 of Chapter 6. The basic model is unchanged but parts (a), (b), and (c) now become the following: (a) Suppose that you own this fleet now. You intend to retire at time T and

sell it. Thus s(T) is free but nonnegative, and you must add to the maxi- mand e~5TPTs(T), where PT>0 is the current unit price for the fleet at time T. Apply the maximum principle and interpret all conditions, in par- ticular the transversality condition.

(b) Draw a phase diagram in the (0,5) space, identifying the region where x = 0. Describe the optimal policy for various selected values of PT and S0.

(c) Repeat part (b) with p = 1, 6 = 0.1, m = 0.4, F(s) = 2(5 + 0.5)1 / 2- VI, and C(x) = (x +1)2 - 1. Find the intercepts of <j> = 0 with the axes (</> and 5, say).

CHAPTER 8

Discontinuities in the optimal controls

It was stated at the end of Chapter 4 and again in Section 6.1 that control variables are required only to be piecewise-continuous. This means that they can exhibit jump discontinuities at a finite number of dates along the horizon. These discontinuities in the control may in turn result in discontinuities for the time derivatives of the state and costate variables, but the state and costate variables are themselves piecewise-differentiate (i.e., there may exist a finite number of points where the left- and right- hand-side derivatives differ from one another). We claimed that this fea- ture greatly enlarged the variety of problems that optimal control the- ory could handle and proved our point with a simple example in Section 6.1. In electrical engineering such discontinuities result in the operation of various circuit switches; in economics this takes the form of policy switches.

In the examples studied so far we have restricted the Hamiltonian to be strictly concave in the controls and the optimal trajectories have been continuous. Difficulties arise when we deal with problems that are linear in the controls (or can be made so), at least over some ranges; in these cases there are often bounds on the control variables, either imposed exog- enously or generated endogenously. Except for a few theorems on time- optimal problems (see Pontryagin, 1962, pp. 120-4), there are no general results for dealing with these problems, so we have chosen to illustrate them with various examples. The reader should be aware that linearity in the controls and/or bounds on the control variables may generate discon- tinuities and be alert for them.

8.1 A classical bang-bang example

This example is found in Pontryagin et al. (1962) and has been reproduced by many writers; in our treatment we rely on phase diagram analysis and transversality conditions instead of the traditional algebraic solution, be- cause it is simpler and makes the optimality of the bang-bang solution clear.

The problem is to reach an equilibrium position in minimum time when controlling not the speed of movement but the acceleration only; moreover,

263

264 8 Discontinuities in the optimal controls

there are lower and upper bounds to the values the acceleration can take. The optimal solution turns out to be: set the acceleration at one of the bounds and then the other, never in between - hence the name "bang- bang." The problem is to maximize

Jo dt (8.1a)

subject to

51 = c, (8.1b)

s2 = su (8.1c)

- 1 < C < 1 , (8.Id)

5!(0),52(0) fixed, Sl(T) = s2(T) = 0. (8.1e)

If we interpret s2(t) as the distance to the origin at time t, then s{(t) is the speed and c(t) the acceleration at that time. Note that c may be negative here, so that it is possible to decelerate by putting the engine in reverse, as in a ship, and this may eventually make the ship go backward. The requirement that both s{ and s2 be zero at time T means that we must reach the origin and have a zero speed at that time. If s{ were not required to be zero, the system would still move since we do not control 5j directly. Note that the integral is — T a n d that the value of T is obviously free; note also that this is an autonomous problem in which time does not appear as an independent argument; these two observations determine the trans- versality condition below. The Hamiltonian is

H=-1 + TCIC+TC2SI. (8.2)

The necessary conditions are

Maximize H of (8.2) subject to - 1 < c < 1, (8.3a)

*! = -dH/dsx = -TT29 (8.3b)

ir2=-dH/ds2 = 0, (8.3c)

H=-l + ir1(t)c(t) + ir2(t)s1(t) = 0, W e [ 0 , T], (8.3d)

plus (8.1b) and (8.1c). We obtain condition (8.3d) by noting that the Ham- iltonian is constant over the horizon (because dH/dt = dH/dt = 0 by equa- tion (6.17) for an autonomous problem without discounting), while free terminal time implies that the Hamiltonian is nil at the end of the horizon (equation (7.116c)); hence, it is nil at all times. Condition (8.3a) can be ex- panded by taking the partial derivative of H with respect to c, dH/dc = 7TJ, and taking the bounds on c into account. If dH/dc is positive (resp.

8.1 A classical bang-bang example 265

negative), we go to the upper (resp. lower) bound; only dH/dc = 0 is com- patible with an interior solution. Formally,

dH/dc=T1>0**c=l9 (8.4a)

d / / / d c = 7 r 1 < 0 = * c = - l , (8.4b)

dH/dc=irl = 0 <= 0 < c < 1. (8.4c)

Suppose that (8.4c) prevails over some interval of time. Then 7rj = 0; thus, TTI = 0 as well, and hence 7r2 = 0 also by (8.3b). Substituting these values into (8.3d) we obtain H= — 1 = 0 over that interval, a clear contradiction. Therefore, (8.4c) never occurs and either c= 1 or c= — 1 at any one time. This indicates that discontinuities may arise. We now show that there can be at most one such discontinuity. At that time, say t*, the Hamiltonian is still zero, whether c = l o r c = — 1 . Therefore, we have both

H(t*) = - 1 + 71-! + 7 ^ = 0

and

H(t*) = -1-Tcl + Tc2sl = 0.

It follows that TTi(t*) = 0. However, we know from (8.3c) that TT2 is con- stant; hence, (8.3b) implies that 7rj is monotone. It can therefore be equal to zero only once and t* is unique.

In order to investigate further we construct a phase diagram in the (sus2) space, where we distinguish between trajectories with c = l and those with c = — 1. This is done in Figure 8.1. Begin with c= 1; then from (8.1) 5*] > 0 and s2 has the sign of su with 5j = 0 forming the s2 = 0 locus. Those trajectories are drawn as full lines. The case of c = — 1 is similar, but now Si < 0 throughout; these trajectories are drawn as dashed lines.

The boundary conditions dictate that the optimal trajectory end at the origin. There are two and only two trajectories that reach the origin, PO and NO. Therefore, with an arbitrary starting point (not on either one of these two trajectories) there must be a way to reach one of these trajec- tories. Consider, for instance, the initial point A. The only way to reach a stable branch leading to the origin is to follow the full line until TV, whence the dashed line to the origin (ANO trajectory). Therefore, in this case, as with all starting points below the PON line, which is called the switch line, it is optimal to set c = 1 until the switch line is reached, then c= - 1 until the origin is reached. The opposite applies to points above the switch line, such as B\ the optimal policy is then c = — 1 initially and c = 1 at the end (BPO). Clearly, the discontinuities in the control occur at the switch points. Each optimal trajectory has at most one switch point and some have none (when the starting point is on the switch line), as already demonstrated.

266 8 Discontinuities in the optimal controls

Figure 8.1

We now modify this example in order to illustrate the statement made earlier that discontinuities may occur when the problem is not linear in the control but can be made so. We maximize

[~l]dt Jo

subjectto Si = / i ( s 2 ) , s2 = / 2 ( c ) , - 1 < C < 1 , (8.5)

where f/>09 ft(0) = 0, / = l,2, and the boundary conditions of (8.1e) apply. This problem is clearly not linear in the control variable. The Ham- iltonian is

/ / = - l + 7r1/1(52) + 7r2/2(c).

Some of the necessary conditions are

"̂1 = 0, 7r2=— ir\f{(s2), and H=0 at all times.

If an interior solution is optimal for some time, then dH/dc=ir2f2(c) = 0, which implies -K2 = 0, which again implies -k2 = 0 and thus TCX = 0. Sub- stituting, we have H= — 1 = 0, a contradiction, and again the only optimal

8.2 The beekeeper's problem 267

choices are c = 1 or c= — 1: a bang-bang solution emerges, although the problem was not linear in the control. Because the / functions are strictly increasing and go through the origin, it would be possible to transform (8.5) into a linear problem by redefining a new control variable C = f2(c). If this feature is not detected at the outset, a clue to the existence of a dis- continuous solution is the fact that the first-order condition correspond- ing to the control variable (such as -K2fi(c)) cannot be set to zero by choosing the control. Then changes in other variables, state or costate, will dictate whether the control should be set at the upper or lower bound. We now turn to a more complex example.

8.2 The beekeeper's problem

In this section we consider a bee population as a renewable resource that is exploited for its honey production. The apicultural process is perforce simplified, and the model can be interpreted in other ways. A bee popula- tion s(t) produces a flow of honey f(s(t)); q(t) of this honey is harvested and x(t) of it is left for the bees and their young to feed on. The bee pop- ulation grows naturally at an exponential rate n, but growth can be in- creased or decreased depending on whether x(t) exceeds or is below some fixed ration x. The beekeeper's aim is to obtain the largest possible rev- enue from the sale of honey during the fixed season [0, T]. The price of honey is unity, and there is a positive discount rate 5 that reflects market forces. We assume that the bee population must be returned to its initial value at the end of the season; any other fixed level would result in sim- ilar policies. Finally, both x(t) and q(t) are required to be nonnegative; since x cannot be negative, q cannot exceed f(s) (because q+x = f(s))9 which means that one cannot gather more honey than is produced at any one time. The nonnegativity of q means that x cannot exceed f(s); thus, one cannot feed the bees more honey than they produce at any time. This implies that once honey has been harvested it cannot be fed back to the bees, nor can commercial honey be bought from outside to feed them. We now state the problem formally as finding x(t) and q(t) that maximize

[Tq(t)e-*'dt (8.6a) Jo

subject to

s(t) = ns(t)+x(t) -x, (8.6b)

x(t) + q(t) = f(s(t))9 (8.6c)

x(t)>0, q(t)>0, (8.6d) s(0) = s(T) = s0. (8.6e)

268 8 Discontinuities in the optimal controls

We assume / ' > 0, / " < 0; some other restrictions will be^plaeed on the slope of / in order to distinguish several outcomes, but they are not es- sential at this stage. Some restrictions will also be placed on s0 to make the problem feasible, but these will arise naturally in the course of the analysis. Eliminating q from the problem and skipping all time argu- ments, we maximize

\T[f(s)-x]e-btdt (8.7a) Jo

subject to

s = ns+x-x, (8.7b)

0 < * < / ( * ) , (8.7c)

s(0) = s(T) = s0. (8.7d) The upper bound on x reflects the nonnegativity of q and (8.6c). The Hamiltonian and the Lagrangean are, respectively,

H=[f(s)-x]e~8t+Tr[ns+x-x], (8.8)

£ = [f(s)-x]e- 8t-hir[ns^x-x]^\[f(s)-x]. (8.9)

Applying the maximum principle we obtain (8.7b) and

7 r = - ( e - 6 / + X)/'(5)-A27r, (8.10)

Maximize H of (8.8) subject to (8.7c). (8.11)

Consider the possibility of an interior solution in (8.11). If 0 < x < f(s), then X = 0 since the upper bound is slack, and the first-order condition is dH/dx= -e-bt+ir = 0. Differentiating this, we get ic = -be~bt and (8.10) becomes

-be-st=_e-btf,(s)_ne-bt^

or

f\s) = b-n. (8.12)

Assuming for the time being that 6 < n, it follows that (8.12) is never satis- fied; hence, an interior solution never occurs, indicating a possible dis- continuity. The two remaining possibilities are

dH/dx=Tr-e-bt>0=*x = f(s), (8.13)

dH/dx = ir-e-dt<0^x = 0.

In the first eventuality x = f'[s] and x has the sign of s, while in the sec- ond one x = 0. We now proceed to construct a phase diagram in the (x, s)

8.2 The beekeeper's problem 269

x = f(s)

Figure 8.2

space for further analysis; this is done in Figure 8.2. The locus of s = 0 is obtained from (8.7b). It is a straight line intersecting the axes at x and x/n. We also plot the graph of x = f(s). There are only two possibilities: either the trajectory is on the graph of x = f(s) or it is on the s axis. Above the 5 = 0 line, s increases (and so does x when following the x = f(s) graph); under that line, s decreases (and so does x unless it keeps to zero values). The area above the f(s) graph is ruled out by (8.7c), and anywhere between the graph and the s axis is ruled out by our previous argument. (Thus, the presumed 5 = 0 line turns out to be a misnomer.) The equilibrium point is at E. We now proceed to restrict the values s0 may take to ensure the existence of a solution with the specific boundary conditions (8.7d). If s0 > x/n, s will always increase and we cannot satisfy (8.7d). Define the value s by ns+f(s) = x that marks the intersection of the 5 = 0 line with x = f(s). Again if s0<s we cannot satisfy (8.7d) as s always decreases. Hereafter, we assume s<s0<x/n. There are two dis- tinct possibilities. The optimal policy for the indicated 50 value could be to first set x = 0 and after some time to jump up onto x = f(s) and return to the s0 line (type II). Alternatively, we could begin with x = f(s) and, after a jump down to x = 0, proceed to the s0 line (type I). Such jumps are sketched in Figure 8.2. In order to resolve this dilemma we must turn to another phase diagram, in the (i/s s) space, where \l/ = e8tir is the current-

270 8 Discontinuities in the optimal controls

Figure 8.3

value costate (Figure 8.3). Following the technique of Section 4.4 we ob- tain a differential equation for \p by differentiating \p and using (8.10),

^ = ( 5 - / ! W - ( l + M ) / m (8.14) where fi = e8t\ is the current value of the multiplier. Equation (8.13) becomes

yj/>l=>x = f(s) and ^ < l = > x = 0. (8.15)

This takes the place of the first-order condition when discontinuities occur, and it seems difficult to use it to replace the x term by a ^ term in the s equation (8.7b). However, careful inspection reveals this to be unneces- sary. In the relevant region (between s = s and s = x/n) it is always true that x = f(s) implies s > 0 and x = 0 implies s < 0. Therefore, using (8.15) we have

yp > 1 => s > 0 and xp < 1 =* s < 0.

8.2 The beekeeper's problem 271

Note that when yj/>l, x = f(s) 3inds = ns+f(s)—x>09 and this changes abruptly to s = ns—x < 0 when xp < 1. There is a discontinuity in 5 due to the one in x, and the line \p = 1 is not really an s = 0 locus. We call it "5 sign" and draw the trajectories with a kink when they cross it since their slope is dxp/ds = \p/s and exhibits a discontinuity when 5 does.

We now turn to xp = 0; this yields 0 = - (1 + ii)f'(s)/(n - b); the denom- inator is nonnegative by the assumption S</? (to be relaxed later). If b = n, then ^ < 0 by (8.14). If b < n, the denominator is positive, so that xp as given by this equation is negatively valued; hence, \p < 1 and ix = 0. The \p = 0 locus has become \p = —f'(s)/(n — d). Given our assumptions on / ( / ' > 0, / " < 0), this has the shape depicted in Figure 8.3. From (8.14) \p is positive only when xp is negative enough, that is, below the xp = 0 locus, and xp is negative above the locus. This and the information gathered on s enable us to draw the diagram, at least between 5 and x/n. This is suffi- cient to resolve the dilemma we had. Clearly, any trajectory starting below xp = 1, whether at xp positive or xp negative, entails 5 decreasing through- out; this conflicts with (8.7d). Therefore, we must start above xp = l, and this means x = f(s) at first, followed by a jump to x = 0. In Figure 8.2 policies of type I are optimal. Note that no matter how large T is, it is possible to find a trajectory of type I that lasts long enough, because after the downward jump the path can be arbitrarily close to the equi- librium point E and its motion very slow.

We now relax our assumption 6 < n and examine the case where b > n. Equation (8.12) is no longer inconsistent but defines some value s* by f'(s*) = 5 — n, where the slope of / is equal to 5 — n. Thus, an interior solution may occur, but during the time that it prevails s is fixed at s* and 5 = 0; hence, by (8.7b) x is also fixed at x* = x—ns*. The point (x*, s*) is, of course, on the presumed 5 = 0 line in an (x,s) phase diagram. The possibility of an interior solution thus does not rule out discontinuities, since jumps are required in order to pass from x = 0 to x = x* to x = f(s). To discover the nature of the optimal solution requires once again the use of the (i/s5) diagram, which we have constructed in Figure 8.4. (In that figure we have assumed s<s*< x/n; if 5* falls outside the relevant area, we obtain somewhat different outcomes, which will be briefly examined shortly.) From equation (8.14) the ^ = 0 locus is given by the expression \p = (\ + IL)f'(s)/(b — n)>0 under our current assumption. While ^ < 1 , li = 0 and this expression simplifies to y}/ = f'(s)/(b — n). When ^ > 1, we have x > 0; hence, d£/dx = 0 and \p = 1 + /*. This and (8.14) imply that the \j/ = 0 locus above yp = 1 is the vertical line 5 = 5*. In the relevant area, yp = 1 is the 5-sign line that separates s > 0 from 5 < 0, and we have a new "equilibrium" at / with two stable arms and two unstable arms. This equi- librium is somewhat peculiar. It is true that yp approaches 0 smoothly but

272 8 Discontinuities in the optimal controls

^ = 0

(* = f (s)/(6 - n))

Figure 8.4

s does not: as we jump from x — f(s*) or x = 0 to x = x*, there is a dis- continuity in 5. The arms (stable and unstable) have been drawn with a horizontal slope at / since \j/ - hence the slope dx/z/ds = \p/s - tends to zero around / . This is an important point, because equilibrium / can then be reached in finite time; we shall need this observation later. While at / itself, s = s*9 x = x*, and of course 5 = 0. We use these arms to define regions (A) through (D) in the relevant area of the plane, as in Figure 8.4. Any trajectory that begins in region (B) or (D) entails a monotone motion for s; hence, it is ruled out. Trajectories in region (A) have the popula- tion s increasing at first and then decreasing and are of type I, as in Fig- ure 8.2. However, trajectories in region (C) are of type II, and if s0 is larger than s*9 type II policies are expected to emerge. The economic in- tuition behind this result is that when the discount rate was low, it paid to let the bee population increase at first so that full harvesting would take place on the larger bee population. However, if the discount rate is large enough, the preference for earlier harvesting reverses the argument.

We now illustrate the consequences of this analysis in the (x,s) space in Figure 8.5 for several eventualities. With an initial value such as s01, trajectories will be of type I, moving from Gx to Hu down to Jx with a

8.2 The beekeeper's problem 273

Figure 8.5

jump and on to Lx. With an initial value such as s02, trajectories will be of type II: L 2 G 2 / 2 ^ 2 - Other possibilities also emerge because of the pe- culiarities of equilibrium point / discussed earlier. Since s does not tend to zero when approaching / , it is possible to reach this equilibrium in finite time, and this gives rise to new types of policies when s0 is close enough to s* and T is long enough. Suppose the initial value is s0 3 and suppose that we ride the stable arm separating regions (A) and (B) in Fig- ure 8.4; there is still much time left when / is reached, so that leaving / immediately along the unstable arm between regions (A) and (D) would fjorce us to reach s03 too soon. Then there must be a rest period at / itself ip between travel along thd arms. This is a new type (type III) of policy \yhich is depicted as G3H3\IJ3L3 in Figure 8.5. This involves two jumps from H3 to / and from / to \J3 with a rest period at / in between. A type IV policy with a long horizon and s04 just above s* could also be described; We leave this to the readerj. Let us now give the economic intuition be- Hind these new types of policies. At point / , we have f'(s*) = b — n and s = ns*+x*—x = (ty. The first expression is the golden rule for this prob- lem: the marginal physical product of the resource plus its natural rate of growth is matched1 with the rate of discount, and this could be maintained forever with x = x*. Thus, when initial (and terminal) stock is close enough to the golden rule level and there is enough time, it becomes optimal to reach that point and remain there as long as possible. Finally, note that

274 8 Discontinuities in the optimal controls

we have not exhausted all eventualities in this problem. For instance, 5 could be close enough to n so that s* may be well above the relevant re- gion (or not exist at all) and all policies would be of type I. Conversely, if b is very large, s* could be below s and only type II policies would pre- vail. Finally, freeing the terminal bee population from the constraint of being equal to the initial one would further broaden the range of optimal policies encountered here. The bulk of the analysis remains unchanged; the drawing of conclusions is left to the reader as an exercise. To facili- tate this task the directions of trajectories outside the relevant [s,x/n] interval have been drawn in all the diagrams.

In this section we used a simple renewable resource problem to gen- erate a rich array of discontinuous policies for the analysis of which we needed to study the interaction between the (state, control) and the (state, costate) phase diagrams. Next we shall analyze a problem with two con- trols, two state variables, and two constraints.

8.3 One-sector optimal growth with reserves

This model is similar to the ones analyzed in Sections 4.4 and 6.4, but it has some additional features. We first state the problem and then proceed to describe it. Find c and x that maximize

[Tu(c)e-btdt + e-bTpTsT (8.16) Jo

subject to

s = F(s)-ms + x-c, (8.17)

X=-x, (8.18)

0 < x < l ; (8.19)

/?r,r,5 , 0,A

r 0exogenously specified; srfree; XT>0. (8.20)

We assume u' > 0, u" < 0, and w'(0) = oo, and the same for F. The nota- tion is as in Section 6.4, but now we have a stock X of the good on which we can draw; there are bounds of 0 and 1 on the flow of good from this source. Note that we are disregarding potential constraints on the use of capital stock for consumption (c<F(s)+x) or on the use of reserves to augment capital stock ( c > x ) . Although these are interesting features, they would bring the number of inequality constraints to four and com- plicate the analysis so as to redirect our main focus away from the discon- tinuities. As another simplification we have not included a scrap value for XT. This would only lengthen the analysis without altering its substance. A rationale for this assumption is that the reserves are firm-specific and of no value to others outside it. The Hamiltonian is

8.3 One-sector optimal growth with reserves 275

H=u(c)e-8t+Tr[F(s)-ms+x-c]-\x. (8.21)

The necessary conditions are (8.17)-(8.20) plus

W ' ( c ) e - 6 ' = 7 r ; (8.22)

dH/dx=ir-\>0 = > x = l ,

dH/dx=>ir-\<0=>x = 0, (8.23)

dH/dx=ir-\ = 0*= 0 < x < l ;

ic = Tr(m-Ff(s))9 TrT = pTe- 8T, (8.24)

X = 0, XT>0, X > 0 , \XT = 0. (8.25)

First suppose that \ = 0; then XT>0. By (8.22) we know that T T > 0 ; hence, -K > X at all times and (8.23) implies x = 1 for the whole interval. This will happen if X0 > T. Hereafter, we analyze the case X0 < T. Note also that (8.24) and (8.22) imply u'(cT)=pT, which fixes the optimal value of cT.

Suppose now that 0 < x < 1 for some time; then by (8.23) n = X and ir is constant; (8.24) implies ir(m—F'(s)) = 0, and we distinguish two cases:

(a) In the case where F'(s) > m for all s, the preceding equation im- plies 7r = 0, which contradicts (8.22). In this case interior solutions for x are ruled out and x=l and x = 0 are the only possibilities. Since also 7r > 0 implies -k < 0 in this case, we must have x = 1 at first and x = 0 later. The optimal policy is clear: set x = 1 until date X0, where the reserves are exhausted; thereafter, set x = 0. Differentiating (8.22) and using (8.24) yield

d=^-[F'(s)-m-8]. (8.26) u

Therefore, the consumption path may or may not be monotone. This case will be illustrated after the analysis of case (b).

(b) We now turn to the case where there exists some value s* such that m = F'(5*). Then 0 < x < 1 is compatible with s = s*; of course, s = 0 dur- ing that time interval. Therefore, the optimal controls in this interval are given by c*(t) and x*(t) defined by

w ' ( c * ( 0 ) e " 6 ' = X (from (8.22) and TT = X) (8.27) and

0 = F(s*) - ms* + x*(t) -c*(t) (from (8.17)). (8.28)

Outside this interval equation (8.26) applies with x set at either 0 or 1. We can now construct the phase diagram in Figure 8.6. Equation (8.26)

defines the c = 0 locus as s = s, where F'(s) = m + d; hence, s<s*. Two

276 8 Discontinuities in the optimal controls

Figure 8.6

5 = 0 loci are drawn, one for x = 0 and the other for x = 1, and so labeled, using (8.17). This equation and (8.26) enable us to find the directions of trajectories in the usual manner. The only complication occurs in the re- gion between the two s = 0 loci: we must now label trajectories as x = 0 or x= 1 (or 0 and 1) since they go in different directions. Note that at any point of intersection of two trajectories of a different type, the s compo- nent in the x = 1 path is higher than in the x = 0 path; therefore, its slope in the (c,s) plane is algebraically larger. There are two saddle-point equi- libria Ex and E0, neither of which is attainable in finite time. We now turn to the interior solution. This can happen only when s = s*; hence, from (8.28), x* = c*9 which is negative by (8.27). Furthermore, this trajectory is feasible only between the s = 0 loci since any other point on the s = s* line would violate (8.19). For instance, at a point above H, we would have

8.4 Highest consumption path 277

c = F(s*)-ms* + l + a, ( a > 0 ) say, and (8.28) would yield x * = l + a, a contradiction. Similar reasoning rules out sections under point L.

There are many possible policies, depending on the values of T, X0, s0, and so on, and it would take too much space to categorize them all. We have instead selected a sample and invite readers to augment it so as to increase their understanding of the problem. Some of these policies keep well away from s = s* and will serve as illustrations of case (a). Consider the initial value s0l and a scrap value pT such that terminal consumption is at cT{. One possible optimal path is ABC, although the switch from x- 1 to x = 0 may occur earlier (smaller X0) or later (larger X0). A dif- ferent initial capital stock such as 50 2 may yield the path DFG with the switch at F. Consider now an initial capital s01 with terminal consumption cT2; then the optimal path could be JIK, which exhibits several changes in the signs of c and s. All of these examples could apply to case (a) since s* plays no role. Consider now an initial capital of s0 3

a n d terminal con- sumption cTl, the optimal path could be NMPQ with MP section corre- sponding to an interior solution. It is advantageous to follow this path rather than pursue the NM trajectory further to meet with an x = 0 path to the right of s*, because it is wasteful to go past s* (the net production function F(s) — ms is decreasing there); if it is possible to keep -w constant until a feasible x = 0 path can be followed to the left of s*, this will be done. Therefore, the policy, which applies only to case (b), is: set x = 1 at first, then stay at s*, adjusting x to keep 5 = 0, and finally move off to x = 0. This policy involves two jump discontinuities because in the interior of the segment HL, x is strictly between 0 and 1 and over the whole horizon x takes on the values 1, [a, j8], and 0 with 1 > a > (3 > 0. This policy could be called a bang-slide-bang solution; it shows that inte- rior solutions can coexist with discontinuities. It is also worth noting that the special features of this model have enabled us to describe the solu- tion of a two-state, two-control problem in a two-dimensional phase dia- gram. Thus, this is a hint that sometimes adding to a model rigidities that yield bang-bang types of solutions may be a first step toward analyzing otherwise intractable problems.

8.4 Highest consumption path

This is a special case of Example 6.4.1, with a linear utility function. Be- cause of the linearity of the utility function, a golden rule equilibrium attainable in finite time appears. This is in contrast with the results of Section 6.4, in which the equilibrium could be reached only in infinite time. Therefore, our analysis here will highlight the fact that the equi- librium of Section 6.4 was not reached in finite time, not because this

subject

[Tce-btdt Jo

to

s = F(s) — ms — c,

0 < c < F ( s ) ;

T, s0, sT exogenously specified.

278 8 Discontinuities in the optimal controls

would have been unfeasible, but because it was suboptimal to do so. We now state the problem; the notation is identical to that of Section 6.4. Find c that maximizes

(8.29)

(8.30)

(8.31)

(8.32)

We assume F ( 0 ) = 0, F'> 0, F" < 0, and F ' ( 0 ) > 5 + m > F'(oo). The Hamiltonian is

H=ce-8t + <jr[F(s)-ms-c], (8.33)

and the Lagrangean is

£ = H+n[F(s)-c]. (8.34)

The necessary conditions are (8.30)-(8.32) plus

dH/dc = e-8t-ir<0 => c = 0,

dH/dc = e-8t-ir>0 => c = F(s), (8.35)

dH/dc = e-dt-Tr = 0<= 0<c<F(s)

(the last two cases of (8.35) can also be stated as e~bt = ir + fi), as well as

7r = m7r-(7r + ^ ) F , ( 5 ) , (8.36)

H>0, F(s)-c>0, n[F(s)-c] = 0. (8.37)

We shall also use the current-value costate variable ^ — eht-K when con- venient.

First suppose that 0 < c < F(s) over some time interval. Then by (8.35) and (8.37) TT = e~bt (or x// = 1) and \K = 0. Substituting these in (8.36) yields *=-5e-6t = e-8t[m-F'(s)]9 or

F,(5*) = m + 6, (8.38)

which defines the value 5*. Therefore, over that time interval, s = s* and 5 = 0, which with (8.30) gives

c* = F(s*)-ms*. (8.39)

Equations (8.38) and (8.39) define the golden rule equilibrium, which is the only point compatible with an interior solution for (8.35). Also recall that ^ * = 1 .

8.4 Highest consumption path 279

c = F(s)

Figure 8.7

We now turn to boundary solutions. Suppose that c = 0; then by (8.35)- (8.37) we have ^ = 0, 7 r > e " 6 ' > 0 (or ^ > 1 ) , TT = ir[m-F'(s)]9 or

\P = \Is[5 + m-F'(s)] (8.40) and

s = F(s)-ms. (8.41)

We take this last expression to be positive since the possibility that, at some high level of capitalization, depreciation overtakes productivity in absolute terms does not seem very sensible - it has little effect on our analysis in any case. Finally, consider the case in which c = F(s). Then by (8.35)-(8.37) we obtain 7 r < e " 6 / ( o r i £ < l ) ; indeed, Tc + n = e~6t9 -k = m-K-e-btF'{s),ox

xP = (d + m)xls-F'(s) (8.42) and

s = -ms<0. (8.43)

We are now ready to construct the phase diagrams that will enable us to select optimal policies. The (c, s) diagram is represented in Figure 8.7. We

280 8 Discontinuities in the optimal controls

i / / = 0

Figure 8.8

have the upper bound c = F(s) and the lower bound c = 0. The 5 = 0 locus, c = F(s) — ms, is also represented, although only the equilibrium point E on it is ever used. As indicated, s > 0 along the s axis and s < 0 along c = F(s). We expect trajectories going past 5 = 5* to jump (up or down) to the golden rule equilibrium, but we need the (\p,s) diagram to con- firm our guess. This is constructed in Figure 8.8. There are two cases to consider: above \J/ = 1, where by (8.41) s increases, and below ^ = 1, where by (8.43) s decreases. Thus, the \{/ = 1 line delineates positive and negative values of 5, although it is not an s = 0 locus because of the dis- continuity in c, hence in 5. Above ^ = 1, equation (8.40) gives the \l/ = 0 locus at 5 = 5*. Below ^ = 1, the yp = 0 locus is given by (8.42), which yields \l/ = F'(s)/(5 + m). Recalling that F' is a positively valued and decreasing function, we have the ^ = 0 locus as represented. The sign of \p can easily be everywhere ascertained, and this is indicated in Figure 8.8 by the usual arrows. Note that trajectories reaching or leaving E have a horizontal slope at E because dyf/fds = 1//5 = 0, since \j/ = 0 and 5 5* 0 wherever ^ 5* 1. (This is similar to the beekeeper's problem.) This explains why E can be

8.5 Concluding comments 281

reached in finite time. We now examine a few optimal trajectories with various endpoint conditions:

(i) Let s0 = Sj and sT = s2 with Si < s* < s2; then with T large enough the optimal policy is to set c = 0 until s* is reached, jump to E, stay there as long as possible, and jump down to c = 0 just in time to reach s2 at time T. Clearly, if T is too small, the problem is unfeasible, whereas if T is very large, most of the time is spent at the equilibrium. This policy is reminiscent of "turnpike" results. In Figure 8.8 the optimal trajectory is (i). Therefore, the equi- librium is approached by the stable arm and abandoned by fol- lowing the unstable arm in the appropriate region. Whenever the optimal path involves E, only the stable or unstable arms are used; no other trajectory is involved.

(ii) When the boundary conditions are reversed, that is, s0 = s2 and sT = Si, trajectory (ii) in Figure 8.8 is used. In the (c,s) plane this involves setting c = F(s) until s* is reached, staying at s*, and jumping again to c = F(s) for the last stretch.

(iii) Let s0 = sT = Si. Then the optimal policy could be (hi) in Figure 8.8 if time is short. This involves setting c = 0 and then switching to c = F(s). If, however, T is very large, the optimal policy could be AEB in Figure 8.8 with some time spent at E. This would in- volve two jump discontinuities.

8.5 Concluding comments

In this chapter we have illustrated a class of problems that can be handled only by the methods of optimal control theory. Conditions under which discontinuities arose were bounded controls and the inability to vary the control variable so as to keep the first-order condition balanced at zero. In many cases the Hamiltonian was a linear function of some of the con- trol variables, but we saw that this was not necessary. A variety of opti- mal policies have emerged with controls set to one of their bounds for some time and then, after a jump discontinuity, moving smoothly along an interior path, or resting temporarily at an equilibrium, or even mov- ing to another bound. The presence of discontinuities has in particular led to the appearance of equilibria that can be reached in finite time and are indeed part of the optimal policy. Thus, discontinuities should not be looked upon as mere oddities that can always be smoothed away (they cannot always be, as in Sections 8.1 and 8.3) but should be seen as con- tributing richness to the behavior exhibited in control problems. Finally, as in Section 8.3 they sometimes allow the analysis of otherwise intractable

282 8 Discontinuities in the optimal controls

problems. We have also used the (state, costate) and the (state, control) phase diagrams jointly in order to elicit the precise optimal path.

Exercises

1. Consider a very simple problem in which the aim is to maximize the present value of the sales from a finite stock of a resource. The flow of sales is bounded, and a scrap value exists for any stock remaining at the end of the horizon. Formally, choose a piecewise-continuous control c(t) to maximize

V=[Tc(t)e-dtdt + e-8Ts(T) Jo

subject to s(t) = ~c(t), 0<c(t)<h s(T)>0, ands(0) = s0, where 6, T, and s0 are specified positive constants. Apply the maximum principle to this prob- lem; pay particular attention to the conditions characterizing the choice of the control that maximizes the Hamiltonian subject to the constraints on c and also to the transversality conditions. Prove by contradiction that an interior solution (i.e., 0 < c ( O < l ) cannot persist for any finite time interval. Show that when T>s0 the solution is bang-bang, and when T<s09 the control is continuous. In each case derive the complete solution, including the value of the switch point when appropriate, and ensure that the transversality condi- tion is satisfied. Give a verbal account of the optimal solution.

2. Repeat exercise 1 when the constraint 0 < c(t) < 1 is replaced by - 1 < c(t) < 1. In other words, buying as well as selling is now permissible. Again show that an interior solution is never optimal and that the control is bang-bang when T>s0 but continuous when T<s0. Give a verbal account of your results.

3. Here we examine the optimal manner of consuming a resource when the cur- rent stock of the resource is an argument of the utility function. This is per- haps because mere ownership pleases the individual or perhaps because the quality of the resource declines with the level of stocks. Find c(t) to maximize

f7c(t)s(t)e~8tdt Jo

subject to s(t) = — c(t), 0 < c(t) < 1, s(T) > 0; s0 is a specified positive con- stant, as is 8. Show that it is never optimal to have 0 < c(t) < 1 for any length of time. Draw the (state, costate) phase diagram; divide the space in two re- gions: one where c = 0 and one where c = l . Describe the optimal solution; show that c is continuous when s0 > T but discontinuous when s0 < T. Show that it is never optimal to enter the interior of the region where c = 0.

4. Reconsider the problem of exercise 3 modified by the introduction of a scrap value: the maximand is now

[T c(t)s(t)e~8t dt + e~dTps(T)9 Jo

where p is a positive constant. Show that the results are similar to those of exer- cise 3 except that it now may be optimal to reach the interior of the c = 0 region.

Exercises 283

Give a verbal account of the optimal policy and explain why it might be opti- mal to consume no resource for some time while a positive stock is available. Modify the problem of exercise 4 by introducing depreciation. The state equa- tion is now s(t) = —c(t) — ms(t), where AW is a specified positive constant. How are the results of exercise 4 modified, if at all? In this exercise there is a most preferred value of the state variable with a penalty applying when the system is not at it. The rate of change in the state variable is the control and it is bounded. Find c(t) to maximize

\2-(s(t)-\)2dt Jo

subject to s(t) = c(t), — 1 < c(t) < 1, and s(t) = 50, where s0 is a specified con- stant and 5(2) is free. Show that — 1 < c(t) < 1 never occurs unless s(t) = 1. Draw a phase diagram in the (state, costate) space. Derive the optimal solu- tion when s0 = 0 and when s0 = 2. What is the sign of the costate variable in these two cases? Explain your result. If a discount factor e~ht is applied, how are the results altered? In this exercise we examine the consumption of a growing resource that might suffer from overcrowding. We seek to choose c(t) to maximize

V c(t)dt Jo

subject to s(t) =s(t)(100-s(t))-c(/), 0 < c(t) < 3,000, 5(0) = 50, and s(T) = 5 r , where s0 and sT are specified positive constants. Show that an interior solution (0 < c< 3,000) is compatible with only one pair of values for c and 5. Draw the phase diagrams in the (c,s) space and the (ir,s) space, where 7r is the costate variable. Identify a steady state with positive consumption that can be reached in finite time and where it is optimal to stay as long as possible provided that T is large enough. Choose a few values for T, 50, and sT and illustrate the various types of optimal paths. Modify the problem of exercise 7 by imposing a smaller upper bound on the consumption for the resource, namely, 0 < c(t) < 900. All other data remain unchanged. Show that an interior solution 0 < c ( / ) < 9 0 0 is never optimal now. Draw the phase diagrams. How many steady states with a positive con- sumption are there now? Are they attainable in finite time? Choose some s0 and sT values so that the optimal control has two jump discontinuities. De- scribe the optimal path when s0 = 70, sT = 95, and T is relatively large. Reconsider the model of Section 8.3. Construct a phase diagram in the (<p,s) space in which <p(t) = -K(t)ebt and match the trajectories with those of Figure 8.6. Show that shifts from x = 1 to x = 0 can occur only when s(t) < s* and shifts from x = 0 to x = 1 can occur only when s(t) > s*. Discuss why this is so. Reconsider the model of Section 8.4. Derive the exact solution when 6 = 0.1, F(s) = 2yfs, 50 = 64, sT = 144, T= 12, and m = 0. Identify the intervals over which c> 0 and those over which c = 0. Suppose now that 5 = 0.06, m = 0.04, F(5) = 2vT, T=75, 50=100e, and 5 r =100e

- 1 5 , where e is the exponential number. What is the optimal solution now?

C H A P T E R 9

Infinite-horizon problems

In economics it is often convenient to postulate that the time horizon is infinite. One of the reasons for adopting this assumption is that one avoids the problem of specifying the end-of-horizon stocks or a scrap value function. Also, this formulation quite often leads to simplified for- mulas and appealing results; for example, the idea of a long-run equi- librium can be given a precise treatment. To critics who point out that the world is predicted to end in finite time, one may offer the following de- fense: provided that the optimal path for an infinite-horizon problem does not differ significantly from the solution of a control problem with a very large but finite horizon, the convenience of working with an infinite- horizon model is worth the loss of "realism."

There are, however, certain technical difficulties associated with opti- mal control problems having an infinite time horizon. In particular, the finite-horizon trans versality conditions do not carry over to the infinite- horizon case. Furthermore, it is possible that the integral does not con- verge for all feasible paths. If this case arises, how would one rank alter- native paths? These, and other issues, will be examined in this chapter.

9.1 Optimality criteria

Consider the problem of attempting to find the control vector c(t) that maximizes the integral

V=\°°v(s(t),c(t),t)dt (9.1) Jo

subject to

*/ = / ' " ( s ( 0 , c t f U ) , / = l , 2 , . . . , / i , (9.1a)

gj(s(t),c(t),t)>0, y = l , 2 , . . . , m ' , (9.1b)

gh(*(t)Mt),t) = 0, h = m'+l,...,m, (9.1b')

Si(0) = si0, i = l , 2 , . . . , / i . (9.1c)

We assume that there are no restrictions on the limiting behavior of the state variables, but alternative specifications are possible; see equation

285

286 9 Infinite-horizon problems

(9.4) in the next section. The constraints (9.1b) and (9.1b') are assumed to satisfy the rank condition stated in Chapter 6.

A solution (s(t),c(t)) of (9.1a)-(9.1c) is called a feasible path. If for all feasible paths (s(t), c(t)) the integral (9.1) converges, then the optimal path is clearly one that yields the highest value for the integral. (Note that convergence is ensured if v(s, c, t) takes the form v(s, c, t) = e~8tu(s, c, t)9 where «(s, c, t) is bounded and 8 > 0.)

Suppose the integral (9.1) does not converge for all feasible paths. How would one compare any two feasible paths (s*(0, c*(/)) and (s(/), c(/))? In order to answer this question, let us consider, for any finite t, the dif- ference of the cumulative performances up to that time:

Z(t) = (' i;(s*, c*, r) dr- [' v(s, c, r) dr. (9.2) Jo Jo

(We have skipped the time arguments for simplicity.) Clearly, if there exists some T such that for all t > T, z(t) > 0, then (s*, c*) is better (or no worse) than (s,c). This leads to the following definition.

Definition 9.1.1: overtaking criterion. A path (s*,c*) is said to be "no worse" than path (s, c) under the overtaking criterion if the difference in cumulative performances z(t) is nonnegative for all t sufficiently large. A path is optimal under the overtaking criterion if under that criterion it is no worse than any other feasible path (s, c).

The overtaking criterion, while intuitively appealing, is in practice very difficult to apply, as we shall see in Section 9.3. Furthermore, it fails to rank any pair of paths whose difference z{t) changes sign periodically. Consider the following example, in which z(t) takes the form z(t) = (sin t)/t. In this example, the difference in cumulative performances oscil- lates around zero but vanishes (approaches zero) as t becomes very large. This suggests another definition of "no worse" in the sense that

l i m z ( O ^ 0 . t-+ao

However, such a definition would still be too restrictive, because in gen- eral z(t) may not approach a limit, as the following example illustrates:

z(t) = sint ( > 0 ) for 2mll<t<(2m + l)ll

= (sint)/t (<0)for (2m + l)II<t<(2m + 2)II (9.3)

for all m = 0,1,2, In this more complicated example, z(t) = 1 periodi- cally, but for all / i n [(2ra + l)n,(2ra + 2)II], ra = 0,l,2,...,z(/) is non- positive; thus, a limit does not exist. However, in a sense (s*, c*) is no

9.2 Necessary conditions 287

worse (or indeed strictly better) than (s,c), because in intervals of time when z(t) is negative, its value is bounded below, and the lower bound approaches zero as t tends to infinity. To formalize this idea, let inf z(t) denote the greatest lower bound of the graph of z(t')9 for all t'> t. By definition inf z(t) is a nondecreasing function of t, so that while z(t) may oscillate, inf z(t) does not, and hence the limit

lim[infz(f)l

exists (it may, of course, be infinite). We can now define optimality ac- cording to the catching-up criterion.

Definition 9.1.2: catching-up criterion. A path (s*9 c*) is said to be "no worse" than path (5, c) under the catching-up criterion if

lim[infz(O]>0. / - + 0 0

A path is optimal according to the catching-up criterion if under that cri- terion it is no worse than any other feasible path.

One advantage of the catching-up criterion is that sufficient conditions are easily verified, as we shall see in Section 9.3.

A moment's reflection will convince the reader that if a path is optimal under the overtaking criterion, then it is also optimal under the catching- up criterion, and that both criteria are equivalent to the maximization of the integral (9.1) if it converges for all feasible paths. The choice of optimality criteria depends on personal tastes. In practice, it is recom- mended that one proceed under the assumption that the integral con- verges. If convergence does not occur, one may look for paths that satisfy the overtaking criterion. If none exists, one can fall back on the catching- up criterion. One can even set up weaker criteria (see, e.g., Seierstad and Sydsaeter, 1977).

9.2 Necessary conditions

It is clear that if (s*, c*) is optimal under any one of the criteria mentioned in the preceding section, then the "truncated path" (s*(0,c*(0), t<T must be an optimal path for the truncated problem of finding c(t) that maximizes

V= [ v(s,c, t)dt Jo

subject to (9.la)-(9.1c) and the terminal condition

288 9 Infinite-horizon problems

s(D = s*(r). It follows that all the necessary conditions (Theorem 6.5.1) for finite- horizon problems (with the exception of transversality conditions) are also necessary for the infinite-horizon problems (for a formal proof, see Halkin, 1974). It remains to find out the appropriate transversality condi- tions. Suppose that the following terminal conditions are imposed on the state variables:

lim Si(t) = sh / = 1,2,...,£, (9.4a)

lim Si(t) >si9 i = k+1,..., k+p, (9.4b)

and no condition on Sj(t), i = k+p + l,...,n9 as f-»oo. One might expect that the following "transversality conditions" are

necessary:

lim7r,(0^0, i = k+l9...,k+p, (9.5a) t-KX>

limn7(0 = 0, i = k+p + l,...,n. (9.5b) t-KX>

Unfortunately, in general, conditions (9.5) are not necessary conditions. Only with considerable restrictions on the functions v, f\ gJ does one ob- tain transversality conditions (9.5) or something like them; see Seierstad (1977b), Benveniste and Scheinkman (1982), and Michel (1982). Since the restrictions are too strict to be applicable to most problems of economic interest, we will not reproduce them here. Instead, we recommend the use of sufficiency theorems to identify optimal paths. This is the subject matter of the next section.

9.3 Sufficient conditions

The following theorem states the sufficient conditions for optimality under the catching-up criterion (and also for cases in which convergence occurs).

Theorem 9.3.1: sufficiency. Let (s*,c*, 7r*, X*) satisfy the necessary con- ditions of Theorem 6.5.1 and the terminal conditions (9.4a)-(9.4b). Then (s*, c*) is optimal if the Lagrangean is concave in (s, c) and if

lim T T * ( O * [ S U ) - S * ( / ) ] > 0 , (9.6) /->oo

where s(t) is any other feasible path satisfying the terminal conditions (9.4) and the expression in (9.6) is the inner product of ir* and s — s*. (Note that (9.6) can be replaced by a weaker condition, where "lim" is replaced by "lim inf.")

9.4 Autonomous problems 289

To prove this result, observe that for any given T the sufficiency argu- ments in Chapter 6 yield

Z(T) = [ Tv(s*, c*, t) dt- [Tv(s, c, t) dt

Jo Jo

= [r[(/r-»••$•)-(#-**•*)] dt Jo

= (r[(//*+^s*)-(//+^s)]^+7r*(r).[s(r)-s*(r)] Jo Jo

>7r*(r).[s(r)-s*(r)]. Hence, if (9.6) is satisfied, then (s*, c*) is optimal according to the catching- up criterion.

Corollary 9.3.1. For optimality under the overtaking criterion, (9.6) must be modified to a more stringent condition: there exists some T such that for all t > T9

x*(O-[s(O-s*(O]^0. (9.7)

This result follows clearly from the proof of Theorem 9.3.1.

There are many different cases in which condition (9.6) is satisfied. The following corollary displays some of these. In applying them it is important to keep in mind the ever-present possibility of making a change of variable from Sj(t) to §i(t) = —St(t), which results in a costate #,(/) = —71-/(0-

Corollary 9.3.2. In Theorem 9.3.1 condition (9.6) can be replaced by (9.8a)-(9.8c):

(i) For / = 1,2,...,k, either \ir*(t)\<N for some N, or

limTrf ( / ) [ * / ( ' ) - * ? ( O U 0 . (9.8a) / - • o o

(ii) For / = Ar-h 1,..., /?,

limx/*(0tf(0 = 0, (9.8b) t->oo

lim Trf (t) > 0, and 0 < st(t) < M for some M. (9.8c)

9.4 Autonomous problems

A special class of infinite-horizon problems often encountered is that in which none of the functions / ' and gh, gJ contains t as an argument and in which t>(s, c, t) takes the form

290 9 Infinite-horizon problems

v(s9c9t) = e~ 8tu(s9c)9

where 6 > 0 is called the discount rate. These problems are called "auton- omous problems" because, as described in Chapter 4, the necessary condi- tions, stated with the use of current-value costate variables, do not con- tain an independent time argument and the resulting system of differential equations is autonomous in the sense defined in Chapter 2. When the time horizon is infinite, an additional feature of this class of problems is that the optimal value of each control variable at any time depends only on the values of the state variables. Before formalizing this idea, let us consider an example.

Example 9.4.1. Find c(t) that maximizes

V(b9t0)=\° Oe-8t(\nc(t))dt (9.9)

subject to

s = rs-c9 (9.9a)

s(to) = b>09 (9.9b)

lim s(t) = 09 (9.9c)

where we assume 6 > r > 0. Since the problem is autonomous, we can form the cur rent-value Ham-

iltonian

H=\nc+\[/(rs-c)

and obtain the necessary conditions (9.9a)-(9.9c) and

dft/dc=l/c-f = 09 (9.9d)

\!/ = 5\l/-dH/ds = (d-r)t. (9.9e)

Differentiate (9.9d) with respect to t:

( l / c 2 ) c = - ^ ,

or

c/c=-^c=-yp/^ = r-b. (9.10)

Solving (9.10),

c(t) = Keir~d)t9 K = const = c(t0)e (d-r)to. (9.11)

Substituting (9.11) into (9.9a):

s-rs = -Ke{r~d)t9

9.4 Autonomous problems 291

or

e-rt(s-rs) = -Ke~8t. (9.12)

Integrating both sides from t0 to any arbitrary T yields

\T e-rt(s-rs)dt = e-rTs(T)-e-rt°s(t0) = -^(e- dt<>)[l-e-HT-to)].

Rearranging terms after multiplying both sides by ert°, we get

5(r)^-^-^ = [5(/o)-c(^)/6] + [c(r0)/a]e- 6(r-H

or

^ ( r ) = [ 5 ( / o ) - c ( / o ) / 5 ] ^ r ( r - / o ) + [ c ( / o ) / 5 ] e ^ - 6 ) ( r - ^ . (9.13)

Taking the limit T-+ oo and noting that the last term vanishes because we assumed r — b < 0, we must have

c«0) = s(t0)8 = b5 (9.14)

in order to satisfy (9.9c). Substituting this into (9.11) and (9.13), respec- tively, we get

c*(t) = b5e{r-6){t-t°\ (9.15a)

s*(f) = te(r~6)('~'o). (9.15b)

Thus,

c*(t) = 8s*(t) for all t>t0. (9.16)

We are able to express the optimal control as a function of the cur- rent value of the state variable. This property holds for all autonomous infinite-horizon problems as we shall see shortly. First, however, let us calculate the value function V(b, t0). Substitute (9.15a) into (9.9):

V(b9to)=\~e- 5i[Qnb6) + (r-6)(t-to)]dt

= [6-le-8t((r-d)t0-5-\r-5)-\nb8-t(r-5))]?0

This is true in particular when t0 = 0; hence,

> - 8 V(b90) = d~

l

It follows that

+ lnZ>6

V(b,t0) = e- 6toV(b,0). (9.17)

292 9 Infinite-horizon problems

In other words, the value of this autonomous infinite-horizon program starting at t0 with s(t0) = b is equal to e~

dt° times the value of the same infinite-horizon program starting at t = 0 with s(0) = b. Let us generalize this property to any autonomous infinite-horizon problem.

A typical autonomous problem

Let

V(b, t0) = max (°°e" 6'w(s,c) dt (9.18)

subject to

*/ = / / ( s , c ) , 1 = 1,2,...,n, (9.19a)

g ' ( s , c ) 2 > 0 , y = l , 2 , . . . , / n ' , (9.19b)

g*(s,c) = 0, /i = ra'+l,...,m, (9.19c)

Si(t0) = bh I = 1 , 2 , . . . , / I , (9.20a)

lim,s'/(/) = J /, / = 1,2,...,A:, (9.20b) f->oo

lim 5 / ( 0 ^ 5 / , / = fc+l,...,£+/?, (9.20c) / - > 0 0

no restriction on the terminal behavior of

Sj(t), i = k+p + l,...,n. (9.20d)

(Note that the cases A: = 0 and p = 0 are admitted.) It is immediately clear that

V(b,t0) = e- 8tomax\0°e-d{t-t°)u(s,c)dt

c J ' 0

= e" 5 / °max e"6 Tw(s,c)rfr,where T = t-t0, c Jo

= e~6'oK(b,0), (9.21)

where

F ( b , 0 ) = m a x f ° ° e - 6 ^ ( s , c ) * r J 0

subject to (9.19a)-(9.19c), (9.20b)-(9.20d), and

Si(0) = &,, z = l , 2 , . . . , / i .

Thus, we have proved (9.17) for any autonomous infinite-horizon prob- lem. Let us proceed a step further and define the (current-value) return function

9.4 Autonomous problems 293

W(b,t0)me 6'°V(b9t0). (9.22a)

Then from (9.21)

W(b9t0) = V(b90); (9.22b)

in other words, Wis independent of t0and can be written as W(b). Thus, we have

W(b) = edt°V(b,t0), where s(f0) = b . (9.22c)

Since this is true for any starting time and any starting capital stock, we can write (9.22c) more generally as

W(st) = e 8tV(stJ). (9.23a)

Now recall that in Chapter 4 under the assumption of differentiability of V, we proved a relationship between the costates and the function V,

**(*)= K s ( s „ 0 . (9.23b)

If yp(t) denotes the current-value costates, it follows from (9.23a) and (9.23b) that

r(0 = Ws(st). (9.23c)

Thus, for autonomous infinite-horizon problems, the optimal current- value costates at any time depend only on the values of the state variables at that time. This implies that the optimal values of the control variables depend on the current states alone. Thus, we have provided an informal proof of the following result.

Theorem 9.4.1. In an autonomous infinite-horizon problem such as (9.18)-(9.20), the maximum present-value return function has the fol- lowing property:

j / ( b , t0) = e~ 8t°V(b, 0). (9.24a)

It follows that the maximum current-value return function depends only on the initial values of the state variables and not on the starting date; hence, it can be expressed as W(b). Consequently, the current-value vec- tor \p*(t) and the optimal control vector c*(t) can be expressed as func- tions of the current-state vector s*(t):

r(t) = $(s*(t)) = Ws{s*(t))9 (9.24b)

c*(0 = «(inO,s'(0) = <o(0(s*(O), s*(0) - G(s*(0). (9.24c)

It is important to stress that this result cannot be obtained in the case of autonomous finite-horizon problems. (For example, in problem (4.38), equations (4.44), (4.46), and (4.48) yield

294 9 Infinite-horizon problems

c*(t) = 5[s*(t)-ert(sTe- rt-s0e-

8T)(l-e-8Trl],

showing that c*(t) depends on t.) The reason is that in the finite-horizon case the step preceding (9.21) fails.

9.5 Steady states in autonomous infinite-horizon problems

For autonomous problems, it is more convenient to work with the current- value Hamiltonian and current-value costates. The resulting differential equations in (s, $) are autonomous (i.e., do not contain t as an indepen- dent argument). Quite often these differential equations yield an equilib- rium point (s°°, \[/°°) with the saddle-point property. If the discount rate is positive and the Lagrangean is concave in (s, c), then the sufficiency theo- rem stated in Section 9.3 implies that the path leading to (s00, ̂ °°) is an optimal path, provided that all feasible paths for s are bounded (or s is required to be nonnegative and ^°° > 0). In Chapter 4 we studied various versions of the optimal growth model that have the saddle-point property. It turns out that for the general autonomous problem (9.18) with the dis- count rate 5 > 0, if a steady state (s°°, ^°°) exists, then it cannot be locally stable in the (state, costate) space; that is, at least one of the roots is posi- tive (or has positive real part), so that we have either only conditional stability (in the sense of saddle point) or only complete instability; see Kurz (1968) for a proof. In the special case where there is only one vari- able, it can also be shown that the optimal path of the state variable is monotone. An informal proof of this result is offered below.

Recall that along an optimal path, the control vector can be expressed as a function of the state variables. When there is only one state variable, using (9.24) we can describe the evolution of the optimal path of the state variable by a single first-order differential equation:

si(t) = fl[sUt),G(sUt))]. (9.25)

If the optimal path of sx were nonmonotone, there would exist tx and t2 such that slUi) = sl(t2) and the sign of sx(t2) would be opposite to that of &\(t\). But this is impossible, because we know that fl in (9.25) takes on the same value at s^ti) and Si(t2) for Si(ti) = Si(t2). This argument establishes that s*(t) must be monotone. Notice that the argument relies on the assumption that there is only one state variable; if there were sev- eral state variables, then Sj (yV 1) would in general appear in (9.25), so that Si(t\) = sx(t2) is consistent with sx(t{) J* sx(t2). The following theorem summarizes these results.

Theorem 9.5.1. Steady states of autonomous infinite-horizon problems with a positive discount rate either are unstable or exhibit the saddle-point

9.5 Steady states in autonomous problems 295

property. In the special case where there is only one state variable, the optimal path for the state variable must be monotone.

To reinforce the readers' grasp of this theorem, we provide a simple example.

Example 9.5.1: saddle point in an infinite-horizon autonomous problem. Let s(t) denote the biomass of a fish species. Without commercial ex- ploitation, the rate of growth of s(t) is assumed to take the following form:

s = s(l— s).

Introducing exploitation, let x(t) denote the rate of landing (harvest). We assume the following relationship between the output x and the inputs s and n (where n stands for "effort"):

x = 2sl/2n1^2. (9.26)

The fish are sold at the fixed price p — 1, and effort costs w dollars per unit. The optimization problem is to find n(t) that maximizes

V(s0) = [°°e- rt(2s1/2nl/2-wn) dt (9.27)

Jo

subject to

A * > 0 , (9.27a)

s = s(\-s)-2sl/2nV2, (9.27b)

s(0) = s0>0 (9.27c) (r and w are positive constants, and we assume r < 1). There is no restric- tion on the limiting behavior of s(t). Note that (9.27b) implies that s(t) can never become negative.

The current-value Hamiltonian is

H=2sl/2nl/2-wn + \[/(s-s2-2sl/2nl/2), (9.28)

and the necessary conditions are

dH/dn = (l-\ls)sV2n-x/2-w<:0,

A2>0, [(l-\ls)sl/2n-l/2-w]n = 09 (9.29a)

t = ryls-dH/ds = yl/(r-\ + 2s)-{\-yl/)s-x/2nx/2, (9.29b)

s = dH/dx/y = s(l-s) - 2sl/2nV2. (9.29c) From (9.29a) if i£ > 1 or s = 0, then n = 0. If ^ < 1 and s > 0, then n is

given by

296 9 Infinite-horizon problems

Figure 9.1

n = w-2s(\-yP)2 (9.30)

n = G(s,f) = (9.31)

Thus, the optimal value of the control is a function of s and ^ :

0 if ^ > 1 or 5 = 0,

5(1 - t/02/w2 if i£ < 1 and s > 0.

Substituting (9.31) into (9.29b) and (9.29c), we have a pair of autonomous differential equations in (s, x//); that is, there is no independent time term. Figure 9.1 is the phase diagram for this system. There are two regions. In region I, which consists of the area defined by \ t > 1 and 5 > 0 , we have n = 0. In region II ( ^ < 1 and s > 0 ) , n is positive. Let us examine the properties of the \p = 0 locus first. In region I, this is the vertical line s = (1 — r)/2 (we have assumed r < 1). In region II, substitution of (9.31) into (9.29b) yields, f o r s > 0 ,

^ = ^ ( r - l + 2 5 ) - ( l - ^ ) 2 / w = M ( 5 , ^ ) . (9.32a)

Thus, the locus \j/ = 0 is a downward-sloping curve, starting at (s, \{/) =

9.5 Steady states in autonomous problems 297

((1 —r)/2,1) and approaching the horizontal axis asymptotically^XTo see this, note that when \p goes to zero in (9.32a) with ^ = 0, s must become very large.) M(s, \p) takes on positive values to the right of that curve, because dM/ds = 2 ^ > 0 for all 0 > 0. Thus, \j/ > 0 (resp. < 0) to the right (resp. left) of the \p = 0 locus.

Turning to the s > 0 locus, we note that in region I, n = 0, so that s > 0 if and only if 0 < s < 1, and s = 0 if and only if s = 0 or s = 1. In region II, substitution of (9.30) into (9.29c) yields

s = s(\-s)-2s(l-\l,)/w = N(s,\ls), (9.32b)

so that s = 0 if s = 0 or

yP = \-(\-s)w/2. (9.33)

(9.33) is the equation of a straight-line segment joining (s9 \j/) = (0,1 - vv/2) with (s, \p) = (1,1). To the right of this line, s < 0, because dN/ds = — 1 < 0.

The intersection of the s = 0 locus (9.32b) with the curve M(s, \l/) = 0 yields a unique equilibrium point, denoted by (s°°, ̂ °°), where 0 < ^°° < 1 and ( l - r ) / 2 < s ° ° < l . There is another equilibrium point, namely, the origin, but starting from any positive s there is no path leading to it.

The point (s°°, x//00) is a saddle point, as can be seen from the phase dia- gram. To confirm this, we linearize (9.32a) and (9.32b):

where

• * 1 r ^ ^ i J\ [MS M J

p-5°°" U-\T_

7V,= [ l - 5 ° ° - 2 ( l - ^ 0 0 ) / w ]

 lA = 2 5 o o / w > 0 ,

- 5 o o = - 5 ° o < 0 ,

M 5 = 2 ^ ° ° > 0 ,

M ^ = ( / - - 1 + 2 S ° ° ) + 2 ( 1 - V O / M > > ( ) ,

and where the simplification of Ns comes from setting 5 = 0 in (9.32b) and the sign of M^ is obtained by recalling that s°° > (1 — r)/2. The trace is Ns-\-Mrp = r>0, and the determinant is negative. Therefore, the roots are real and have opposite signs; hence, the equilibrium is a saddle point.

Starting from any s 0 > 0, the optimal policy is to choose an appropriate i^(0) so that (s0, \K0)) is located on the stable branch of the saddle point. The system will approach the equilibrium point along the stable branch. Notice that the state variable s*(t) is monotone, thus verifying Theorem 9.5.1. However, one should bear in mind that in models with two or more state variables, it is possible that their time paths are nonmonotone; in fact, limit cycles may sometimes be optimal (see Ryder and Heal, 1973; Benhabib and Nishimura, 1979).

298 9 Infinite-horizon problems

9.6 Further properties of autonomous infinite-horizon problems

In Theorem 9.4.1 we have shown that autonomous infinite-horizon prob- lems have the property

V(b,t0) = e- 8toV(b,0) = e-8toW(b).

Hence, for any tx and s(/j) = Sj

V(sutl) = e- 8t^V(sl90) = e-^W(sl). (9.34)

We now use this identity to prove an important relationship between the value function V and the Hamiltonian, assuming that the former is dif- ferentiable.

Consider problem (9.18). Let (s*,c*) denote the optimal path. Then for any tu

K(b, t0) = [ l e~8tu{s\ c*) dt + V&Ut), tx). (9.35)

Use (9.34) in (9.35):

j / ( b , t0) = J' 1 e~btu(s\ c*) dt + V(s*(tx), 0)e~

8tK (9.36)

Since (9.36) is true for all tu differentiating both sides with respect to t{ yields

0 = e -a/iH(s*(f 0 , c*(^)) - SV(sm(tx)9 0)e ~ 6'i

+ e-8ti[dV(s*(t1),0)/ds].(ds*/dtl). (9.37)

But

and

f!^=/(S*(/I),c*(r1)) at i

e-8tidV(s*(tx), 0)/ds = d[V(s*(t{), tx)]/ds = **(ti).

Hence, (9.37) becomes

ms*(h), h) = e-t'WVi), c * ^ ) ) + »•(* i)-f (s*(f 0 , c*(ti)) - mh), (9.38)

or equivalently, by (9.34),

ff(ti) = u(s*(tx)9 c*(tx)) + ntd'HsW, c'tf,)) = 8K(s*(/!), 0), (9.39)

where ^*(/i) is the vector of current-value costates. If 5 = 0, equation (9.39) implies that along an optimal path the current-

value Hamiltonian of an autonomous infinite-horizon problem is identi- cally zero. If 8 > 0 , then (9.34) and (9.38) imply that

9.6 Further properties of autonomous problems 299

lim//(/) = 0. (9.40) / - • o o

(We can prove condition (9.40) without relying on the assumed differen- tiability of V\ see Michel, 1982. However, note also that in our definition of the Hamiltonian we ignore anomalous cases in which the multiplier associated with the integrand cannot be set at unity.) We summarize these results in the following theorem.

Theorem 9.6.1. The Hamiltonian and the value function of the autono- mous infinite-horizon problem (9.18) satisfy the following properties:

H(s*9 c*, i H = 6F(s*, 0) a 5W(s*), (9.41)

If 5 > 0 then lim H(t) = 0. (9.42) f->oo

When 5 > 0 , (9.41) has an interesting economic interpretation. Recall that V(s*(ti), 0) is the value of the maximization problem

[°e-d{i-^u(s9e)dt

subject to s(t1) = s*(tl) (9.19a-9.19c) and (9.20b-9.20d). It is thus the "stock of total wealth," measured in utility units. If the discount rate is interpreted as an interest rate, then H is the income earned as interest on total wealth (see Weitzman, 1976; Kemp and Long, 1982).

Remark (a). It is instructive to relate (9.41) to the Hamilton-Jacobi- Bellman equation (5.63) of Chapter 5. Since for autonomous infinite- horizon problems V(s,t) can be written as V(s90)e~

8t = W(s)e~8t, that equation becomes

0 = max[e-8tu(s,c) + e-8tWs(s)*f(s,c)-8e- 8tW(s)], (9.43)

c(/)

or equivalently,

SW(s) = max[ i*(s, c) + FF,(sW (s, c)]. (9.44) c(/)

But recalling that Ws(s) is equal to the current-value costate (equation (9.23c)), it can be seen that (9.44) is identical to (9.41).

Remark (b). Equation (9.41) is consistent with the transversality condi- tion for free-terminal-time problems with a scrap value function (see Sec- tion 7.6). Consider problem (9.18). Suppose we have solved the problem and obtain the optimal time path s*(t) and the value function V(st, t) and hence W(st); see (9.23a). For any tx and s, let V0(s0, §, tx) denote the value of the following fixed-time, fixed-endpoint problem:

300 9 Infinite-horizon problems

maxf'1 u(s,c)e~dt dt (9.45)

subject to (9.19a)-(9.19c) and s(t0) = s0, s(^) = s. Then by definition, V(s0, t0) must be the value of the following free-time, free-endpoint prob- lem with the scrap value function ebtW($t) = V(st, t):

max[F0(So,Mi) + e - 6 / l ^ ( s ) ] . (9.46)

M i

Clearly, because the scrap value function in (9.46) is not any arbitrary function, but is the value function of the remaining portion of the orig- inal autonomous infinite-horizon problem, any t\ will solve (9.46), pro- vided that that s is chosen to be s*(/j), the optimal value of the state vari- able at time tx. The necessary conditions for (9.46) are

dV0/dtx - be ~ 5/i W(s*(tx)) = 0, (9.47a)

dV0/di + e- 6^Wn(s*(tl)) = 0. (9.47b)

Condition (9.47a) is a special case of the transversality condition (7.92) of the free-terminal-time problems discussed in Section 7.6. Recalling that dV0/dti = H(ti), we see that (9.47a) is identical to (9.41). Condition (9.47b) simply says that the costate variables are continuous (recall that dV0/ds = -ir(tn and e-

h^WMh)) = *(t?)).

The result of Theorem 9.6.1 can also be useful in deriving the solution to some problems. We now present one such problem, which also incor- porates a control parameter. (Necessary conditions that characterize the optimal choice of control parameters are stated in Section 7.11.)

Example 9.6.1: optimal resource depletion under the maximin criterion. An economy produces an output q using a stock of capital K and a flow of extracted resource x. The production function is

q(t) = [x(t)r[K(t)]l-«9 (9.48)

where we assume 0 < a < 0 . 5 ; we shall see later why this assumption is crucial. Gross output is allocated between consumption c(t) and invest- ment I(t). There is no capital depreciation, so that

K(0 = I(t). (9.49)

We are dealing with an exhaustible resource; denoting the current stock of resource by R(t), we have

R(t) = -x(t). (9.50)

9.6 Further properties of autonomous problems 301

We require I(t) > 0, x(t) > 0, and c(t) > b for all t, where b is a control parameter. The planner's objective is to select the highest possible value for the lower bound on consumption, b; this is why it is called the maxi- min criterion. (Another name for it is the Rawlsian criterion because it equates society's welfare with that of the poorest generation.) The objec- tive can be stated as choosing the largest constant b or maximizing

W= [°°5be-8tdt = b, S > 0 , (9.51) Jo

because \™be~btdt = \ for any 6 > 0 . The maximization is subject to (9.49), (9.50), and

/ ( r t + c(/) = [ * ) ] a [ ^ ( 0 ] 1 " ° , (9.52a)

c(t)*>b9 (9.52b)

7 ( 0 ^ 0 , x ( / ) > 0 , (9.52c)

K(0) = K0>0, R(0) = R0>0, and \imR(t) = 0. (9.52d) / - • o o

We can use (9.52a) to eliminate c(t) and restate (9.52b) as

[x(t)]a[K(t)]l-°-I(t) > b. (9.52e)

The Hamiltonian of this problem is

H=5be-8t-Trlx+<K2I, (9.53a)

and the Lagrangean is

£ = H+\(xaKl-a-I-b). (9.53b)

The necessary conditions are (9.49), (9.50), (9.52c)-(9.52e), and

M = 7 r 2 - X < 0 , 7 > 0 , 7 ^ = 0, (9.54a)

^ • = - x 1 + a X j r a - 1 A ' 1 - a < 0 > x > 0 , x ^ = 0, (9.54b)

dx dx

^ = xaKl-a-I-b>0, X > 0 , X ^ - = 0, (9.54c) oX oX

^ = 0, (9.54d)

it2=-(l-a)\x aK-a, (9.54e)

\°°^§-dt = 0. (9.54f) Jo do

302 9 Infinite-horizon problems

Conditions (9.54a)-(9.54e) are the familiar ones; condition (9.54f) re- lates to the optimal choice of the control parameter b according to Theo- rem 7.11.1. It can be rewritten as

\">(5e-8t-\(t))dt = 0, Jo

or

[°°\(t)dt = l. (9.55) Jo

At this stage the existence of a solution to the problem appears by no means assured. Our strategy will be to suppose that there exists a positive (x>0, 7 > 0 , b>0) solution, use the necessary conditions to derive it, and establish its optimality with the sufficiency results of Theorem 9.3.1. Although it is possible to use only the necessary conditions stated above, it is much more efficient to make use of Theorem 9.6.1, which states

e5tH=6W,

or, using (9.51) and (9.53a),

hence,

When

Vipnrp l l C l l V C )

8b-

ll 7T2

-edtTTlx+e 8tir2I=

I x'

--8b;

x> 0 and / > 0, (9.54a) and (9.54b) yield

7T2

/ =

= axa-lKl~a;

axaKl~a.

(9.56)

(9.57)

(9.58)

Investment is seen to equal a constant fraction a of gross output. We sup- pose further that c(t) = b for all t. (Recall that any consumption in excess of b contributes nothing to the objective criterion of (9.49).) Then (9.58) and (9.54c) yield

b = (l-a)xaKl-a9 (9.59)

and this enables us to express I(t) in terms of the constant b:

I(t) = -2-b. (9.60) 1 —a

Therefore, K=ab/(l-a) and

9.6 Further properties of autonomous problems 303

K(t) = K0+-2-bt. (9.61) 1 —a

Manipulating (9.59) and (9.61) yields

*<> = ( l Z ^ H - * o ) ( y ^ ) • (9.62)

To determine the value of b we note that (9.50) and (9.52d) imply R0 = lo*(t) dt, which, with (9.62), gives

\ ( 2 a - l ) / a - | o o / j _ ^ v ( a - l ) / a j

# 0 = [(^4 T(x) = ( J f o ) ( 2 « - i ) / « / l l £ L \

2 a - l ( a - l ) / a |

l - 2 a

because a < 0 . 5 implies that (2 a —l)/a is negative. Finally, we obtain

b = (l-a)[(l-2(x)R0] a^-a)(K0)

{l-2a)/V-a\ (9.63)

which we can substitute into (9.60)-(9.62) to obtain the precise solutions for / , K, and x.

We now turn to the task of determining the solutions for the multiplier and the costates. From (9.56) and (9.60) we deduce that

*2(t) = *ilzjrx(t). (9.64) OLD

Since b and -KX are constant and $™x(t)dt = R0, we can integrate both sides of (9.64) and use (9.55) and (9.54a) (with I> 0) to obtain

! oo poo 1 —rv P°° 1 — a.

\(t)dt=\ T2(t)dt = Tll—2-\ X(t)dt = Tl!—£-Ro; o Jo ab JO ocb hence,

x1 = 7 1 ^ V - (9.65) This can be substituted into (9.64) and used in conjunction with (9.62) to get, after simplification,

The expressions in (9.65) and (9.66) with b given by (9.63) complete the solution to the problem.

It remains to verify that this solution satisfies the condition of Theorem 9.3.1. First consider l i m , ^ Tc{(t)[R(t)-R*(t)]=Lu where irf(t) is the

304 9 Infinite-horizon problems

constant of (9.65), R*(t) is the optimal path determined by (9.62) and (9.50), and R(t) is any other feasible path. Since -KX is constant and R(t) is required to satisfy (9.52d), we have

Li = *-i ]im[R(t)-R*(t)] = 0.

Next consider L2 = \imt_00Tr$(t)[K(t)-K*(t)]i where T T | ( 0 and K*{t) are given by (9.66) and (9.61) and K(t) is any feasible path:

L2 = lim Tc5(t)K(t)-1im -K$(t)K*(t)

l/ct

lim ir*2(t)K(t)-Mm -t-(-^-

= lim ici(t)K(t) since ^5L_1 < 0

, n(2a-l)/a - — t o + t f 0 1 —a:

by (9.61) and (9.66)

/ - • o o a

2*0,

as irZV) > 0, and any feasible ^ ( 0 must be positive because K(t) = I(t)> 0 and K(0) = K0>0. Therefore, Ll+L2> 0, and the solution gives a max- imum. It is interesting that the optimal path does not tend to a steady state, since K*(t) increases without bound.

Exercises

1. The problem of optimal income transfer in a growing economy was first con- sidered by Hamada (1967). The government's objective is to maximize the inte- gral of discounted utility of the representative worker, V= iolu(c(t))e~8t] dt, where c(t)>0 is consumption per worker. The capitalists' income is Y(t) = N(t)[f(k(t)) — c(t)]9 where N(t) is the number of workers and k(t) is the capital/labor ratio K(t)/N(t). We require Y(t) > 0. The rate of growth of the number of workers is exogenous, N(t) = nN(t), and the capitalists' propensity to save is constant; hence, K(t) = sY(t). We take 5, n, and s to be specified positive constants. (a) Show that this problem can be reduced to a single-state variable prob-

lem in which V is maximized subject to k(t) = s[f(k(t)) — c(t)] — nk(t), f(k(t))-c(t)>0, c(t)>0.

(b) Assume that k(0) = k0>0, lim,^+0O k(t) > 0, and u and / have the fol- lowing properties:

/ ' ( 0 ) = +oo, / ' ( s ) > 0 , /'(+oo) = 0, / " ( * ) < 0 ,

M'(0) = +oo, w'(c)>0, w'(+oo) = 0, w"(c)<0.

Exercises 305

Show that there exists a steady state at k*, where /'(£*) = (5 + n)/st and construct a phase diagram in the (<p, k) space. Show that the steady state is a saddle point and the long-run optimum.

(c) Rework the problem under the assumption that the utility function is lin- ear (u(c) = c).

2. The rate of change of a country's net worth A(t) is given by

A(t) = R(A(t))-c(t)+y,

where R(A(t)) denotes interest income, c(t) is aggregate consumption, and y is an exogenous flow of income. It is assumed that R(0) = 0, R'(A) > 0, and R"(A) < 0. A negative net wealth implies that the country is a debtor. The country's aim is to maximize welfare represented by W= j ~ U(c(t))e~dt dt sub- ject to the above constraints; U"(c)<09 and U'(0) = +oo. The following as- sumptions are made. Let AM be the negative number defined by R(AM) + y — 0; assume that there exists a value A* > AM such that R'(A*) = 8 and that A(0) = A0>AM; we require lim,^*, A(t) > AM. Derive the necessary conditions, con- struct a phase diagram, and show that the optimal path converges to (A*, \p*), where \p*= U'(y+R{A*)). Verify that this is a saddle point.

3. A country uses capital to extract a resource, which is used as an input in the production of a final output. The two production functions are R(t) = K2(t) and Q(t) = [R(t)Kx(t)]

x/2. R(t) denotes the rate of extraction of the resource. Kx(t) and K2(t) are the amounts of capital used in the two industries; they are control variables and can be freely chosen at any time, subject to the constraint Kx(t) + K2(t) = K(t)t where K(t) is the total stock of capital, a state variable. We have K(t) = I(t), S(t) = -R(t), and C(t) = Q(t)-I(t). S(t) is the exist- ing stock of resource; Q(t) denotes final output, which is shared between con- sumption C(t) and investment I(t);we assume that I(t) is unrestricted in sign. The utility function is U(C(t)) = (1 - ^)_1(C(0)1",? (v > 0, y * 1). The country's objective is to maximize J* U(C(t))e~bt dt subject to the above constraints. In addition, it is required that lim,^*, AXO^O and l i m , ^ S(/)>0; 5(0) and AX0) are specified. Set up the problem as a control problem and eliminate by substitution the controls Q(t), I(t)9 Kx(t), and K2(t). Show that if 5(0) > 0.5K{0)rj/(8 — 0.5) and 6>0.5, the following solution path satisfies all the necessary and sufficient conditions: (i) The costate variable associated with 5 is zero for all t; hence, R(t) =

0.5K(t); (ii) -k(t) = —0.5ic(t), where ir(t) is the costate for capital stock; (iii) C(O = C(0)exp((0.5-6)/M (iv) I(t) = 0.5K(t)-C(t); (v) C(0) = K(0)[0.5-(0.5-5)/v].

4. Reconsider the yabbies problem of exercises 8 and 9 in Chapter 6. Let T= +oo and require lim,^^ s(t) > 0. Determine the optimal path in all three cases. For the special case 6 = 5, R(c) = c(l — 0.5c), f(s) = 10s(l — s), find all steady-state equilibria and show that one of the equilibria is an unstable focus, which is not part of the optimal path because the Hamiltonian is strictly convex in the

306 9 Infinite-horizon problems

state variable in that region. (What is the sign of the costate variable at that point?)

5. Reconsider the model of exercise 6 in Chapter 6. Describe the optimal policy when E(0) = E0>0 is specified, E(T)>0 is free, and T= +oo. Carry out the exercise again under the added assumption that equipment stock depreciates at the rate m>0 (i.e., E(t) = b(t)-mE(t)).

CHAPTER 10

Three special topics

10.1 Problems with two-state variables

Nearly all the models hitherto encountered in this book have contained a single state variable. (Exceptions are the models of Sections 8.1, 8.3, and 9.6.) We have relied very heavily on phase diagrams in shedding light on the optimal solution. When there are two state variables, however, the (state, costate) space is four-dimensional and cannot be represented straightforwardly. It must be understood that, given the usual regularity conditions, we have in the maximum principle a set of necessary and suffi- cient conditions for an optimum, whatever the size of the problem, and if all functional forms and other restrictions were fully specified, we could - possibly using numerical methods - provide an explicit solution to the problem. However, since most models of interest in economic theory in- volve some unspecified functional forms, an explicit solution is normally unobtainable. This is why phase diagrams are such a useful device for pulling together all the pieces of information contained in the maximum principle.

Since they fail us here, we must devise other means of synthesizing the information. Unfortunately, this is often quite difficult, and in many cases a complete characterization of the solution escapes us. This is not to say that we cannot offer a partial characterization of the solution. It is the aim of this section to illustrate what can indeed be done. First note that in the models of Sections 8.1 and 8.3, the analysis was reduced to a two-dimensional phase diagram. The reader is referred to those sections.

Reduction of a two-sector growth model to a one-state-variable model

This is a straightforward generalization of the one-sector growth model introduced in Chapter 4 and further analyzed in subsequent chapters. There are now two industries (or plants, or regions, etc.) that can inde- pendently produce the consumption good, each with its own technology. The problem is to find cx(t) > 0 and c2(t) > 0 that maximize

307

308 10 Three special topics

[C°u(cl + c2)e- btdt (10.1a)

Jo subject to

sx = Fx(sx)-mxsx-ch (10.1b)

Si = ^2(^2) -m2s2-c2\ (10.1c)

S\{0),s2(0) exogenously specified. (lO.ld)

Using the current-value costate variables fa and fa the Hamiltonian is

H=u(cx + c2) + fa[Fx(sx)-mxsx-cx] + fa[F2(s2)-m2s2-c2]. (10.2)

As long as the usual strict concavity requirements are met, with deriva- tives ranging from + 00 to 0, and nonnegativity conditions are ignored, the following conditions are optimal: (10.1b), (10.1c), and

u'-fa = 0, (10.3a) u'-fa = 0, (10.3b)

fa = fa[8 + mx-F{], (10.3c)

lfe = lM8 + ™2-F2'], 00.3d)

where the prime denotes a derivative. These conditions would normally require a four-dimensional phase dia-

gram, but this problem exhibits some particular features. Clearly, fa = fa; hence, the net marginal products are the same in both industries: F{—mx=F2—m2. This suggests that the two industries could be operated as one, maximizing their joint net product subject to the total availability of capital. This aggregation would not be possible if the two kinds of capital could not be measured in the same units. Define

F(s) = max[Fx(sx)-mxsx+F2(s2)-m2s2\sx+s2 = s]. (10.4) sx,s2

The envelope theorem (Theorem 1.2.8) and the optimality conditions im- mediately yield

F'(s)=F{(sl)-ml = Fi(s2)-m2. (10.5)

Consider the aggregated problem of maximizing

ru{c)e~btdt Jo

subject to

s = F(s)-c,

s(0) = Sl(0)+s2(0).

10.1 Problems with two-state variables 309

It has the Hamiltonian H=u(c)-\-\j/[F(s) — c] and the optimality condi- tions

11'=^, s = F(s)-c, and f = \l,(6-F'(s)). (10.6)

Letting i// = ^1 = ^2 and F(s) as defined by (10.4), we see that condi- tions (10.6) along with (10.5) can be used to duplicate the optimality con- ditions of problem (10.1). The analysis can be carried out in the (s, \[/) space, and (10.5) is used to get the optimal path of sx and s2. The phase diagram in the (s, \[/) space is similar to Figure 6.5 without region A and presents no difficulties. Suppose s(0)<s; we then follow the stable arm to the equilibrium. However, problems arise when the path is to be repre- sented in the (si,s2) space. Equation (10.5) defines what we shall term an efficient locus by its second equality; it is an upward-sloping curve in the (sus2) plane along which the net marginal products of both industries are the same. The equilibrium values of Si and s2 are found at the inter- section of the efficient locus with the line of equation sx+s2 = s. While an interior solution (both cx and c2 positive) prevails, the optimal path fol- lows the efficient locus to the equilibrium point. A difficulty arises if the initial values sx(0) and s2(0) are not on the efficient locus. Then clearly a corner solution must be considered (a diagram is helpful). Suppose, for instance, that the initial point is below the efficient locus; that is, s2(0) is relatively too small. Then in order to correct this imbalance we shall set c2 = 0 at first. The optimality conditions become (10.1b), (10.3a), (10.3c), (10.3d), and

h = ^2(^2) - ™2s2, (10.7a) w'<lfe- (10.7b)

We can use (10.7a) and (10.3d) to determine the path of s2 and î 2, an<l

(10.1b), (10.3a), and (10.3c) can yield a phase diagram in the (su 1/̂ ) space. The resulting optimal trajectory in the (S\,s2) space now has an arm be- ginning at the initial point and reaching up to the efficient locus. A set of initial conditions with ^(O) relatively small would result in a similar ini- tial phase with c{ = 0; the possibility that cx = c2 = 0 can be ruled out by assuming infinite marginal utility of nil consumption.

The reduction of a two-state-variable problem to a single-state-variable one is often far more difficult than in the preceding example. One very interesting treatment is that of Hadley and Kemp (1971, ch. 6), which analyzes an economy with a consumption good industry and a capital good industry, with nontransferable capital between the two industries. Many two-state-variable models, in anticipation of difficulties with their solution, have been structured in such a way that several distinct phases

310 10 Three special topics

can be distinguished, each with a solution simpler than the whole problem. Examples of such models are Takayama (1985, pp. 627-37) and Pitchford (1977). We find other examples in this book (Sections 8.1 and 8.3) where the linearities of the model were exploited for that purpose.

10.2 Trade in capital goods: jumps in the state variables

In the models studied thus far capital goods have been essentially home- grown: no trade in capital took place within the horizon, although the in- clusion of a scrap value implied the sale of capital at the end of the hori- zon. We maintained the requirement that state variables be continuous, indeed differentiable, albeit not continuously so. This is an appropriate assumption in cases where the planning unit does not have access to ex- terior capital markets, but not otherwise. For instance, there is no reason that an individual firm may not purchase or sell some capital asset at any time. In order to take this possibility into account we must allow jump discontinuities to occur for the state variables. This is a departure from the control problem format expounded here and requires a new result. This was provided by Vind (1967).

10.2.1 The general case

We first state the result for a problem with one state variable, no con- straints, and upward jumps only. Many state variables and constraints introduce no new complications; downward jumps will be treated later.

We alter the problem stated at the beginning of Chapter 4 to take into account the possibility of jumps in the state variable. Find c(t), Oj, s(d~), and s(0f), j = 1,..., 7, that maximize

K= [ r «(j(/),C(fl,/)d/+ J p(«y)[5(J,-)-5(^)] (10.8) JO j = i

subject to

Xt)=f(s(t),c(t),t), exceptat^,y = l , . . . , / , (10.9)

s(0f)*s(0j-)9 j = l,...9J, (10.10) and

5(0) = s0, s(T) = sT. (10.11)

The dates 6j at which jumps will occur are to be chosen optimally. The values s(d~) and s(0/) are, respectively, the values of the state variable immediately before and after a jump. A price p(0j) is paid for an extra unit of capital at time Oj. Thus, the state variable follows the differential equation (10.9) at all times except when jumps occur. Equation (10.10)

10.2 Jumps in the state variables 311

reflects the fact that only upward jumps are allowed; the cost of pur- chases of capital has been subtracted from the objective function (10.8).

The Hamiltonian for this is

H = u(s9 c, t) + TT/(S, C, t). (10.12)

Theorem 10.2.1: upward jumps in the interior of the horizon. An opti- mal solution to the above problem, with upward jumps at dj9 j = 1,..., / , must necessarily satisfy (10.9)-(10.11) plus

m a x / / c(t)

Jc = -dH/ds

and in addition

p(t)>ir(t)9 all fe [0,71, p(0y) = 7r(0y), y = l , . . . , / , (10.15)

H(oJn-H(o;)+p(eJ)[s(dJn-s(o;)]=o if 0je(o9T)9j = i9...9j9 (10.16)

where

H(0j-) = u{s(0j-)9 c(0/), Oj) + T{0j)f(s(0j-), c(0j-), 0j) and

me;)=u(s(o;)9 c(o;)9 0 , ) + x ^ x / w e / ) , c(o;)9 Oj).

The ingenious method devised by Vind and adopted by others (e.g., Ar- row and Kurz, 1970) was to introduce an artificial time that coincides with natural-time outside jumps but keeps running while natural time stands still at jump points. We shall use instead a simpler method of derivation that relies on the economic content of jumps. To begin with, let us in- terpret equation (10.15). It indicates that the internal price of capital, 7r, never exceeds the outside price p\ at jump points the two prices are equal. Presumably no purchase takes place when the outside price is higher and if a purchase occurs (at a jump), the two prices must be equal. To under- stand equation (10.16) it must be thought of as characterizing the choice of Oj. The last term is the "gain" made by postponing the purchase of [s(0;)—s(0j~)] units by one instant (supposing p(dj)<0). To under- stand the first two terms, recall that we showed in Section 7.5 that H(T) was the marginal contribution of an increase in T to the maximand V. Here we use the difference H(0;)—H(dj~) to evaluate the loss of post- poning the injection of [s(0/)—s(0/")] units of capital by an instant.

This discussion brings out similarities between conditions (10.15) and (10.16) with transversality conditions. Indeed, problems with free end- point and a scrap value function actually allow a jump from sT to zero

(10.13)

(10.14)

312 10 Three special topics

at terminal time, and T may also be free. The same can be said about problems with free initial point and an initial purchase cost. These ob- servations are the basis for the strategy we adopt in proving (10.15) and (10.16). We consider a problem with a single jump at time 0 in order to simplify the notation (several jumps at Oj•, j = 1,..., J, simply require sum- mation o v e r y ) .

Let c, s(0~), s(d+), and 6 maximize

V=[e u(c,s9t)dt+p(6)[s(0-)-s(0 +)] + [Tu(c,s,t)dt (10.17)

Jo J0

subject to s =f(c, s, t) except at 6, s0 and sT are fixed, and s(0 +) > s(d~).

Using integration by parts with an arbitrary w function, as in equation (4.77), we easily obtain

V= [ (H+*s)dt + iroso-ir(6-)s(e-) + <jr(0 +)s(e+)-TrTsT

Jo

+ [T(H+irs)dt+p(0)[s(d-)-s(e+)]. (10.18) Jfl

For an optimal choice of 6 we partially differentiate Fwith respect to 8:

dV cd[ ds dc

{ T f Hs dc \(Hs + *)—+Hc—

+ ^^-s(6+)+p(d)[s(6-)+s(e+)}.

We now substitute the optimal x, c, and 5 functions and use the other nec- essary conditions, Hs + f = 0,Hc = 0. We also note that x(0~) = dir(6~)/dd and x ( 0 + ) = dir(d+)/de and set the derivative to zero if 0 e (0, T):

^ = H(6-)-H(6+) +p(6)[s(0-) -s(6+)] = 0. (10.19)

This is condition (10.16). In order to choose s(d+) and s(6~) to maximize F w e need to form a

Lagrangean, £ = V+\[s(d+)-s(e~)]. We obtain

_ ^ - = - x ( 0 - ) + / > ( 0 ) - X = O( ds(0 )

d%- = w(d+)-p(8) + \ = 0,

dt

&s(0+) and

s ( 0 + ) - s ( 0 - ) > O , A > 0 , \[s(0+)-s(6-)] = O.

10.2 Jumps in the state variables 313

Therefore, if a jump occurs, X = 0 and ir(6+) = 7r(0~) = /?(0), whereas if no jump occurs, p(d) — 7r(0) = X > 0. Indeed, we must never have/?(0 < ir(t), since this is inconsistent with a maximum of V. To sum up, we have p(t)> w(t) everywhere, and at a jump, where 5 ( 0 + ) > 5 ( 0 ~ ) , we have p(6) = 7r(0); this is condition (10.15).

We now provide a simple example of a problem in which an upward jump is optimal within the interior of the horizon.

Example 10.2.1: an interior jump upward. A firm has a stock of an ex- haustible resource 5(0) = 2. The rate of extraction is c(t): s(t) = —c(t). The extraction is costless, and the extracted resource is transformed into a final output y — 2c1 / 2/V1 / 2, where N is the quantity of a fixed factor, say land. Output price is 1 (per unit) and N=l. The firm cannot sell its re- source stock, but it can acquire additional resource at the (buying) price p(t) per unit of stock, where/?(0 = (̂ —1)2 + 1, f ^ 0 . Therefore, the low- est buying price ever i s p ( \ ) — 1. Suppose also that the planning horizon is [0,2] and that 5(2) must equal 1. The firm must solve the following prob- lem: find c that maximizes

\22cV2dt+Zp(0j)[s(e-)-s(e;)] J0 j

subject to

s = — c,

5(0) = 2, 5(2) = 1,

5 ( 0 / ) > 5 ( 0 / ) .

H = 2 c 1 / 2 — 7rc and the maximum principle yields C = TT~2 and -k = 0. Thus, c and 7r are constant. This is expected: with a zero discount rate and a concave objective, it is optimal to spread extraction evenly. In the absence of jumps JQ cdt = 2 — 1 = 1 yields c = \ and 7r = (2) 1 / 2 . We now show that an upward jump at 0 = 1 is optimal, using conditions (10.15) and (10.16). Recall that for an upward jump we require p(t) > ir(t) at all times and equality at the jump time. We have shown that -K is constant, hence a jump will occur, if at all, at the minimum value of /?, namely, p(\) = 1. This determines the (constant) values of -w and c: Tr(t) = 1 and c(t) = 1 for all t e [ 0 , 2 ] . We can now evaluate the size of the jump: c(t) = 1 for 0 < t < 1, with 5(0) = 2, implies 5(1") = 1 and c(t) = 1 for 1 < / < 2, with 5(2) = 1, requires 5(1+) = 2. Hence, the firm purchases one unit of the re- source. To check on the optimality of this policy we calculate

/ / ( l - ) - / / ( l + ) + / ) ( l ) [ 5 ( r ) - 5 ( l + ) ]

= ( 2 x l - l x l ) - ( 2 x l - l x l ) + 0 [ l - 2 ] = 0,

314 10 Three special topics

as required by (10.16). The maximum revenue is now

f22tf/ + l ( - l ) = 3 Jo

instead of

\l2l2dt = 2^ previously.

Let us now extend these results to allow for downward jumps as well. First note that if the same price p(t) is used for buying (upward jump) or selling (downward jump) capital, we will necessarily have p(t) = ir(t) everywhere. This unrestrained opening of the model destroys it, since the path of the costate is then fixed at the outset. Under these circumstances no solution would exist for most problems.

There is, however, a more sensible way of modeling two-way trade in capital: we must introduce two sets of prices, a buying price and a selling price of capital, say pb(t) and ps(t), respectively, with pb{t)>ps(t) to avoid infinitely large profits. Some distortion of a free market such as transaction costs is presumably the cause of this discrepancy. We must replace the expression in (10.17) by

V= f1 u(c, 5, t) dt -pb(ex)[s(et) -s(0f)] Jo

+ \62u(cis9t)dt+p s(d2)[s(d2)-s(0t)] + \

T u(c9s,t)dt J0! J02

= \e\H+ics)dt + ir0s0-ir(dns(en-p b(Oi)[s(Ol-)-s(en] Jo

+ f 2 ( / / + 7T5) dt + Ir(0i+)S(01+) - 7T(02-)S(02-)

+ f (H+TS)dt + Tr(eZ)s(0Z)-TrTsT

+Ps(O2)[s(02)-s(eZ)], (10.20) with

5(01 +)-5(0f)>O,

s(d2)-s(0})>O.

(The form of the expression in (10.20) presumes, without loss, that the upward jump - at 6\ - must occur before the downward jump - at 02.)

We apply the same technique as before with £ = V+ X1[5(0j f) —s(6^)] +

X2[s(02l—s(02~)]. If an upward jump occurs at 6h we differentiate £ with respect to s(dr) and 5,(01

+):

10.2 Jumps in the state variables 315

\,>o, S(o?)-s(or)*o, \i[s(ot)-s(er)]=o. If a downward jump occurs at 02, we differentiate <£ with respect to s(62) and s(0}):

-Tr(d2)+p s(62) + \2 = 09

*(0})-PS(02)-\2 = 0, \ 2 > 0 , s(d2)-s(6t)>0, \2[s(0})-s(0;)] = 0.

Therefore, in either case at a jump, ic(0j~) = 7r(0/), j = 1,2, and we have

ps(t) < TT(0 < /? V ) w e [0, T]. (10.21)

Moreover, differentiating (10.20) with respect to dx and 02 yields two ex- pressions similar to (10.19):

H(en-H(e^)-pb(dl)ls(e^)-s(en] = o9 H(e2)-H(et)+p

s(e2)[s(d2)-s(el-)] = o. We can now gather our results. At an upward jump,

s(0t)>s(0r) and pb(0l) = ic(0l)>p s{0l)9

H(dn-H(6t)-pb(0i)[s(e^)-s(dn] = O.

At a downward jump,

s(0})<s(62) and p s(62) = ir(e2)<p

b(e2),

H(e2)-H(e})+p s(e2)[s(e2)-s(et)]=o.

These results indicate that the internal valuation of capital is bracketed by the buying and selling prices. Clearly, if the internal valuation is strictly within that interval, no trade takes place, and if trade does take place, the internal price equals the buying or selling price, depending on whether a purchase or a sale takes place, respectively. These are no more than generalized market-clearing conditions which state that the demand price never exceeds the supply price and that both prices are equal when a trade takes place, with the internal valuation of capital playing the role of a demand price vis-a-vis the buying price and of a supply price vis-a-vis the selling price. We now state these results formally.

Theorem 10.2.2: upward and downward jumps in the interior. Consider the problem of maximizing

V=\dlu(c9s9t)dt-p b(d1)[s(0l-)-s(en]

Jo

+ \°2u(c9s9 t)dt+p s(e2)[s(e2)-s(d})] + \

T u(c,s9 t)dt J0J J 0 2

316 10 Three special topics

subject to

s(0t)-s(0r)*09

s =f(s, c, t) except at 0y, j = 1,2,

s(0) = s0 and s(T) = sT,

while allowing the possibility of an upward jump at time dx or a down- ward jump at time 02, 0/ e (0, T), j = 1,2. Let H(t) = u(s, c, 0 + TT/(S, C, /) be the Hamiltonian evaluated along an optimal path. Assume pb(t) and ps(t) to be differentiate. If an upward jump in s is optimal at time 6h then (10.21) and (10.23) hold, as do the usual necessary conditions. If a downward jump in s is optimal at time 02, then (10.21) and (10.24) hold, as do the usual necessary conditions.

The generalization to several state variables and many jumps is straight- forward, at the cost of adding some notation.

Notation

The dates of jumps are 0y, j = 1,..., 7. The set of variables that exhibit upward jumps at date 0y is 6 j ;

Qj = li\si(ej-)<si(0;)l The set of variables that exhibit downward jumps at date 0y is

ej;ej = u\si(eJ-)>si(e;)}. Note that while we can have both 0 j and Qj nonempty, their intersec-

tion is always empty because some variables may have upward jumps and other variables downward jumps at the same date, but any one variable cannot jump both upward and downward at the same time.

Corollary 10.2.1. If many jumps are optimal and follow the pattern de- scribed in the notation above and if pf(t) and pf(t) are everywhere differ- e n t i a t e , then in addition to the maximum principle, the following condi- tions apply:

pM^TCiW^pfV), i = l , . . . , / , te[0,T]9 (10.25)

5 / ( 0 / ) > 5 / ( 0 " ) , p}'(0j) = ici(0j)>pH0j), / e e j , y = l , . . . , 7 , (10.26a)

si(0J-)>si(0f), P?(0j)>Ti(0j)=pf(0j)9 ieej,j = l,...,J, (10.26b)

H(ej-)-H(e;)- s phWsiWfr-stf]-)) ieO)

+ !jH0j)(Si(0D-si(d;)) = O9 y = l , . . . , 7 . (10.27) ieQf

10.2 Jumps in the state variables 317

(b)

Ps(t)

i

hNsp« ,(t)

v **<*> }

P W I

(c) (d)

Figure 10.1

In Figure 10.1 we illustrate various cases. The costate variable before jumps were allowed is denoted by 7r*(0, while the variable when jumps are allowed is denoted by it(t)\ 6 is the jump time and pb(t) and ps(t) are the buying price and the selling price of capital, respectively. Case (a)

318 10 Three special topics

illustrates an upward jump; the previous path ir*(t) is no longer optimal, for during some interval it exceeds the price at which stock can be pro- cured from outside; 1t(t) is now the optimal path with pb(6) = t ( 0 ) . Case (b) illustrates a downward jump at time 0. Case (c) illustrates the possibil- ity that although jumps are allowed, they are suboptimal: the original valuation of capital ir*(t) is bracketed by the buying price and the selling price at all times; thus, no trade in capital takes place. Case (d) illustrates the possibility that the introduction of some capital price patterns may result in a problem without a solution. In the case illustrated, as in Ex- ample 10.2.1, the value of the costate must be constant, and this is incom- patible with it remaining not above the buying price and not below the selling price.

This presentation makes it clear that while it is possible for a state vari- able to exhibit several jumps (some upward and some downward), this oc- currence is highly unlikely for arbitrarily chosen buying and selling prices. We now illustrate downward jumps on a variant of Example 10.2.1.

Example 10.2.2: an interior jump downward. It is now assumed that no resource can be purchased but that instead it can be sold at price ps(t) = 2 — (t — 1 )2. This price path exhibits a maximum at t = 1: p( 1) = 2. The pre- vious optimality conditions still prevail; c = TT~2 is constant. If a sale takes place at f = 1, we have ir(t)=p(l) = 2 for all te [0,2] and c= \. There- fore, JQ cdt - \(2) = \ and 0.5 unit is sold at time 1. The costate variable remains above the price path ps(t) everywhere but at t = 1. The reader is invited to verify that H(r)-H(l+)+ps(l)[s(\-)-s(l+)] = 0.

Remark: jumps at the boundary. In Section 10.2.1 we have hitherto re- stricted our attention to jumps occurring at some time 0 e (0, T). A slight modification is needed for jumps that occur at time 0 or time T. We must return to equation (10.19), which reflects the choice of the timing of the jump; that equation was valid if 0 e (0, T). If we wish to restrict 0 properly to the closed interval [ 0 , 7 ] , we must add dV/dd < 0 if 0 = 0 and dV/dd > 0 if 0 = T. More precisely, letting

H(0~) = u(c(0), s0,0) + T T ( 0 ) / ( C ( 0 ) , s 0 , 0 ) and

H(T+) = u(c(T), sTi T) + ir(T)f(c(T),sT, T),

we require

/ / ( 0 " ) - / / ( 0 + ) + ^ ( 0 ) [ 5 0 - 5 ( 0 + ) ] < 0 forajumpat 6 = 0,

H(T-)-H(T+)+p(T)[s(T-)-sT]>0 for a jump at 8 = T.

The other conditions are unaltered. We now illustrate boundary jumps with an example.

10.2 Jumps in the state variables 319

Example 10.2.3: jumps at the boundary. In this example a naturally wast- ing r e s o u r c r x a i r b e harvested, but larger harvests hasten wastage. The problem is to choose c to maximize

V=[l2\ncdt, Jo

subject to

s = -2s~l/2c,

5(0) = 4, s ( l ) = l.

The Hamiltonian / / = 21nc — 2irs~l/2c is everywhere strictly concave in c, and maximizing it yields c = 7r_ 151 / 2. Substitution yields the maximized Hamiltonian H(s,ir) = \ns — 2(lmr) — 2, which is strictly concave in 5, given 7r. The state and costate obey the differential equations

7r = — s~l and 5 = —27T-1.

To solve this system of nonlinear equations, differentiate the second one to get S = 2TT~27T and by substitution s = — 0.5s~l(s)2, or s/s = —0.5s/s, which can easily be integrated (twice). The general solution is

s ( 0 = ( / 3 - a 0 2 / 3 , (1 0-2 9) c(t) = a / 3 , a > 0 , / 3 > 0 .

In the first instance let no jumps be allowed; the boundary conditions 5(0) = 4 and 5(1) = 1 then yield a = 7, 0 = 8, and the solution is ir*(t) = f ( 8 - 7 0 1 / 3 , s*(t) = (8-lt)2/\ c*(t) = | , with *•*(/) ranging from f to f over the horizon and the value function V=2lnc —1.6945.

In order to generate an upward jump at time T= 1 say, all we need do is offer a buying price that is lower than the current internal valuation at time T. Since currently 7r*(l) = | , let us use pb(t) = 2 . 3 - 2 / (we will also need to check that pb{t) remains above the new ir(t) path once the latter is determined). The general solution (10.29) is still valid for t e [0,1) and using the boundary condition 5(0) = 4 and 7r(l) =pb(l) = 0.3, we obtain j8 = 8 and a - 7.567. The solution is

s(t) = (8 - 7.56702 / 3, TT(0 = 0.3965(8 - 7.567/)1/3, c(t) = 2.5223;

s(l~) = 0.57234 and since 5(1) = 1, there is a purchase of 0.42766 unit of stock at T= 1, at a price of 0.3 per unit. The value function is V= 2 In c — 0.3(0.42766) = L722, an improvement, and finally

H(r) - / / ( 1 + ) +^(5(1") - 5 0 = ln(0.57234) - In 1 - 2(-0.42766)

= 0.2973 > 0 ,

320 10 Three special topics

6. 7

0.7929

jl \

F^^^<! £ ( 0 ^

pb(t)

1 1 1 1 1 1 1

V \ l 3 v \ 1 7

1 1 1 1 1

(a)

0.9764

(b)

Figure 10.2

as required for jumps at the terminal time. The paths of ir*(t), t(t), and pb(t) are plotted in Figure 10.2a; it can be verified that 7t(t) < pb(t) every- where.

We now present a downward jump at T=l. Let the selling price be ps(t) = 0.5 + 0.1/; ps(l) = 0.6 > TT*(1) = f, indicating that a downward jump at T= 1 is warranted. We calculate the solution on t e [0,1) by us- ing the general solution (10.29) and the boundary conditions 5(0) = 4, TT(1) = 0.6. We obtain 0 = 8 and a ^ 6.145. The solution is

10.2 Jumps in the state variables 321

s(t) = (8 - 6.14502 / 3, TT(0 = 0.4882(8 - 6.14501 / 3, c(t) = 2.0483;

5(1") = 1.50972 and since 5(1) = 1, there is a sale of 0.50972 unit of stock at T — 1, at a price of 0.6 per unit. The value function is V — 21nc + 0.6(0.50972) = 1.7399, an improvement over no jumps, and finally

/ / ( l - ) - / / ( l + ) + / ) ( 5 ( r ) - 5 1 ) = ln(1.50972)-ln 1 + 0.1(0.50972)

= 0.4629 > 0

as expected. The paths of 7r*(0, n(t), and ps(t) are plotted in Figure 10.2b; note that 7r(/) >ps(t) everywhere.

10.2.2 The case of the strictly concave Hamiltonian

We again turn our attention to the possible occurrence of jumps within the interior of the planning horizon. We begin our discussion with a theo- rem that modifies a proposition of Arrow and Kurz (1970, p . 57).

Theorem 10.2.3. Let J / ° ( s , TT, t) = m a x c / / ( c , s, TT, t). If i / ° ( s , TT, /) is strictly concave in s and if the exogenous price paths of capital, p(/), are continuously differentiate, jumps in the state variables may be optimal only if they take place at the initial time or at the terminal time.

Proof. Consider a hypothetical jump point 0 where variables sh i e /, ex- hibit a jump. For those doing an upward jump (5,(0+) >5/(0~)) we must have pf(t) > 71-/(0, all / and pf(0) = TT/(0). This implies that pf must de- crease toward 7T/ before time 0 and must increase away from 717 after time d. In terms of slopes these restrictions can be expressed as

A*(fl")^*/(n and />ft0+)^*/(0+>; hence,

TT/(0+) - */(0~) ^ A*(0+) -P?(0~) = 0 by differentiability of pf.

Therefore,

[ * / ( f l + ) - * i ( n ] [ j | ( f l + ) - 5 / ( n i ^ o (10.30)

if 5/ exhibits an upward jump at instant 0. For a downward jump (5/(0+) < 5/(0")), we must have pf(t) < 717(f), all

/ and pf(0) = 7t-/(0); therefore, pf must increase toward 717 before time 0 and decrease aWay from it after time 0. This leads to

TT/(0+) - 7T/(0") > pf (0+) - pf (0") = 0 by differentiability of pf.

Hence, (10.30) is also valid at a downward jump, and we can sign the in- ner product

322 10 Three special topics

[x(fl+)-*(n]-[s(fl+)-s(ni^o, 00.31) where the elements of s(0 + ) and s(0~) are such that

Si(0+)*Si(0-)9 iel, and Sj(0+) = Sj(0-)9 j*l.

We know from the maximum principle that 717 = —//j); hence,

M6+) ~ * / ( n = - / / i ^ ( s ( 0 + ) , *, 0) + //j>(s(0~), 7T, 0).

Substituting this into (10.31), we obtain

- [ / / s ° ( s ( 0 + ) , 7 r , 0 ) - / / s ° ( s ( n , ^ ^ ] - [ s ( 0

+ ) - s ( r ) ] ^ O .

This inequality contradicts the assumed strict concavity of H in s, which requires

[//s°(s(0 +), 7T,0) - / / s

o ( s ( 0 " ) , * , ^ ) ] - [ s ( 0 + ) - s ( 0 " ) ] < 0

for any values of s(0 + ) and s(0~), thus proving the theorem. •

We can present a geometric interpretation of the preceding proof in the case where a single state variable, say sh jumps at time 0. Consider first an upward jump, that is, Si(0+) >5j(0"), for which P\(t) must just "touch" ir(t) from above at time 0. Yet we know that T r ^ " ) = - i / i ) 1 ( 5 1 ( 0 " ) , . . . , 0) and 7r!(0+) = —//i)1(51(0

+),..., 0). The implication of the strict concavity of H° in Si is that -kx increases at an upward jump - because the first de- rivative of H° with respect to Sj decreases. The geometric interpretation of this configuration is that there is a kink in ir{ at time 0 and that, at the kink, the two branches of -KX form an angle of less than 180° (from above). Therefore, it is impossible for a smooth /?f curve to just touch -KX at 0 from above since this would require the graph of ir\ to be below the tangent to p\ at 0.

A similar argument can be made for downward jumps where p\ must touch 7T! from below and where -KX exhibits a kink forming an angle of less than 180° (from below).

Remark. The conditions of applicability of Theorem 10.2.3 could be weakened to the requirement that H° be strictly concave in each sh indi- vidually, if we are considering jumps in one state variable at a time, a most likely occurrence.

Under the differentiability assumption for outside prices, Theorem 10.2.3 severely restricts the applicability of our previous results on jumps. It is relatively seldom that one encounters problems where 7/°(s,7r,/) is not strictly concave in s, as we recall that concavity is the most common re- striction ensuring sufficiency of the maximum principle. Note, however, that a problem with a nonstrictly concave Hamiltonian (it was linear)

10.2 Jumps in the state variables 323

was encountered in Examples 10.2,1 and 1 & 2 ^ , and jumps occurred in the interior of the horizon with differentiate price paths.

We wish to offer two other approaches to the modeling of the impor- tant phenomenon of trade in capital goods. The first approach introduces nondifferentiable price paths, while the second pursues the implications of smoothness and does away with jumps.

Nondifferentiable price paths. Our proof of Theorem 10.2.3 makes it clear that we can devise nondifferentiable price paths that induce jumps in the interior of the horizon even with a strictly concave Hamiltonian. There is no particular reason to insist on differentiability since it is mainly an assumption of convenience without any real economic content here. For concreteness and to investigate the anatomy of jumps in such problems we will present an example. Before we do this, however, we must state precisely the optimality conditions corresponding to jumps with nondif- ferentiable price paths. This will entail a modification of Theorem 10.2.2.

Theorem 10.2.4. Consider the problem described in Theorem 10.2.2 but allow pb(t) and ps(t) to be piecewise-differentiable. Then the usual neces- sary conditions hold and

ps(t)*T(t)<pb(t), W e [ 0 , 7 1 . (10.32)

Furthermore, if an upward jump in s is optimal at time 6U

s(0t)>s(0r), p\ex) = TT(^) >p s(dx)\

H(0r)-H(0t)-Pb(0n[s(6t)s(0r)] ^ 0 and

men-mot) -pb(et)[s(ot) - ^ m < o. If a downward jump is optimal at time d2,

s(0})<s(62), p s(62) = ir(d2)<p

b(e2); (10.35)

H(d2)-H(e})+p s(e2)[s(62)-s(8})]>0 (10.36a)

and H(62) - H(0Z) +p

s(el)[s(B2) -s(0t)] < 0. (10.36b)

Note that p(0~) and p(6+) represent left- and right-hand-side derivatives of the appropriate p function in the event that it is not differentiable at 0. Up is differentiable at 0, these derivatives are identical and the (a) and (b) parts of (10.34) and (10.36) reduce to an equality as in Theorem 10.2.2.

Proof. We can transform the objective function into expression (10.20). Conditions (10.32), (10.33), and (10.35) are obtained as before from the optimal choice of s(0+) and s(dj~). In order to obtain (10.34) and (10.36),

(10.33)

(10.34a)

(10.34b)

324 10 Three special topics

which are new, we must take the derivative of (10.20) with respect to 0, and evaluate it at df and at 0+, noting that only for the nondifferentiable prices will this distinction be relevant. Since we seek a maximum, we must let the derivative evaluated at Of be nonnegative while the right-hand-side derivative is nonpositive. We obtain

mon-moft+pwniswn-siofn^o, 1=1,2, j=s,b, and

mon-moft-pwtnsioft-swrn^o, 1 = 1,2, j=s,b9 where H(0f) = H(s(6D, c(0D> *(0/)t 0/)» a n d so on. These conditions can be specialized to (10.34) and (10.36). D

Theorem 10.2.4 can be extended to several variables and many jump dates, as was done in Corollary 10.2.1.

Corollary 10.2.2. Suppose that it is optimal for variables to exhibit jumps according to the notation given on page 316 but pf(t) and pf(t) are only piecewise-differentiable. Then, in addition to the maximum principle, the following conditions are also necessary: (10.25), (10.26), and

mop-mo?)- s pftornsAofr-SiVj-)] iee)

+ S o A - ( ^ r ) [ ^ ( ^ ) - ^ ^ / ) ] ^ 0 , y = l , . . . , / , (10.37a)

ieQf

mop-mof)- s pHofnsiiofr-sAOj-)] ieO)

+ 2 pnofnsiiop-Siio;)]^, y = i,...,y. 00.37b) ieQf

Theorem 10.2.4 and Corollary 10.2.2 can be specialized for problems with a concave Hamiltonian - the bulk of problems encountered in the economics literature. It turns out that conditions (10.37) (or (10.34) and (10.36)) are superseded by other conditions, as we now show.

Corollary 10.2.3. Consider the problem described in Theorem 10.2.4 and assume that //°(s, TT, t) = max H(s, c, TT, t) is a concave function of s. Then if an upward jump is optimal at date dh conditions (10.34) are replaced by

/)*(0f)<7r(0r) and /)6(01 +)>x((91

+). (10.38)

If a downward jump takes place at time 02, conditions (10.36) are re- placed by

/>5(02~)>7r(02~) and p s(6}) < TT(02+). (10.39)

10.2 Jumps in the state variables 325

More generally if there are several state variables and several jumps as in the problem of Corollary 10.2.2 and H°(s, TT, /) is jointly concave in all sh i = 1,...,/, the following condition replaces (10.37):

pt(0]-)**i(0y) and A * ( 0 / ) ^ / ( 0 / ) > ^ 0 j , y = l,...,7, (10.40a) /)f(0/)>*/(0/) and pf(0;)*in(0;)9 160?, j = l,...,J. (10.40b)

Proof, It is clear that the necessary conditions (10.25) and (10.26) them- selves imply (10.40), as argued in the proof of Theorem 10.2.3; hence, (10.40) is a necessary condition. Furthermore, we now show that, under concavity of H°, (10.40) implies (10.37), thereby superseding it.

When H°(s) is a concave function, we know that

(S2-Siy.H^S2)^H 0(s2)-H°(Sl)^(S2-SlY^(Sl), VS,*S2.

Hence, letting s2 be s(0/) and s^ be s(0j~), where 6j is any interior jump date, and noting that Hs = —-k, we have

[s(o;)-s(Of)Y**(o;)*H0(Of)-H0w;) *[s{0;)-s(0j-)]'.i(0j-). (10.41)

Condition (10.40) implies that

[ ^ ( 0 / ) - ^ ( 0 7 ) ] A ( ^ ) ^ [ ^ ( ^ ) - ^ ( ^ " ) ] ^ / ( ^ ) and

[ * / ( 0 / ) - * / ( 0 / ) ] A ^ ^ where Pi=p? when i e 0 j and /),=/>/ when / e 0y. Therefore, with the same summary notation,

[ s ( 0 / ) ^ s ( ^ ) ] ^ p ( ^ ) < [ S ( 0 / ) - s ( 0 7 ) ] ' . i r ( 0 7 ) , [ s ( 0 / ) - s ( 0 7 ) ] ' . p ( 0 / ) > [ s ( 0 / ) - s ( 0 - ) ] ' . ^ ( 0 / ) .

Together (10.41) and (10.42) imply (10.37). We can in turn specialize this to (10.38) and (10.39) by selecting / = 1, / = 2, 0} = 1, 0^ = 0, 0? = 0, Q22 = 1; this means that the single state variable has an upward jump at d\ and a downward jump at 02-

Example 10.2.4: interior jump with a strictly concave Hamiltonian. We use again the model of Example 10.2.3. The general solution (10.29) still applies and, without jumps, the costate ranges from f to | . Let us con- struct an example that will make a downward jump optimal at / = 0.5, say. Since 7r*(0.5) —0.70756, let us choose a larger price, say /?5(0.5) = 0.8. In order to solve for the new solution consistent with a jump at t = 0.5, we must distinguish two branches: the first is valid on [0,0.5), while the second holds on (0.5,1]. s{t) will be discontinuous at t = 0.5, but 1t(t) will not be. It is essential to understand that the lack of discontinuity of

326 10 Three special topics

# ( / ) at t = 0.5 is consistent with the existence of two distinct branches of 7r(/), which will meet with a kink at ^ = 0.5. Both branches must obey 7r(0.5) = 0.8; this imposes the following restriction on a and 0:

3a"1(0-o.5a)1/3 = o.8.

In addition, the first branch must satisfy 5(0) = 4, or

0 2 / 3 = 4, 0 = 8,

while the second branch must satisfy 5(1) = 1, or

(P-a)2/3 = l, 0 = a + l.

Substitution yields, for the first branch,

3 ( 8 - 0 . 5 a ) 1 / 3 = 0.8a, or a = 6.339, 0 = 8,

and for the second branch,

3(1 + 0.5a) 1 / 3 = 0.8a, or a = 5.937, 0 = 6.937.

Using the pairs of values for a and 0, we can calculate the values of s near the jump point; we have 5(0.5") = 2.8575 and 5(0.5+) = 2.5066 - thus a downward jump of 0.3509. We can also verify that the costate values, TT(0.5") = 0.800015 and T T ( 0 . 5 + ) = 0.800012, are approximately equal at the jump time, using these a and 0 values. The control in the first half is c — 2.113 and that in the second half is c —1.979. The max- imum value is V= 2(ln 2.113 + In 1.979)/2 + 0.8(0.3509) = 1.7114, an im- provement over the lack of jumps. In order to choose an appropriate price path we need to know the slope of w before and after the jump. We have TT(0.5") = - 0 . 3 4 9 9 and T T ( 0 . 5 + ) = - 0 . 3 9 8 9 . We must choose a price path with a slope less steep than that of TT before the jump and steeper than that of 7r after the jump. Furthermore, we must have ps(t) < ir(t), 0 < t < 0.5, and ps(t) < 7r(0, 0.5 <t < 1. For instance,

5 _ f 0 . 9 0 - 0 . 2 / , 0 < / < 0 . 5 , P ^ 1 . 1 - 0 . 6 / , 0 . 5 < / < l .

Note that ps(t) is continuous and that /?5(0.5) = 0.8. The relative shapes of ps(t) and 7t(t) before and after the jump are plotted approximately in Figure 10.3a, and the jump area is scaled up in Figure 10.3b.

To illustrate Corollary 10.2.3 we now verify that (10.36) applies. We have

H°(0-) - H°(0+) = In 5 « T ) - In s(6+)

here; hence,

10.2 Jumps in the state variables

J

0.9465 0.9 0.8

i

if(t) v ^

Ps(t) ^

:

^ ^ 1 0 . 5 0 5 3 ,0.5

0.5

(a)

(b)

- t

Figure 10.3

/ / ° ( 0 . 5 - ) - / / 0 ( 0 . 5 + ) + p ( 0 . 5 - ) [ 5 ( 0 . 5 _ ) - s ( 0 . 5 + ) ]

= ln(2.8575/2.5066) + (-0.2)(2.8575 - 2.5066)

= 0.13102-0.07018 = 0.06084 > 0;

/ / 0 ( 0 . 5 - ) - / / ° ( 0 . 5 + ) + j o ( 0 . 5 + ) [ s ( 0 . 5 - ) - s ( 0 . 5 + ) ]

= 0.13102-0.6(2.8575-2.5066)

= - 0 . 0 7 9 5 2 < 0 .

328 10 Three special topics

Smooth trading in capital goods. We wish to offer an alternative to jumps in the state variables, which still allows trading on the capital market. We argue that this possibility, not the discontinuities themselves, is the im- portant feature. After all, a continuous-time model is at best an idealiza- tion of reality, and similarly an instantaneous transaction can be repre- sented by a very fast, but smooth change in the value of the state variable.

Consider the (strictly concave) control problem of finding ct that max- imizes

\Tu(ct,st)e- 8tdt + e-8T$(sT)

Jo subject to

st=f(Ct,st), s0 fixed, 57 free.

Suppose we wish to consider altering the capital stock s by outside pur- chases or sales. We propose to introduce a new state variable Xt that indi- cates the net stock of capital purchased from outside, to date. This trade is not instantaneous in the sense that there is an upper bound on the num- ber of units of stock that can be traded in an instant of time. We maximize

\T[u(ct,st+Xt)-tfxt + tizt]e- btdt + e-bT<)>(sT+XT) (10.43) Jo

subject to

st =f(ct9 (st+Xt))9 s0 fixed, sT free, (10.44) Xt = xt-zt, X0 = 0, A> free, (10.45)

Jt>.x:,>0, z > Z / > 0 .

The current-value Hamiltonian is

H=u(c,s+X)-vbx + tfz + Trf(c,s+X) + <p(x-z). (10.46)

The optimal solution is characterized by (10.44), (10.45), and

(c, x, z) maximize H in (10.46) subject to 0 < x < Jc, 0 < z < z; (10.47) 7r = -w^-h(6-/2

,)7r, 7rr = 0f (10.48) and

where

<P = - u i + d<p-irfi, <pT = <l>T, (10.49)

U2~W^X-y f2-~d(^X~V * r ~ d(sT + XT) ' Quite obviously -K = <p along the optimal path, so that the Hamiltonian can be expressed as

H=u(c,s+X)-vbx+\>sz + Tr[f(c,s+X)+x-z].

10.2 Jumps in the state variables 329

We can reduce this problem to one with a single state variable, say y = s+X. We maximize

\T[u{ct,yt)-tixtWtzt]e- btdt + e-bT<l>(yT) Jo

subject to

$t = f(Ct,yt)+Xt-zt9 y0 = s0, JYfree, (10.50)

0 < xt < x, 0 < z, < z.

The current-value Hamiltonian is

H=u(c,y)-pbx + tfz + t[f(c,y)+x-z], (10.51) and the optimality conditions are (10.50) and

(c, x, z) maximize H (in 10.51) subject to 0 < x < Jc, 0 < z < ?, (10.52)

* = -wi + (5-/ 2 'W, tfr = 0f. (10.53)

These conditions are identical to those of the preceding problem with the Hamiltonian (10.46), y = s+X and IT = cp = yp. The values of the bounds x and z are arbitrary; thus, the transactions can be made as abrupt as de- sired, but not instantaneous. We must assume p6 > p5; otherwise, the firm would simply buy and sell at the maximum rate throughout the horizon. Expanding (10.52) yields

wf+lfc/i = 0; (10.54)

- p * + ^ < 0 => JC = 0, - p * + 0 > O =>* = *, (10.55) - p * + ^ = 0«= 0<x<x;

p * _ ^ < 0 =>z = 0, p 5 - 0 > O = * z = z, (10.56) p 5 - ^ = 0^= 0<z<z.

It is impossible for x and z to be positive at the same time, since this im- plies yp > p*7 from (10.55) and \J/ < p5 from (10.56), which together contra- dict p^ > p5.

We now specialize the model for illustrative purposes by taking u(c,y) = lnc and f(c,y) = f(y) — c; p5 and p^ are constant. Equations (10.53)- (10.56) become

+ = V-f'(yM, +T = <l>tl 00.57)

c~l = i, cT = (^y\ (10.58)

330 10 Three special topics

Figure 10.4

-pZ 7 + ^ < 0 o r c > ( p z , ) _ 1 =>x = 0, - p ^ + ^ > 0 o r c < ( p ^ ) " 1 =>* = *, (10.59) - p ^ + ^ = 0or c = (p^)_1 <= 0<x<x;

p 5 - ^ < 0 o r c < ( p 5 ) - 1 =*z = 0, p 5 - 0 > 0 or 0 ( p V => z = ?, (10.60) p 5 - ^ = 0or c = (p5)- 1 <= 0 < z < z .

From these a phase diagram can easily be constructed; this is done in Fig- ure 10.4. When c is above (p5)"1, capital is sold at the maximal rate z\ when c lies between (p5)"1 and (p^)_1, no trade in capital takes place; when c is below (p6)- 1, capital is purchased at the maximal rate x. We can have 0 < x < x only along the c = (p*7)-1 line and 0 < z < z along the c = (p5)- 1 line. There are potentially three y — 0 loci, but only the thick portions are relevant because of the above restrictions on capital trading. Similarly, only along the thick portions of the c = (p5)- 1 and c = ($b)~l

lines does y change sign (it does not become zero, however); along other portions of these lines, trajectories have a kink but no change in the direc- tion of y. The values of p5 and p*7 affect the topography of the diagram; only one instance is represented in Figure 10.4. There is never a sudden

10.2 Jumps in the state variables 331

shift from buying to selling (or vice versa), but an intermediate no-trade phase occurs between the two, although not all three phases need be pres- ent. The buying or selling of capital need not be bunched at the beginning of the horizon. If z and x are taken to be large (so as to better approxi- mate a jump) the curves c=f(y) + x and c=f(y) -z become irrelevant.

One final remark concerns the scrap value function. We had set a cur- rent value </>(jr) arbitrarily for a terminal stock yT, but perhaps some argument can be provided to tie the scrap value to that of other exoge- nous prices. Let us call </>(y) the current scrap value of y units of capital; p5 and p^ are the prices of one unit of capital. Presumably whoever buys y can generate an economic rent of $sf(y) forever without altering the capital stock. (We have implicitly assumed that depreciation, if any, is accounted for by / ; see exercise 11.) Thus, it seems reasonable to define the capitalized value of y as

<t>(y) = \°°e-dt[ff(y)]dt Jo

VsAy) = ̂ Jjf±. (10.61)

When this scrap value function is used, the transversality condition is

\l/T = d(j>/dyT = )() sS-lf'(yT)i

or cT = tf)-

xb/f'{yT). (10.62)

Recall that y* is defined by b=f'(y*). Therefore, if yr = y*> then cT = (p5)-1, and if yT>y* (<y*)f then c r > ( p

5 ) - 1 (<(p5)- 1) by concavity of / . This is illustrated in Figure 10.5; the crossed line represents the trans- versality condition (10.62). It is now possible to restrict trajectories fur- ther. Inspection shows that all optimal trajectories must lie in the shaded area. A different configuration is drawn in Figure 10.6. We observe that in both these figures, buying or selling occurs first, if at all, followed (but not always) by a no-trade period. Therefore, in this model, and for fixed trad- ing prices and this scrap value function, trading does take place toward the beginning of the horizon. Our method of analysis, if not the results, can be applied to other models in which trade in capital goods is possible.

10.2.3 A final remark

In this section we have attempted to model additions (or subtractions) to the stock of capital from outside sources. In some cases it was presumed that lumps of capital could simply be added onto existing capital. This is not always reasonable; for instance, if the existing capital stock is the

332 10 Three special topics

Figure 10.5

total reserves of ore in a mine, the purchase of another mine would not normally just add to the stock (unless, perhaps, it were an adjacent strip mine). In some such cases the formulations of this section would be in- appropriate.

10.3 Constraints on the state variables

In all preceding chapters and sections we assumed that there were no con- straints of the form

0 * ( s , O ^ O , k = l,2,...9K. (10.63)

This is a constraint on the state variables. It differs from the constraints introduced in Chapter 6, which were of the form

gj(s9c9t)^09 y = l , 2 , . . . , m , (10.64)

in that it does not involve control variables. As explained in Section 6.1 constraints such as (10.64) restrict the controls, for given values of s and /,

10.3 Constraints on the state variables 333

Figure 10.6

whereas (10.63) simply restricts s, given /. Because of this, such constraints do not satisfy the constraint qualifications of Chapter 6, which required, among other things, that the vector of partial derivatives dgJ/dc be non- zero. As a consequence we are unable to use the control variables to satisfy these constraints, and the path of s(t) may lead to a "collision" with some constraints. When this occurs, the costate variables may exhibit jump discontinuities.

There are several methods of dealing with problems involving state vari- able inequality constraints. The method presented in this chapter consists of attaching a multiplier to each constraint; it yields relatively simple nec- essary conditions that are valid if the optimal path is "well-behaved," in a sense to be made clear later. This method can be used to identify candi- dates for the optimal path; if one of these also satisfies Theorem 10.3.2 (sufficiency), it is an optimal path. References to other, more complicated methods are given at the end of the section in case the above fails.

334 10 Three special topics

We now state the problem. Find a vector of controls c(t) that maximizes

V=[Tv(s,c9t)dt (10.65) Jo

subject to

i , = / ' ( s , c , / ) , i = l , 2 n,

gJ\s,c,t)>0, 7 = 1,2 m,

<t>k(s,t)>0, k = \,2,...,K,

Si(0) = si0, i = 1 , 2 , . . . , « ,

Sj(T) = siT, / = 1,2 n\

Si(T)>siT, i = i t ' + l , . . . , * ' ,

Sj(T) free, / = « " + ! , . . . , « .

(10.66a)

(10.66b)

(10.66c)

(10.66d)

(10.66e)

(10.66f)

(10.66g)

As usual we assume that the functions v, f, g, <f> are twice differentiable and that the constraint qualifications on g are satisfied (see Section 6.1).

The dates tj9 J= 1, ...,M, at which some state variable constraints be- come binding or cease to be so are called junction points; the initial and terminal times (0 and T) are also classified as junction points. The costate variables corresponding to the state variables involved at a junction point may exhibit a jump discontinuity - except that for fixed initial 5/(0), there can be no jump in 717 at t = 0. In what follows we also assume that the controls are continuous while ^ ( s , t) = 0.

10.3.1 Necessary conditions

The method expounded here is due to Jacobson, Lele, and Speyer (1971, p. 267). First form the Lagrangean

m K

£ = / / + £ \jgJ+ 2 iik4> kmH+\'*g +?'•</>, (10.67)

j = 1 k = 1 where

H=v+i ^ / ' W + Tr'-f. / = i

Theorem 10.3.1: necessity. Let (s*, c*) be a solution to the problem (10.65)-(10.66); then there exist costate variables TT and multipliers X and fi such that

(i) c* maximizes H subject to (10.66b); (10.68a) (ii) X y > 0 , g V , c * , 0 ^ 0 , X y g V , c * , 0 = 0,y = l,2,...,m; (10.68b)

(iii) ^ > 0 , r t s V ) ^ , ^ V , 0 = 0 , ^ l , . . . , * ; (10.68c)

10.3 Constraints on the state variables 335

(iv) 7T/ is both continuous and piecewise-differentiable except possibly at junction points; whenever 7T/(/) exists, it satisfies

TTi = -d£/dSi

= -Vsg-v'^fsj-X^gsj- n'94>si9 / = l,...,fl, (10.68d)

and at junction points tj9 J= 1, ...,M, the jumps in the costate variables are given by

7r/(/7)-MO+) = 0'(O)-<^, / = 1, ...,*, (10.68e)

where £ d<l>k

k = \ °Si

and t3k(tj) satisfies

^ ( O ) > 0 , Pk(tj)<l> k(s*(tj),tj) = 09 7 = 1 , . . . , M ; (10.68f)

(v) 7r,(r)>0, in(T)[snT)-siT] = 09 i = /!'+l,...,/!", (10.68g) TCi(T) = 0, i = n"+l,...,n. (10.68h)

Conditions (10.68e) and (10.68f) relating to the jumps in costate variables at junction points are the only new elements. Because of the form taken by these conditions, one must often first make a guess about the optimal path and then check it against the necessary conditions, as the following numerical example illustrates.

Example 10.3.1: a law-abiding speed enthusiast. A driver wishes to max- imize his enjoyment of a ride, and this depends on speed s(t) and acceler- ation c(t). There is a speed limit, and acceleration is also restricted. Find c(t) to maximize

V= (2[2(s(0)1/2 + 0.005(c(0-0.1)]tff (10.69) Jo

subject to

&(t) = c(t)-0.1, (10.70) l . l - c ( 0 > 0 , (10.71a) 2 - 5 ( 0 ^ 0 , (10.71b) 5(0) = 1 and 5(2) free. (10.71c)

Condition (10.71b) is the constraint on the state variable (speed), which is required to hold at all times. Since the maximand is an increasing func- tion of speed s, it is natural to guess that the speed limit should be reached in minimum time and thereafter maintained. Specifically,

336 10 Three special topics

c(t) = 1.1 until time tu at which s(tx) = 2;

c ( 0 = 0.1 w h e n ^ < ^ < 2 .

To find th set

[hs(t)dt = 2-s(0) = [t\lA-0A)dt = 2-\; hence, tx = l. Jo Jo

If we follow policy (10.72) we have

s*(t) = t + l, 0 < f < l , and s*(t) = 2, l < f < 2 .

We now proceed to verify that this solution satisfies Theorem 10.3.1. Let t£ = 2 5

1 / 2 + 0.005(c-0.1) + 7r(c-0.1) + X ( l . l - c ) + ^ ( 2 - 5 ) ; we require

4^- = 0.005 + TT - X = 0, (10.73a) dc

s = c-0A, (10.73b)

\ > 0 , l . l - c > 0 , X ( l . l - c ) = 0, (10.73c)

^ > 0 , 2 - 5 > 0 , fi(2-s) = 0, (10.73d)

7r(2) = 0 and 7r = — S~1/2 + IL whenever Sexists, (10.73e) and at each junction point tj

*(tT)-*(tj+) = P(tj)(-l), P(*j)*0, j8(O)(2-5(O)) = 0. (10.73f)

According to our solution, there are two junction points at t = 1 and t — 2. In the second half of the horizon, the constraint on c is not binding; hence, by (10.73c) X = 0 and by (10.73a) TT = - 0 . 0 0 5 . This contrasts with (10.73e), and there is a jump in -K at t = 2. Specifically, TT(2~) — 7r(2) = / 3 ( 2 ) ( - l ) = - 0 . 0 0 5 ; hence, 0(2) = 0.005 > 0 and s(2) = 2, satisfying (10.73f). Since TT = - 0 . 0 0 5 when 1 < / < 2, (10.73e) yields /* = 1/vl on that interval. In the first half (0 < t < 1) /i = 0 and -k = -(t + 1 ) " 1 / 2 ; thus, 7T = — 2(^ + 1 ) 1 / 2 + A If there is no jump in -K at the junction point f = 1, this must satisfy 7r(l) = —0.005; hence,

7r(O = - 2 ( / H - l ) 1 / 2 + 2 V 2 - 0 . 0 0 5 .

Substituting this in (10.73a) yields X = 2 V 2 - 2 ( / + l ) 1 / 2 > 0 when 0 < f < 1 and (10.73c) is satisfied. Therefore, there is no jump in w at / = 1 and j8(l) = 0. The only jump occurs at t = 2. In the next section we shall see that the sufficient conditions are also met and that (10.72) is indeed the unique optimal policy. Before we leave this example, note that if we had required that 5(2) = 2 in (10.71c), the transversality condition 7r(2) = 0 would have disappeared and no jump in TT would have been required, al- though the outcome would have been unchanged.

10.3 Constraints on the state variables 337

10.3.2 Sufficiency results

Theorem 10.3.2. If a path (s*, c*) and associated costates and multipliers satisfy the conditions of Theorem 10.3.1 and also

(i) T*(T)[Si(T)-s?(T)] > 0 for all feasible s^T), i = 1,..., n\ (ii) the Lagrangean is concave in (s,c); and

(iii) ( ^ ( s , t) has the property (k = 1,...,K)

</>*(s2, t) >4> k(sl91) => 0*(slf 0 * ( s 2 ~ s i ) > 0 , all s! and s 2

(which is automatically satisfied if <t>k is concave in s),

then this path represents an optimal solution to the problem (10.65)- (10.66).

Proof. For simplicity we assume that there are only two junction points t\ and t2, in addition to T. We have the by now familiar arguments,

[T(v*-v)dt=[T[(H*-Tr*'S*)-(H-ir**s)]dt Jo Jo

>[T(£*-£)dt+[TTT*'{s-s*)dt Jo Jo

<[(^)'-(s*-s)+( (by (10.66a) and (10.66b))

3c -T

l o

•(c*-c) dt

+ 1 ir*»(s—s*) dt, (by concavity of <£) Jo

>[T[ir*'(s-s*) + ir*'(s-s*)]dt (by(10.68d)) Jo

- ( ' Jo

T-%-[***(s-s*)]dt dt

= - * * ( 0 ) . [ s ( 0 ) - s * ( 0 ) ] + [ x ^ r ) - * * ( / i + ) ] « t s ( ^ ) - s * ( ^ ) ]

+ [**(t2)-ic*(tt)h[s(t2)-s*(t2)] + **(T-)'[s(T)-s*(T)]

> 0 . (10.74)

The right-hand side of (10.74) is the sum of nonnegative terms. To see this recall the following:

(i) s(0) = s*(0). (ii) For each tj (J= 1,2, T in our case),

[ * * ( / / ) - * * ( 0 + ) ] - [ s ( 0 ) - s * ( 0 ) ]

= 2 0k(tj)[<t>*(s*(tj),tj))-[s(tj)-s*(tj)] by(10.68e). k = \

338 10 Three special topics

In order to sign this expression, first note that &k(tj) > 0 by (10.68f); second, recall that at a junction point 4>k(s*(tj), tj) = 0, whereas <t>k(s(tJ)itJ) > 0 since s is feasible; then <l>

k(s(tj),tj) - <t>k(s*(tj), tj) > 0 and assumption (iii) of Theorem 10.3.2 implies that the expression is nonnegative.

(iii) ir*(r-).[s(r)-s*(r)] = [7r*(r-)-7r*(r)].[s(r)-s*(r)] + i r * ( 7 > [ s ( r ) - s * ( r ) ] .

The first term is nonnegative by the same argument as used in (ii). (7Ms a junction point and by convention ir(T) = ir(T+y, the second term is nonnegative by assumption (i) of Theorem 10.3.2.) •

Corollary 10.3.1. For infinite-horizon problems, replace assumption (i) of Theorem 10.3.2 with

lim Tr?(T)[Si(t)-s?(t)]>0 for all feasible st(t)9 / = 1,...,/I, (10.75a)

or, more generally, in case a limit does not exist for (10.75a),

lim inf Tc?(t)[Si(t)-s?(t)] > 0 for all feasible 57(f), '~*°° I = 1,...,/I. (10.75b)

For more general sufficiency results the reader is referred to Seierstad and Sydsaeter (1977). We now turn to a diagrammatic analysis of a con- trol problem involving a simple constraint on the state variable and show how the sufficiency results can be applied.

Example 10.3.2. Consider the fishery model of Section 9.5 and assume that the firm exploiting this renewable resource is required by law to main- tain the stock of fish above a predetermined level, say s - we must have s < 1 since a stock s > 1 cannot be maintained indefinitely (see Section 9.5 for notation and assumptions). The problem facing the firm is to find n(t) that maximizes

V= re- rt(2sl/2nl/2-wn)dt (10.76)

Jo subject to

« > 0 , (10.76a) s = s(\-s)-2sl/2nl/2, (10.76b) s(t)>s, (10.76c) s(0) = s0(>s). (10.76d)

Applying Theorem 10.3.1 we obtain a set of necessary conditions with the proviso that we work with the current-value Lagrangean; thus, (10.68d)

10.3 Constraints on the state variables 339

Figure 10.7

is modified to \j/ = r\p — d£/ds. The current-value Lagrangean is £ = 2sl/2nl/2-wn + \l/[s(l-s)-2sl/2nl/2] + ti(s-s) and we obtain

d£/dn = (l-\l,)(s{/2n-l/2)-w<:0 (=0ifn>0)9 (10.77a)

/ * > 0 , fi(s-s) = 0, s-s > 0 , (10.77b)

^ = - ( l - ^ ) 5 - 1 / 2 A ? 1 / 2 - h \ K r - l - h 2 5 ) - / i , (10.77c)

and at junction points

HtT)-Ht}) = Htj)y (10.77d)

where

0 ( 0 ) > 0 , 0 ( O ) ( 5 - 5 ) = O, 5 - 5 > 0 . (10.77e)

It is often advisable to build on the simpler analysis of the case without state constraints. Here we begin by ignoring (10.77b) and the multiplier /i. Then the phase diagram of Figure 9.1 is applicable; it is reproduced here in Figure 10.7, except that only the region to the right of the vertical line 5 = 5 is relevant. For concreteness assume that s>s°°, where the latter is the steady-state stock of the unconstrained problem of Section 9.5 (a

340 10 Three special topics

lower bound s < s°° would have no effect). In Figure 10.7, path A leads to the steady state, but it is now blocked by the state variable constraint s > s. We claim that the optimal trajectory, for any initial stock size 5(0) > 5, follows path B, which is above path A and leads to the inter- section of the s = 0 locus with s = 5, at point /. The intuition behind this guess is that the steady state is now out of bounds and a substitute must be found. At /, s = 0 but it is off the \j/ = 0 locus. However, equation (10.77c) reveals that \p can be adjusted by a change in [i when / is reached, which is in finite time. Note that \p does not jump at that point.

We now proceed with explicit calculations. For coordinates, the inter- section point / has ($,5) = (1 — (1 — s)w/2,s); this follows from (9.31). At this point, if there were no /*, \j/ would take the value given by (9.29), with

^ = ^ ( r - H - 2 5 ) - ( l - ^ ) 2 / w , (10.78)

which is positive because / is above the \p = 0 locus where \j/ > 0. The jump in /*, to the value equal to the right-hand side of (10.78), just makes \j/ — 0. The value of ^ does not jump and /? = 0. Note that Theorem 10.3.2 with Corollary 10.3.1 applies to this example because £ is concave in (5, n), the state constraint (10.76c) is linear, and the transversality condition (10.75a) is satisfied because all s values are bounded by s and 5(0) and

l i m 7 r * ( 0 = l i m e " r ^ = 0. / ->oo t ->oo

The economic implications of imposing the state constraint s > s are interesting. Denote the catch by c = 2s1 / 2«1 / 2; then along either path A or path B we have, by (10.77a), c = 2s(l — yp)w~x and since s decreases and ^ increases, the catch must decrease through time on the way to equilib- rium. Along s = 0, the catch must necessarily be c = 5(1 —s). Hence, the long-run value of the catch depends on s alone and is largest at s = 0.5. We know that (1 — r)/2 < s°°< s < 1, but we cannot ascertain whether s°° or s yields the higher catch. Since \f/ is higher along path B, we know that for the same s values, the catch is smaller than on path A, but it may con- verge to a larger equilibrium value.

10.3.3 Concluding comments

We attempt to provide an intuitive explanation of condition (10.68e) that indicates possible discontinuities of the costate variables at junction points. For simplicity we consider a problem with one inequality constraint on the state variables. Assume further that the time horizon is [0, T] and that over [0, t\) the state constraint is slack while it binds over [th T], Let (s*, c*) be an optimal path. Then over [0, t\), we must find c that maximizes

10.3 Constraints on the state variables 341

^o(s0, s*(f 0) = j ^ 1 y(s, c, t) dt (10.79)

subject to

s = f ( s , c , 0 , (10.80a) s(0) = s0, s(tl) = s*(tx) fixed. (10.80b)

Note that the state constraint </>(s, 0 ^ 0 can be ignored in this problem as 0(s*, t) > 0 by assumption. Denoting the costate variables by ir(t)9 we know that

dV0/dsf(tl) = -ici(tr). (10.81)

The second subproblem is to find c that maximizes

Vl(s*(ti),sT) = \ Tv(s,c9t)dt (10.82)

subject to

s = f ( s , c , 0 , (10.83a)

</>(s,/)>0, (10.83b) s(t{) = s*(t{), s(T) = sT. (10.83c)

Again, we know that if the costate variables for this problem are p(0, we have

dVx/dst(tx)=Pi(tf). (10.84)

If, however, s*(^) is optimal, it must be the solution of the "static" prob- lem to find s*(̂ i) that maximizes

^ ( s o ^ ^ + ^ s * ^ ) ^ ) (10.85)

subject to

<Ms*(f!Ui)^0. (10.86)

Let $(tx) be the multiplier associated with (10.86). Then problem (10.85) yields

dv0/dsr(t1)+dvl/dsr(tl)+pi(tl)dHdsr{tl)=o. oo.87) From (10.81), (10.84), and (10.87),

-x/(/r)+A-Ui +) + l8|Ui)0,/(s*(r1)^1) = O, (10.88)

which we recognize as a special case of (10.68e). It is interesting to observe the parallel between jumps in state variables

when costates hit the price boundary p(t) (the selling or buying price of the capital stocks), as in Section 10.2, and jumps in the costate variables when the state variables hit the admissible state boundary </>(s, 0 = 0. In

342 10 Three special topics

both cases we were able to explain the jumps by using the value function. It should be borne in mind that this approach is not meant to be rigorous and that in general value functions may not be differentiable.

Finally, we note that there exist more general methods for dealing with state constraints. One such method uses the observation that while a state constraint (j>k(s, t)>0 is binding, its time derivative is identically zero; hence,

or 0 $ . f ( s , c , O + */* = O. (10.89)

Condition (10.89) is a constraint involving the control variables and can replace the state constraint on the relevant interval. For details see Neu- stadt (1976) and Russak (1970).

Exercises

1. Consider the two-state-variable problem of maximizing $lU(su s2, cu c2)e~ btdt

subject to sl=F l(sus2,cl), s2 = F

2(sus2,c2) with conditions on ^(0), s2(0), sx(T)/s2(T). Assume that U is homogeneous of degree 0 in (sus2) and that both Fl and F2 are homogeneous of degree 1 in (sus2). Reduce this problem to a one-state-variable problem. (Hint: Let 0 = sx/s2\ find 0 and use the homo- geneity assumptions.)

2. Reconsider the problem of Example 10.2.1 with a buying price of capital p(t) = 0.8-h(/ — l)2. Show that an upward jump takes place at t = 1; find the size of the jump and the total revenue during the planning horizon.

3. Reconsider the problem of Example 10.2.1 when discounting is introduced. The Hamiltonian is now H = 2cl/2e~°-25t — nc. The buying price remains p(t) = 1 -I- (/ — 1 )2. Show that an upward jump still takes place at / = 1. Find the size of the jump; how does it compare with the jump in Example 10.2.1? Verify that all conditions of Theorem 10.2.1 apply.

4. Reconsider the problem of Example 10.2.1 with depreciation. The state equa- tion now reads s = — c — 0.3s, and the buying price is p(t) = e03l(\ + (t — \)2). Find the timing and size of an upward jump.

5. Consider the problem of maximizing \22cx/2e~bt dt + p(d)(s(6+) — s(B~)) sub- ject to s = -c — ms, s(0) = 2, and s(2) = 1, where m and h are specified posi- tive constants and p(t) is a specified function. Suppose that a downward jump takes place at time 0 and derive the values of s(6+) and s(6~) in terms of m, 6, 6, and p(B). Indicate the restrictions to be placed on p(t), p(6) and the values of 6, m9 6, and p(6) which guarantee that the jump is optimal and that s(6~) > s(6+) >0. Choose a functional form for p(t) and values for 6 and m that in- duce a downward jump at some point 6, O < 0 < 2 , and calculate the size of that jump.

6. Obtain the general solution (i.e., with two arbitrary constants) to the following problem: Maximize Ĵ 3(2)-2/3(s(0),/3(c(/))1/3tf/ subject to s(t) = -c(t) with

Exercises 343

5(0) and 5(2) to be specified. Show that the path of the state variable is of the form s(t) = (At + B)l/2, where A and B are arbitrary constants with A < 0. Check that the maximized Hamiltonian H°(s(t),ir(t)) is strictly concave in s(t). In the remainder of the exercise use 5(0) = 4 and 5(2) = 0. Derive the exact solution for s(t), 7r(0, and c(t) when no jumps are permitted. Calculate 7r(l).

Suppose now that the capital good s can be bought at price p(t) = 0.5er(1~°, where

_ f l 0 < / < l , r ~ [ o . 4 ' \<t<2.

Show that an upward jump at time 0 = 1 is optimal. Calculate the size of the jump. Compare the slopes of p(t) and ir(t) before and after the jump.

Alternatively, suppose that the capital good can be sold at price p(t) = er{X~t\ where

f0.2 0 < / < l , r ~ [ o . 6 l < / < 2 .

Show that a downward jump at time 0 = 1 is optimal. Calculate the size of the jump. Compare the slopes of p(t) and ir(t) before and after the jump.

7. Obtain the general solution (i.e., with two arbitrary constants) to the following problem: Maximize J* V3(s(0)1/6(c(0)1/2<# subject to s(t) = -c(t) with 5(0) and 5(2) to be specified. Show that the path of the state variable is of the form s(t) = (At +B)3/4, where A and B are arbitrary constants, with A < 0. Check that the maximized Hamiltonian H°(s(t), 7r(/)) is strictly concave in s(t). In the remainder of the exercise use 5(0) = 8 and 5(2) = 0. Derive the exact solu- tion for 5(/), 7r(0, and c(t) when no jumps are permitted. Calculate 7r(l).

Suppose now that the capital good can be bought at price p(t) = (0.06)1/4 x er{l~l\ where

f l . l 0 < / < l , r " [ o . 2 i < a < 2 .

Show that an upward jump is optimal at time 0 = 1. Calculate the size of the jump. Compare the slopes of ic(t) and p(t) before and after the jump. Choose another price path that would elicit the same jump (e.g., a piecewise-linear path).

Alternatively, suppose that the capital good can be sold at price p(t) = (0.44)1/4er(1~/), where

fO.l 0 < / < l , r " [ o . 3 l < / < 2 .

Show that a downward jump at time 0 = 1 is optimal. Calculate the size of the jump. Compare the slopes of p(t) and ir(t) before and after the jump. Choose another price path that would elicit the same jump (e.g., a piecewise-linear path).

8. Consider the problem of maximizing J Q ( 1 2 ) 1 / 3 ( 5 ( 0 ) 1 / 6 ( C ( / ) ) 1 / 3 ^ subject to s(t) = —c{t), 5(0) and 5(9) to be specified. Showthat the-general form of the

344 10 Three special topics

solution for the state is s(t) = (An B)2/3. Solve the problem with 5(0) = 16, 5(9) free and calculate 5(9). Now suppose that s(t) must never go below 1; i.e., we have a pure state constraint s(t)>\. Show that it is optimal for s(t) to remain above 1, for all t<9. What happens to the value of ir(t) when / reaches 9?

9. Consider the problem of maximizing ^2(s(t))l/2(c(t))l/2 dt subject to s(t) = 1 — c(t), with boundary conditions to be specified. Derive the solution to this problem. Show that the general solution for the state variable is 5(0 = 2 / - A + B^jA — lt, where A and B are arbitrary constants. In the first instance let T = 2 , 5(0) = 4, and 5(2) free. Derive the specific solution. Now impose the pure state constraint 5 ( 0 ^ 3 , W. Obtain the necessary conditions. Examine the following two possible solutions: (a) Follow the previous solution until 5 = 3; switch to c — 1 henceforth; (b) reach s = 3 at time 2, which is a junction point. Determine which proposal is optimal. Calculate the paths of all vari- ables and point out all discontinuities.

10. Reconsider the general solution to the problem of exercise 9, but now use the specification T= 1.5, 5(0) = 2, 5(7") = 2. In the first instance calculate the solution without constraints on the state variable. Now impose the capacity constraint s(t) < 2.16, W. Obtain the necessary conditions. Show that the opti- mal solution involves two junction points at which both the control and the costate are discontinuous - a phase diagram in (7r, S) might give a hint. Calcu- late the paths of all variables.

11. The model where smooth trading in capital goods was analyzed did not men- tion depreciation explicitly; this is the object of this exercise. We replace equations (10.44) and (10.45) by s = F(c,s + X) — ms and X — x — z — mX, respectively. Discuss how these pairs of equations differ. Derive the optimal- l y conditions in the case presented here; can you reduce it to a one-state- variable problem?

Bibliography

Arrow, K. J., and M. Kurz. Public Investment, the Rate of Return, and Optimal Fiscal Policy. Baltimore, Johns Hopkins University Press, 1970.

Athans, M., and P. L. Falb. Optimal Control. New York, McGraw-Hill, 1966. Bellman, R. Dynamic Programming. Princeton, N.J., Princeton University Press,

1957. Bellman, R., and S. Dreyfus. Applied Dynamic Programming. Princeton, N.J.,

Princeton University Press, 1962. Benhabib, J., and K. Nishimura. "The Hopf Bifurcation and the Existence and

Stability of Closed Onbits in Multisector Models of Optimal Economic Growth." Journal of Economic Theory 20 (1979), 421-44.

Bensoussan, A., E. G. Hurst, and B. Naslund. Management Applications of Modern Control Theory. Amsterdam, North Holland, 1974.

Benveniste, L. M., and J. A. Scheinkman. "Duality Theory for Dynamic Opti- mization Models of Economics: The Continuous Time Case." Journal of Economic Theory 27 (1982), 1-19.

Bliss, G. A. Lectures on the Calculus of Variations. Chicago, University of Chi- cago Press, 1946.

Brauer, F., and J. A. Nohel. Qualitative Theory of Ordinary Differential Equa- tions. New York, Benjamin, 1969.

Calvo, G. A., and M. Obstfeld. "Optimal Time-Consistent Fiscal Policy with Finite Lifetimes." Econometrica 56 (1988), 411-32.

Clark, Colin W. Mathematical Bioeconomics: The Optimal Management of Re- newable Resources. New York, Wiley, 1976.

Clark, C. W., H. C. Frank, and G. R. Munro. "The Optimal Exploitation of Re- newable Resource Stocks: Problems of Irreversible Investment," Economet- rica 47 (1979), 25-47.

Coddington, Earl A., and Norman Levinson. Theory of Ordinary Differential Equations. New York, McGraw-Hill, 1955.

Cohen, D., and P. Michel. "How Should Control Theory Be Used to Calculate a Time Consistent Government Policy?" Review of Economic Studies 54 (1988), 263-74.

Das, S. P., and Y. Niko. "A Dynamic Analysis of Protection, Market Structure and Welfare." International Economic Review 27 (1986), 513-23.

Dasgupta, P., and G. Heal. "The Optimal Depletion of Exhaustible Resources." Review of Economic Studies, 1974 Symposium, 3-29.

Diewert, W. E. "Duality Approaches to Microeconomic Theory." In K. J. Arrow and M. D. Intriligator (eds.), Handbook of Mathematical Economics, vol. 2. Amsterdam, North Holland, 1982.

345

346 Bibliography

Dixit, A. Optimization in Economic Theory. Oxford, Oxford University Press, 1976^

Dorfman, Robert. "An Economic Interpretation of Optimal Control Theory." Americal Economic Review 59 (1969), 817-31.

Dorfman, R., P. A. Samuelson, and R. M. Solow. Linear Programming and Economic Analysis. New York, McGraw-Hill, 1958.

Feichtinger, G. (ed.). Optimal Control Theory and Economic Analysis. Amster- dam, North Holland, 1982.

Feichtinger, G. (ed.). Optimal Control Theory and Economic Analysis, vol. 2. Amsterdam, North Holland, 1985.

Feichtinger, G. (ed.). Optimal Control Theory and Economic Analysis, vol. 3. Amsterdam, North Holland, 1988.

Fourgeaud, C , B. Lenclud, and P. Michel. "Technological Renewal of Natural Resource Stocks." Journal of Economic Dynamics and Control 4 (1982), 1-36.

Forster, Bruce A. "On a One State Variable Optimal Control Problem: Consump- tion-Pollution Trade-Offs." In J. D. Pitchford and S. J. Turnovsky (eds.), Applications of Control Theory to Economic Analysis. Amsterdam, North Holland, 1977.

Goldberg, Samuel. Introduction to Difference Equations. New York, Wiley, 1958.

Guesnerie, R., and J.-J. LafTont. "A Complete Solution to a Class of Principal- Agent Problems with an Application to the Control of the Self-managed Firm." Journal of Public Economics 25 (1984), 329-69.

Hadley, George. Linear Programming. Reading, Mass.: Addison-Wesley, 1962. Hadley, G., and M. C. Kemp. Variational Methods in Economics. Amsterdam,

North Holland, 1971. Halkin, Hubert. "Necessary Conditions for Optimal Control Problems with In-

finite Horizons." Econometrica 42 (1974), 267-72. Hamada, K. "On the Optimal Transfer and Income Distribution in a Growing

Economy." Review of Economic Studies 34, no. 3 (1967), 295-9. Harris, Milton. "Optimal Planning under Transaction Costs: The Demand for

Money and Other Assets." Journal of Economic Theory 12 (1976), 298-314. Hartl, R. "A Simple Proof of the Monotonicity of the State Trajectories in Auton-

omous Control Problems." Journal of Economic Theory 41 (1987), 211-15. Hestenes, Magnus R. Calculus of Variations and Optimal Control Theory. New

York, Wiley, 1966. Hirsch, M. W., and S. Smale. Differential Equations, Dynamical Systems, and

Linear Algebra. New York, Academic Press, 1974. Hotelling, H. "The Economics of Exhaustible Resources." Journal of Political

Economy 39 (1931), 137-75. Jacobson, D. H., M. M. Lele, and J. L. Speyer. "New Necessary Conditions of

Optimality for Control Problems with State Variable Inequality Constraints." Journal of Mathematical Analysis and Applications 35 (1971), 255-84.

Jensen, R., and M. Thursby. "A Strategic Approach to the Product Life Cycle." Journal of International Economics 21 (1986), 269-84.

Bibliography 347

Jovanovic, B., and S. Lach. "Entry, Exit and Diffusion with Learning by Doing." American Economic Review 79 (1989), 690-9.

Kamien, M. I., and N. L. Schwartz. "Optimal Exhaustible Resource Depletion with Endogenous Technical Change." Review of Economic Studies 45 (1978), 179-96.

Kamien, M. I., and N. L. Schwartz. Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management. Amster- dam, North Holland, 1981.

Kemp, M. C , and N. V. Long. "Optimal Control Problems with Integrands Dis- continuous with Respect to Time." Economic Record 53 (1977), 405-20.

Kemp, Murray C , and N. V. Long. Exhaustible Resources, Optimality and Trade. Amsterdam, North Holland, 1980.

Kemp, M. C , and N. V. Long. "On the Evaluation of National Income in a Dynamic Economy." In G. Feiwell (ed.), Samuelson and Neoclassical Eco- nomics. Boston, Nijhoff, 1982.

Kemp, Murray C , and N. V. Long (eds.). Essays in the Theory of Exhaustible Resources. Amsterdam, North Holland, 1984.

Kemp, M. C , and N. V. Long. "Union Power in the Long Run." Scandinavian Journal of Economics 89 (1987), 103-13.

Koopmans, T. C. "Proof of a Case When Discounting Advances Doomsday." Review of Economic Studies 41 (1974), 117-20.

Kurz, M. "The General Instability of a Class of Competitive Growth Processes." Review of Economic Studies 35 (1968), 155-74.

Kydland, F. E., and E. C. Prescott. "Dynamic Optimal Taxation, Rational Ex- pectations and Optimal Control." Journal of Economic Dynamics and Con- trol 2 (1980), 79-91.

Lefschetz, S. Stability of Nonlinear Control Systems. New York, Academic Press, 1965.

Leland, H. E. "The Dynamics of a Revenue Maximizing Firm." International Economic Review 13 (1972), 376-85.

Leonard, Daniel. "The Signs of the Costate Variables and Sufficiency Conditions in a Class of Optimal Control Problems." Economic Letters 8 (1981), 321-5.

Leonard, Daniel. "Costate Variables Correctly Value Stocks at Each Instant: A Proof." Journal of Economic Dynamics and Control 11 (1987), 117-22.

Leonard, Daniel. "Market Behaviour of Rational Addicts." Journal of Economic Psychology 10, no. 2 (1989), 117-44.

Long, N. V. "International Borrowing for Resource Extraction." International Economic Review 15 (1974), 168-83.

Long, N. V. "Resource Extraction Under the Uncertainty About Possible Nation- alization." Journal of Economic Theory 10 (1975), 42-53.

Long, N. V. "Optimal Exploitation and Replenishment of a Natural Resource." In J. D. Pitchford and S. J. Turnovsky (eds.), Applications of Control The- ory to Economic Analysis. Amsterdam, North Holland, 1977.

Long, N. V., and H. Siebert. "Optimal Foreign Borrowing: The Impact of the Planning Horizon on the Half and Full Debt Cycle." Zeitschrift fur Nat\onal- okonomie 49 (1989), 279-97.

348 Bibliography

Long, N. V., and N. Vousden. "Optimal Control Theorems." In J. D. Pitchford and S. J. Turnovsky (eds.), Applications of Control Theory to Economic Analysis. Amsterdam, North Holland, 1977.

Mangasarian, O. L. "Sufficient Conditions for the Optimal Control of Nonlinear Systems." SI AM Journal of Control 4 (1966), 139-52.

Mangasarian, Olvi. Nonlinear Programming. New York, McGraw-Hill, 1969. Manning, R. "Optimal Aggregative Development of a Skilled Workforce." Quar-

terly Journal of Economics 89 (1975), 504-11. Michel, P. "On the Transversality Condition in Infinite Horizon Optimal Control

Problems." Econometrica 50 (1982), 975-85. Milne, Frank. "The Adjustment Cost Problem with Jumps in the State Variable."

In J. D. Pitchford and S. J. Turnovsky (eds.), Applications of Control The- ory to Economic Analysis. Amsterdam, North Holland, 1977.

Mirrlees, J. A. "Optimum Growth When Technology Is Changing." Review of Economic Studies 34 (1967), 95-124.

Mirrlees, J. A. "The Optimum Town." Swedish Journal of Economics 74 (1972), 114-35.

Neustadt, L. W. Optimization: A Theory of Necessary Conditions. Princeton, N.J., Princeton University Press, 1976.

Nordhaus, W. "The Political Business Cycle." Review of Economic Studies 42 (1975), 165-90.

Pitchford, J. D. Population in Economic Growth. Amsterdam, North Holland, 1974.

Pitchford, John D. "Two State Variable Problems." In J. D. Pitchford and S. J. Turnovsky (eds.), Applications of Control Theory to Economic Analysis. Amsterdam, North Holland, 1977.

Pitchford, J., and S. Turnovsky (eds.). Applications of Control Theory to Eco- nomic Analysis. Amsterdam, North Holland, 1977.

Pollak, R. A. "Consistent Planning." Review of Economic Studies 35 (1968), 201-8.

Pontryagin, L. S. Ordinary Differential Equations. Reading, Mass.: Addison- Wesley, 1962.

Pontryagin, L. S., V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko. The Mathematical Theory of Optimal Processes. New York, Wiley, 1962.

Quyen, N. V. "The Optimal Depletion and Exploration of a Nonrenewable Re- source." Econometrica 56 (1988), 1467-71.

Ramsey, F. "A Mathematical Theory of Savings." Economic Journal (1928); re- printed in K. J. Arrow and T. Scitovsky (eds.), Readings on Welfare Eco- nomics. Homewood, 111., Irwin, 1969.

Russak, I. B. "On General Problems with Bounded State Variables." Journal of Optimization Theory and Applications 6 (1970), 424-52.

Ryder, H a d E., Jr., and Geoffrey M. Heal. "Optimal Growth with Intertempor- ally Dependent Preferences." Review of Economic Studies 40 (1973), 1-31.

Sampson, A. A. "A Model of Optimal Depletion of Renewable Resources." Jour- nal of Economic Theory 12 (1976), 315-24.

Bibliography 349

Seierstad, A. "A Sufficient Condition for Control Problems with Infinite Hori- zons." Memorandum from Institute of Economics, University of Oslo, Jan. 26, 1977a.

Seierstad, A. "Transversality Conditions for Control Problems with Infinite Hor- izons." Memorandum from Institute of Economics, University of Oslo, Jan. 27, 1977b.

Seierstad, A. "Sufficient Conditions in Free Terminal Time Optimal Control Problems." Journal of Economic Theory 32 (1984), 367-71.

Seierstad, Atle, and Knut Sydsaeter. "Sufficient Conditions in Optimal Control Theory." International Economic Review 18 (1977), 367-91.

Seierstad, A., and K. Sydsaeter. Optimal Control Theory with Economic Appli- cations. Amsterdam, North Holland, 1987.

Sethi, S. P., and G. L. Thompson. Optimal Control Theory Applications to Man- agement Science. Boston, Nijhoff, 1981.

Shell, Karl (ed.). Essays on the Theory of Optimal Economic Growth. Cam- bridge, Mass., MIT Press, 1967.

Smith, Vernon L. "An Optimistic Theory of Exhaustible Resources." Journal of Economic Theory 9 (1974), 384-96.

Stokey, N. L. "Learning by Doing and the Introduction of New Goods." Journal of Political Economy 96 (1988), 701-17.

Strotz, R. H. "Myopia and Inconsistency in Dynamic Utility Maximization." Re- view of Economic Studies 23 (1955-6), 165-80.

Takayama, Akira. Mathematical Economics. Cambridge University Press, 1985. Tu, Pierre N. V. Introduction to Optimization Dynamics: Optimal Control with

Economics and Management Science Applications. Berlin, Springer, 1984. Uzawa, H. "Optimal Growth in a Two-Sector Model of Capital Accumulation."

Review of Economic Studies 31 (1964), 1-24. Vind, Karl. "Control Systems with Jumps in the State Variables." Econometrica

35 (1967), 273-7. Vousden, Neil. "Basic Theoretical Issues of Resource Depletion." Journal of Eco-

nomic Theory 6 (1973), 126-43. Vousden, Neil. "Resource Depletion with Possible Nonconvexities in Produc-

tion." In J. D. Pitchford and S. J. Turnovsky (eds.), Applications of Control Theory to Economic Analysis. Amsterdam, North Holland, 1977.

Weitzman, M. "Welfare Significance of National Product in a Dynamic Econ- omy." Quarterly Journal of Economics 90 (1976), 156-62.

Index

admissible controls, 187 advertising, 261 asymptotic stability, 90 autonomous differential equations, 89, 90,

143, 167 autonomous problems, 149, 289, 292, 294,

295, 298

backward induction, 176 bang-bang solution, 263 beekeeper's problem, 267 Bellman's equation, 176 Bernoulli equation, 93 binding constraint, 52 biomass, 107 borrowing, optimal, 118 boundary conditions, 89, 142 bounded optimization, 53, 54, 201, 265,

268, 275, 278 bounded set, 75 business cycle, political, 260

calculus of variations, 169 catching-up criterion, 286 center, 100 characteristic equation, 96 characteristic roots, 96, 173 closed set, 75 closed-loop control, 181 closed-loop solution, 181 comparative statics, 43-52 compensated price change, 47 competitive equilibrium, 41, 43 complementary slackness, 199, 230 concave function, 1, 3-7, 12, 13 concave Hamiltonian, 163, 217 concave Lagrangean, 31, 32, 213, 214, 251,

337 concave programming, 61 conditional stability, 99 constrained control problem, 189 constraint qualification, 58 consumption: constant, 197; optimal, 133,

135, 199, 277; suboptimal, 120, 125 control parameters, 253, 300, 301

control problem, 127 control variable, 127 convex function, 1, 8-9, 12 convex set, 4, 7, 8, 75 cost function, 32-3, 38, 43-7, 84 costate variable, 128 costate variables as prices, 152 current-value costate, 149, 269, 279, 299 current-value Hamiltonian, 149, 150, 206,

295 cusp, 23, 58, 59, 60

decay, see depreciation demand function, 33; Hicksian, 47;

Marshallian, 47 depreciation, 121, 123, 159 discount factor, 122, 123, 151, 179 discounting, 121, 122, 123, 135 doomsday, 261 dual problem, 71 dual variable, 55; and economic

interpretation, 67 duality theory, 38 dynamic inconsistency, 151 dynamic programming, 169, 173, 182

education, 48 efficient allocation of resources, 38, 67, 80 endpoint: constraint on, 229, 235; fixed,

222; free, 228 envelope theorem, 36, 38, 64, 215 equality constraints, 20, 192 equilibrium: dynamic, 90, 145; in finite

time, 272, 281 Euler's equation, 170, 172, 184,185 Euler's theorem, 17, 78, 219 exhaustible resource, 194, 203, 245, 258,

274, 282, 300, 305, 313 existence of solution, 18, 89, 165

feasible set, 52 first-order condition, 2, 8, 21, 129 fiscal policy, 119, 124 fisheries, 105, 295, 338 fishing, commercial, 219, 262

351

352 Index

focus, 99, 107 free initial conditions, 248, 249 free initial time, 247 free terminal time, 240, 244 functional recurrence equation, 175, 182

general solution to differential equations, 89 global maximum, 2, 3, 7, 31 golden rule, 273, 280 growth, optimal, 145, 158 growth model, 117

Hamilton-Jacobi-Bellman equation, 169, 182, 183, 185, 299

Hamiltonian: defined, 128; maximized, 163, 214, 217, 319, 321, 343

Hamiltonian systems, 107-11, 215, 319 Hessian matrix, 3, 7, 76 homogeneous differential equation, 91 homogeneous function, 16, 78, 218 Hotelling's rule, 231

implicit function theorem, 78 imputed value, 35, 155 inconsistent planning, 151, 157 inequality constraints, 52, 198 initial conditions, 89 inner product, 76 input demand functions, 38 integral constraints, 190 integration by parts, 111 integration by substitution, 112 "invisible hand," 35, 83 irreversible investment, 199 isosector, 103

jumps: in control variables, 190, 265, 269, 277, 281; in costate variables, 341; in state variables, 310-27, 341

junction point, 334, 336

Kuhn-Tucker conditions, 52, 55, 63, 65

Lagrange, method of, 20-43 Lagrange multipliers, see multipliers Lagrangean, 21, 192 Leibniz's rule, 113 level curve, 77 limit cycles, 101, 107, 297 linear expenditure system, 33 linear programming, 7 0 - 4 lower contour set, 8, 11 macroeconomic model, 117 marginal physical product, 41, 76

marginal value product, 41, 76 maximin criterion, 300, 301 maximization: equality-constrained,

20-43; inequality-constrained, 57-67; unconstrained, 1-3, 8

maximum principle, 127, 129; derivation of, 161; in discrete time, 129; economic interpretation of, 151, 155-7; with equality constraints, 192; with inequality constraints, 198

maximum value function, 38 mineral spring, 166 minor, principal, 77 mixed problems, 63, 210 multipliers, 20, 193; as prices, 34 mushroom grower's problem, 205, 216, 218

negative-definite matrix, 76 node, 97, 99 nondifferentiable price paths, 323 nonlinear programming, 52

open-loop control, 181 open set, 75 optimal saving, 236, 242 optimality criteria in infinite horizon, 285 overtaking criterion, 287

parameter, 42, 44, 113, 153, 155, 253 Pareto optimum, 41 partial differential equation, 184 particular solution, 89 peakload policy, 69, 85, 257 phase diagram, 94, 95, 97, 101, 137-49,

201-10, 216-17 piecewise continuous, 165, 170, 190, 263 piecewise differentiable, 145, 170, 263 planner, central, 34, 155, 156, 157, 158 pollution, 102, 114, 224, 228 positive-definite matrix, 76 predator-prey model, 115 present value, 121, 122 primal problem, 71 principle of optimality, 169, 174, 182 profit maximization, 15, 17-19, 166, 219,

230, 243

quadratic form, 17, 76

rank condition, 21, 58, 60, 189 Rawlsian criterion, 301 regular maximum, 28 resource: nonrenewable, 168, 258;

renewable, 168, 267, 274, 283, 338 return function, 175

Index

returns to scale, 16,18 reversibility of investment, 120

saddle point, 13, 15, 72, 99, 276, 293, 297 salvage value, see scrap value function scrap value function, 226, 235, 240, 244 second-order conditions, 26 shadow price, 35, 69, 155 slack constraint, 152 Slater's condition, 58, 61 species, competing, 107 stability of equilibria, 90 stable path, 141, 147 state variable, 127 steady state, in autonomous problems, 294 strictly concave function, 7 strictly concave Hamiltonian, 319, 321 strictly convex function, 8 switch line, 265

353

tax, 119 Taylor's expansion, 2, 3, 4, 77 terminal time, optimal, 240, 244 time preference, 151, 157 total derivative, 78 total differential, 2, 23, 77 trade in capital goods, 310, 328 transition equation, 174 transversality condition, 221, 248, 254 trap, 106

unconstrained optimization, 1 upper contour set, 6,10, 11

value function, 33, 151

welfare economics, 39-43

yabbies, 220, 305